Overview
Content
The Core Idea
The silktorrent packet is a tar-file that contains folders
payload
header
In software projects and HTML pages the content of the folders payload
and header can be referenced by using the
tree
encoding:
<tar-file name>/payload/<the file or folder relative to the
folder payload>
<tar-file name>/header/<the file or folder relative to the
folder header>
The <tar-file name> can be the name of a local folder or some
folder at some publicly hosted web page. The include/require/src/uses
statements of various programming languages and configurations files, including
HTML, can be modified by changing the prefix of the <tar-file name>. That
allows an HTML-page to be switched from using JavaScript libraries from one
site to using the very same JavaScript libraries from another site.
http://www.first_site.com/<tar-file name>/payload/the_JavaScript_library.js
http://www.second_site.com/<tar-file name>/payload/the_JavaScript_library.js
Due to the possibility to use regular expressions, the proposed solution
does not require code generation. It might be usable with many existing IDE-s
without requiring any additional tools or plugins. The solution is also
programming language agnostic.
Theory
The Core Idea has an assumption that backwards compatibility of
libraries does not exist, even if advertised. Whenever a software component
dependency is declared by stating that the version of the dependency is
"newest" or "greater than X", a flaw is introduced. That is the reason, why
Linux/BSD/etc. package collections are "unstable" and NEVER WILL BE "stable".
For built/compiled software components each combination of build parameters,
compiler, environment set-up is actually a separate version of the software
component.
Implementation Notes
Storage Allocation Policy
To make the Silktorrent network of package/packet hosting servers more
reliable, the package/packet hosting servers should use multiple storage
allocation policies simultaneously. There is one allocation agent per
policy. Each allocation agent has its own, fixed, size of storage space, disk
space, which will be allocated to Silktorrent packets according to the policy
that the agent implements. Some agent, bot, may sell paid subscriptions like
the Dropbox and alike use. Some agent might run a mirroring service in favor of
some public library or operating system packages repository. Some agent might
store Silktorrent packets according to popularity. Some agent may offer
personal storage service to the owner of the server. Some agent may service
some Silktorrent based messaging service. Some agent might service Silktorrent
based "web" (PDF-files, LibreOffice files, all-in-one-HTML-documents, etc.)
To avoid duplicate copies of the same Silktorrent packet at the same
storage server, the agents of a single storage server may use a single "storage
engine" that keeps track of the storage space quotas of the agents and
physically deletes the Silktorrent packet only, if no agent on this sotorage
server wants to store that Silktorrent packet.
Storage Size Requirements
The hashes within the Silktorrent packet names depend on the tar files,
silktorrent packets. Hash of a tar-file depends on the file attributes,
including the date-and-time attributes. That's why it is not possible to
recreate a Silktorrent packet from unmodified content of an unpacked
Silktorrent packet without some special hacking, meaning: copies of downloaded
Silktorrent packets must not be deleted. To use the downloaded Silktorrent
packets without un-tar-ing and waiting for the slow HDD operation to finish, an
unmodified set of un-tar-red Silktorrent packets must be stored. Given the huge
amount of Silktorrent packets, the packets must be distributed among a set of
folders, because otherwise the file systems are slow to use. The recursive
folder names may be derived from the first letters of Silktorrent packet names.
The Silktorrent packet names were intentionally designed to contain equally
distributed letters at the start of their packet names, because that allows
database indices that index the Silktorrent packet names to probabilistically
work faster.
Silktorrent packets, tar-files, that contain software/datacollection forks
or newer versions of the software/datacollection, might consist of many
bitstreams that match with the original version of the software/datacollection
Silktorrent packets. Storage space for storing a collection of tar-files that
contain relatively long, common, bitstreams, can be reduced by storing the
common, relatively long, bitstreams only once. The search for the common
bitstream tokens can be greatly mitigated by explicitly telling, which
Silktorrent packets, tar-files, are version wise or fork wise closely related.
The arising
knapsack
problems might be solved by using specialized
open source libraries.
Partial list of Similar Software