Update of "Experiment: mmmv_silkexec"
Not logged in
Overview

Artifact ID: 90ce818db49ff8277850de72738beb0d413c4a58
Page Name:Experiment: mmmv_silkexec
Date: 2017-06-02 01:05:10
Original User: martin_vahi
Parent: 1438ed9d7a42499140b991919a82eebba98ab89a (diff)
Next b20adfca7c119446b281763c1d803a8ff74f5e11
Content


The Current and Incomplete Version of this Specification

Currently code for the mmmv_silkexec does not exist. The flaws that the current version must eliminate are described at the page of the most current deprecated version.


Deprecated Versions of this Specification


Partial Draft of the Current Specification

mmmv_silkexec application (hereafter: silkexapp) is wrapper to other applications, including other mmmv_silkexec applications. The purpose of the wrapping is to use tags and user specific configuration parameters for using the wrappable application. Each silkexapp is distributed as a Silktorrent packet. Silkexapp declares its dependencies either by listing the Silktorrent packets that the dependencies have been wrapped to or by listing the names of the command line programs that must be on the PATH. Silkexapps are searched from a path that is described at the environment variable named MMMV_SILKEXEC_TRUSTED_REPOSITORY. Optionally there may also be a folder path referencing environment variable named MMMV_SILKEXEC_UNTRUSTED_REPOSITORY, but Silktorrent packets at that folder are expected to be accessed only for scrutinizing/auditing. 

Silkexapps must use/set their own build specific and execution specific values for the LD_LIBRARY_PATH and PATH. The general ideology is that almost nothing is assumed of the silkexapps and the silkexapps are expected to optionally use the environment that is offered to them, except that the silkexapp packaging format.


Silkexapp Format

None of the silkexapp standard file and folder names contain any spaces, tabulation characters, line breaks. All silkexapp configuration text files, the ones at the silkexapp etc folder, are preprocessed by removing all spaces, tabulation characters, single-line-comments that start with either "#" or "//". After the single-line-comments deletion preprocessing step all of the linebreaks are deleted, except for the list_of_immediate_dependencies.txt.

<silktorrent packet name>

  |

  +--payload

      |    

   +--silkexapp
     |
     +--<A folder that has the silkexapp format version as its name, for example, format_version_01.>
        |
        # The rest according to the silkexapp format version

Silkexapp Format Version 01

format_version_01
  |
  +--Rakefile
  |
  +--bin
  |
  +--bonnet 
     |
     +--silkexapp_standard_interface.rb # A Ruby file that contains the derived class.
     |
     +--list_of_immediate_dependencies_silkexapps.txt # Optional. If it exists, then 
     |                                                # there is one Silktorrent packet name per line.
     |                                                # All of the Silktorrent packets are required to 
     |                                                # conform to the Silkexapp format.
     |                                                # The list includes build dependencies.
     |
     +--list_of_immediate_dependencies_on_PATH.txt  # Optional. If it exists, then 
     |                                              # there is a semicolon-separated list of 
     |                                              # command line programs. The list can be placed to 
     |                                              # multiple lines. The last command line program name 
     |                                              # at the end of the line may be left without the
     |                                              # semicolon suffix.
     |
     |


Silkexapp is implemented as a Ruby class that is derived from a silkexec specific base class.

The silkexec uses reflection to study, whether the derived class meets silkexec specification.

The silkexapp Ruby class is required to override silkexec API related methods.

The Rakefile of the silkexapp calls the same silkexapp specific Ruby library that the siliexapp Ruby class calls, but

the silkexapp Rakefile may also call the silkexec API calls by calling the Ruby API of the silkexec command line tool.


Action plan

Create a reference implementation of the mmmv_silkexec specification and document the reference implementation. The documentation will be in the role of the mmmv_silkexec specification. Preliminary ideas about the implementation:



Template text from Older spec for Copying Parts of it


A mmmv_silkexec application (hereafter: silkexapp) is wrapper to other applications, including other mmmv_silkexec applications. The purpose of the wrapping is to use extra tags and user specific configuration parameters for using the wrappable application.

The mmmv_silkexec project consists of the following tools:

mmmv_silkexec <full path of or a name on the PATH> <the command line args of the silkexapp>

No technical solution is going to compensate for crappy work, because technical tools can be switched off or the original authors of software may leave edge cases out of consideration. The main ideas behind the mmmv_silkexec are to allow the execution of only those silkexapps that have been tested and/or verified by trusted parties, the NixOS package manager idea that each application is executed in an environment, where the LD_LIBRARY_PATH and the PATH have silkexapp specific prefixes, the reliance on operating system services is limited to the UNIX standard to give the mmmv_silkexec a chance to be ported to all operating systems that have the UNIX layer (Linux, BSD, GNU Hurd, Genode OS, Minix3), the mmmv_silkexec must be fully usable without having any root access.



Silkexapp Format (Old version for Copy-Pasting parts of it)

None of the silkexapp standard file and folder names contain any spaces, tabulation characters, line breaks. All silkexapp configuration text files, the ones at the silkexapp etc folder, are preprocessed by removing all spaces, tabulation characters, single-line-comments that start with either "#" or "//". After the single-line-comments deletion preprocessing step all of the linebreaks are deleted, except for the list_of_immediate_dependencies.txt.


<silktorrent packet ID as folder name>
    |
    +--bin  # is a symlink to ./build/bin . The symlink gets created by
    |       # the mmmv_silkexec after the building of the silkexapp has succeeded. 
    |       # The mmmv_silkexec deletes this symlink before the execution of the build_script.bash .
    |       # If the build_script.bash exited with an error, the mmmv_silkexec does not create this symlink.
    |
    +--doc  # is an optional symlink to ./build/doc . The symlink gets created by
    |       # the mmmv_silkexec after the building of the silkexapp has succeeded. 
    |       # The mmmv_silkexec deletes this symlink before the execution of the build_script.bash .
    |       # If the build_script.bash exited with an error, the mmmv_silkexec does not create this symlink.
    |
    +-- etc # is an optional folder
    |    +--runtime_environment # is an optional folder
    |    |    |
    |    |    +--overriding_values_of_environment_variable_values   # the existence of this folder is optional
    |    |    |    |  # All files in this folder are optional. File names without the ".txt" suffix
    |    |    |    |  # must match with the environment variable name. File names are case sensitive.
    |    |    |    |  
    |    |    |    +--LD_LIBRARY_PATH.txt
    |    |    |    +--PATH.txt
    |    |    |    +--CFLAGS.txt
    |    |    |    +--<other environment variables>
    |    |    |
    |    |    +--prefixes_of_environment_variable_values   # the existence of this folder is optional
    |    |         |  # All files in this folder are optional. File names without the ".txt" suffix
    |    |         |  # must match with the environment variable name. File names are case sensitive.
    |    |         |  
    |    |         +--LD_LIBRARY_PATH.txt
    |    |         +--PATH.txt
    |    |         +--CFLAGS.txt
    |    |         +--<other environment variables>
    |    |
    |    +--buildtime_environment # is an optional folder.
    |    |                        # The content of this folder has the same format
    |    |                        # as does the folder ./runtime_environment
    |    |
    |    +--list_of_immediate_dependencies.txt # Optional. If exists, then 
    |    |                                     # there is one Silktorrent packet name per line.
    |    |                                     # All of the Silktorrent packets are required to 
    |    |                                     # conform to the Silkexapp format.
    |    |                                     # The list includes build dependencies.
    |    |
    |    +--list_of_aggregate_Silktorrent_packets.txt # Optional. If exists, then 
    |                                                 # there is one Silktorrent packet name per line.
    |                                                 # From the point of view of this mmmv_silkexec
    |                                                 # specification the content of those Silktorrent packets
    |                                                 # is not determined.
    |
    +--src # After unpacking it is recursively set to file permissions 04AB, 
    |    | # where A,B inSet{4,0} and each of them, the A and the B, is 
    |    | # determined by mmmv_silkexec settings, which are manipulated by 
    |    | # mmmv_silkexec sys-commands. Everything in the this folder, the src,
    |    | # is automatically copied to the build folder before building. 
    |    |
    |    +--silkexapp_control
    |    |    |
    |    |    +--build_script.bash # is executed as a copy in the subfolder of the build folder. 
    |    |    |                    # Building always includes bootstrapping. If the build fails,
    |    |    |                    # then the build_script.bash must exit with a non-0 error code.
    |    |    |
    |    |    +--run_all_tests_and_exit_with_0_if_all_tests_pass_t1.bash # should describe failures at stdout or stderr
    |    |    |
    |    |    +--si_storage_space_size_in_bytes_without_including_dependencies.bash # Prints a number to console.
    |    |    |                                                                     # The printout may include 
    |    |    |                                                                     # a line break.
    |    |    |
    |    |    +--si_recursive_storage_space_size_of_all_immediate_dependencies_in_bytes.bash 
    |    |    |                                                                     # Prints a number to console.
    |    |    |                                                                     # The printout may include 
    |    |    |                                                                     # a line break.
    |    |    |
    |    |    +--s_silkexapp_core_version.bash # Prints a string to console. To facilitate 
    |    |    |                                # the use of signatures, the string is allowed to  
    |    |    |                                # include line breaks, spaces, tabulation characters.
    |    |    |                                # 
    |    |    |                                # The idea is that different silkexapps can use the same  
    |    |    |                                # version of an application source, but the application source 
    |    |    |                                # can be built by using different build parameters. 
    |    |    |                                # Different silkexapps that build the application by 
    |    |    |                                # using the same build parameters, can have 
    |    |    |                                # different silkexapp scripts and different developers.
    |    |    |
    |    |    +--sb_is_ready_4_silkexapp_package_format_tests.bash # Prints a string from the set {"t","f"}, 
    |    |    |                                                    # where "t" stands for "true" 
    |    |    |                                                    # and "f" stands for "false".
    |    |    |                                                    # The string does not end with a line break.
    |    |    |                                                    # 
    |    |    |                                                    # The "f" indicates that not all 
    |    |    |                                                    # silkexapp format specific scripts and 
    |    |    |                                                    # folders are present or ready to be used.
    |    |    |                                                    # The "t" indicates that the current 
    |    |    |                                                    # specification is expected to be met
    |    |    |                                                    # to the extent that the silkexapp scripts
    |    |    |                                                    # can be tested. The output value can be
    |    |    |                                                    # calculated during runtime, dynamically. 
    |    |    |                                                    # For example, the script may study the 
    |    |    |                                                    # value of environment variables and the presence
    |    |    |                                                    # and content of files.
    |    |    |                                                    # 
    |    |    |                                                    # An optional command line parameter, 
    |    |    |                                                    # "explanation", without the quotes, 
    |    |    |                                                    # is expected to change the output mode 
    |    |    |                                                    # from the "t" and "f" to a freestyle 
    |    |    |                                                    # text that explains the circumstances 
    |    |    |                                                    # that effect the choice between the 
    |    |    |                                                    # "t" and "f". The freestyle text can
    |    |    |                                                    # contain line brakes, tabulation characters and 
    |    |    |                                                    # spaces.
    |    |    |
    |    |    +--delete_cache_content_t1.bash # sets silkexapp state to as if it were installed the 1. time
    |    |                                    # If build succeeds and this file exists, then 
    |    |                                    # it is automatically called by the mmmv_silkexec
    |    |                                    # after the build. It must be able to handle a case, 
    |    |                                    # where there is no copy of the cache.
    |    |
    |    +--upstream_deliverables # is a folder for unpatched, unmodified, originals
    |
    +--build  # is deleted by the mmmv_silkexec every time before the silkexapp is built .
         |    # The mmmv_silkexec sets its file permissions recursively 
         |    # to 07AB before executing the build_script.bash. 
         |    # The A and the B are the same as with the src folder.
         | 
         +--bin # is the target of the silkexapp bin folder symlink. This folder 
         |      # is created automatically before the build_script.bash is run.
         |
         +--doc # an optional folder that is meant to be created by the build_script.bash .
         |      # If this folder exists and the build_script.bash has 
         |      # exited without any errors, id est with error code 0, 
         |      # then mmmv_silkexec creates a symlink to this folder 
         |      # from the silkexapp folder.
         |
         +--silkexapp_control  # copy of the folder from the src

Silkexapps are installed/unpacked to

To make it possible to share silkexapps between different users of a same system, packages are allowed to assume that they have write access only to 

The mmmv_silkexec preprocesses silkexapp names by removing a single instance of the following prefixes: "sys_", "usr_". A silkexapp that is on PATH and has a name of "sys_foo" should be executed as "mmmv_silkexec usr_sys_foo". The prefix "sys_" is reserved for mmmv_silkexec commands. Preprocessing is not applied, when the silkexapp executable is referred through a full path.  The mmmv_silkexec sets the values of silkexapp runtime and buildtime enironment variables according to the following precedence: 

  1. If configuration exists, override the value of the environment variable.
  2. Prefix the value from the step 1 with the value from the configuration file.
  3. Prefix the value from the step 2 with the full path of the silkexapp bin folder.
  4. <Execute the executable from the silkexapp bin folder or silkexapp_control folder.>



Testing (Including Verification and Audits)

Silkexecapp is executed only, if all of its dependencies have been recursively audited/tested by trusted parties. Different parties trust different other parties. Trust is a multidimensional value. A very kindhearted and not corrupt person can be totally untrustworthy from their capabilities point of view. On the other hand, sometimes, at some narrow contexts, enemies can be more trustworthy than friends. For example, enemies might have a good track record of fine skills and being rigorous and they might use some component, software package, at some security wise extremely critical role, while being very rigorous at the construction of their software component.

Given that the only proper way to verify/review code is to study it in a fully automated way(archival copy), there has to be a way to subscribe to the testing/verification system of the trusted parties. To avoid a Denial of Service attack by hacking into the testing/verification systems of one of the trusted parties, the local settings of the mmmv_silkexec should treat a test failure of a formerly accepted component as a timed warning, where the warning state moves to a blocked/failed state with a delay. The delay gives the trusted party time to handle the hack.



Components to Study



Experimental Subspecifications


References