Before you begin
If you have an earlier version of Singularity installed, you should remove it before executing the installation commands.
These instructions will build Singularity from source on your system. So you will need to have some development tools installed. If you run into missing dependencies, try installing them like so:
$ sudo apt-get update && \ sudo apt-get install \ python \ dh-autoreconf \ build-essential
$ sudo yum update && \ sudo yum groupinstall 'Development Tools'
Install the master branch
The following commands will install the latest version of the GitHub repo master branch to
$ git clone https://github.com/singularityware/singularity.git $ cd singularity $ ./autogen.sh $ ./configure --prefix=/usr/local --sysconfdir=/etc $ make $ sudo make install
Note that the installation prefix is
/usr/local but the configuration directory is
/etc. This ensures that the configuration file
singularity.conf is placed in the standard location.
If you omit the
--sysconfdir option , the configuration file will be installed in
/usr/local/etc. If you omit the
--prefix option, Singularity will be installed in the
/usr/local directory hierarchy by default. And if you specify a custom directory with the
--prefix option, all of Singularity’s binaries and the configuration file will be installed within that directory. This last option can be useful if you want to install multiple versions of Singularity, install Singularity on a shared system, or if you want to remove Singularity easily after installing it.
Install a specific release
The following commands will install a specific release from GitHub releases page to
$ VER=2.2.1 $ wget https://github.com/singularityware/singularity/releases/download/$VER/singularity-$VER.tar.gz $ tar xvf singularity-$VER.tar.gz $ cd singularity-$VER $ ./configure --prefix=/usr/local --sysconfdir=/etc $ make $ sudo make install
Install the development branch
If you want to test a development branch feature the routine above should be tweaked slightly:
$ git clone https://github.com/singularityware/singularity.git $ cd singularity $ git fetch $ git checkout development $ ./autogen.sh $ ./configure --prefix=/usr/local --sysconfdir=/etc $ make $ sudo make install
Remove an old version
Let’s say that we installed Singularity to
/usr/local. To remove it completely, you need to hit all of the following:
$ sudo rm -rf /usr/local/libexec/singularity $ sudo rm -rf /usr/local/etc/singularity $ sudo rm -rf /usr/local/include/singularity $ sudo rm -rf /usr/local/lib/singularity $ sudo rm -rf /usr/local/var/lib/singularity/ $ sudo rm /usr/local/bin/singularity $ sudo rm /usr/local/bin/run-singularity $ sudo rm /usr/local/etc/bash_completion.d/singularity $ sudo rm /usr/local/man/man1/singularity.1
If you modified the system configuration directory, remove the
singularity.conf file there as well.
If you installed Singularity in a custom directory, you need only remove that directory to uninstall Singularity. For instance if you installed singularity with the
--prefix=/some/temp/dir option argument pair, you can remove Singularity like so:
$ sudo rm -rf /some/temp/dir
Overview of the Singularity Interface
Singularity is a command line interface that is designed to interact with containerized applications as transparently as possible. This means you can run programs inside a container as if they were running on your host system. You can easily redirect IO, use pipes, pass arguments, and access files, sockets, and ports on the host system from within a container.
Once you have Singularity installed, you should inspect the output of the
--help option as follows:
$ singularity --help USAGE: singularity [global options...] <command> [command options...] ... GLOBAL OPTIONS: -d --debug Print debugging information -h --help Display usage summary -q --quiet Only print errors --version Show application version -v --verbose Increase verbosity +1 -x --sh-debug Print shell wrapper debugging information GENERAL COMMANDS: help Show additional help for a command selftest Run some self tests to make sure Singularity is installed and operating properly CONTAINER USAGE COMMANDS: exec Execute a command within container run Launch a runscript within container shell Run a Bourne shell within container test Execute any test code defined within container CONTAINER USAGE OPTIONS: see singularity help <command> CONTAINER MANAGEMENT COMMANDS (requires root): bootstrap Bootstrap a new Singularity image from scratch copy Copy files from your host into the container create Create a new container image expand Grow the container image export Export the contents of a container via a tar pipe import Import/add container contents via a tar pipe mount Mount a Singularity container image CONTAINER REGISTRY COMMANDS: pull pull a Singularity Hub container to $PWD For any additional help or support visit the Singularity website: http://singularity.lbl.gov/
Notice the first line marked
USAGE: and the placement of the options. Option placement is very important in Singularity. It ensures that the right options are being parsed at the right time. For instance, if you were to run a command inside the container called
foo -v, then Singularity must be aware that the option
-v that you are passing to the command
foo is not intended to be parsed or interfered with by Singularity.
For example, you may pass the
-v option twice, once in the Singularity global options and once for the command that you are executing inside the container. The final command may look like:
$ singularity -v exec container.img foo -v
This means that debugging looks like the following:
$ singularity --debug create container.img
Quiet is the same position:
$ singularity --quiet create container.img
But if we were to bind a directory within a container, it must come after the command because
--bind is not a global option!
$ singularity run --bind /local/path:/container/path container.img
The take home message here is that option placement is exceedingly important. The algorithm that Singularity uses for option parsing for both global options as well as subcommand options is as follows:
- Read in the current option name
- If the option is recognized do what is needed, move to next option (goto #1)
- If the paramater is prefixed with a
-(hyphen) but is not recognized, error out
- If the next option is not prefixed with a
-(hyphen), then assume we are done with option parsing
This means that options will continue to be parsed until no more options are listed.
note: Options that require data (e.g.
--bind <path>) must be separated by white space, not an equals sign!
As the above
USAGE: describes, Singularity will parse the command as follows:
- Singularity command (
- Global options
- Singularity subcommand (
- Subcommand options
- Any additional input is passed to the subcommand
You can get additional help on any of the Singularity subcommands by using any one of the following command syntaxes:
$ singularity help <subcommand> $ singularity --help <subcommand> $ singularity -h <subcommand> $ singularity <subcommand> --help $ singularity <subcommand -h
Shell into a Non-Persistent Container
At this point, you can easily test Singularity by downloading and running a non-persistent container. As mentioned earlier, Singularity has the ability to interface with the main Docker Registry, so let’s start off by pulling a container down from the main Docker Registry and launching a shell inside of a given container:
$ singularity shell docker://ubuntu:latest Importing: base Singularity environment Cache folder set to /home/vanessa/.singularity/docker Exploding layer: sha256:6d9ef359eaaa311860550b478790123c4b22a2eaede8f8f46691b0b4433c08cf.tar.gz Exploding layer: sha256:9654c40e9079e3d5b271ec71f6d83f8ce80cfa6f09d9737fc6bfd4d2456fed3f.tar.gz Exploding layer: sha256:e8db7bf7c39fab6fec91b1b61e3914f21e60233c9823dd57c60bc360191aaf0d.tar.gz Exploding layer: sha256:f8b845f45a87dc7c095b15f3d9661e640ebc86f42cd8e8ab36674846472027f7.tar.gz Exploding layer: sha256:d54efb8db41d4ac23d29469940ec92da94c9a6c2d9e26ec060bebad1d1b0e48d.tar.gz Exploding layer: sha256:fe44851d529f465f9aa107b32351c8a0a722fc0619a2a7c22b058084fac068a4.tar.gz Singularity: Invoking an interactive shell within container... Singularity ubuntu:latest:\w> cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS" Singularity ubuntu:latest:\w> which apt-get /usr/bin/apt-get
In this example, you started on your (insert OS here) host operating system, ran Singularity as a non-root user and used a URI which tells Singularity to pull a given container from the main Docker Registry and execute a shell within it. In this example, we are not telling Singularity to use a local image, which means that any changes we make will be non-persistent (e.g. the container is removed automatically as soon as the shell is exited). It’s also salient to note that you don’t need to have Docker installed to use an image from Docker Hub.
You may select other images that are currently hosted on the main Docker Hub Library.
Where do we keep metadata and things? The metadata folder is located within the image under the root directory at
Singularity ubuntu:latest:\w> ls /.singularity.d actions env labels.json runscript
runscript is linked to the file
env we have different files that were sourced by the container to generate the
/environment file at the root of the image, in the order specified.
ls /.singularity.d/env 01-base.sh 10-docker.sh
labels.json is metadata about the container, both from the bootstrap or creation, and from Docker (if relevant). These files should not be edited manually. Instead, add variables to your bootstrap specification file that will automatically generate these files.
Creating a New Singularity Image
Singularity’s primary objectives are:
- archiving environments
These features are realized via the Singularity image file. As explained earlier, Singularity images are individual files which can be copied, shared, and easily archived along with relevant data. This means that the all computational components can be easily replicated, utilized, and extended by other users.
The first part of building your reproducible container is to create the raw Singularity image file:
$ singularity create container.img Creating a new image with a maximum size of 768MiB... Executing image create helper Formatting image with ext3 file system Done.
Think of this as an empty file system that you can fill with an OS, apps, and data. You can override the default size (which varies somewhat according to which version of Singularity you are using) by specifying the
--size option in MiB as follows:
$ singularity create --size 2048 container.img Creating a new image with a maximum size of 2048MiB... Executing image create helper Formatting image with ext3 file system Done. $ ls -l container.img -rwxr-xr-x 1 user group 2147483680 Dec 25 06:00 container.img
Note the permissions of the resulting file. While the
umask is adhered to, you should find that the file is executable. At this point there is nothing to execute within that image. But once this image contains a proper operating system, you can define what it will do when it is executed directly.
Bootstrapping a Singularity Image
The final step in creating a fully functional Singularity container is to execute the
bootstrap command. This installs an operating system, apps, and creates an environment within the image.
To bootstrap an image, you first need a definition file. This acts as a set of blueprints describing how to build the container. The Singularity repository contains an
examples subdirectory with several definition files. Assuming that you
git cloned the Singularity repository into your current working directory and you already executed the
create command above, you could bootstrap your image like so. (The exact path to the definition file may vary depending on the version of Singularity you installed.)
$ sudo singularity bootstrap container.img singularity/examples/ubuntu/Singularity
You will see a lot of standard output as Singularity downloads and installs all the pieces for Ubuntu into your image. After it completes, you will have a fully functional container!