Installation Quick Start
Note that this quickstart is intended for using Singularity on your personal workstation, where you have installed Singularity and have sudo. If you only have access to Singularity on a shared cluster resource, you will be able to go through all parts of this tutorial that do not require writing to an image. First, if you are on your local machine, let’s install Singularity.
git clone https://github.com/singularityware/singularity.git cd singularity ./autogen.sh ./configure --prefix=/usr/local make sudo make install
Installing Singularity as a user, or without sudo, will not produce software that works properly. If you want Singularity on your shared cluster resource, you should ask an administrator to install it for you!
Command Quick Start
This first section of commands can be done on a shared resource, or your personal computer. You don’t need sudo to create, import, or shell into containers.
Make a container
singularity create centos7.img Initializing Singularity image subsystem Opening image file: centos7.img Creating 768MiB image Binding image to loop Creating file system within image Image is done: centos7.img
Dump docker layers into it! Nope, you don’t need sudo.
singularity import centos7.img docker://centos:7 Cache folder set to /home/vanessa/.singularity/docker Importing: base Singularity environment Importing: /home/vanessa/.singularity/docker/sha256:785fe1d06b2d42874d3e18fb0747ad8c9ed83d04e7641279a4d5ae353f27eff9.tar.gz Importing: /home/vanessa/.singularity/docker/sha256:a90ac515821d5b70fe202c201485396ba95305348f9f7f52813e2873d3c72eee.tar.gz
Pull a container
Another easy way to obtain and use a container is to pull it directly. Here we can pull centos directly from Docker Hub, or pull an image from Singularity Hub.
singularity pull docker://centos:latest Initializing Singularity image subsystem Opening image file: centos-latest.img Creating 336MiB image Binding image to loop Creating file system within image Image is done: centos-latest.img Docker image path: index.docker.io/library/centos:latest Cache folder set to /home/vanessa/.singularity/docker [1/1] |===================================| 100.0% Importing: base Singularity environment Importing: /home/vanessa/.singularity/docker/sha256:d5e46245fe40c2d1ab72bfe328de28549b605b2587ab2fa8715f54e3e2de9c5d.tar.gz Importing: /home/vanessa/.singularity/metadata/sha256:6b8bbe197a20c88d065c265cf6f6f8b4e3695f104d1f47f01a1298b3566f27fe.tar.gz Done. Container is at: centos-latest.img
Did you notice anything interesting about the output to the terminal? If you noticed that the above output for “pull” is similar to the output of “create” and “import” combined, you nailed it on the head! In the context of docker, running the
pull command is an easy shortcut to create and import docker layers to an image. Whereas Docker creates and imports layers into an image on pull, when we pull (or run, or shell) an image from Singularity Hub, the entire image is downloaded. This is one of the main differences between Docker and Singularity:
singularity run shub://vsoch/hello-world Progress |===================================| 100.0% RaawwWWWWWRRRR!! $ ls vsoch* vsoch-hello-world-master.img
Shell into container
Now let’s go back to the
centos7.img we created, and shell inside.
singularity shell centos7.img Singularity: Invoking an interactive shell within container... # I am the same user inside as outside! Singularity centos7.img:~/Desktop> whoami vanessa Singularity centos7.img:~/Desktop> id uid=1000(vanessa) gid=1000(vanessa) groups=1000(vanessa),4(adm),24,27,30(tape),46,113,128,999(input)
Want to keep the container’s environment contained, meaning no sharing of host environment?
singularity shell --contain centos7.img
exec will send a custom command for the container to run, anything that you like! Unlike docker exec, the container doesn’t have to be actively running. So, to list the root of the image (
/), we could do the following:
singularity exec centos7.img ls / anaconda-post.log etc lib64 mnt root singularity tmp bin home lost+found opt run srv usr dev lib media proc sbin sys var
Working with Files
Files on the host can be reachable from within the container
echo "Hello World" > $HOME/hello-kitty.txt singularity exec centos7.img cat $HOME/hello-kitty.txt Hello World
By default, most configurations will mount
/tmp and the home directories by default. On a research cluster, you probably want to access locations with big datasets, and then write results too. For this, you will want to bind a folder to the container. Here, we are binding my Desktop to
/opt in the image, and listing the contents to show it worked. We use the command
--bind to do this.
$ singularity exec --bind /home/vanessa/Desktop:/opt centos7.img ls /opt centos7.img researchapps-matlab-sherlock-master.img hello-kitty.txt singularity-recipe-demo.mp4 party_dinosaur.gif
Commands needing root
The next set of actions, namely anything with
--writable or bootstrap, do require you to use sudo. Well, they require sudo for most things. We can shell into a container, with
--writable without sudo and write to locations for which we have permission to do so. For example, let’s shell in to our container using
--writable without sudo and try to write a
/data folder at the root of the image:
singularity shell --writable centos7.img Singularity: Invoking an interactive shell within container... Singularity centos7.img:~/Desktop> mkdir /data mkdir: cannot create directory '/data': Permission denied
In the above, we see that while it’s possible to shell in with
--writable sans sudo, the content that you can write depends on the permissions set in the container. Let’s now try creating a file in the present working directory:
Singularity centos7.img:~/Desktop> touch file.txt
This we are allowed to do, however for most things you will likely need to use sudo with writable, discussed next. At this point, if you have been working on your shared resource, you will need to move to your personal laptop (and install Singularity if you haven’t yet).
Writing in the container
While we discourage making tweaks on the fly to containers (you should properly define all edits to the container in a boostrap specification file, shown later) you can add
--writable to any command to write inside the container. Assuming we have our
centos7.img on our local resource with sudo, let’s try again to make that
sudo singularity shell --writable centos7.img Singularity centos7.img:/root> mkdir /data Singularity centos7.img:/root> touch /data/noodles.txt exit
We made the data! And the noodles! But after we exit, is the file still there?
singularity exec centos7.img ls /data noodles.txt
We (ideally) would have done this action with bootstrap, discussed next.
For a reproducible container, the recommended practice is to build by way of a bootstrap file. This also makes it easy to add files, environment variables, and install custom software, and still start from your bootstrap of source (e.g., Docker). Here is what a basic bootstrap file looks like for Singularity 2.3:
Bootstrap: docker From: ubuntu:latest %runscript exec echo "The runscript is the containers default runtime command!" %files /home/vanessa/Desktop/hello-kitty.txt /data/hello-kitty.txt /home/vanessa/Desktop/party_dinosaur.gif /tmp/the-party-dino.gif %environment VARIABLE=MEATBALLVALUE export VARIABLE %labels AUTHOR email@example.com %post apt-get update && apt-get -y install python3 git wget mkdir /data echo "The post section is where you can install, and configure your container."
The above bootstrap definition can then be run with singularity. Assuming that the definition was saved as
Singularity and an image file
ubuntu.img exists, the following will build the container.
sudo singularity bootstrap ubuntu.img Singularity
How might you go through this entire process without having singularity installed locally, or without leaving your cluster? You can build images using singularity hub.