Apptainer/Singularity is a container system designed for use on High Performance Computing (HPC) systems. It allows users to run both Docker and Singularity containers.
From the Docker website: "A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings."
singularity (apptainer is the official command). Thus, user should continue running containers on OSC systems without any issue: SINGULARITY_ and SINGULARITYENV_ environment variables.APPTAINER_ and APPTAINERENV_ counterparts in their job environments.Apptainer/Singularity is available on all OSC clusters. Only one version is available at any given time. To find out the current version:
singularity version
Check the release page for the changelog: https://github.com/apptainer/apptainer/releases
Apptainer/Singularity is available to all OSC users.
Apptainer project, established as Apptainer a Series of LF Projects LLC; 3-clause BSD License
No setup is required. You can use Apptainer/Singularity directly on all clusters.
See HOWTO: Use Docker and Singularity Containers at OSC for information about using Apptainer/Singularity on all OSC clusters, including some site-specific caveats.
Example: Run a container from the Singularity hub
[owens-login01]$ singularity run shub://singularityhub/hello-world INFO: Downloading library image Tacotacotaco
You might encounter an error while pulling a large Docker image:
ERROR: toomanyrequests: Too Many Requests.
or
You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.
On November 20, 2020, Docker Hub puts rate limits on anonymous and free authenticated pull requests. The rate limits for anonymous and authenticated pulls are 100 per 6 hours and 200 per 6 hours, respectively. Anonymous users have limits enforced via IP. Since all computing nodes at OSC share the same IP, anonymous pull rate limit is shared by all OSC users if you are not authenticated.
If you encounter this error and want to get rid of it, please consider setting up authenticated access to Docker Hub: https://apptainer.org/docs/user/main/endpoint.html?highlight=endpoint#ma....
You might encounter an error while pulling a large Docker image:
[owens-login01]$ singularity pull docker://qimme2/core FATAL: Unable to pull docker://qiime2/core While running mksquashfs: signal: killed
The process could be killed because the image is cached in the home directory which is a slower file system or the image size might exceed a single file size limit.
The solution is to use other file systems like /fs/ess/scratch and $TMPDIR for caches and temp files to build the squashfs filesystem:
[owens-login01]$ sinteractive -n 1 -A PAS1234 bash-4.2$ export APPTAINER_CACHEDIR=$TMPDIR bash-4.2$ export APPTAINER_TMPDIR=$TMPDIR bash-4.2$ singularity pull docker://qiime2/core:2019.1 INFO: Converting OCI blobs to SIF format INFO: Starting build... ... INFO: Creating SIF file... bash-4.2$ exit
You might encounter an error while run a container directly from a hub:
[owens-login01]$ singularity run shub://vsoch/hello-world Progress |===================================| 100.0% FATAL: container creation failed: mount error: can't mount image /proc/self/fd/13: failed to find loop device: could not attach image file too loop device: No loop devices available
One solution is to remove the Singularity cached images from local cache directory $HOME/.apptainer/cache.
singulariy cache clean
Alternatively, you can change the Singularity cache directory to different location by setting the variable APPTAINER_CACHEDIR. For example, in a batch job:
#/bin/bash #SBATCH --job-name="singularity_test" #SBSTCH --ntasks=1 export APPTAINER_CACHEDIR=$TMPDIR singularity run shub://vsoch/hello-world