singularityware.github.io

Singularity 2.5.2 Release

Tue, 03 Jul 2018 00:00:00 +0000

This release contains fixes for a high severity security issue affecting Singularity 2.3.0 through 2.5.1 on kernels that support overlay file systems (CVE-2018-12021). A malicious user with network access to the host system (e.g. ssh) could exploit this vulnerability to access sensitive information on disk and bypass directory image restrictions like those preventing the root file system from being mounted into the container.

Singularity 2.5.2 should be installed immediately, and all previous versions of Singularity should be removed. The vulnerability addressed in this release affects kernels that support overlayfs. If you are unable to upgrade immediately, you should set enable overlay = no in singularity.conf.

In addition, this release contains a large number of bug fixes. Details follow:

Removed the option to use overlay images with singularity mount. This flaw could allow a malicious user accessing the host system to access sensitive information when coupled with persistent ext3 overlay.
Fixed a race condition that might allow a malicious user to bypass directory image restrictions, like mounting the host root filesystem as a container image

Bug fixes

Fix an error in malloc allocation #1620
Honor debug flag when pulling from docker hub #1556
Fix a bug with passwd abort #1580
Allow user to override singularity.conf “mount home = no” with –home option #1496
Improve debugging output #1535
Fix some bugs in bind mounting #1525
Define PR_(S|G)ET_NO_NEW_PRIVS in user space so that these features will work with kernels that implement them (like Cray systems) #1506
Create /dev/fd and standard streams symlinks in /dev when using minimal dev mount or when specifying -c/-C/–contain option #1420
Fixed * expansion during app runscript creation #1486

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.5.1 Release

Thu, 03 May 2018 00:00:00 +0000

This is a bug fix point release to the 2.5 feature branch.

Bug fixes

Corrected a permissions error when attempting to run Singularity from a directory on NFS with root_squash enabled
Fixed a bug that closed a socket early, preventing correct container execution on hosts using identity services like SSSD
Fixed a regression that broke the debootstrap agent

And as always, report any bugs to: https://github.com/singularityware/singularity/issues/new

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.5.0 Release

Fri, 27 Apr 2018 00:00:00 +0000

This release includes fixes for several high and medium severity security issues. It also contains a whole slew of bug fixes including the much awaited docker aufs whiteout file fix. It’s a new release instead of a point release because it adds a new dependency to handle this bug, includes some new (albeit minor) feature enhancements, and changes the behavior of a few environment variables (see below).

Singularity 2.5 should be installed immediately and all previous versions of Singularity should be removed. Many of the vulnerabilities fixed in this release are expected to affect all Linux distributions regardless of whether they implement overlayfs. There are no mitigations or workarounds for these issues outside of updating Singularity.

Additionally, Singularity 2.5 drops support for hosts that do not support the prctl() function PR_SET_NO_NEW_PRIVS. The PR_SET_NO_NEW_PRIVS feature was added to prctl() in the Linux 3.5 kernel. Various distributions have since backported this feature to currently maintained kernels (for example, Red Hat added this feature to RHEL 6.7 with the 2.6.32-504.16.2 kernel). Kernels that do not have this feature are inherently insecure in many ways. They do not implement container runtimes securely. Blocks have therefore been put in place to prevent Singularity 2.5 from building or running on vulnerable kernels.

Patches are provided to prevent a malicious user with the ability to log in to the host system and use the Singularity container runtime from carrying out any of the following actions:

Create world writable files in root-owned directories on the host system by manipulating symbolic links and bind mounts
Create folders outside of the container by manipulating symbolic links in conjunction with the –nv option or by bypassing check_mounted function with relative symlinks
Bypass the enable overlay = no option in the singularity.conf configuration file by setting an environment variable
Exploit buffer overflows in src/util/daemon.c and/or src/lib/image/ext3/init.c (reported by Erik Sjölund (DBB, Stockholm University, Sweden))
Forge of the pid_path to join any Singularity namespace (reported by Erik Sjölund (DBB, Stockholm University, Sweden))

Implemented enhancements

Restore docker-extract aufs whiteout handling that implements correct extraction of docker container layers. This adds libarchive-devel as a build time dep. At runtime libarchive is needed for whiteout handling. If libarchive is not available at runtime will fall back to previous extraction method.
Changed behavior of SINGULARITYENV_PATH to overwrite container PATH and added SINGULARITYENV_PREPEND_PATH and SINGULARITYENV_APPEND_PATH for users wanting to prepend or append to the container PATH at runtime

Bug fixes

Support pulls from the NVIDIA cloud docker registry (fix by Justin Riley, Harvard)
Close socket file descriptors in fd_cleanup
Fix conflict between –nv and –contain options
Throw errors at build and runtime if NO_NEW_PRIVS is not present and working
Reset umask to 0022 at start to correct several errors
Verify docker layers after download with sha256 checksum
Do not make excessive requests for auth tokens to docker registries
Fixed stripping whitespaces and empty new lines for the app commands (fix by Rafal Gumienny, Biozentrum, Basel)
Improved the way that working directory is mounted Fixed an out of bounds array in src/lib/image/ext3/init.c

And as always, report any bugs to: https://github.com/singularityware/singularity/issues/new

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.4.6 Release

Fri, 06 Apr 2018 00:00:00 +0000

This release addresses a high severity security issue with bind mounts on hosts using overlayfs. This fixes a vulnerability that could allow a malicious user to create files and directories outside of a Singularity container. Special thanks to Lars Viklund (HPC2N, Umeå University, Sweden) for identifying and helping test fixes for this bug.

But fixes include:

Fix for check_mounted() to check parent directories #1436
Free strdupped temporary variable in joinpath #1438 Please note that this release is being made with minimal community testing to allow administrators to expedite the patch process. Without full community testing, this release may not be completely stable. It’s up to administrators to decide if they value stability or security when choosing whether to install 2.4.6.

And as always, report any bugs to: https://github.com/singularityware/singularity/issues/new

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.4.5 Release

Mon, 19 Mar 2018 00:00:00 +0000

This is a security-related point release, bringing the following fix thanks to Justin Riley (@jtriley):

PR1387/1397 - python: strip “Authorization” header on (urllib) redirects to different domains

The security fix prevents Singularity from leaking credentials if:

You are logging in to a Docker registry with credentials
The registry redirects you to a 3rd party host (e.g. S3 for download of layers)

The fix ensures that in this situation the HTTP “Authorization” header is stripped from the redirected request, to prevent leaking of registry credentials to the 3rd party.

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.4.2 Release

Tue, 05 Dec 2017 00:00:00 +0000

Singularity 2.4.2 is released with minor bug fixes to the original 2.4. Important notes include the following:

We fixed an issue for support of older distributions and kernels with regards to the setns() function.
Also fixed autofs bug path (lost during merge)

For the full release announcement and downloads, please see the release on GitHub.

Singularity 2.4 Release

Mon, 02 Oct 2017 00:00:00 +0000

Singularity 2.4 is released, and here are the important notes for you to know about!

Singularity Build Building an image is now more intuitive with the introduction of our build command. You don’t need to worry about using create, or bootstrap, you just build. It looks like this:

sudo singularity build container.simg Singularity

If you are tired of needing to estimate your image size, that issue goes away with build. The default format for a Singularity image base is now a smaller, compressed file format called SquashFS. This format is also a much better fit for reproducible containers, as it is immutable. For more details, see our build documentation.

Build Bases We originally supported just using an existing distribution package or docker as a base for building images. With 2.4, we have extended that set to include another Singularity image (localimage), or a Singularity Hub or Registry image (shub).

Instance Support Singularity now has support for instances, which means services! Images instances can be started, stopped, and listed. Along with instances comes Network Namespace Isolation. We are excited to see the kinds of use cases that this will afford for our users.

Singularity Registry If you want to host your own Registry, we listened and you are in luck! Singularity Registry is ready for your use, along with a portal for you to “register your registry” to make it easier to share your images. Singularity Hub, along with improved building and updated builders, will follow this release.

Internal Modularity This release includes support for the Standard Container Integration Format (SCI-F) that makes it easy for you to produce single, internally modular and programmatically discoverable containers. More information is provided on it’s main site.

Better Documentation We’ve added an entire page on the different customizations you can do to the build environment, including changing the cache and specifying credentials for Docker. We now also have new sidebars for the Image and Instances command groups.

Deprecated We are deprecating the original create command to be replaced with image.create, along with bootstrap in favor with build. These commands will still work for 2.4, however we recommend you start testing the updated ones.

In addition, here are some tiny changes for your information!

Persistent Overlay
Container checks
Tests for instance support
Wrapper for create
Group instance commands
Group image commands
Bash completion updates

For the full release announcement and downloads, please see the release on GitHub.

Wowee, way to go team, contributors, and the community! Please report any issues or questions that you might have to our issue board.

Thank you!

Scientific Filesystem (SCIF) Final Draft

Tue, 26 Sep 2017 00:00:00 +0000

Hi Singularity Community!

Thanks to everyone that provided comments on the early draft for SCIF. The goals were initially:

write specification draft (this was done via a Google Doc)
implement into Singularity (now in development branch)
get feedback on both, adjust implementation and draft
do several (N=4) implemented use cases for SCIF, write ups (available here)
make an interactive, open source repository to contribute and share SCI-F apps https://sci-f.github.io/apps/
write up the final draft formally https://sci-f.github.io

I’m (@vsoch) happy to report that I’ve finished the above, and I’d like to ask the following:

if you contributed, please check the AUTHORS file in the SCI-F repository to make sure that I added you, and spelled your name correctly (apologies in advance).
Any further suggestions, additions, or entire new contributions can be added to the draft by way of pull request. I’m not in any rush, and if you have a good contribution, I want to help.
If you have not contributed yet and would like to, see the ideas linked on the specification.

The plan is (as of now) to submit this no later than the end of October, and I want to make sure everyone that wants to contribute has had ample chance. Indeed, with Singularity there are many ways to cook your container, and SCIF definitely exemplifies that.

Thanks again everyone!

Singularity 2.3.2 Quick Fix Release

Fri, 15 Sep 2017 00:00:00 +0000

Release 2.3.2 This dot release includes a fix for a change that Docker implemented to their registry RESTful API which broke compatibility with Singularity (among several other low minor fixes).

What happened?

Where does Docker image metadata come from? The Docker Registry serves metadata about images via manifests, where each image has a manifest that tells Singularity important information like environment, labels, and entrypoints and running command. Importantly, the image manifest serves the actual layers that should be obtained to create the image. The manifests come in different flavors, or schema versions. Version 1 serves the primary load of metadata (labels, environment) while the first release of version 2 served layers, and had the addition of size. This addition made it possible to pull an image with Singularity and calculate the size on the fly for images generated with support for this manifest.

Singularity had an old default to retrieve the version 2 (done by way of a header asking for it), and ask it for layers. If the remote registry could only offer version 1, we were still able to obtain a list of layers (under a different key, fsLayers instead of layers) and metadata via the older (schema 1) manifest. However, with the update, the API version 2 schema can now return a list of manifests. This meant that when we checked the response for the keys layers or fslayers, they wouldn’t be found, because we needed to look under manifests, and then do a second call to retrieve the manifest of interest based on a system architecture and OS. This of course broke most import, shell, pull, exec, because all of these commands require retrieving the layers.

The Patch

A super quick fix would have been to fall back to the version 1 manifest, always, but then we would lose the automatic calculation of size, which is important to many users. The “better” fix is obvious - the code needs to:

check for a manifests key
if found, select a default manifest to use
retrieve it, and continue!
if not found, fall back to version 1

This means that we’ve added environment variables SINGULARITY_DOCKER_OS and SINGULARITY_DOCKER_ARCHITECTURE to specify this choice, with defaults linux and amd64. This is a pretty exciting change, because it means down the line you (as a user!) can specify the specifics of the layers you want returned, given an image that has this metadata available. We can see the fix running as follows:

DEBUG 
*** STARTING DOCKER IMPORT PYTHON  ****

This is the initialization of the Docker client, it’s parsing the image name provided

DEBUG Starting Docker IMPORT, includes env, runscript, and metadata.
VERBOSE Docker image: ubuntu:14.04
VERBOSE2 Specified Docker ENTRYPOINT as %runscript.
DEBUG Headers found: Content-Type,Accept
VERBOSE Registry: index.docker.io
VERBOSE Namespace: library
VERBOSE Repo Name: ubuntu
VERBOSE Repo Tag: 14.04
VERBOSE Version: None

The initial poke to Docker hub asks for tags, to determine if we need some kind of token

VERBOSE Obtaining tags: https://index.docker.io/v2/library/ubuntu/tags/list
DEBUG GET https://index.docker.io/v2/library/ubuntu/tags/list

401 means that we do, and with the response the API sends back the scope and other details we need to make to request it

DEBUG Http Error with code 401

Here we are requesting it

DEBUG GET https://auth.docker.io/token?service=registry.docker.io&expires_in=9000&scope=repository:library/ubuntu:pull
DEBUG Headers found: Content-Type,Authorization,Accept

And here is the new bit. Instead of blindly doing one call, we have a function to update two versions of the manifest - the older (with metadata) and some version of the newer (with layers and size) with a fallback to use the version 1

DEBUG Updating manifests.

# Here is version 2+
DEBUG MANIFEST (Primary): not found, making initial call.
VERBOSE Obtaining manifest: https://index.docker.io/v2/library/busybox/manifests/latest
DEBUG GET https://index.docker.io/v2/library/busybox/manifests/latest

# Here is version 1 (Metadata)
DEBUG MANIFEST (Metadata): not found, making initial call.
VERBOSE Obtaining manifest: https://index.docker.io/v2/library/busybox/manifests/latest
DEBUG GET https://index.docker.io/v2/library/busybox/manifests/latest

Notice that the two calls are the same - that’s because the headers are actually different, to request different ones.

And here is the (new) indication that we found a list

DEBUG Image manifest version 2.2 list found.

Here is the default architecture / os

DEBUG Obtaining architecture: amd64, OS: linux

And the specific call to get it

VERBOSE Obtaining manifest: https://index.docker.io/v2/library/busybox/manifests/sha256:030fcb92e1487b18c974784dcc110a93147c9fc402188370fbfd17efabffc6af
DEBUG GET https://index.docker.io/v2/library/busybox/manifests/sha256:030fcb92e1487b18c974784dcc110a93147c9fc402188370fbfd17efabffc6af

With this fix - we can again use pull/import, etc, and also have a working singularity pull docker://busybox that estimates the size automatically.

Please report any additional bugs to our issues board.

Announcement: Problem downloading from Docker Hub to be resolved soon

Wed, 13 Sep 2017 00:00:00 +0000

To all Singularity users,

On Tuesday September 12, Docker released a new version of Docker image metadata. This means that any new images built on Docker Hub cannot currently be downloaded using a singularity pull or other commands like shell, exec, and bootstrap when updated Docker registries are queried.

Vanessa (@v) has created an interim fix for the problem and we have merged it into the development branch. Pending further testing we plan to merge this fix into master and create a new minor release (2.3.2). We will make another announcement as soon as it is ready to install.

Thanks for your patience!

The Singularity team

singularityware.github.io

Singularity 2.5.2 Release

Security related fixes

Bug fixes

Singularity 2.5.1 Release

Bug fixes

Singularity 2.5.0 Release

Security related fixes

Implemented enhancements

Bug fixes

Singularity 2.4.6 Release

Singularity 2.4.5 Release

Singularity 2.4.2 Release

Singularity 2.4 Release

Scientific Filesystem (SCIF) Final Draft

Singularity 2.3.2 Quick Fix Release

What happened?

The Patch

Announcement: Problem downloading from Docker Hub to be resolved soon