How to Use Multiple Docker Build Contexts to Streamline Image Assembly


Graphic showing the Docker logo

Docker’s concept of the “build context” is one of its most restrictive and misunderstood characteristics. The build context defines the local files and folders you can reference in your Dockerfile. Content outside it cannot be used, which often hinders complex build procedures.

BuildKit v0.8 improves this situation by letting you use multiple contexts with each build you perform. This makes it easier to reference files that may reside in completely separate locations, such as a file within your working directory and a dependency at a remote URL.

In this article we’ll explain why multiple build contexts are useful and how you can use them with the latest Docker CLI release. First let’s recap what the build context is and why so many people have run into issues in the past.

Purpose of the Build Context

Docker is daemon-based. The process that runs your image builds is independent of the CLI process that issues the command. The daemon could be located on a remote host which can’t directly access your machine’s filesystem.

The build context refers to the files that are transferred to the Docker daemon when a build occurs. This is why only content within the context can be referenced by your Dockerfile.

It’s common to run docker build with . as its argument, which makes your working directory the build context:

docker build -t my-website:latest .

This permits references to any path inside your working directory:

FROM httpd:latest
COPY index.html /var/www/html/index.html

You can’t reach out to copy anything above the working directory in your filesystem:

FROM httpd:latest
COPY index.html /var/www/html/index.html
COPY ../company-css/company.css /var/www/html/company.css

Every file you need in your container image must exist under a single directory that you can use as the build context. This can be problematic in situations like the one shown above, where you want to pull in dependencies from sources that aren’t in your project’s tree.

Using Multiple Build Contexts

Multiple build contexts are now supported in BuildKit v0.8 and newer when you opt-in to Dockerfile syntax v1.4. These releases are shipped with the Docker CLI starting from version 20.10.13. You should be able to use them today if you’re running the latest version of Docker.

You must build your image with BuildKit to use multiple contexts. They aren’t supported by the legacy builder. Use the docker buildx build command instead of plain docker build:

$ docker buildx build -t my-website:latest .

Now you can use the --build-context flag to define multiple named build contexts:

$ docker buildx build -t my-website:latest . 
    --build-context company-css=../company-css 
    --build-context company-js=../company-js 

Adjust your Dockerfile to reference content from these contexts:

#syntax=docker/dockerfile:1.4
FROM httpd:latest
COPY index.html /var/www/html/index.html
COPY --from=company-css /company.css /var/www/html/company.css
COPY --from=company-js /company.js /var/www/html/company.js

This illustrates how you can copy in files and folders that lie outside your main build context, irrespective of their position in your filesystem tree.

The Dockerfile v1.4 syntax declaration is required to enable support for the feature. You can then use the --from option with ADD and COPY instructions to pull in files from named build contexts, similarly to referencing a resource in an earlier build stage.

Priority Order

Multiple build contexts modify the resource resolution order for the --from flag. Docker will now match the key you supply (--from=key) using the following procedure:

  • Look for a named build context set with the --build-context flag.
  • Look for an earlier build stage created with FROM my-image:latest AS stage-name.
  • Create a new inline build stage using the given key as the stage’s image.

This means you can use named contexts to override remote dependencies defined using build stages.

Consider this example:

#syntax=docker/dockerfile:1.4
FROM my-org/company-scss:latest AS css
RUN sass company.scss company.css

FROM httpd:latest
COPY index.html /var/www/html/index.html
COPY --from=css /company.css /var/www/html/company.css

This Docker image pulls in some remote resources from another shared Docker image. This can create difficulties when you’re testing your project – there could be a bug in the dependency which you want to quickly patch.

Named build contexts let you override the css stage name to supply a local file instead:

$ docker buildx build -t my-website:latest . --build-context css=css/

This will copy your working directory’s css/company.css file into the final image, instead of the version supplied by the my-org/company-scss:latest dependency.

The resolution order means overrides can be applied even if your image doesn’t use named build stages. By defining a build context with the same name as an image, your Dockerfile will pull in content from that context, instead of the original registry image.

$ docker buildx build -t my-website:latest . --build-context my-org/company-scss:latest=css/

Remote URLs

Named build contexts support all the sources that docker build already accepted:

  • --build-context my-context=../local/path – A path in your filesystem.
  • --build-context my-context=https://github.com/user/repo.git – A remote Git repository.
  • --build-context my-context=https://example.com/data.tar – A remote tarball provided by an HTTP server.
  • --build-context my-context=docker-image://busybox:latest – The content of another Docker image.

Remote sources further simplify dependency overrides. You can point directly to a forked Git repository or a different Docker image tag, all while leaving your Dockerfile unchanged.

Mounting Files From a Build Context

Named build contexts work with RUN instructions too. You can use --mount=from to run an executable from another build context.

#syntax=docker/dockerfile:1.4
RUN --mount=from=name-of-build-context demo-executable

This mounts the file without copying it into the current layer, helping to improve performance. demo-executable won’t exist in the final image.

Precisely Rebuilding Images

Another use case for named build contexts concerns rebuilding images in the future. Dockerfiles with instructions like FROM alpine:3.15 aren’t fully reproducible. Image tags are mutable so alpine:3.15 may contain different content in the future, after a new patch is released. This means rebuilt images aren’t guaranteed to produce the same layers as their original versions.

You can solve this problem by inspecting the first build’s metadata to discover the exact base image that was used:

$ docker buildx imagetools inspect --format '{{json .BuildInfo}}' my-image:latest
{
    ...
    "sources": [
        {
            "type": "docker-image",
            "ref": "docker.io/library/alpine:3.15",
            "pin": "sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454"
        }
    ]
    ...
}

Now you can define a named build context called alpine:3.15 that points to the exact version that was previously used:

$ docker buildx build -t my-image:latest . --build-context alpine:3.15=docker-image://alpine3.15@4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454

This simplifies creating a precise rebuild of a previously created image, without having to modify its Dockerfile.

Conclusion

Multiple build contexts give you more options for organizing complex Dockerfiles and project directory trees. They solve the longstanding usability challenges that you may experience with a single build context.

Named build contexts let you include out-of-tree dependencies and perform ad-hoc overrides. They work well alongside Docker’s existing named build stages. Combining the two features helps you create modular Dockerfiles that can be customized at build time.

You can get started with multiple build contexts today by updating to Docker 20.10.13 or newer and using docker buildx to create your images. Standalone BuildKit distributions are available too when you don’t want to install the whole Docker CLI.





Source link

Previous articleBinance CEO Warns ‘We Could Disable Wazirx Wallets’ — Advises Investors to Transfer Funds to Binance – Exchanges Bitcoin News
Next articleThis iPhone 14 launch leak will really worry Apple fans, set alarm bells ringing