Container images may be built only via an API call, without registering a git repository. This is discouraged, however, because it will make it hard to track your changes. But for a quick and dirty test this is a good starting point.

To use the API endpoint you need to create API credentials (one time setup) using the developer portal.

  • Add a new application (choose a name you like, e.g., container-image-builder)
  • Go to the OAuth2 Tokens tab, and click at the bottom on Generate tokens. Note down the Consumer Key and Consumer secret.
  • Go to the Subscriptions tab and subscribe to ciext-container-builder.

After your application is created the new the tab APIs will appear at the top of the developer portal, which allows you to inspect the API for ciext-container-builder. The section Documents is very helpful as it contains the endpoint documentation for /container/build POST.

Refer to the documentation there for all parameters. Basically the endpoint can work in two different modes:

  1. Send just a Dockerfile
  2. Send a full build context as a tar.gz-archive and tell the API where the Dockerfile inside the tarball is.

Create access token

To send any request to the API endpoint, we first need to create an access token, which can be done with the Consumer Key and Consumer secret.

$ ACCESS_TOKEN="$(curl  -u <your-consumer-key>:<your-consumer-secret> --silent -X POST https://auth.cscs.ch/auth/realms/firecrest-clients/protocol/openid-connect/token -d "grant_type=client_credentials" | jq -r '.access_token')"

If the consumer secret is omitted on the command line you will be prompted for it when you run the command. The token is stored in the variable ACCESS_TOKEN. This token has only a short validity, so you need to create a fresh access token, whenever the current one becomes invalid (about 5 minutes validity).

Mode 1 - Send a Dockerfile

Now we can call the API endpoint:

$ curl -H "Authorization: Bearer $ACCESS_TOKEN" --data-binary @path/to/Dockerfile "https://api.cscs.ch/ciext/v1/container/build?arch=x86_64"

It is mandatory to specify for which architecture you want to build the container. Valid choices are:

  • x86_64 - Intel/AMD architecture - Correct for all nodes that are not Grace-Hopper at CSCS
  • aarch64 - ARM architecture - Correct for Grace-Hopper

The API call above sends the Dockerfile to the server, and the server will reply with a link, where you can see the build log. The final container image will be pushed to JFrog, a CSCS internal container registry. Once the container image is built, it can be pulled from any CSCS machine.

If you want to push the image to your Docker Hub account, you need to create a Docker Hub access token with write permissions, and then use the API call (similarly for other OCI registry providers)

$ curl -H "Authorization: Bearer $ACCESS_TOKEN" -H "X-Registry-Username <your-dockerhub-username>" -H "X-Registry-Password: <your-dockerhub-token>" --data-binary @path/to/Dockerfile "https://api.cscs.ch/ciext/v1/container/build?arch=x86_64&image=docker.io/<your-dockerhub-username>/my_image_name:latest"

Mode 2 - Full tar.gz-archive as build context

Lastly, we want to present the second mode, where you send a full tar.gz archive to the server. This is needed when you use  COPY or ADD statements inside your Dockerfile, since the files being placed in the container won't otherwise exist on the server. To send a full archive the easiest is via the API call

$ cd path/to/build_context
$ tar -czf - . | curl -H "Authorization: Bearer $ACCESS_TOKEN" --data-binary @- "https://api.cscs.ch/ciext/v1/container/build?arch=x86_64&dockerfile=relative/path/to/Dockerfile"

This is similar to

$ docker build -f relative/path/to/Dockerfile .

Of course you can also specify a custom image, see the above example for a custom image name.