A few months back AWS Codebuild release Batch builds. A very easy way to build multiple things at the same time, with or without dependencies or order between each other. Very convenient in order to avoid creating multiple projects, with different settings, simply define these settings directly in the buildspec.yml file.
Last year I read a blog post published by AWS, around docker images build for multiple architecture using AWS CodeBuild and AWS CodePipeline.
So this is a follow-up, in some regards, to that article, to further demonstrate AWS Codebuild capabilities.
It is very simple to get going with AWS CodeBuild batch, and it very well integrates into CICD Pipelines. Here what I had to do is simply have a specific buildspec definition file to distinguish building the docker images and one for building the manifest.
Let's go step by step on how I approached the implementation
At the start of the project I had only aimed to build for python3.7 images. But, I realized, why stop there. Given that some extra commands are necessary for installing python3.8 with Amazon Linux, I thought the easiest thing for all is just to have two different files.
But in essence, they are doing the same thing: update the packages installed, install python, set the new python as default.
AWS CodeBuild supports multiple configurations and is very versatile. Here we want to build 4 docker images, and gather these images in groups of two manifests. So, we are going to have one build per configuration and therefore one image per.
Each build will end by publishing the image build to AWS ECR, which our final stage will use.
Here with a batch-graph configuration, we can define dependencies between builds. So here, our manifest step will only be executed once the others are finished.
You can define whether they all need to succeeed or not to progress.
batch: fast-fail: false build-graph: - identifier: amd64_py37 env: compute-type: BUILD_GENERAL1_LARGE privileged-mode: true variables: VERSION: 3.7 ARCH: amd64 buildspec: build_images.yml - identifier: arm64v8_py37 env: type: ARM_CONTAINER image: aws/codebuild/amazonlinux2-aarch64-standard:2.0 compute-type: BUILD_GENERAL1_LARGE privileged-mode: true variables: ARCH: arm64v8 VERSION: 3.7 buildspec: build_images.yml - identifier: amd64_py38 env: compute-type: BUILD_GENERAL1_LARGE privileged-mode: true variables: VERSION: 3.8 ARCH: amd64 buildspec: build_images.yml - identifier: arm64v8_py38 env: type: ARM_CONTAINER image: aws/codebuild/amazonlinux2-aarch64-standard:2.0 compute-type: BUILD_GENERAL1_LARGE privileged-mode: true variables: ARCH: arm64v8 VERSION: 3.8 buildspec: build_images.yml - identifier: manifest env: compute-type: BUILD_GENERAL1_LARGE privileged-mode: true depend-on: - amd64_py37 - arm64v8_py37 - amd64_py38 - arm64v8_py38
All the buildspec batch specifications are available here.
Once the build has finished, if succeeded, you should see
And that is it, this is really that simple.
Here I use the same image base for both python 3.7 and 3.8. So instead of doing 1 build for each, I could have simply build both images in 1 go per architecture. But for the purpose of this example, it seemed clearer that way to demonstrate the potential of AWS Codebuild for your multi-arch and multi-os builds.