In modern software development practice, it is common to dockerize our applications. Therefore, it will be convenient if a docker image can be automatically built when we push some code to the repository. This is part of the continuous integration/continuous deployment (CI/CD) practice. For example, if we want to schedule some jobs in Airflow, we can use the KubernetesPodOperator to run the docker image of our application. With Cloud Build, the docker image can be automatically built for us whenever we push some new code.
Cloud Build is a service that executes your builds on Google Cloud Platform’s infrastructure. Cloud Build can import source code from a variety of repositories or cloud storage spaces, execute a build to your specifications, and produce artifacts such as Docker containers.
With cloud build, you can write a build config file to provide instructions to Cloud Build on what tasks to perform. Cloud Build executes your build as a series of build steps, where each build step is run in a Docker container.
To build your Docker image using a build config file, we need to create a file named cloudbuild.yaml in the same directory that contains your application source code.
It is recommended to use the GCP Cloud Shell to follow the instructions in this article because the environment has been set up in the Cloud Shell and all commands needed such as docker and gcloud can be used directly. For demonstration purpose, we will use a very simple Dockerfile to build an image:
FROM alpine
CMD ["echo", "Hello World!"]
FROM sets the base docker image for subsequent instructions. alpine is a minimal Docker image based on Alpine Linux.
CMD specifies which command should be run when a container is started from your image.
To build the docker image directly in the Cloud Shell, we can run this command:
docker build -t helloworld:latest .
To run a container with the image built, run
docker run --rm helloworld
You can see Hello World! in the terminal, which means our Dockerfile is valid and the image was successfully built.
Let’s now build the docker image with Cloud Build. A sample build config file is like this:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'eu.gcr.io/$PROJECT_ID/helloworld:latest', '.' ]
images: [eu.gcr.io/$PROJECT_ID/helloworld:latest]
In this example, we use the traditional Container Registry to store the built image. You can also use the more recent Artifact Registry if you like.
With the Container Registry, the name of a docker image (eu.gcr.io/$PROJECT_ID/helloworld:latest) is composed of four parts.
The first part is the hostname, which can be gcr.io, eu.gcr.io, us.gcr.io and asia.gcr.io. You can choose the one which is closest to you.
The second part is the project ID, which can be found on the home page of the GCP console. In Cloud Build, we can use the environment variable PROJECT_ID will be resolved to your project ID in the building process.
The third part is the repository name, which is also the root of the image name.
The fourth part is the tag of the image. A docker image can have multiple tags. If no tag is specified, the default tag will be latest.
Let’s take a deeper look into the sample build config file. A build config file can be written as a YAML file or JSON file. I personally prefer the YAML file because it is easier to write and more readable. A build config file can have multiple build steps. Each build step specifies an action that you want Cloud Build to perform. For each build step, Cloud Build executes a docker container as an instance of docker run. By default, Cloud Build executes all steps of a build serially on the same machine.
Cloud Build runs its build steps in a Docker container. To use an image from Docker Hub, just specify the name of the image, such as ubuntu. To use an image from other registries, specify the full registry path of the image, such as gcr.io/cloud-builders/docker.
Add a name field and point it to a container image to execute your task. Cloud Build provides several container images with common tools and languages installed in them. You can use any of these images. Formally, these images are called Cloud Builders in GCP. You can configure Cloud Build to run a specific command within the context of these builders.
The args field of a build step takes a list of arguments and passes them to the builder referenced by the name field. If the builder used in the build step has an entrypoint, args will be used as arguments to that entrypoint. If the builder does not define an entrypoint, the first element in args will be used as the entrypoint, and the remainder will be used as arguments. In this example, the docker builder has an entrypoint which is docker, the arguments specified by the args will be passed to it. If you compare it to the command we run in the Cloud Shell to build a docker image, you can see that the two commands are actually the same.
images stores the images built to Container Registry. The images field in the build config file specifies one or more Docker images to be pushed by Cloud Build to Container Registry.
Start the build using the build config file:
gcloud builds submit --config [CONFIG_FILE_PATH] [SOURCE_DIRECTORY]
Where:
[CONFIG_FILE_PATH]is the path to the build config file.[SOURCE_DIRECTORY]is the path or URL to the source code.
If you don’t specify a [CONFIG_FILE_PATH] and [SOURCE_DIRECTORY] in the gcloud builds submit command, Cloud Build assumes that the config file and the source code are in the current working directory.
We can run this command to start the build:
gcloud builds submit --config cloudbuild.yaml .
You may need to enable the Cloud Build API when you run the command. After the build process is finished, you can see the built image in the Container Registry console, which can be used in any application now.
To make Cloud Build useful, we need to set up a trigger to let Cloud Build automatically build the image when we push some new code to the repository.
First, we need to connect Cloud Build to our GitHub repository. You can fork this demo repository for demonstration. You can click the CONNECT REPOSITORY link on the Cloud Build page in the GCP console to connect to a GitHub repository.
You would need to authenticate yourself in this process. If after authentication, you still can’t see your GitHub repositories, you need to login into your GitHub account and go to settings -> Applications. Choose to configure Cloud Build and enable the READ/WRITE permissions for Cloud Build.
After you click Save, you will be redirected to GCP and be prompted to select a repository.
After the repository is selected, you can create a trigger for it. You can always create new triggers in the Cloud Build dashboard.
Let’s create a push trigger for the repository now. After the trigger is created, we can check it in the Cloud Build console:
We can click the three vertical dots on the right side to configure the trigger. The default settings are normally enough, but you may want to fine-tune it according to your actual need. For example, you may want to invoke the trigger only if you push a new tag, or you may only want to invoke the trigger if a new code is pushed to the main/master branch.
Now the trigger is set up. We can trigger it manually or by pushing a new commit to the main branch. For simplicity, you can modify the README file and create a new commit on GitHub. After the commit is made, you can see that the trigger is invoked.
You can check the newly built images in the Container Registry.
In this article, you learn to build a Dockerfile, Cloud Build config file (cloudbuild.yaml) and Cloud Build trigger step by step. With this simple example, I’m sure you can apply the techniques to your practical projects and streamline your workflow.
SuperDataMiner 
Leave a comment