I am currently refactoring my project, SmartFactory v2, and I want to set up a CI/CD pipeline to automate the testing and deployment process. In the past, I used GitHub Actions for CI, but I wanted to try a solution that is more Kubernetes native. I decided to use Tekton Pipelines.

1. What is Tekton?

Tekton is a Cloud Native CI/CD tools that allows you to build, test, and deploy your applications on Kubernetes.

Some important concepts in Tekton:

• Task: A reusable, loosely coupled number of steps that perform a specific task.
• Pipeline: A collection of tasks that are executed in a specific order.
• PipelineRun: An instance of a pipeline. It defines the execution of a pipeline.
• TaskRun: An instance of a task. It defines the execution of a task.

2. Prerequisites

• A Kubernetes cluster (Minikube, Docker Desktop, or any other Kubernetes cluster)
• kubectl CLI tool
• Tekton CLI tool (tkn cli)
• Tekton Pipelines installed on your Kubernetes cluster (Installation Guide)
• Tekton Dashboard (Optional)

3. Creating CI Pipeline

I have this repository that contains 4 microservices: data-collector-service, notification-service, subscription-service and user-management-service. I want to create a CI pipeline that will build and test each microservice.

Our pipeline will have the following steps:

• Clone the repository: Clone the repository from GitHub.
• Build the microservices: Build the microservices using Maven.
• Run the tests: Run the tests using Maven.
• Build and push the Docker images: Build the Docker images and push them to Docker Hub.

So, we need a Tekton Task for each of these steps.

3.1. Create Tasks - Tekton Hub

Tekton Hub is a collection of reusable Tekton tasks and pipelines. You can find a lot of tasks and pipelines that you can use in your CI/CD pipeline.

For our pipeline, we will use the following tasks:

• git-clone: Clone a git repository.
• maven: Build and test a Maven project.
• kaniko: Build and push Docker images.

Let’s install these tasks on our Kubernetes cluster:

1
2
3
tkn hub install task git-clone
tkn hub install task maven
tkn hub install task kaniko

List the installed tasks:

1
tkn task list

1
2
3
4
NAME        DESCRIPTION              AGE
git-clone   These Tasks are Git...   2 minutes ago
kaniko      This Task builds a ...   2 minutes ago
maven       This Task can be us...   2 minutes ago

3.2. Create the Pipeline

A Tekton Pipeline is a collection of tasks that are executed in a specific order.

Let’s create a pipeline.yaml file that defines our pipeline:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: smartfactory-ci-pipeline
spec:
  description: |
    This pipeline clones a git repo, then echoes the README file to the stout.
  params:
    - name: REPO_URL
      type: string
      description: The git repo URL to clone from.
    - name: PROJECT_DIR
      type: string
      description: The subdirectory to clone from the git repo.
      default: ""
    - name: IMAGE
      type: string
      description: The image name to build.
    - name: TAG
      type: string
      description: The image tag to build.
      default: "latest"
    - name: ACCOUNT
      type: string
      description: The docker account to push the image to.
  workspaces:
    - name: shared-workspace
      description: |
        This workspace contains the cloned repo files, so they can be read by the
        next task.
    - name: maven-settings
      description: |
        This workspace contains the maven settings file, so it can be used by the
        maven task.
    - name: maven-local-m2
      description: |
        This workspace contains the maven local repository, so it can be used by the
        maven task.
    - name: docker-credentials
      description: |
        This workspace contains the docker credentials, so it can be used by the
        kaniko task.
  tasks:
    - name: fetch-repository
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: shared-workspace
      params:
        - name: url
          value: "$(params.REPO_URL)"
        - name: deleteExisting
          value: "true"
    - name: maven-run
      taskRef:
        name: maven
      runAfter:
        - fetch-repository
      params:
        - name: MAVEN_IMAGE
          value: "maven:3.8.8-eclipse-temurin-21-alpine"
        - name: CONTEXT_DIR
          value: "$(params.PROJECT_DIR)"
        - name: GOALS
          value:
            - clean
            - test
            - package
      workspaces:
        - name: maven-settings
          workspace: maven-settings
        - name: source
          workspace: shared-workspace
        - name: maven-local-repo
          workspace: maven-local-m2
    - name: build-push
      runAfter:
        - maven-run
      taskRef:
        name: kaniko
      workspaces:
        - name: source
          workspace: shared-workspace
        - name: dockerconfig
          workspace: docker-credentials
      params:
        - name: IMAGE
          value: "$(params.ACCOUNT)/$(params.IMAGE):$(params.TAG)"
        - name: DOCKERFILE
          value: "$(params.PROJECT_DIR)/Dockerfile"
        - name: CONTEXT
          value: "$(params.PROJECT_DIR)"

In this pipeline, we have defined three tasks:

• fetch-repository: Clones the git repository.
• maven-run: Builds the Maven project.
• build-push: Builds and pushes the Docker image.

We have also defined some parameters that are used by the tasks:

• REPO_URL: The git repo URL to clone from.
• PROJECT_DIR: The subdirectory to clone from the git repo. (We have 3 microservices in our repository, so we need to specify the subdirectory for each microservice)
• IMAGE: The image name to build.
• TAG: The image tag to build.
• ACCOUNT: The docker account to push the image to.

Workspaces are used to share files between tasks.

3.3. Create the PipelineRun

Now that we have defined our pipeline, we need to create a PipelineRun that will execute the pipeline.

We need to create a pipelinerun.yaml file that defines our PipelineRun for each microservice:

Before creating the PipelineRun, we need to create a secret that contains the Docker credentials that will be used by the kaniko task to push the Docker image to Docker Hub.

1
2
3
4
5
6
apiVersion: v1
data:
  config.json: <base64 encoded docker config.json>
kind: Secret
metadata:
  name: docker-credentials

To create the secret, you need to create a config.json file that contains your Docker credentials and then encode it to base64:

1
2
3
4
5
6
7
8
9
10
{
  "auths": {
    "https://index.docker.io/v1/": {
      "username": "<username>",
      "password": "<token>",
      "email": "<email>",
      "auth": "<base64 encoded username:token>"
    }
  }
}

Let’s create a pipelinerun.yaml file for the notification-service microservice:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: smartfactory-ci-notification-service-
spec:
  pipelineRef:
    name: smartfactory-ci-pipeline
  podTemplate:
    securityContext:
      fsGroup: 65532
  workspaces:
    - name: shared-workspace
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi
    - name: maven-local-m2
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 10Gi
    - name: maven-settings
      emptyDir: { }
    - name: docker-credentials
      secret:
        secretName: docker-credentials
  params:
    - name: REPO_URL
      value: https://github.com/DragomirAlin/smartfactory
    - name: PROJECT_DIR
      value: "notification-service"
    - name: IMAGE
      value: "notification-service"
    - name: TAG
      value: "latest"
    - name: ACCOUNT
      value: "dragomiralin"

In this PipelineRun, we have defined the following parameters:

• REPO_URL: The git repo URL to clone from.
• PROJECT_DIR: The subdirectory to clone from the git repo. (notification-service)
• IMAGE: The image name to build. (notification-service)
• TAG: The image tag to build. (latest)
• ACCOUNT: The docker account to push the image to. (dragomiralin)

We have also defined some workspaces:

• shared-workspace: Contains the cloned repo files.
• maven-local-m2: Contains the maven local repository.
• maven-settings: Contains the maven settings file.
• docker-credentials: Contains the docker credentials.

4. Run the Pipeline

Now that we have defined our pipeline and pipeline runs, we can run the pipeline using the Tekton CLI tool.

Let’s create the pipeline:

1
kubectl apply -f pipeline.yaml

See the list of pipelines:

1
2
3
tkn pipeline list
NAME                       AGE            LAST RUN                                        STARTED          DURATION   STATUS
smartfactory-ci-pipeline   3 minutes ago   -                                               -                 -      Succeeded

Let’s run the pipeline for the notification-service microservice:

1
kubectl create -f pipelinerun-notification-service.yaml

See the list of pipeline runs:

1
2
3
tkn pipelinerun list
NAME                                            STARTED          DURATION   STATUS
smartfactory-ci-notification-service-8l75f      4 minutes ago   1m45s      Succeeded

We can check the logs of the pipeline run:

1
tkn pipelinerun logs smartfactory-ci-notification-service-8l75f

The result of the pipeline run is a Docker image that is pushed to Docker Hub:

Tekton has a nice dashboard that you can use to see the status of your pipelines and pipeline runs. You need to install the Tekton Dashboard on your Kubernetes cluster.

Next Steps

• Use Tekton Triggers to trigger the pipeline when a new commit is pushed to the repository.