task-loops

Task Loop Extension

The Task Loop Extension for Tekton allows users to run a Task in a loop with varying parameter values. This functionality is provided by a controller that implements the Tekton Custom Task interface.

This is an experimental feature. The purpose is to explore potential use cases for looping in Tekton and how they may be achieved.

• Install
• Usage
• Limitations
• Uninstall

Install

The Task Loop Extension does not have any published releases. You can build and deploy it using ko.

Usage

Two resources are required to run a Task in a loop:

• A TaskLoop defines the Task to run and how to iterate it.
• A Run executes the TaskLoop and provides parameters to pass to the Task.

Configuring a TaskLoop

A TaskLoop definition supports the following fields:

• Required:
  • apiVersion - Specifies the API version, custom.tekton.dev/v1alpha1.
  • kind - Identifies this resource object as a TaskLoop object.
  • metadata - Specifies the metadata that uniquely identifies the TaskLoop, such as a name.
  • spec - Specifies the configuration for the TaskLoop.
    • taskRef or taskSpec - Specifies the Task to execute.
    • iterateParam - Specifies the name of the Task parameter that holds the values to iterate.
• Optional:
  • timeout - Specifies a timeout for the execution of a Task.
  • retries - Specifies the number of times to retry the execution of a Task after a failure.
  • concurrency - Specifies the number of TaskRuns that are allowed to run concurrently.

The example below shows a basic TaskLoop:

apiVersion: custom.tekton.dev/v1alpha1
kind: TaskLoop
metadata:
  name: echoloop
spec:
  taskSpec:
    params:
      - name: message
        type: string
    steps:
      - name: echo
        image: ubuntu
        script: |
          #!/usr/bin/env bash
          echo "$(params.message)"
  iterateParam: message

Specifying the target task

To specify the Task you want to execute, use the taskRef field as shown below:

spec:
  taskRef:
    name: message-task

You can also embed the Task definition directly using the taskSpec field:

spec:
  taskSpec:
    params:
      - name: message
        type: string
    steps:
      - name: echo
        image: ubuntu
        script: |
          #!/usr/bin/env bash
          echo "$(params.message)"

Specifying the iteration parameter

The iterateParam field specifies the name of the Task parameter which varies for each execution of the Task. This is what controls the loop.

• The parameter type as defined in the Task must be string.
• The parameter value as defined in the Run can be an array or a string. Strings will be split into arrays, one element for each line in the original string.

For example, suppose you have a Task that runs tests based on a parameter name called test-type.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: testtask
spec:
  params:
    - name: test-type
      type: string
  steps:
    - name: run-test
      image: docker.hub/...
      args: ["$(params.test-type)"]

If you want to run this Task for multiple test types, your TaskLoop would look like this:

apiVersion: custom.tekton.dev/v1alpha1
kind: TaskLoop
metadata:
  name: testloop
spec:
  taskRef:
    name: testtask
  iterateParam: test-type

Your Run would look like this:

apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
  generateName: testloop-run-
spec:
  params:
    - name: test-type
      value:
        - codeanalysis
        - unittests
        - e2etests
  ref:
    apiVersion: custom.tekton.dev/v1alpha1
    kind: TaskLoop
    name: testloop

This Run would result in three TaskRuns being created to run the Task testtask. In the first TaskRun the parameter test-type would be set to codeanalysis. In the second TaskRun the parameter test-type would be set to unittests. In the third TaskRun the parameter test-type would be set to e2etests.

Using a parameter of type string is particularly useful when the array of options is dynamically generated:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: pipeline-with-tests
spec:
  tasks:
    - name: test-selector
      taskSpec:
         results:
           - name: listoftests
             description: the list of tests to execute
         steps:
           - name: producer
             image: busybox
             script: |
               # Some logic to select tests
               echo "codeanalysis" > $(results.listoftests.path)
               echo "unittests" >> $(results.listoftests.path)
    - name: loop
      taskRef:
        apiVersion: custom.tekton.dev/v1alpha1
        kind: TaskLoop
        name: testloop
      params:
        - name: test-type
          value: $(tasks.test-selector.results.listoftests)

Specifying a timeout

You can use the timeout field to set each TaskRun's timeout value. If you do not specify this value, Tekton's global default timeout value applies. If you set the timeout to 0, each TaskRun will have no timeout and will run until it completes successfully or fails from an error.

The timeout value is a duration conforming to Go's ParseDuration format. For example, valid values are 1h30m, 1h, 1m, 60s, and 0.

See Configuring the failure timeout for more information about how TaskRun processes the timeout.

Specifying retries

You can use the retries field to specify the number of times to retry the execution of a Task when it fails. If you don't explicitly specify a value, no retry is performed.

Specifying concurrency

You can use the concurrency field to specify the number of TaskRuns that are allowed to run concurrently. The default is 1. If you specify 0 or a negative value, then the TaskRuns for all iterations are allowed to run concurrently.

Configuring a Run

A Run definition supports the following fields:

• Required:
  • apiVersion - Specifies the API version, tekton.dev/v1alpha1.
  • kind - Identifies this resource object as a Run object.
  • metadata - Specifies the metadata that uniquely identifies the Run, such as a name.
  • spec - Specifies the configuration for the Run.
    • ref - Specifies the type and name of the TaskLoop to execute.
    • params - Specifies the execution parameters for the Task.
• Optional:
  • serviceAccountName - Specifies a ServiceAccount object that provides custom credentials.
  • workspaces - Specifies the physical volumes to use for the Workspaces required by the Task.
  • podTemplate - Specifies a Pod template to use to configure pods created by the TaskRuns.

Specifying the TaskLoop

Your Run must reference a TaskLoop. It must include the apiVersion, kind and name fields as shown below.

apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
  generateName: run-
spec:
  params:
    # […]
  ref:
    apiVersion: custom.tekton.dev/v1alpha1
    kind: TaskLoop
    name: mytaskloop

Specifying parameters

Your Run can provide any parameters that are defined by the Task that is referenced by the TaskLoop. The parameters are passed through as is to each TaskRun with the exception of the iteration parameter named by iterateParam in the TaskLoop.

• In the Run, the iteration parameter value must be an array.
• A TaskRun is created for each array element with the iterate parameter value set to the element.
• In the Task the iteration parameter type must be string.

Specifying workspaces, service account, and pod template

A Run object can specify workspaces, a service account name, or a pod template. These are passed through as is to each TaskRun.

Monitoring execution status

As your Run executes, its status field accumulates information on the execution of each TaskRun as well as the Run as a whole. This information includes the complete status of each TaskRun under status.extraFields.taskRuns.

apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
  generateName: run-
# […]
status:
  completionTime: "2020-09-24T17:33:10Z"
  conditions:
  - lastTransitionTime: "2020-09-24T17:33:10Z"
    message: All TaskRuns completed successfully
    reason: Succeeded
    status: "True"
    type: Succeeded
  extraFields:
    taskRuns:
      run-nt4p7-00001-zhtc8:
        iteration: 1
        status:
          # TaskRun status for iteration 1 is here
      run-nt4p7-00002-674jw:
        iteration: 2
        status:
          # TaskRun status for iteration 2 is here
  startTime: "2020-09-24T17:32:51Z"

For more information about monitoring Run in general, see Monitoring execution status.

Cancelling a Run

To cancel a Run that's currently executing, update its status to mark it as cancelled. The running TaskRun is cancelled.

Example of cancelling a Run:

apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
  name: run-loop
spec:
  # […]
  status: "RunCancelled"

Limitations

The following limitations exist. These limitations may be addressed in future issues based on community feedback.

• The value of only one Task parameter can be varied between TaskRun executions.

• If a TaskRun fails, the execution of the TaskLoop stops. TaskRuns for remaining iteration values are not created.

• Task results are not collected into Run results (run.status.results). However the results of each TaskRun can be seen in the TaskRun status under run.status.extraFields.

• There are no metrics specific to Run.

Uninstall

Use the command kubectl delete -f config\ to delete the Kubernetes resources for this project.

Want to get involved?

Visit the Tekton Community project for an overview of our processes.