Difficulty: Easy
Estimated Time: 10min

The central concept of any CI/CD system is to run tasks, but 99% of these tasks run as part of pipelines.

It is recommended to be familiar with the core configuration of tasks before moving on to this scenario where we deep dive on pipeline configuration.

Basic concourse and client configuration have been initiated as part of this introduction and should be ready for you to use soon.

In this scenario we've learned how to work with pipelines.

Next we will learn how to define pipeline parameters and how to work with credential managers.

Pipeline basics

Step 1 of 5

Working with Pipelines

Pipelines are configured entirely via the fly CLI. There is no GUI.

Let's see how to upload a pipeline, run it and watch its execution all through the command line.

The provided pipeline.yml (click to open) configuration defines a basic pipepline using the hello world task as covered in "Task Basics" scenario of this course.

This first pipeline is unimpressive - a single job job-hello-world with no inputs from the left and no outputs to its right, no jobs feeding into it, nor jobs feeding from it and no triggers to kick off the job. It is the most basic pipeline.

To submit a pipeline configuration to Concourse from a file on your local disk you use the set-pipeline (alias sp) command.

The set-pipeline command takes the name you want to store the pipeline as (-p) and the yaml configuration of the pipeline (-c):

fly -t tutorial sp -p hello-world -c pipeline.yml

You will be prompted to apply any configuration changes you made in the yaml file each time with a clear diff of what you changed.

apply configuration? [yN]:

Press y

Note: Unfortunetaly the link to view the pipeline won't work in this Katacoda learning environment and we will use the terminal only.

You should see:

pipeline created!
you can view your pipeline here: http://docker:8080/teams/main/pipelines/hello-world

the pipeline is currently paused. to unpause, either:
  - run the unpause-pipeline command
  - click play next to the pipeline in the web ui

New pipelines start paused as you might not yet be ready for triggers to fire and trigger jobs to run.

To view all defined pipelines use the pipeplines (alias ps) command:

fly -t tutorial ps

To unpause the pipeline use the unpause-pipeline (alias up) command:

fly -t tutorial up -p hello-world

As our pipeline did not define any triggers for the job, we need to manually trigger the job, this will queue the job for execution.

Note: Triggering a job in a paused pipeline will keep the job pending until the pipeline itself is unpaused!

Use the trigger-job (alias tj) command to trigger the job-hello-world in our hello-world pipeline.

We may pass in the -w flag to monitor the execution of the job at the same time:

fly -t tutorial tj -j hello-world/job-hello-world -w

Once the job has completed, we may list all the builds for a pipeline using the builds ( alias bs) command with the -p flag:

fly -t tutorial bs -p hello-world

To review the output of the last build of our job, use the watch (alias w) command with the -j flag:

fly -t tutorial w -j hello-world/job-hello-world

To review the output of a particular build of our job add the -b flag:

fly -t tutorial w -j hello-world/job-hello-world -b 1

We have succesfully created, ran and watched our first basic pipeline, next we will create more advanced pipelines.