CodePipeline
8 minute read
Introduction
CodePipeline is a continuous integration/continuous delivery (CI/CD) service offered by AWS. CodePipeline can be used to create automated pipelines that handle the build, test and deployment of software.
LocalStack comes with a bespoke execution engine that can be used to create, manage, and execute pipelines. It supports a variety of actions that integrate with S3, CodeBuild, CodeConnections, and more. The available operations can be found on the API coverage page.
Getting started
In this guide, we will create a simple pipeline that fetches an object from an S3 bucket and uploads it to a different S3 bucket. It is for users that are new to CodePipeline and have a basic knowledge of the AWS CLI and the awslocal wrapper.
Start LocalStack using your preferred method.
Create prerequisite buckets
Begin by creating the S3 buckets that will serve as the source and target.
$ awslocal s3 mb s3://source-bucket
$ awslocal s3 mb s3://target-bucketIt is important to note the CodePipeline requires source S3 buckets to have versioning enabled. This can be done using the S3 PutBucketVersioning operation.
$ awslocal s3api put-bucket-versioning \
--bucket source-bucket \
--versioning-configuration Status=EnabledNow create a placeholder file that will flow through the pipeline and upload it to the source bucket.
$ echo "Hello LocalStack!" > file$ awslocal s3 cp file s3://source-bucket
upload: ./file to s3://source-bucket/file
Pipelines also require an artifact store, which is also an S3 bucket that is used as intermediate storage.
$ awslocal s3 mb s3://artifact-store-bucketConfigure IAM
Depending on the specifics of the declaration, CodePipeline pipelines need access other AWS services. In this case we want our pipeline to retrieve and upload files to S3. This requires a properly configured IAM role that our pipeline can assume.
Create the role and make note of the role ARN:
# role.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "codepipeline.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
$ awslocal iam create-role --role-name role --assume-role-policy-document file://role.json | jq .Role.Arn
"arn:aws:iam::000000000000:role/role"
Now add a permissions policy to this role that permits read and write access to S3.
# policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": "*"
}
]
}
The permissions in the above example policy are relatively broad. You might want to use a more focused policy for better security on production systems.
$ awslocal iam put-role-policy --role-name role --policy-name policy --policy-document file://policy.jsonCreate pipeline
Now we can turn our attention to the pipeline declaration.
A pipeline declaration is used to define the structure of actions and stages to be performed. The following pipeline defines two stages with one action each. There is a source action which retrieves a file from an S3 bucket and marks it as the output. The output is placed in the intermediate bucket until it is picked up by the action in the second stage. This is a deploy action which uploads the file to the target bucket.
Pay special attention to roleArn, artifactStore.location as well as S3Bucket, S3ObjectKey, and BucketName. These correspond to the resources we created earlier.
# declaration.json
{
"name": "pipeline",
"executionMode": "SUPERSEDED",
"pipelineType": "V1",
"roleArn": "arn:aws:iam::000000000000:role/role",
"artifactStore": {
"type": "S3",
"location": "artifact-store-bucket"
},
"version": 1,
"stages": [
{
"name": "stage1",
"actions": [
{
"name": "action1",
"actionTypeId": {
"category": "Source",
"owner": "AWS",
"provider": "S3",
"version": "1"
},
"runOrder": 1,
"configuration": {
"S3Bucket": "source-bucket",
"S3ObjectKey": "file",
"PollForSourceChanges": "false"
},
"outputArtifacts": [
{
"name": "intermediate-file"
}
],
"inputArtifacts": []
}
]
},
{
"name": "stage2",
"actions": [
{
"name": "action1",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "S3",
"version": "1"
},
"runOrder": 1,
"configuration": {
"BucketName": "target-bucket",
"Extract": "false",
"ObjectKey": "output-file"
},
"inputArtifacts": [
{
"name": "intermediate-file"
}
],
"outputArtifacts": []
}
]
}
]
}
Create the pipeline using the following command:
$ awslocal codepipeline create-pipeline --pipeline file://./declaration.jsonVerify pipeline execution
A ‘pipeline execution’ is an instance of a pipeline in a running or finished state.
The CreatePipeline operation we ran earlier started a pipeline execution. This can be confirmed using:
$ awslocal codepipeline list-pipeline-executions --pipeline-name pipeline
{
"pipelineExecutionSummaries": [
{
"pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7",
"status": "Succeeded",
"startTime": 1745486647.138571,
"lastUpdateTime": 1745486648.290341,
"trigger": {
"triggerType": "CreatePipeline"
},
"executionMode": "SUPERSEDED"
}
]
}
Note the trigger.triggerType field specifies what initiated the pipeline execution. Currently in LocalStack, only two triggers are implemented: CreatePipeline and StartPipelineExecution.
The above pipeline execution was successful. This means that we can retrieve the output-file object from the target-bucket S3 bucket.
$ awslocal s3 cp s3://target-bucket/output-file output-file
download: s3://target-bucket/output-file to ./output-file
To verify that it is the same file as the original input:
$ cat output-file
Hello LocalStack!
Examine action executions
Using the ListActionExecutions, detailed information about each action execution such as inputs and outputs can be retrieved. This is useful when debugging the pipeline.
$ awslocal codepipeline list-action-executions --pipeline-name pipeline
{
"actionExecutionDetails": [
{
"pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7",
"actionExecutionId": "e38716df-645e-43ce-9597-104735c7f92c",
"pipelineVersion": 1,
"stageName": "stage2",
"actionName": "action1",
"startTime": 1745486647.269867,
"lastUpdateTime": 1745486647.289813,
"status": "Succeeded",
"input": {
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "S3",
"version": "1"
},
"configuration": {
"BucketName": "target-bucket",
"Extract": "false",
"ObjectKey": "output-file"
},
"resolvedConfiguration": {
"BucketName": "target-bucket",
"Extract": "false",
"ObjectKey": "output-file"
},
"region": "eu-central-1",
"inputArtifacts": [
{
"name": "intermediate-file",
"s3location": {
"bucket": "artifact-store-bucket",
"key": "pipeline/intermediate-file/01410aa4.zip"
}
}
]
},
"output": {
"outputArtifacts": [],
"executionResult": {
"externalExecutionId": "bcff0781",
"externalExecutionSummary": "Deployment Succeeded"
},
"outputVariables": {}
}
},
{
"pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7",
"actionExecutionId": "ae99095a-1d43-46ee-8a48-c72b6d60021e",
"pipelineVersion": 1,
"stageName": "stage1",
"actionName": "action1",
...
Note
LocalStack does not use the same logic to generate external execution IDs as AWS so there may be minor discrepancies. The same is true for status and error messages produced by actions.Pipelines
The operations CreatePipeline, GetPipeline, UpdatePipeline, ListPipelines, DeletePipeline are used to manage pipeline declarations.
LocalStack supports emulation for V1 pipelines. V2 pipelines are only created as mocks.
Tip
Emulation for V2 pipelines is not supported. Make sure that the pipeline type is explicitly set in the declaration.Pipeline executions can be managed with StartPipelineExecution, GetPipelineExecution, ListPipelineExecutions and StopPipelineExecutions.
When stopping pipeline executions with StopPipelineExecution, the stop and abandon method is not supported. Setting the abandon flag will have no impact. This is because LocalStack uses threads as the underlying mechanism to simulate pipelines, and threads cannot be cleanly preempted.
Action executions can be inspected using the ListActionExecutions operation.
Tagging pipelines
Pipelines resources can be tagged using the TagResource, UntagResource and ListTagsForResource operations.
$ awslocal codepipeline tag-resource \
--resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline \
--tags key=purpose,value=tutorial
$ awslocal codepipeline list-tags-for-resource \
--resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline
{
"tags": [
{
"key": "purpose",
"value": "tutorial"
}
]
}
$ awslocal codepipeline untag-resource \
--resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline \
--tag-keys purposeVariables
CodePipeline on LocalStack supports variables which allow dynamic configuration of pipeline actions.
Actions produce output variables which can be referenced in the configuration of subsequent actions. Make note that only when the action defines a namespace, its output variables are availabe to downstream actions.
Tip
If an action does not use a namespace, its output variables are not available to downstream actions.CodePipeline’s variable placeholder syntax is as follows:
#{namespace.variable}
As with AWS, LocalStack only makes the codepipeline.PipelineExecutionId variable available by default in a pipeline.
Actions
You can use runOrder to control parallel or sequential order of execution of actions.
The supported actions in LocalStack CodePipeline are listed below. Using an unsupported action will make the pipeline fail. If you would like support for more actions, please raise a feature request.
CodeBuild Source and Test
The CodeBuild Source and Test action can be used to start a CodeBuild container and run the given buildspec.
CodeConnections Source
The CodeConnections Source action is used to specify a VCS repo as the input to the pipeline.
LocalStack supports integration only with at this time. Please set the environment configuration option CODEPIPELINE_GH_TOKEN with the Personal Access Token to be able to fetch private repositories.
ECR Source
The ECR Source action is used to specify an Elastic Container Registry image as a source artifact.
ECS CodeDeploy Blue/Green
The ECS CodeDeply Blue/Green action is used to deploy container application using a blue/green deployment.
LocalStack does not accurately emulate a blue/green deployment due to limitations in ELB and ECS. It will only update the running ECS service with a new task definition and wait for the service to be stable.
ECS Deploy
The ECS Deploy action creates a revision of a task definition based on an already deployed ECS service.
Manual Approval
The Manual Approval action can be included in the pipeline declaration but it will only function as a no-op.
S3 Deploy
The S3 Deploy action is used to upload artifacts to a given S3 bucket as the output of the pipeline.
S3 Source
The S3 Source action is used to specify an S3 bucket object as input to the pipeline.
Limitations
- Emulation for V2 pipeline types is not supported. They will be created as mocks only.
- Rollbacks and stage retries are not available.
- Triggers are not implemented. Pipelines are executed only when CreatePipeline and StartPipelineExecution are invoked.
- Execution mode behaviours are not implemented. Parallel pipeline executions will not lead to stage locks and waits.
- Stage transition controls are not implemented.
- Manual approval action and PutApprovalResult operations are not available.