This post is one of four tutorials that help you put into practice concepts from Microservices March 2023: Start Delivering Microservices:
Automating deployments is critical to the success of most projects. However, it’s not enough to just deploy your code. You also need to ensure downtime is limited (or eliminated), and you need the ability to roll back quickly in the event of a failure. Combining canary deployment and blue‑green deployment is a common approach to ensuring new code is viable. This strategy includes two steps:
If you’re unfamiliar with the different use cases for distributing traffic between different versions of an app or website (traffic splitting), read How to Improve Resilience in Kubernetes with Advanced Traffic Management on our blog to gain a conceptual understanding of blue‑green deployments, canary releases, A/B testing, rate limiting, circuit breaking, and more. While the blog is specific to Kubernetes, the concepts are broadly applicable to microservices apps.
In this tutorial, we show how to automate the first step of a canary blue‑green deployment using GitHub Actions. In the four challenges of the tutorial you use Microsoft Azure Container Apps to deploy a new version of your application, then use Azure Traffic Manager to shift traffic from the old environment to the new environment:
Note: While this tutorial uses Azure Container Apps, the concepts and techniques can be applied to any cloud‑based host.
If you want to do this tutorial in your own environment, you need:
Create and configure the necessary base resources. Fork and clone the repository for the tutorial, log in to the Azure CLI, and install the extension for Azure Container Apps.
In your home directory, create the microservices-march directory. (You can also use a different directory name and adapt the instructions accordingly.)
Note: Throughout the tutorial the prompt on the Linux command line is omitted, to make it easier to copy and paste the commands into your terminal.
mkdir ~/microservices-marchcd ~/microservices-march
Fork and clone the Microservices March platform repository to your personal GitHub account, using either the GitHub CLI or GUI.
If using the GitHub GUI:
If using the GitHub CLI, run:
gh repo fork microservices-march/platform -–clone
Login to the Azure CLI. Follow the prompts to log in using a browser:
az login
[
{
"cloudName": "AzureCloud",
"homeTenantId": "cfd11e0f-1435-450s-bdr1-ffab73b4er8e",
"id": "60efapl2-38ad-41w8-g49a-0ecc6723l97c",
"isDefault": true,
"managedByTenants": [],
"name": "Azure subscription 1",
"state": "Enabled",
"tenantId": "cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde",
"user": {
"name": "user@example.com",
"type": "user"
}
}
]
Install the containerapp
extension:
az extension add --name containerapp -upgradeThe installed extension 'containerapp' is in preview.
In this initial challenge, you create an NGINX Azure Container App as the initial version of the application used as the baseline for the canary blue‑green deployment. Azure Container Apps is a Microsoft Azure service you use to easily execute application code packaged in a container in a production‑ready container environment.
Create an Azure resource group for the container app:
az group create --name my-container-app-rg --location westus{
"id": "/subscriptions/0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg",
"location: "westus",
"managedBy": null,
"name": "my-container-app-rg",
"properties": {
"provisioningState": "Succeeded"
},
"tags": null,
"type": "Microsoft.Resources/resourceGroups"
}
Deploy the container to Azure Container Apps (this step may take a while):
az containerapp up \ --resource-group my-container-app-rg \
--name my-container-app \
--source ./ingress \
--ingress external \
--target-port 80 \
--location westus
...
- image:
registry: cac085021b77acr.azurecr.io
repository: my-container-app
tag: "20230315212413756768"
digest: sha256:90a9fc67c409e244195ec0ea190ce3b84195ae725392e8451...
runtime-dependency:
registry: registry.hub.docker.com
repository: library/nginx
tag: "1.23"
digest: sha256:aa0afebbb3cfa473099a62c4b32e9b3fb73ed23f2a75a65ce...
git: {}
Run ID: cf1 was successful after 27s
Creating Containerapp my-container-app in resource group my-container-app-rg
Adding registry password as a secret with name "ca2ffbce7810acrazurecrio-cac085021b77acr"
Container app created. Access your app at https://my-container-app.delightfulmoss-eb6d59d5.westus.azurecontainerapps.io/
...
In the output in Step 2, find the name and the URL of the container app you’ve created in the Azure Container Registry (ACR). They’re highlighted in orange in the sample output. You will substitute the values from your output (which will be different from the sample output in Step 2) for the indicated variables in commands throughout the tutorial:
Name of container app – In the image.registry
key, the character string before .azurecr.io
. In the sample output in Step 2, it is cac085021b77acr
.
Substitute this character string for <ACR_name>
in subsequent commands.
URL of the container app – The URL on the line that begins Container
app
created
. In the sample output in Step 2, it is https://my-container-app.delightfulmoss-eb6d59d5.westus.azurecontainerapps.io/
.
Substitute this URL for <ACR_URL>
in subsequent commands.
Enable revisions for the container app as required by for a blue‑green deployment:
az containerapp revision set-mode \ --name my-container-app \
--resource-group my-container-app-rg \
--mode multiple
"Multiple"
(Optional) Test that the deployment is working by querying the /health endpoint in the container:
curl <ACR_URL>/healthOK
In this challenge, you obtain the JSON token that enables you to automate Azure container app deployments.
You start by obtaining the ID for the Azure Container Registry (ACR), and then the principal ID for your Azure managed identity. You then assign the built‑in Azure role for ACR to the managed identity, and configure the container app to use the managed identity. Finally you obtain the JSON credentials for the managed identity, which will be used by GitHub Actions to authenticate to Azure.
While this set of steps may seem tedious, you only need to perform them once when creating a new application and you can fully script the process. The tutorial has you perform the steps manually to become familiar with them.
Note: This process for creating credentials for deployment is specific to Azure.
Look up the principal ID of your managed identity. It appears in the PrincipalID
column of the output (which is divided across two lines for legibility). You’ll substitute this value for <managed_identity_principal_ID>
in Step 3:
az containerapp identity assign \ --name my-container-app \
--resource-group my-container-app-rg \
--system-assigned \
--output table
PrincipalId ...
------------------------------------ ...
39f8434b-12d6-4735-81d8-ba0apo14579f ...
... TenantId
... ------------------------------------
... cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde
Look up the container app’s resource ID in ACR, replacing <ACR_name>
with the name you recorded in Step 3 of Challenge 1. You’ll substitute this value for <ACR_resource_ID>
in the next step:
az acr show --name <ACR_name> --query id --output tsv/subscriptions/60efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr
Assign the built‑in Azure role for ACR to the container app’s managed identity, replacing <managed_identity_principal_ID>
with the managed identity obtained in Step 1, and <ACR_resource_ID>
with the resource ID obtained in Step 2:
az role assignment create \ --assignee <managed_identity_principal_ID> \
--role AcrPull \
--scope <ACR_resource_ID>
{
"condition": null,
"conditionVersion": null,
"createdBy": null,
"createdOn": "2023-03-15T20:28:40.831224+00:00",
"delegatedManagedIdentityResourceId": null,
"description": null,
"id": "/subscriptions/0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr/providers/Microsoft.Authorization/roleAssignments/f0773943-8769-44c6-a820-ed16007ff249",
"name": "f0773943-8769-44c6-a820-ed16007ff249",
"principalId": "39f8ee4b-6fd6-47b5-89d8-ba0a4314579f",
"principalType": "ServicePrincipal",
"resourceGroup": "my-container-app-rg",
"roleDefinitionId": "/subscriptions/60e32142-384b-43r8-9329-0ecc67dca94c/providers/Microsoft.Authorization/roleDefinitions/7fd21dda-4fd3-4610-a1ca-43po272d538d",
"scope": "/subscriptions/ 0efafl2-38ad-41w8-g49a-0ecc6723l97c/resourceGroups/my-container-app-rg/providers/Microsoft.ContainerRegistry/registries/cac085021b77acr",
"type": "Microsoft.Authorization/roleAssignments",
"updatedBy": "d4e122d6-5e64-4bg1-9cld-2aceeb0oi24d",
"updatedOn": "2023-03-15T20:28:41.127243+00:00"
}
Configure the container app to use the managed identity when pulling images from ACR, replacing <ACR_name>
with the container app name you recorded in Step 3 in Challenge 1 (and also used in Step 2 above):
az containerapp registry set \ --name my-container-app \
--resource-group my-container-app-rg \
--server <ACR_name>.azurecr.io \
--identity system
[
{
"identity": "system",
"passwordSecretRef": "",
"server": "cac085021b77acr.azurecr.io",
"username": ""
}
]
Look up your Azure subscription ID.
az account show --query id --output tsv0efafl2-38ad-41w8-g49a-0ecc6723l97c
Create a JSON token which contains the credentials to be used by the GitHub Action, replacing <subscription_ID>
with your Azure subscription ID. Save the output to paste in as the value of the secret named AZURE_CREDENTIALS
in Add Secrets to Your GitHub Repository. You can safely ignore the warning about --sdk-auth
being deprecated; it’s a known issue:
az ad sp create-for-rbac \ --name my-container-app-rbac \
--role contributor \
--scopes /subscriptions/<subscription_ID>/resourceGroups/my-container-app-rg \
--sdk-auth \
--output json
Option '--sdk-auth' has been deprecated and will be removed in a future release.
...
{
"clientId": "0732444d-23e6-47fb-8c2c-74bddfc7d2er",
"clientSecret": "qg88Q~KJaND4JTWRPOLWgCY1ZmZwN5MK3N.wwcOe",
"subscriptionId": "0efafl2-38ad-41w8-g49a-0ecc6723l97c",
"tenantId": "cfda3e0f-14g5-4e05-bfb1-ffab73b4fsde",
"activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
"resourceManagerEndpointUrl": "https://management.azure.com/",
"activeDirectoryGraphResourceId": "https://graph.windows.net/",
"sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
"galleryEndpointUrl": "https://gallery.azure.com/",
"managementEndpointUrl": "https://management.core.windows.net/"
}
In this challenge, you add secrets to your GitHub repo (used to manage sensitive data in your GitHub Action workflows), create an Action workflow file, and execute the Action workflow.
For a detailed introduction to secrets management, see the second tutorial for Microservices March 23, How to Securely Manage Secrets in Containers on our blog.
To deploy a new version of the application, you need to create a series of secrets in the GitHub repository you forked in Set Up. The secrets are the JSON credentials for the managed identity created in Challenge 2, and some sensitive deployment‑specific parameters necessary to deploy new versions of the NGINX image to Azure. In the next section you’ll use these secrets in a GitHub Action to automate the canary blue‑green deployment.
If using the GitHub GUI:
Type the following values in the indicated fields:
AZURE_CREDENTIALS
Repeat Steps 3–5 three times to create the secrets listed in the table. Type the values from the Secret Name and Secret Value columns into the GUI’s Name and Secret fields respectively. For the third secret, replace <ACR_name>
with the name assigned to the container app which you recorded in Step 3 of Challenge 1.
Secret Name | Secret Value |
---|---|
CONTAINER_APP_NAME |
my-container-app |
RESOURCE_GROUP |
my-container-app-rg |
ACR_NAME |
<ACR_name> |
If using the GitHub CLI:
At the root of your repo, create a temporary file.
touch ~/creds.json
Create the secret:
gh secret set AZURE_CREDENTIALS --repo <your_GitHub_account>/platform < ~/creds.json
Delete creds.json:
rm ~/creds.json
Repeat this command to create three more secrets:
gh secret set <secret_name> --repo <your_GitHub_account>/platform
For each repetition, replace <secret_name>
with one of the values in the Secret Name column in the table. At the prompt, paste the associated value from the Secret Value column. For the third secret, replace <ACR_name>
with the name assigned to the container app which you recorded in Step 3 of Challenge 1.
Secret Name | Secret Value |
---|---|
CONTAINER_APP_NAME |
my-container-app |
RESOURCE_GROUP |
my-container-app-rg |
ACR_NAME |
<ACR_name> |
With the managed identity and secrets in place, you can create a workflow file for a GitHub Action that automates the canary blue‑green deployment.
Note: Workflow files are defined in YAML format, where whitespace is significant. Be sure to preserve the indentation shown in the steps below.
Create a file for the Action workflow.
If using the GitHub GUI:
If using the GitHub CLI, create the .github/workflows directory and create a new file called main.yml:
mkdir .github/workflowstouch .github/workflows/main.yml
Using your preferred text editor, add the text of the workflow to main.yml. The easiest method is to copy in the text that appears in Full Workflow File. Alternatively, you can build the file manually by adding the set of snippets annotated in this step.
Note: Workflow files are defined in YAML format, where whitespace is significant. If you copy in the snippets, be sure to preserve the indentation (and to be extra sure, compare your file to Full Workflow File.
Define the workflow’s name:
name: Deploy to Azure
Configure the workflow to run when a push or pull request is made to the main branch:
on:
push:
branches:
- main
pull_request:
branches:
- main
In the jobs
section, define the build-deploy
job, which checks out the code, logs into Azure, and deploys the application to Azure Container App:
jobs:
build-deploy:
runs-on: ubuntu-22.04
steps:
- name: Check out the codebase
uses: actions/checkout@v3
- name: Log in to Azure
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Build and deploy Container App
run: |
# Add the containerapp extension manually
az extension add --name containerapp --upgrade
# Use Azure CLI to deploy update
az containerapp up -n ${{ secrets.CONTAINER_APP_NAME }}\
-g ${{ secrets.RESOURCE_GROUP }} \
--source ${{ github.workspace }}/ingress \
--registry-server ${{ secrets.ACR_NAME }}.azurecr.io
Define the test-deployment
job, which obtains the staging URL of the newly deployed revision and uses a GitHub Action to ping the API endpoint /health to ensure the new revision is responding. If the health check succeeds, the Azure Traffic Manager on the container app is updated to point all traffic at the newly deployed container.
Note: Be sure to indent the test-deployment
key at the same level as the build-deploy
key you defined in the previous bullet:
test-deployment:
needs: build-deploy
runs-on: ubuntu-22.04
steps:
- name: Log in to Azure
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Get new container name
run: |
# Add the containerapp extension manually
az extension add --name containerapp --upgrade
# Get the last deployed revision name
REVISION_NAME=`az containerapp revision list -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --query "[].name" -o tsv | tail -1`
# Get the last deployed revision's fqdn
REVISION_FQDN=`az containerapp revision show -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision "$REVISION_NAME" --query properties.fqdn -o tsv`
# Store values in env vars
echo "REVISION_NAME=$REVISION_NAME" >> $GITHUB_ENV
echo "REVISION_FQDN=$REVISION_FQDN" >> $GITHUB_ENV
- name: Test deployment
id: test-deployment
uses: jtalk/url-health-check-action@v3 # Marketplace action to touch the endpoint
with:
url: "https://${{ env.REVISION_FQDN }}/health" # Staging endpoint
- name: Deploy succeeded
run: |
echo "Deployment succeeded! Enabling new revision"
az containerapp ingress traffic set -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision-weight "${{ env.REVISION_NAME }}=100"
This is the complete text for the Action workflow file.
name: Deploy to Azure
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build-deploy:
runs-on: ubuntu-22.04
steps:
- name: Check out the codebase
uses: actions/checkout@v3
- name: Log in to Azure
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Build and deploy Container
run: |
# Add the containerapp extension manually
az extension add --name containerapp -upgrade
# Use Azure CLI to deploy update
az containerapp up -n ${{ secrets.CONTAINER_APP_NAME }} \
-g ${{ secrets.RESOURCE_GROUP }} \
--source ${{ github.workspace }}/ingress \
--registry-server ${{ secrets.ACR_NAME }}.azurecr.io
test-deployment:
needs: build-deploy
runs-on: ubuntu-22.04
steps:
- name: Log in to Azure
uses: azure/login@v1
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}
- name: Get new container name
run: |
# Install the containerapp extension for the Azure CLI
az extension add --name containerapp --upgrade
# Get the last deployed revision name
REVISION_NAME=`az containerapp revision list -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --query "[].name" -o tsv | tail -1`
# Get the last deployed revision's fqdn
REVISION_FQDN=`az containerapp revision show -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision "$REVISION_NAME" --query properties.fqdn -o tsv`
# Store values in env vars
echo "REVISION_NAME=$REVISION_NAME" >> $GITHUB_ENV
echo "REVISION_FQDN=$REVISION_FQDN" >> $GITHUB_ENV
- name: Test deployment
id: test-deployment
uses: jtalk/url-health-check-action@v3 # Marketplace action to touch the endpoint
with:
url: "https://${{ env.REVISION_FQDN }}/health" # Staging endpoint
- name: Deploy succeeded
run: |
echo "Deployment succeeded! Enabling new revision"
az containerapp ingress traffic set -n ${{ secrets.CONTAINER_APP_NAME }} -g ${{ secrets.RESOURCE_GROUP }} --revision-weight "${{ env.REVISION_NAME }}=100"
If using the GitHub GUI:
If using the GitHub CLI:
Add main.yml to the Git staging area:
git add .github/workflows/main.yml
Commit the file:
git commit -m "feat: create GitHub Actions workflow"
Push your changes to GitHub:
git push
Monitor the progress of the workflow:
gh workflow view main.yml --repo <your_GitHub_account>/platform
In this challenge, you test the workflow. You first simulate a successful update to your Ingress load balancer and confirm the application has been updated. You then simulate an unsuccessful update (which leads to an internal server error) and confirm that the published application remains unchanged.
Create a successful update and watch the workflow succeed.
If using the GitHub GUI:
In the location
/health
block near the end of the file, change the return
directive as indicated:
location /health {
access_log off;
return 200 "Successful Update!\n";
}
Successful
Update!
message.You can confirm the message by starting a terminal session and sending a health‑check request to the app, again replacing <ACR_URL>
with the value you recorded in Step 3 of Challenge 1:
curl <ACR_URL>/healthSuccessful Update!
If using the GitHub CLI:
Create a new branch called patch-1:
git checkout -b patch-1
In your preferred text editor open ingress/default.conf.template and in the location
/health
block near the end of the file, change the return
directive as indicated:
location /health {
access_log off;
return 200 "Successful Update!\n";
}
Add default.conf.template to the Git staging area:
git add ingress/default.conf.template
Commit the file:
git commit -m "feat: update NGINX ingress"
Push your changes to GitHub:
git push --set-upstream origin patch-1
Create a pull request (PR):
gh pr create --head patch-1 --fill --repo <your_GitHub_account>/platform
Monitor the progress of the workflow:
gh workflow view main.yml --repo <your_GitHub_account>/platform
When the workflow completes, send a health‑check request to the app, replacing <ACR_URL>
with the value you recorded in Step 3 of Challenge 1:
curl <ACR_URL>/health
Successful Update!
Now create an unsuccessful update and watch the workflow fail. This basically involves repeating the steps in Make a Successful Update but with a different value for the return
directive.
If using the GitHub GUI:
Change the return
directive as indicated:
location /health {
access_log off;
return 500 "Unsuccessful Update!\n";
}
When the workflow completes, navigate to your container app at the <ACR_URL>/health endpoint, where the <ACR_URL> is the URL you recorded in Step 3 of Challenge 1.
Notice the message is Successful
Update!
(the same as after the previous, successful update). Though that may seem paradoxical, it in fact confirms that this update failed – the update attempt resulted in status 500
(meaning Internal
Server
Error
) and did not get applied.
You can confirm the message by starting a terminal session and sending a health‑check request to the app, again replacing <ACR_URL>
with the value you recorded in Step 3 of Challenge 1:
curl <ACR_URL>/healthSuccessful Update!
If using the GitHub CLI:
Check out the patch-1 branch you created in the previous section:
git checkout patch-1
In your preferred text editor open ingress/default.conf.template and again change the return
directive as indicated:
location /health {
access_log off;
return 500 "Unsuccessful Update!\n";
}
Add default.conf.template to the Git staging area:
git add ingress/default.conf.template
Commit the file:
git commit -m "feat: update NGINX ingress again"
Push your changes to GitHub:
git push
Monitor the progress of the workflow:
gh workflow view main.yml --repo <your_GitHub_account>/platform
When the workflow completes, send a health‑check request to the app, replacing <ACR_URL>
with the value you recorded in Step 3 of Challenge 1:
curl <ACR_URL>/healthSuccessful Update!
It may seem paradoxical that the message is Successful
Update!
(the same as after the previous, successful update). Though that may seem paradoxical, in fact it confirms that this update failed – the update attempt resulted in status 500
(meaning Internal
Server
Error
) and did not get applied.
You probably want to remove the Azure resources you deployed in the tutorial to avoid any potential charges down the line:
az group delete -n my-container-app-rg -y
You can also delete the fork you created if you wish.
If using the GitHub GUI:
If using the GitHub CLI:
gh repo delete <your_GitHub_account>/platform -yes
Congratulations! You’ve learned how to use GitHub Actions to perform a canary blue‑green deployment of a microservices app. Check out these articles in the GitHub docs to continue exploring and growing your knowledge of DevOps:
If you’re ready to try out the second step of a canary deployment (testing in production) then check out the tutorial from Microservices March 2022, Improve Uptime and Resilience with a Canary Deployment on our blog. It uses NGINX Service Mesh to gradually transition to a new app version. Even if your deployments aren’t yet complex enough to need a service mesh, or you’re not using Kubernetes, the principles still apply to simpler deployments only using an Ingress controller or load balancer.
To continue your microservices education, check out Microservices March 2023. In Unit 3, Accelerate Microservices Deployments with Automation, you’ll learn more about automating deployments.
"This blog post may reference products that are no longer available and/or no longer supported. For the most current information about available F5 NGINX products and solutions, explore our NGINX product family. NGINX is now part of F5. All previous NGINX.com links will redirect to similar NGINX content on F5.com."