Solution Overview
This project tryting to setup CI-CD with AWS Cloud Formation, AWS System Manager, Docker Hub, Docker and GitHub Actions. AWS Cloud Formation will help to provision AWS EC2 infra which contain two EC2 instances, EC2 auto scaling group, 3 IAM policies, 2 security groups and one VPC. Github CI-CD will build and generate docker file for example nodejs application and deploy image on EC2 instances via AWS System Manager.
This lab only cost 2$ for me.
The solution utilizes the following services:
- GitHub Actions – Workflow Orchestration tool that will host the Pipeline.
- AWS Auto Scaling – AWS Service to help maintain application availability and elasticity by automatically adding or removing Amazon EC2 instances.
- Amazon EC2 – Destination Compute server for the application deployment.
- AWS CloudFormation – AWS infrastructure as code (IaC) service used to spin up the initial infrastructure on AWS side.
- IAM OIDC identity provider – Federated authentication service to establish trust between GitHub and AWS to allow GitHub Actions to deploy on AWS without maintaining AWS Secrets and credentials.
- Amazon System Manager – to run Ad-Hoc command inside EC2 instances.
Architecuture Overview
- Developer commits code changes from their local repo to the GitHub repository.The GitHub action is triggered automatically.
- GitHub action triggers the build stage.
- GitHub uses the AWS Access Key to authenticate to AWS and access resources.
- GitHub action uploads build the simple nodejs application docker image.
- GitHub action upload continaer image to docker hub.
- GitHub action execute linux docker run command to EC2 instances via AWS System Manager.
- Prerequisties
- An AWS account with permissions to create the necessary resources.
- An GitHub account with permission to Configure GitHub repositories, Create workflows, and configure GitHub secrets
- A Git client to clone the provide source code.
Install on Ubuntu
1
2
3
4
5
6
type -p curl >/dev/null || sudo apt install curl -y
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null \
&& sudo apt update \
&& sudo apt install gh git wget -y
Install on Centos
1
2
3
sudo dnf install 'dnf-command(config-manager)'
sudo dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo
sudo dnf install gh git wget -y
- Task List
- Clone Github Repo
- Prepare DockerHub
- Provision AWS infra with Cloud Formation
- Create user and grant SSM access
- Create github secret for AWS IAM key and secret
- Bulild and Deploy
- Github Action
- Dockerfile
- Access the application
- Cleaning
Clone Github Repo
- Clone the githubrepository lillianphyo/nodejs-demo
1
git clone https://github.com/lillianphyo/nodejs-demo.git
- Create empty repository in your github personal account.
1 2 3 4 5 6 7 8 9 10 11
mkdir ghcicd cp -r nodejs-demo/. ghcicd cd ghcicd git remote remove origin rm -rf .git gh repo create ghcicd --public --source=. -y git remote add origin https://github.com/<github-username>/ghcicd.git git branch -M main git add . git commit -m "fist commit" git push -u origin main
GitHub Action
Prepare Docker Hub
- if you have docker hub account? SignIN Here: SignUP Here;
- create repo via Create repository.
Then you will get like this.
create docker secret to push image via github action.
Save this credential to use later.
- Update Docker Hub credentilas to repo.
Provision AWS infra with Cloud Formation
To provision EC2 instance with atuo scaling grop and configure ALB from AWS Cloud Formation Template. Application steps are as follow.
- Open AWS CloudFormation console, Enter your account ID, username and passowrd.
- Check your region is ap-southease-1 (singapore).
- Create New Stack
- Select Template is Ready
- Select Upload a template file
- Choose File under “ghcicd/cloudformation/deployment.yml”
- Select deployment.yml file and select next.
- In stack detail,
- stack name: nodejs-demo
- VPC and Subnets: (these are pre-populated for you) you can change these values if you prefer to use your own Subnets)
- GitHubRepoName: Name of your GitHub personal repository which you created.
Create user and grant SSM access
In this session, IAM user create and grant for AWS System Manager permission for github action.
Create github secret for AWS IAM key and secret
Bulild and Deploy
Check EC2 instance id from AWS console and update the instance ids at githubcicd/.github/workflows/deploy.yaml. If you build this in different region,you can change your region at AWS_REGION envioronment variable. My region is ap-southeast-1.
1
2
3
4
5
6
env:
REPO: nodejs-demo
AWS_REGION: ap-southeast-1
instance_ids: |
i-0b74b27c04b52197b
i-011d84c1e4eff7d8d
Later, enter the repo directory from command line and finish the tasks.
1
2
3
4
cd ghcicd
git add .
git commit "first build"
git push -u origin main
Process will flow like this. 
OR you can run manually.
Detail Stuffs
May skip and check later if you want to go through details.
Github Action
There is three parts in github action. First build the application, second create docker image and push it to docker hub repository, Finally, deploy docker image on EC2 instances. We can Declare aws secrets variables in environment variables as option.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# This is a basic workflow to help you get started with Actions
name: CI-CD
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
env:
REPO: nodejs-demo
AWS_REGION: ap-southeast-1
instance_ids: |
i-0b74b27c04b52197b
i-011d84c1e4eff7d8d
As first step, github action workflow will do task named build-test.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains multiple jobs
build_test:
# The type of runner that the job will run on
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x,16.x]
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: setup node
uses: actions/setup-node@master
with:
node-version: $
# install applicaion dependencies
- name: Install dependencies
run: |
npm install
npm ci
As second, github action workflow will do task named push_to_Docker_Hub. This task will build Dockerfile and create container image. Later, it will access to dockerhub and push/submit the image to dockerhub.
‘push_to_Docker_Hub’ task depend on ‘build_test’. means, if build_test failed, push_to_Docker_Hub task may not work.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
push_to_Docker_Hub:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# build docker image and push to docker hub
# only if the app build and test successfully
needs: [build_test]
steps:
- name: checkout repo
uses: actions/checkout@v2
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: $
password: $
- name: Build and push
uses: docker/build-push-action@v2
with:
context: ./
file: ./Dockerfile
push: true
tags: $/nodejs-demo:latest
- name: Run the image in a container
uses: addnab/docker-run-action@v3
with:
image: $/nodejs-demo:latest
run: |
echo "runing the docker image"
echo "Testing the nodejs app endpoints"
echo $
As third, github action workflow will do task named deploy. This task will access EC2 instances via AWS System Manager and run the docker command to deploy created container image from docker hub.
‘deploy’ task depend on ‘push_to_Docker_Hub’. means, if push_to_Docker_Hub failed, deploy task may not work.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
deploy:
needs: [push_to_Docker_Hub]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: AWS SSM Send-Command
uses: peterkimzz/aws-ssm-send-command@master
id: ssm
with:
aws-region: $
aws-access-key-id: $
aws-secret-access-key: $
instance-ids: $
working-directory: /tmp
command: docker pull 270596/nodejs-demo && docker run -d -p 8080:8080 --name nodejs-demo 270596/nodejs-demo
comment: run application
# Catch SSM outputs
- name: Get the outputs
run: echo "The Command id is $"
Dockerfile
Github action task2 ‘push_to_Docker_Hub’ will use this file to generate container image. In this dockerfile, specify application working direcotry within container, copy application file, running the application and expose application to specific port. This is all heart exit. If this not work, other setup are non-sense.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
FROM node:16
# Create app directory
WORKDIR /usr/src/app
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./
RUN npm install
# If you are building your code for production
# RUN npm ci --only=production
# Bundle app source
COPY . .
EXPOSE 8080
CMD [ "node", "index.js" ]
Access the application
- Your AWS EC2 Load Balancer address
- nodej-appli-10bj2utmxi5z3-294453507.ap-southeast-1.elb.amazonaws.com
Proof Check
Cleaning
Clean the lab by deleting nodejs-demo stack in Cloud Formation and delete IAM user gh-action.
If this step missed, aws will charge you for this lab.
REF:
https://aws.amazon.com/blogs/devops/integrating-with-github-actions-ci-cd-pipeline-to-deploy-a-web-app-to-amazon-ec2/
https://github.com/marketplace/actions/aws-ssm-send-command
Thank You!