Step-by-Step: Install Unified CloudWatch Agent on EC2 and Create Log Groups and Dashboards

One of our DevOps engineers is using the monitoring system. He wants to keep an eye on the CPU, memory, storage, and network usage of an Nginx web server hosted on AWS EC2. He asked me to provide a sample monitoring dashboard. So, I set up the Unified CloudWatch agent on the EC2 instance hosting Nginx and shared the dashboard with him. Let me share this step-by-step.

In this article, you will learn how to install and set up the Unified CloudWatch Agent on EC2 and how to share a CloudWatch dashboard via email. Let's get started.

Path 1 ( Define Security Group and IAM role )

I installed the Unified CloudWatch Agent on an Amazon Linux EC2 instance. I allowed inbound rules for HTTP (port 80) and HTTPS (port 443). Then, I set up an IAM role for this EC2 instance and attached three permissions to this role.

1. AmazonSSMManagedInstanceCore

2. CloudWatchAgentAdminPolicy

3. CloudWatchAgentServerPolicy

After creating the EC2 role, attach this role to the EC2 instance.

Part 2 ( Install and Configure CloudWatch agent on EC2 )

sudo yum update && sudo yum upgrade -y

sudo yum install amazon-cloudwatch-agent -y

sudo systemctl start amazon-cloudwatch-agent.service

sudo systemctl enable amazon-cloudwatch-agent.service

sudo systemctl status amazon-cloudwatch-agent.service

sudo systemctl status amazon-ssm-agent.service

After installing and starting the CloudWatch Agent service, let's configure the CloudWatch Agent.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

In the AWS credential section, I didn't use the SDK key. Instead, I used the AWS Access Key and Secret Key from IAM.

Path 3 ( Install Nginx and configure )

sudo yum install nginx -y

sudo systemctl start nginx

sudo systemctl enable --now nginx

I don't use the default HTML page, so I customized the HTML.

sudo su -

echo "Hola this is DevOps Learning Program" > /usr/share/nginx/html/index.html

cat /usr/share/nginx/html/index.html

curl http://localhost

curl ifocnfig.me ( This command is show your EC2's public ip ) 

sudo su -

tail -f /var/log/nginx/access_log ( This command will show you real time nginx access log )

Open your browser and access the server's public IP address. You should now see our customized HTML page.

Path 4 ( Create Dashboard and Log Group in CloudWatch )

I don’t store my CloudWatch Agent config file on this EC2 server. Instead, I store it in AWS Systems Manager using Parameter Store. That's why I assigned the AmazonSSMManagedInstanceCore permission to the EC2 instance role. After this, you can see how easy it is to redefine parameter values with AWS Parameter Store.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:AmazonCloudWatch-demo-nginx-parameter -s

However, in AWS CloudWatch Log groups, I noticed that the Nginx web server's access and error logs are missing. I accidentally set the wrong configuration for the log file path. Instead of using /var/log/nginx/access.log and /var/log/nginx/error.log, I defined access-log and error-log.

So I redefined the correct value in the AWS SSM Parameter Store. You don't need to redefine it in the local config file. That’s why I love to use AWS SSM Parameter Store.

On Ubuntu, the correct path is /var/log/nginx/access-log. However, on Amazon Linux, the correct path is /var/log/nginx/access.log.

Now you can see Log groups are appear.

After the log groups appear, let's configure the CloudWatch dashboard for CPU, memory, storage, and network usage.

Now it's time to share the dashboard with our DevOps Engineer.

Please check your inbox. AWS has sent an email with a temporary password for dashboard access. You should ask your DevOps Engineer for the dashboard link, as they are the one who set it up. He will provide the link to you.

Now you can see the dashboard.

Horaay. We did it. Don't forget to delete your AWS cloud resources after you've finished testing. I don't want to incur cloud costs for unused resources. If you find my article helpful, please like and share it. I would really appreciate your support. Stay tuned and keep in touch. See you in the next articles.