Monitoring Docker within EC2 with Prometheus
We use plenty of docker containers while we use microservice architecture. As a result, we have to monitor performance metrics for these containers, such as memory, I / O, cpu, etc.
Enter cAdvisor. Out of the box, cAdvisor exposes Prometheus metrics. In this guide we will accomplish the following:
- Initialize an EC2 instance
- Build a Docker Compose multi-container installation that includes containers running Prometheus, cAdvisor, and a Redis server.
Create EC2 instance and SSH into it
First let's create a new EC2 instance and SSH into it (see step by step instructions here on how to create and connect to EC2 instance). Then let's perform the following commands. Next, run the following command to ensure we have the latest
# perform updates
sudo yum update -y
# install docker
sudo amazon-linux-extras install docker
sudo yum install docker
# start docker
sudo service docker start
sudo usermod -a -G docker ec2-user
# install docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Then log out and log back in to enable some of the changes.
First, to scrape metrics from cAdvisor, you’ll need to configure Prometheus. Create and populate a prometheus.yml file with this configuration:
- job_name: cadvisor
Upload the file via ssh
scp -i monitoring-challenge.pem prometheus.yml email@example.com:/home/ec2-user
We will now need to build a Docker Compose configuration that defines which containers are part of our installation, which ports each container is exposed to, which volumes are used, and so on.
Create a docker-compose.yml file and populate it with this Docker Compose configuration:
Docker Compose is instructed by this configuration to run three services, each corresponding to a Docker container:
- The prometheus service uses a local configuration file called prometheus.yml
- The cAdvisor service relies on a number of local volumes (/, /var / run, etc.) and exposes port 8080 (the default port for cAdvisor metrics).
- The Redis service is the standard server for Redis. Without any further setup, cAdvisor will automatically obtain container metrics from this container.
Upload the file via ssh
scp -i monitoring-challenge.pem docker-compose.yml firstname.lastname@example.org:/home/ec2-user
Then run the following command within SSH
Now we can explore a suite of metrics at the following URLs:
- http://localhost:8080 — web UI of cAdvisor is a useful interface to explore the kinds of items that are being tracked by cAdvisor, but it does not have an interface to explore container metrics.
- http:/localhost:9090/graph — provides an interface for exploring container metrics
In this summary, we covered how to use Docker Compose that contains separate containers in a single installation, and how to deploy it within an EC2 instance. For further reading, see Prometheus documentation.