CloudWatch logs, metrics and alarms for a Node.js API

When something breaks in production, you need three answers fast: is it broken, since when, and why. Logs on the server's disk answer none of them once the instance is gone, and an alert that fires in a dashboard nobody watches is the same as no alert.

CloudWatch can answer all three for a Node.js API without another vendor or another agent to run, if it's set up deliberately. Here's the setup I use: structured JSON logs from the app, shipped by Docker's awslogs driver, searched with Logs Insights, turned into metrics with filters, and alarms that reach a person. Every command is included, and it costs a few dollars a month for a small API.

Part of my guide to AWS infrastructure for small SaaS teams.

TL;DR#

• Log JSON to stdout with pino, one line per request with status and response time, secrets redacted.
• Ship it with Docker's awslogs log driver to a log group with a retention period (the default is never to delete anything).
• Search with Logs Insights: recent errors, slowest requests, p95 latency by route.
• Turn logs into metrics with metric filters (5xx count and latency) without changing any code.
• Add the CloudWatch agent for memory and disk, which EC2 doesn't report by default.
• Alarm on errors, p95 latency, an external health check and disk space, through SNS to email or Slack.

What you're building#

Node.js (pino → stdout, JSON)
  └─ Docker awslogs driver ─▶ CloudWatch Logs  /api/production (30-day retention)
                                 ├─ Logs Insights      search and ad-hoc stats
                                 └─ Metric filters ──▶ Api/Http5xx, Api/Latency
CloudWatch agent ─▶ memory and disk metrics           │
Route 53 health check ─▶ HealthCheckStatus            │
                                                      ▼
                                       CloudWatch alarms ─▶ SNS ─▶ email / Slack

Step 1: log JSON to stdout with pino#

Plain-text logs are for humans reading one line at a time. JSON logs can be searched, filtered and turned into metrics by field. pino is the fastest mainstream Node.js logger and writes JSON by default; pino-http adds one line per request with its status code and response time.

import express from 'express';
import pino from 'pino';
import pinoHttp from 'pino-http';

const logger = pino({
  level: process.env.LOG_LEVEL ?? 'info',
  base: { service: 'api', version: process.env.APP_VERSION },
  // Never ship credentials or session cookies to your logs.
  redact: ['req.headers.authorization', 'req.headers.cookie', 'res.headers["set-cookie"]'],
});

const app = express();
app.use(pinoHttp({ logger, autoLogging: { ignore: (req) => req.url === '/health' } }));

Each request produces a line like this:

{"level":30,"time":1760518800000,"service":"api","version":"3f9c2ab","req":{"id":1,"method":"GET","url":"/orders/42"},"res":{"statusCode":200},"responseTime":38,"msg":"request completed"}

Log errors with the error object under err, so the stack trace is kept as a field: req.log.error({ err }, 'payment failed'). Pino writes levels as numbers (30 is info, 50 is error), which makes "everything at error or above" a simple level >= 50 filter. With NestJS, nestjs-pino wires the same logger into Nest's own.

Write to stdout, not to files. Docker collects stdout, and the app doesn't need to know where logs end up.

Step 2: ship logs with the awslogs driver#

Docker's awslogs log driver sends a container's stdout and stderr straight to CloudWatch Logs. No agent needed for application logs.

Create the log group yourself, so you can set its retention. Log groups created automatically keep everything forever, and storage is billed every month:

aws logs create-log-group --log-group-name /api/production
aws logs put-retention-policy --log-group-name /api/production --retention-in-days 30

Give the instance role permission to write to that group, and nothing else:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["logs:CreateLogStream", "logs:PutLogEvents"],
      "Resource": "arn:aws:logs:us-east-1:123456789012:log-group:/api/production:*"
    }
  ]
}

Then start the container with the driver:

docker run -d --name api \
  --log-driver awslogs \
  --log-opt awslogs-region=us-east-1 \
  --log-opt awslogs-group=/api/production \
  --log-opt tag='{{.Name}}/{{.ID}}' \
  --log-opt mode=non-blocking \
  --log-opt max-buffer-size=25m \
  -p 127.0.0.1:3000:3000 \
  123456789012.dkr.ecr.us-east-1.amazonaws.com/api:$TAG

• tag names the log stream after the container, so you can tell instances and releases apart.
• mode=non-blocking buffers log lines in memory instead of blocking the app's writes when CloudWatch is slow to accept them. If the buffer fills, Docker drops lines rather than stalling your API, which is the right trade for a web server.
• docker logs api still works: Docker keeps a local copy for remote drivers.

The same in Docker Compose:

services:
  api:
    image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/api:${TAG}
    logging:
      driver: awslogs
      options:
        awslogs-region: us-east-1
        awslogs-group: /api/production
        tag: '{{.Name}}/{{.ID}}'
        mode: non-blocking
        max-buffer-size: 25m

To make it the default for every container on the host, put the same options in /etc/docker/daemon.json and restart Docker.

Step 3: search with Logs Insights#

CloudWatch Logs Insights queries JSON fields directly. Four queries cover most incidents.

The latest errors, with their messages:

fields @timestamp, msg, err.message, req.method, req.url
| filter level >= 50
| sort @timestamp desc
| limit 50

The slowest requests:

fields @timestamp, req.method, req.url, res.statusCode, responseTime
| filter ispresent(responseTime)
| sort responseTime desc
| limit 20

Request count and p95 latency per endpoint:

filter ispresent(responseTime)
| stats count(*) as requests, pct(responseTime, 95) as p95_ms by req.method, req.url
| sort p95_ms desc
| limit 20

Responses by status class, every five minutes, the shape of an incident:

filter ispresent(res.statusCode)
| fields floor(res.statusCode / 100) as class
| stats count(*) by bin(5m), class

URLs with IDs in them (/orders/42) group badly. If you can, log the route pattern (/orders/:id) as its own field and group by that.

Insights charges by data scanned, so pick the shortest time range that answers the question.

Step 4: turn logs into metrics#

Alarms need metrics. Metric filters create them from the logs you're already sending, with no code changes:

# Count of 5xx responses
aws logs put-metric-filter \
  --log-group-name /api/production \
  --filter-name api-5xx \
  --filter-pattern '{ $.res.statusCode >= 500 }' \
  --metric-transformations metricName=Http5xx,metricNamespace=Api,metricValue=1,defaultValue=0

# Every request's response time, in milliseconds
aws logs put-metric-filter \
  --log-group-name /api/production \
  --filter-name api-latency \
  --filter-pattern '{ $.responseTime >= 0 }' \
  --metric-transformations metricName=Latency,metricNamespace=Api,metricValue='$.responseTime',unit=Milliseconds

defaultValue=0 makes the error count report zero during quiet minutes, so the graph and the alarm see a real "no errors" instead of missing data. Don't set a default on latency: zeros would drag the averages down.

Because the latency filter publishes each request's value, CloudWatch can compute percentiles on it. Alarm on p95, not the average: an average hides the slow requests your users actually notice.

Metric filters can also add dimensions, such as one per route. Keep them few and low-cardinality: every unique combination is a separate custom metric, billed monthly. For many dimensions, the Embedded Metric Format is the better tool.

Step 5: memory and disk with the CloudWatch agent#

EC2 reports CPU, network and disk I/O, but not memory or disk space, because those live inside the operating system. The CloudWatch agent reports them. On Ubuntu (use amd64 instead of arm64 on x86 instances):

wget https://amazoncloudwatch-agent.s3.amazonaws.com/ubuntu/arm64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

A minimal config at /opt/aws/amazon-cloudwatch-agent/etc/config.json:

{
  "metrics": {
    "namespace": "CWAgent",
    "append_dimensions": { "InstanceId": "${aws:InstanceId}" },
    "metrics_collected": {
      "mem": { "measurement": ["mem_used_percent"] },
      "disk": { "measurement": ["used_percent"], "resources": ["/"] }
    }
  }
}

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/etc/config.json

Attach the AWS-managed CloudWatchAgentServerPolicy to the instance role. That's two custom metrics per instance, and the data you need before rightsizing anything: how we cut infrastructure costs by 35% started from exactly these numbers.

Step 6: alarms that reach a person#

Create an SNS topic and subscribe whoever is on call:

aws sns create-topic --name api-alerts
aws sns subscribe \
  --topic-arn arn:aws:sns:us-east-1:123456789012:api-alerts \
  --protocol email \
  --notification-endpoint oncall@example.com

For Slack or Microsoft Teams, connect the topic to Amazon Q Developer in chat applications (formerly AWS Chatbot).

Too many server errors:

aws cloudwatch put-metric-alarm \
  --alarm-name api-5xx-errors \
  --namespace Api --metric-name Http5xx \
  --statistic Sum --period 300 \
  --evaluation-periods 1 --threshold 10 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:api-alerts \
  --ok-actions arn:aws:sns:us-east-1:123456789012:api-alerts

Slow for most of the last 15 minutes:

aws cloudwatch put-metric-alarm \
  --alarm-name api-p95-latency \
  --namespace Api --metric-name Latency \
  --extended-statistic p95 --period 300 \
  --evaluation-periods 3 --datapoints-to-alarm 2 \
  --threshold 800 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:api-alerts

"2 out of 3" periods filters out a single slow spike while still catching a real slowdown within 15 minutes. --ok-actions sends a second message when it recovers, so nobody has to check.

Down, from the outside. Every alarm above depends on the app writing logs. If the instance is dead, there are no logs and no errors. A Route 53 health check calls https://api.example.com/health from several locations, and costs a few dollars a month. Alarm on its HealthCheckStatus metric, which Route 53 publishes in us-east-1 whatever Region your app runs in, and treat missing data as breaching.

With a load balancer, also alarm on its own metrics, which are free: HTTPCode_Target_5XX_Count, TargetResponseTime (p95) and UnHealthyHostCount.

Which alarms to start with#

Tune the thresholds to your own traffic after a week. An alarm that fires every day gets ignored, and then it's worse than none.

What it costs#

The free tier covers 5 GB of log data, 10 custom metrics and 10 alarms a month. A small API logging one line per request at info usually stays within a few dollars. What makes CloudWatch bills grow is debug logging in production, logging request bodies, and log groups without retention.

The checklist#

• The app logs JSON to stdout, one line per request, with secrets redacted
• Log groups are created with a retention period
• Containers use the awslogs driver in non-blocking mode
• The instance role can only write to its own log group
• Saved Logs Insights queries for errors, slow requests and p95 by route
• Metric filters for 5xx count and latency
• The CloudWatch agent reports memory and disk
• Alarms on errors, p95 latency, disk, memory and the database
• An external health check with its own alarm
• Alarms go to an SNS topic someone actually reads, with OK messages too

FAQ#

How do I send Node.js logs to CloudWatch?#

Log JSON to stdout with a logger like pino and run the app in Docker with the awslogs log driver, which sends each container's output to a CloudWatch Logs log group. Create the log group first with a retention period, and give the instance role logs:CreateLogStream and logs:PutLogEvents on it. Without Docker, the CloudWatch agent can ship log files instead.

What is the best logger for Node.js on AWS?#

Pino is a strong default: it's fast, writes structured JSON by default, supports redacting sensitive fields, and pino-http logs each request with its status code and response time. JSON logs let CloudWatch Logs Insights query fields directly and let metric filters turn fields into metrics.

How do I get EC2 memory metrics in CloudWatch?#

Install the CloudWatch agent on the instance, configure it to collect mem_used_percent (and disk_used_percent for disk space), and attach the CloudWatchAgentServerPolicy managed policy to the instance role. EC2 doesn't report memory on its own, because memory usage is only visible inside the operating system.

How do I alert on errors in CloudWatch Logs?#

Create a metric filter on the log group that matches error lines, for example { $.res.statusCode >= 500 } for JSON logs, publishing a value of 1 with a default of 0. Then create a CloudWatch alarm on that metric's sum over five minutes, sending to an SNS topic subscribed by email or connected to Slack.

How much does CloudWatch logging cost?#

In us-east-1, ingesting logs costs $0.50 per GB, storing them $0.03 per GB a month, and Logs Insights $0.005 per GB scanned. Custom metrics are $0.30 a month each and standard alarms $0.10 a month each. The free tier includes 5 GB of logs, 10 custom metrics and 10 alarms a month, which covers many small APIs.

Related reading#

• AWS infrastructure for small SaaS teams: what you actually need
• AWS ALB vs Nginx load balancing: when you need which
• Ubuntu server hardening checklist for a fresh EC2
• Deploying Node.js to production: the complete guide