# AWS CLI Cheat Sheet — Observability: CloudWatch & Logs > **Tool:** AWS CLI v2 > **Category:** Cloud CLIs > **Verified against:** aws-cli/2.33.6, flags verified via `aws help` run locally, 2026-08-29 > **Official docs:** https://docs.aws.amazon.com/cli/ ## What it is and where it fits CloudWatch is AWS's native observability service, split into two halves the CLI's own command structure mirrors: `aws cloudwatch` for numeric time-series data (metrics, alarms, dashboards) and `aws logs` for text log data (log groups, streams, and the CloudWatch Logs Insights query language). Every AWS-managed service ships metrics here automatically at no extra setup cost, which makes CloudWatch the default first place to look when something's wrong in AWS-native infrastructure — even teams running Prometheus/Grafana or a third-party observability stack elsewhere typically still end up reading CloudWatch for anything AWS emits natively (RDS, ELB, Lambda concurrency, EBS IOPS) rather than re-instrumenting it. This page covers reading and publishing metrics, alarming on them, and querying logs — the everyday loop of "is something wrong, and if so, what does the evidence say." ## How metrics, alarms, and logs relate ```mermaid flowchart TD Svc["AWS service
(EC2, RDS, Lambda, ...)"] -->|"emits automatically"| Metrics["CloudWatch Metrics
(numeric time series)"] App["Your application"] -->|"put-metric-data /
put-log-events"| Metrics App -->|"logs"| Logs["CloudWatch Logs
(log groups / streams)"] Metrics --> Alarm["Metric alarm
(threshold + evaluation window)"] Alarm -->|"ALARM state"| Action["SNS topic, Auto Scaling,
Lambda, ..."] Logs -->|"filter-log-events /
Logs Insights query"| Investigate["Root-cause investigation"] Logs -->|"metric filter"| Metrics classDef info fill:#e5f0fa,stroke:#1d6fb8,color:#10161c classDef warn fill:#fbeee0,stroke:#b8650f,color:#10161c classDef crit fill:#fbe8e6,stroke:#b3261e,color:#10161c class Metrics,Logs info class Alarm warn class Action crit ``` A metric filter (not covered as a dedicated section below, but worth knowing exists) can turn a log pattern into a metric — e.g. counting `"ERROR"` occurrences per minute in application logs — which lets you alarm on log content using the same `put-metric-alarm` mechanism as any native AWS metric, bridging the two halves of CloudWatch shown above. ## Reading metrics ```bash aws cloudwatch get-metric-statistics \ --namespace AWS/EC2 --metric-name CPUUtilization \ --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \ --start-time 2026-08-19T00:00:00Z --end-time 2026-08-20T00:00:00Z \ --period 300 --statistics Average aws cloudwatch get-metric-data \ --metric-data-queries '[{"Id":"cpu","MetricStat":{"Metric":{"Namespace":"AWS/EC2","MetricName":"CPUUtilization","Dimensions":[{"Name":"InstanceId","Value":"i-0123456789abcdef0"}]},"Period":300,"Stat":"Average"},"ReturnData":true}]' \ --start-time 2026-08-19T00:00:00Z --end-time 2026-08-20T00:00:00Z ``` `get-metric-statistics` is the simple single-metric query; `get-metric-data` is the newer, batched form (query up to 500 metrics in one call, supports metric math — see below) with generally lower latency per metric returned. Reach for `get-metric-data` for anything beyond a quick one-off check; it's also the form the console's own dashboards use internally. ## Publishing custom metrics ```bash aws cloudwatch put-metric-data \ --namespace MyApp \ --metric-name QueueDepth \ --value 42 --unit Count \ --dimensions Environment=production ``` `--namespace` is the top-level grouping (never starts with `AWS/`, which is reserved for AWS's own service metrics) — pick one per application or logical system, since it's also the unit of cost/isolation in the CloudWatch console's namespace browser. A common mistake is publishing every metric under one generic namespace shared across unrelated apps, which makes the metric browser useless once a team has more than a handful of services. ## Alarms ```bash aws cloudwatch put-metric-alarm \ --alarm-name high-cpu \ --namespace AWS/EC2 --metric-name CPUUtilization \ --dimensions Name=InstanceId,Value=i-0123456789abcdef0 \ --statistic Average --period 300 --evaluation-periods 3 \ --threshold 80 --comparison-operator GreaterThanThreshold \ --alarm-actions arn:aws:sns:us-east-1:111122223333:my-alerts-topic aws cloudwatch describe-alarms --state-value ALARM # only alarms currently firing aws cloudwatch describe-alarms --alarm-name-prefix high- # filter by name prefix ``` `--evaluation-periods 3` with `--period 300` means the alarm needs 3 consecutive 5-minute periods above threshold before it actually fires (15 minutes of sustained breach) — a common first-time misconfiguration is setting `--evaluation-periods 1`, which makes an alarm trigger on a single noisy spike instead of a sustained condition, generating pager fatigue almost immediately. ## Testing an alarm without waiting for a real breach ```bash aws cloudwatch set-alarm-state --alarm-name high-cpu --state-value ALARM --state-reason "Testing notification pipeline" aws cloudwatch describe-alarm-history --alarm-name high-cpu --history-item-type StateUpdate ``` `set-alarm-state` temporarily forces an alarm's state for testing — it does trigger the configured actions (an SNS message really does go out), which makes it the right tool for verifying an on-call notification pipeline actually works end to end, not just that the alarm's threshold math is correct. The alarm returns to its real, metric-derived state on the next evaluation, typically within seconds to minutes depending on `--period`. ## Metric math ```bash aws cloudwatch get-metric-data \ --start-time 2026-08-20T00:00:00Z --end-time 2026-08-21T00:00:00Z \ --metric-data-queries '[ {"Id":"m1","MetricStat":{"Metric":{"Namespace":"AWS/EBS","MetricName":"VolumeReadOps","Dimensions":[{"Name":"VolumeId","Value":"vol-0123456789abcdef0"}]},"Period":300,"Stat":"Sum"},"ReturnData":false}, {"Id":"m2","MetricStat":{"Metric":{"Namespace":"AWS/EBS","MetricName":"VolumeWriteOps","Dimensions":[{"Name":"VolumeId","Value":"vol-0123456789abcdef0"}]},"Period":300,"Stat":"Sum"},"ReturnData":false}, {"Id":"total_iops","Expression":"(m1+m2)/300","Label":"Avg Total IOPS","ReturnData":true} ]' ``` Metric math lets you combine raw metrics with arithmetic/statistical functions server-side instead of pulling raw series and computing client-side. `ReturnData: false` on the input metrics (`m1`, `m2`) hides them from the response and returns only the computed expression — set it `true` on any series you also want returned alongside the math result, useful when you want both the raw inputs and the derived value plotted together. ## Composite alarms ```bash aws cloudwatch put-composite-alarm \ --alarm-name service-degraded \ --alarm-rule "ALARM(high-cpu) AND ALARM(high-error-rate)" \ --alarm-actions arn:aws:sns:us-east-1:111122223333:my-alerts-topic \ --actions-enabled ``` A composite alarm doesn't watch a metric directly — its `--alarm-rule` is a boolean expression (`AND`/`OR`/`NOT`, parenthesized) over the ALARM/OK/INSUFFICIENT_DATA state of *other* alarms. Use it to cut noise: page only when several individually-noisy alarms are in ALARM together, instead of firing one page per underlying alarm — the standard fix for an on-call rotation that's drowning in correlated, redundant pages for what is really one underlying incident. ## Dashboards ```bash aws cloudwatch put-dashboard --dashboard-name my-service --dashboard-body file://dashboard.json aws cloudwatch get-dashboard --dashboard-name my-service aws cloudwatch list-dashboards --dashboard-name-prefix my- ``` `--dashboard-body` is a JSON document of widget definitions (each widget references a metric or a Logs Insights query) — there's no imperative "add a widget" command; `put-dashboard` always replaces the entire dashboard body. Fetch the current body with `get-dashboard`, edit it, and `put-dashboard` it back rather than hand-authoring the whole thing from scratch each time, and treat the JSON as something worth keeping in version control alongside the rest of the service's infrastructure. ## Log groups and streams ```bash aws logs describe-log-groups --log-group-name-prefix /aws/lambda/ aws logs describe-log-streams --log-group-name /aws/lambda/my-function --order-by LastEventTime --descending aws logs put-retention-policy --log-group-name /aws/lambda/my-function --retention-in-days 30 ``` Log groups default to **never expiring** unless you set a retention policy — a common, quietly expensive default left over from first-time Lambda/ECS setups, since CloudWatch Logs storage cost scales with retained volume indefinitely by default. Worth auditing account-wide with `describe-log-groups` periodically, and worth setting `put-retention-policy` explicitly as part of provisioning any new log group from day one rather than as a later cleanup task. ## Reading log events ```bash aws logs get-log-events --log-group-name /aws/lambda/my-function --log-stream-name --start-from-head aws logs filter-log-events --log-group-name /aws/lambda/my-function --filter-pattern "ERROR" --start-time 1755648000000 aws logs tail /aws/lambda/my-function --follow --since 1h --filter-pattern "ERROR" ``` `aws logs tail` is the closest thing to `kubectl logs -f` for CloudWatch — it's a CLI-only convenience command (not a direct API wrapper), it accepts human-readable `--since` values like `1h`/`30m`, and `--follow` streams new events live instead of returning a fixed page. `filter-log-events` searches **across every stream in a log group at once**; `get-log-events` reads **one specific stream**. Use `filter-log-events` (or `logs tail`) when you don't already know which stream/task/container instance produced the log line you're looking for, which in practice is most incident-investigation situations. ## CloudWatch Logs Insights — querying across log groups ```bash aws logs start-query \ --log-group-names /aws/lambda/my-function \ --start-time $(date -d '1 hour ago' +%s) --end-time $(date +%s) \ --query-string 'fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 50' aws logs get-query-results --query-id abc12345-6789-def0-1234-56789abcdef0 ``` `start-query` and `get-query-results` are two separate calls because Insights queries run asynchronously — `start-query` returns a `queryId` immediately, and the query itself may still be running when `get-query-results` is first called (check the `status` field in the response; poll again if it's `Running`, not `Complete`). This two-step, poll-based shape is the same async pattern the credential report in the IAM page uses, and shows up across several AWS APIs whenever an operation takes longer than a typical synchronous request-response cycle. A single query can span up to 10 log groups at once — the tool of choice for "search across every Lambda function's logs for this request ID" rather than checking each function's log group individually. ## Real-world scenario: finding the one bad deploy in a fleet of instances An alarm fires for elevated error rate across a service running on 20 EC2 instances behind an ALB, and CloudWatch alone can't say *which* instance(s) are actually unhealthy without checking each one: ```bash # 1. Confirm the alarm and get its evaluation window aws cloudwatch describe-alarms --alarm-names high-error-rate --query 'MetricAlarms[0].[StateReason,StateUpdatedTimestamp]' # 2. Search application logs across every instance's log stream at once for the error signature aws logs filter-log-events --log-group-name /app/checkout-service \ --start-time $(date -d '20 minutes ago' +%s000) --filter-pattern "\"500 Internal Server Error\"" # 3. Cross-reference which instance IDs appear in the matching log streams aws logs filter-log-events --log-group-name /app/checkout-service --filter-pattern "500" \ --query 'events[].logStreamName' --output text | sort -u ``` `filter-log-events`' `logStreamName` field is often the fastest link between "an error happened" and "on which specific host" when the log stream naming convention includes the instance/task ID (a worthwhile convention to establish deliberately when configuring the log driver, precisely so this kind of query works later). ## Real-world scenario: reducing alarm noise for a genuinely bursty metric A queue-depth metric legitimately spikes for a few minutes during a nightly batch job, triggering a false alarm every night: ```bash aws cloudwatch put-metric-alarm \ --alarm-name queue-depth-sustained \ --namespace MyApp --metric-name QueueDepth \ --statistic Average --period 300 --evaluation-periods 6 \ --threshold 1000 --comparison-operator GreaterThanThreshold \ --datapoints-to-alarm 4 \ --treat-missing-data notBreaching ``` > [!TIP] > **`--datapoints-to-alarm` (M-out-of-N alarming) is the right fix for a bursty-but-not-actually-broken > metric, not simply widening `--period`.** Requiring 4 breaching datapoints out of the last 6 > evaluation periods (instead of all 6 consecutively) tolerates a metric that dips briefly during a > real incident (data collection gaps, a metric emission hiccup) while still catching a genuinely > sustained problem — a materially better signal-to-noise tradeoff than either "alarm on any single > spike" or "require every period to breach with zero tolerance." ## Real-world scenario: setting up log-based alerting without a separate agent An application logs structured JSON but has no CloudWatch agent configured to extract a metric from it: ```bash aws logs put-metric-filter --log-group-name /app/checkout-service \ --filter-name payment-failures --filter-pattern '{ $.level = "ERROR" && $.event = "payment_failed" }' \ --metric-transformations metricName=PaymentFailures,metricNamespace=MyApp,metricValue=1,defaultValue=0 aws cloudwatch put-metric-alarm --alarm-name payment-failure-spike \ --namespace MyApp --metric-name PaymentFailures --statistic Sum --period 60 \ --evaluation-periods 5 --threshold 10 --comparison-operator GreaterThanThreshold \ --alarm-actions arn:aws:sns:us-east-1:111122223333:my-alerts-topic ``` A metric filter turns a log pattern match into an emitted data point every time it matches — no application code change and no separate metrics-emission library needed, since the same log lines the application already writes drive the alarm. `defaultValue=0` matters: without it, the metric simply has no data point in periods with zero matching log lines (instead of an explicit `0`), which can make `--evaluation-periods`/`--treat-missing-data` behave unexpectedly around quiet periods. ## CI/CD recipe: gating a deploy on post-deploy error rate ```yaml # .github/workflows/post-deploy-check.yml name: Post-deploy health gate on: workflow_run: workflows: ["Deploy"] types: [completed] jobs: health-check: runs-on: ubuntu-latest permissions: id-token: write contents: read steps: - uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::111122223333:role/GitHubActionsReadOnlyRole aws-region: us-east-1 - name: Wait 5 minutes then check error rate run: | sleep 300 ERRORS=$(aws cloudwatch get-metric-statistics \ --namespace MyApp --metric-name Errors --statistic Sum \ --start-time "$(date -u -d '5 minutes ago' +%Y-%m-%dT%H:%M:%S)" \ --end-time "$(date -u +%Y-%m-%dT%H:%M:%S)" --period 300 \ --query 'Datapoints[0].Sum' --output text) if [ "$ERRORS" != "None" ] && [ "$(echo "$ERRORS > 50" | bc)" -eq 1 ]; then echo "Error rate exceeded threshold post-deploy: $ERRORS"; exit 1 fi ``` A post-deploy metric gate like this closes the loop a deploy pipeline otherwise leaves open — the `aws ecs wait services-stable` pattern from the compute page confirms tasks came up healthy at the infrastructure level, but only a real metric check confirms the *new code* isn't actively producing errors once it's serving real traffic. ## Common pitfalls - **`--evaluation-periods 1`** — triggers on a single noisy spike; see the Alarms section above for why most alarms should require sustained breach. - **Log groups with no retention policy set** — accumulates cost indefinitely by default; audit with `describe-log-groups` and set `put-retention-policy` on every group. - **Forgetting `get-query-results` needs to be polled** — `start-query` returns before the Insights query has actually finished; a `get-query-results` call made too early returns a `Running` status, not the final result, and a script that treats that as "no results" is wrong, not just slow. - **A metric filter with no `defaultValue`** — leaves genuinely-zero periods with no data point at all instead of an explicit 0, which can distort `--evaluation-periods`/`--treat-missing-data` behavior. - **Reaching for `describe-alarms --state-value ALARM` as the *only* signal something's wrong** — an alarm can be in `INSUFFICIENT_DATA` (no data flowing at all, often worse than a clean breach) without ever entering `ALARM`; check for that state too in any automated health check. ## Exit codes / when to reach for something else CloudWatch/Logs commands exit `0` on a successfully accepted API call — for `put-metric-alarm` and `put-metric-filter` that means the configuration was saved, not that the alarm has actually evaluated yet (there's a real delay before the first evaluation completes). For alarm/dashboard definitions meant to be reviewable and consistent across environments, prefer Terraform/CloudFormation over hand-run `put-metric-alarm`/`put-dashboard` calls; reach for the CLI directly for ad-hoc investigation (`filter-log-events`, `get-metric-data`), scripted post-deploy health gates, and testing an alerting pipeline end to end with `set-alarm-state`. For a team already running its own metrics/log stack (Prometheus + Loki, Datadog, and similar), CloudWatch is still worth querying for anything AWS-managed emits natively that the third-party stack doesn't already scrape — it's rarely an either/or choice in practice.