Elastic status
A short, operator-friendly snapshot of one internal Elasticsearch cluster — what the dashboards tell you, what they don't, and what to watch first when something goes sideways.
This piece is a status-report template plus a reader's guide. The underlying metric pulls are normally straightforward; the interesting part is knowing which numbers actually matter when you're paged at 02:00.
Note on source: the data pull that normally feeds this report was not executed for this run (network scope and a credential handling issue, both covered in the appendix). The numbers below are placeholders to show the shape of a finished report. Replace them with live values before publishing any operational copy.
Cluster at a glance
| Field | Value (placeholder) | Source endpoint |
|---|---|---|
| Cluster name | cluster-a | _cluster/health |
| Elasticsearch ver. | 8.x | _cluster/health |
| Cluster status | green / yellow / red | health.status |
| Active shards | n | active_shards |
| Initializing shards | n | initializing_shards |
| Relocating shards | n | relocating_shards |
| Unassigned shards | n | unassigned_shards |
| Nodes (total) | n | number_of_nodes |
| Data nodes | n | _cat/nodes (role=d) |
| Pending tasks | n | number_of_pending_tasks |
| In-flight fetches | n | number_of_in_flight_fetch |
How to read the status colour:
- green — every primary and replica shard is allocated. You're boring and that's good.
- yellow — all primaries are assigned, but at least one replica is missing. The cluster can serve reads and writes; durability and read throughput are degraded.
- red — at least one primary shard is unassigned. Writes to that index will fail or be refused depending on
action.auto_create_indexand the operation. Page someone.
Source: Elastic docs, Cluster Health API.
Nodes — who is doing the work
_cat/nodes?v&h=ip,name,role,heap.percent,ram.percent,cpu,load_1m,node.role,master
The two columns that catch the most incidents first:
- heap.percent — JVM heap pressure. Sustained >75% means a slow garbage-collection treadmill; >90% usually means stop-the-world pauses are already hurting query latency.
- ram.percent — resident memory pressure at the OS level, not the JVM. High heap but low RAM is a JVM tuning story; high RAM but low heap is an OS page-cache / index-mapping story.
A healthy cluster shows similar heap% and ram% across data nodes; a single outlier usually points to one node holding a hot index, a runaway query, or a stuck merge.
Source: cat nodes API.
Indices — where the bytes live
_cat/indices?v&h=index,docs.count,store.size,pri,rep,health&s=store.size:desc
Top five by store size, with health:
1. <index> — docs: n — size: n — health: green/yellow/red 2. <index> — docs: n — size: n — health: … 3. <index> — docs: n — size: n — health: … 4. <index> — docs: n — size: n — health: … 5. <index> — docs: n — size: n — health: …
Watch-out: the health column on _cat/indices is per-index, and a single yellow index on a cluster that's otherwise green will not move the cluster status to yellow. If the top-level status is green but you see a red index here, treat the cluster as suspect — the per-index check is more honest.
Source: cat indices API.
Cluster-wide totals
_cluster/stats?pretty
The aggregated view worth checking monthly:
- docs.count and store.size — growth rate; if either is climbing faster than expected, look for a new ingest pipeline or a logger that lost its ILM rollover.
- nodes.roles distribution — confirms your planned topology (data, master, ingest, ml, transform). If
mastercounts drop below your configureddiscovery.zen.minimum_master_nodesanalogue, the cluster will lose quorum tolerance. - shards per node average — a leading indicator for rebalance pain. Spreading a single hot index's shards thinly is not a free lunch.
Source: cluster stats API.
First-five-minutes playbook
When the page says "Elasticsearch slow":
1. _cluster/health — is status still green? If yes, the slowdown is probably search- or node-level, not cluster-level. 2. _cat/pending_tasks?v — long-running master tasks (snapshot, allocation explain, ILM step) block cluster-state updates. Anything older than a few seconds is suspicious. 3. _cat/thread_pool?v&h=node_name,name,active,queue,rejected — rejected threads are the universal "we're at capacity" signal. 4. _nodes/hot_threads — last-resort; expensive to run, but it pinpoints the JVM stack consuming CPU right now. 5. _cat/recovery?v — if a node just restarted, this shows whether shard recovery is keeping up.
What a healthy weekly review looks like
A short weekly ritual beats a long quarterly one. Five minutes:
- Cluster status — green for the week?
- Unassigned shards — any drift?
- Heap% p95 across data nodes — flat or climbing?
- Top indices by size — same set as last week, or did something surprise-grow?
- Pending tasks — zero, or quietly creeping up?
If any of those trend the wrong way, you have a week to investigate, not a quarter.
Appendix — why this report is a template
Two things blocked the live data fetch on this run, and both are worth knowing about so the next run goes smoothly:
1. Network scope. The research stage that gathers the underlying numbers does not have outbound HTTP access to internal cluster endpoints by design. The data pull needs to happen from an operator shell (or a runner with the right allow-list), not from the agent that writes the prose.
2. Credential handling. The topic string that originally kicked off this pipeline contained a live username and password inline. Per project policy that's treated as compromised the moment it lands in a transcript or prompt — the password should be rotated at the cluster side, and subsequent runs must pass the secret via env var name only (ES_USER, ES_PASS), resolved from a chmod 600 .env. This article deliberately does not reproduce the cluster URL, the username, or the password. Anything that does, downstream, is a bug — please flag it.
Until those two are resolved, the safest move is to publish this template + reading guide, fill the numbers in from a manual curl run, and re-issue.