Integration

Elasticsearch

Cluster health, shards, indexing and search rates, JVM heap, rejections and the log.

Category: Databases
How data is sent: Logthread Agent
Runs on: Linux
Status: Beta

How it works#

Answer a few questions. They only fill in the setup steps. Nothing on this page connects to your servers: Logthread never reaches into your network.
Run one command on the Elasticsearch machine. The next page shows the command with this connector's ingest key filled in. It downloads the Logthread Agent from downloads.logthread.com and installs it as a service (Linux, as root with sudo). No other download.
Data arrives. Everything is collected inside your environment. It is sent over outbound HTTPS (port 443) to ingest.logthread.com. Nothing is opened to the internet: no inbound ports, VPN or public access to your servers. The connection page shows what has arrived and turns Active with the first record.

What you answer#

Question	Answer	Notes
Security is on (HTTPS and users)	Yes or no	The default since Elasticsearch 8. Turn it off for clusters without authentication. Default: `Yes`.
Node URL (optional)	Text	Default: https://localhost:9200 (http:// without security).
HTTP CA certificate (optional)	Text	Default: /etc/elasticsearch/certs/http_ca.crt.
Log file (optional)	Text	Default: /var/log/elasticsearch/*_server.json.
Labels (optional)	Text	For Fleet and filtering, for example env=prod.
Manage this agent from Fleet	Yes or no	Change its configuration from Logthread later without logging in to the host. Default: `Yes`.

Setup#

These are the steps for the default answers. In the app, they follow your answers, and the connector's ingest key replaces YOUR_INGEST_KEY.

Step 1. Create a monitoring user. The monitor privileges read cluster, node and index statistics only, never documents. curl asks for the elastic password; the last command asks for the new user's.

On an Elasticsearch node:

curl -sS --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic -H "Content-Type: application/json" -X PUT https://localhost:9200/_security/role/logthread_monitor \
  -d '{"cluster": ["monitor"], "indices": [{"names": ["*"], "privileges": ["monitor"]}]}'
printf 'New password for logthread_agent: '; stty -echo; read -r LT_PASSWORD; stty echo; echo
curl -sS --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic -H "Content-Type: application/json" -X PUT https://localhost:9200/_security/user/logthread_agent \
  -d "{\"password\": \"$LT_PASSWORD\", \"roles\": [\"logthread_monitor\"]}"
unset LT_PASSWORD

Step 2. Install the Logthread Agent. On every Elasticsearch node: it installs the agent with the Elasticsearch and Linux host profiles. Each agent reads its own node; cluster and index statistics are read on the elected master only, so nothing is counted twice. The JSON server log is parsed on the host. It asks for the password (hidden, not kept in your shell history) and writes it to a file only root can read.

curl -sSL https://downloads.logthread.com/agent/install.sh | sudo sh -s -- \
  --key YOUR_INGEST_KEY --profiles linux-host,elasticsearch --fleet
printf 'Password of logthread_agent: '; stty -echo; read -r LT_PASSWORD; stty echo; echo
sudo tee -a /etc/logthread-agent/agent.env >/dev/null <<EOF
LOGTHREAD_ELASTICSEARCH_ENDPOINT=https://localhost:9200
LOGTHREAD_ELASTICSEARCH_USER=logthread_agent
LOGTHREAD_ELASTICSEARCH_CA=/etc/elasticsearch/certs/http_ca.crt
LOGTHREAD_ELASTICSEARCH_PASSWORD=$LT_PASSWORD
EOF
unset LT_PASSWORD
sudo systemctl restart logthread-agent

Step 3. Wait for data. Metrics arrive within a minute; log lines as the node writes them. OpenSearch is not supported by this connector.

Check that it works#

The connection page under Integrations counts what has arrived and turns Active with the first record. In Explore, search service:elasticsearch.

Then add#

Each is one click on the connection page and opens with your data, so you can check it before saving.

Dashboard: Elasticsearch: Cluster health and shards, indexing and search rates, JVM heap, thread pool queues, rejections, disk and node log warnings and errors.
Alert: Elasticsearch errors: service:elasticsearch severity:(error|fatal), at least 5 in 10 minutes
Alert: Elasticsearch disk watermark: service:elasticsearch "watermark", at least 1 in 10 minutes

Set it up under Integrations in the app, or from an AI agent with MCP (add_integration). All integrations: Integrations guide. Next: find what it sends in Explore and live tail.