Send data to Logthread

Every way to send logs, traces and metrics to Logthread, with copy-paste settings, the limits, the response codes and what to check when nothing arrives. Each method uses an ingest key (send-only; see Ingest keys and API keys). Your workspace's endpoints are also listed under Settings in Endpoints, and Connect a source in the app builds each configuration for you, with your key filled in.

Running SQL Server, IIS, PostgreSQL, MySQL, Redis, MongoDB, Elasticsearch, Kafka, RabbitMQ, NGINX, Apache, Docker, Kubernetes or Pipecat, or on AWS, Cloudflare, Vercel, Heroku, Google Cloud or Azure, or want your Okta or GitHub audit log? The Integrations guide sets each up with one command or one pasted address, and comes with dashboards and alerts.

Your data is stored in India, in the Mumbai region. Logthread runs in this one region today; EU and US regions are on the roadmap. See Data residency for what is stored where.

How to send the key#

HTTP and gRPC endpoints read the ingest key from Authorization: Bearer <ingest key> (for gRPC, the authorization metadata). If a proxy, gateway or agent in front of your service rewrites or strips the Authorization header, send the key as X-Logthread-Key: <ingest key> instead (the x-logthread-key metadata for gRPC). When both are present and Authorization carries a bearer token, the bearer token is used. Syslog and Fluent Forward take the key as described in their sections.

curl https://ingest.logthread.com/v1/ndjson \
  -H "X-Logthread-Key: $LOGTHREAD_INGEST_KEY" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary $'{"service":"checkout","message":"hello"}\n'

OpenTelemetry (OTLP)#

Any OpenTelemetry SDK or Collector can send logs, traces and metrics. Set these two variables:

OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.logthread.com
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer%20<ingest key>"

• OTLP/HTTP accepts protobuf (application/x-protobuf) or JSON (application/json), optionally gzip-compressed.
• OTLP/gRPC uses ingest.logthread.com:4317 with TLS, and the key goes in the authorization: Bearer <key> metadata.
• OpenTelemetry, any language has the setup for Node.js, Python, Java, Go, .NET, Ruby, PHP and the Collector.

Prometheus remote write#

Logthread accepts remote write 1.0 (every Prometheus 2.x, Grafana Agent, vmagent) and remote write 2.0 (Prometheus 3.0 or later). Change only the URL and the credential. In prometheus.yml:

remote_write:
  - url: https://ingest.logthread.com/api/v1/write
    authorization:
      credentials: <ingest key>

For remote write 2.0, which is more compact and carries metadata and native histograms, add the message type (Prometheus 3.0 or later):

remote_write:
  - url: https://ingest.logthread.com/api/v1/write
    protobuf_message: io.prometheus.write.v2.Request
    authorization:
      credentials: <ingest key>
global:
  scrape_native_histograms: true   # optional

Grafana Agent (static mode):

metrics:
  configs:
    - name: default
      remote_write:
        - url: https://ingest.logthread.com/api/v1/write
          bearer_token: <ingest key>

Grafana Alloy:

prometheus.remote_write "logthread" {
  endpoint {
    url          = "https://ingest.logthread.com/api/v1/write"
    bearer_token = sys.env("LOGTHREAD_INGEST_KEY")
  }
}

Both versions need snappy compression, which every sender uses by default. A successful write answers 204; a 429 or 503 is retried by the sender. See Metrics and traces.

Grafana Loki push#

If you send logs to Grafana Loki today, point the same sender at Logthread: the Loki push API is accepted at https://ingest.logthread.com/loki/api/v1/push, in both its snappy-compressed protobuf and JSON forms. The ingest key goes in as a bearer token, or as the password of HTTP basic auth (any user name). A X-Scope-OrgID header is ignored, because your workspace comes from the key.

How a Loki push becomes Logthread logs:

• Each stream becomes one resource. The service name is the first non-empty stream label out of service_name, service, app, job, container; without one it is unknown_service. The other labels stay as resource attributes you can filter on.
• The line is the log message and the entry timestamp is the log time.
• Structured metadata becomes log attributes. A trace_id or span_id in it links the log to its trace.
• A level, severity or detected_level label or metadata value sets the log level.

Promtail:

clients:
  - url: https://ingest.logthread.com/loki/api/v1/push
    bearer_token: <ingest key>

Grafana Alloy:

loki.write "logthread" {
  endpoint {
    url          = "https://ingest.logthread.com/loki/api/v1/push"
    bearer_token = sys.env("LOGTHREAD_INGEST_KEY")
  }
}

Point your existing loki.source.* components at forward_to = [loki.write.logthread.receiver].

Fluent Bit:

[OUTPUT]
    Name         loki
    Match        *
    Host         ingest.logthread.com
    Port         443
    Tls          On
    Tls.verify   On
    Uri          /loki/api/v1/push
    HTTP_User    logthread
    HTTP_Passwd  ${LOGTHREAD_INGEST_KEY}
    Labels       service=checkout, job=fluent-bit

Use Bearer_Token ${LOGTHREAD_INGEST_KEY} instead of HTTP_User and HTTP_Passwd if you prefer a bearer token. Label_Keys and Auto_Kubernetes_Labels work as with Loki.

Test it with curl:

curl -i https://ingest.logthread.com/loki/api/v1/push \
  -H "Authorization: Bearer $LOGTHREAD_INGEST_KEY" \
  -H "Content-Type: application/json" \
  --data "{\"streams\":[{\"stream\":{\"app\":\"checkout\",\"level\":\"error\"},\"values\":[[\"$(date +%s)000000000\",\"payment failed\"]]}]}"

A successful push answers 204. Only the push API is accepted: queries stay in Logthread (Explore, or the API), not Loki's query endpoints.

JSON lines#

POST one JSON object per line. Gzip is optional.

curl https://ingest.logthread.com/v1/ndjson \
  -H "Authorization: Bearer $LOGTHREAD_INGEST_KEY" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary $'{"service":"checkout","level":"error","message":"payment declined","order_id":"o-42"}\n'

• Logthread recognises the usual names: level/severity, timestamp/time/ts/@timestamp, service/service.name/app, message/msg/body, trace_id, span_id.
• Every other key becomes an attribute. Nested objects become dotted keys, up to 8 levels deep.
• A JSON array of objects sent as application/json works too.
• The reply is 200 {"accepted": N}.

Syslog over TLS#

Send RFC 5424 or RFC 3164 messages over TCP with TLS to ingest.logthread.com:6514, using octet-counting or newline framing. Put the ingest key in any one of these places:

• the structured-data element [logthread@53595 key="<ingest key>"] (recommended);
• a token before the <PRI>;
• the first word of the message.

The key is never stored with your logs.

For rsyslog without TLS syslog output, use its HTTP output to JSON lines instead: Connect a source → rsyslog builds /etc/rsyslog.d/60-logthread.conf for you.

Fluent Forward#

Fluentd, Fluent Bit and the Docker fluentd log driver can send Fluent Forward over TLS to ingest.logthread.com:24224.

• Authentication uses the forward handshake: the username and the shared key are both your ingest key.
• Logthread acknowledges a chunk only after it is safely stored, so Require_ack_response gives you at-least-once delivery.

[OUTPUT]
    Name                  forward
    Match                 app.*
    Host                  ingest.logthread.com
    Port                  24224
    Tls                   On
    Tls.verify            On
    Shared_Key            ${LOGTHREAD_INGEST_KEY}
    Username              ${LOGTHREAD_INGEST_KEY}
    Password              logthread
    Require_ack_response  True
    Compress              gzip

Fluent Bit#

Fluent Bit's HTTP output sends JSON lines:

[INPUT]
    Name              tail
    Path              /var/log/app/*.log
    Tag               app.*

[FILTER]
    Name    modify
    Match   app.*
    Add     service checkout

[OUTPUT]
    Name              http
    Match             app.*
    Host              ingest.logthread.com
    Port              443
    URI               /v1/ndjson
    Format            json_lines
    Json_date_key     timestamp
    Json_date_format  iso8601
    Header            Authorization Bearer ${LOGTHREAD_INGEST_KEY}
    Compress          gzip
    Tls               On
    Retry_Limit       False

Connect a source → Fluent Bit also offers the Forward output above, and builds configs for Vector.

Ready-made connectors (beta)#

For well-known software, Integrations → Software does the setup for you. Today it covers:

• Microsoft SQL Server (Windows and Linux) and IIS;
• Linux hosts;
• NGINX and Apache httpd;
• PostgreSQL, MySQL and MariaDB, Redis, MongoDB and Elasticsearch;
• Apache Kafka and RabbitMQ;
• Docker and Podman: container logs, plus CPU, memory and network per container;
• Kubernetes: two Helm releases of the OpenTelemetry Collector chart.
• One collector runs on each node and collects pod logs, kubelet CPU and memory, and OTLP from your workloads.
• A single cluster-wide collector reports restarts and deployment health.
• Pods take their container name as the service name, unless the application sets its own.
• Pipecat: each voice-agent call as a trace, with response latency, interruptions, and STT, LLM and TTS timing and tokens.

Pipecat puts the transcript and LLM text on its spans, so leave tracing off on pipelines whose conversations must stay on your servers.

• Cloud services, with nothing to install: each sends its logs to a Logthread address with the connector's key.
• AWS: CloudWatch Logs (Lambda, ECS, any log group) through an Amazon Data Firehose stream, or any Firehose stream.
• Cloudflare: HTTP requests or firewall events through Logpush.
• Vercel: function, edge, build, static and firewall logs through a log drain.
• Heroku: app, router and platform logs through an HTTPS log drain.
• Google Cloud: Cloud Logging (Cloud Run, GKE, Cloud Functions, App Engine, Compute Engine, audit logs) through a Pub/Sub push subscription.
• Microsoft Azure: Azure Monitor (activity log, resource diagnostics, App Service and Functions, Entra ID sign-in and audit logs) streamed to an Event Hub that the Logthread Agent reads.
• Audit logs Logthread reads for you, with a read-only token: the Okta System Log and the GitHub organization audit log, polled every 5 minutes.
• Pick the software and answer a few questions: the operating system, how to sign in and an optional label.
• Logthread creates an ingest key for that connector and shows the exact steps with the key filled in: for example the read-only database grants and one install command for the Logthread Agent, the Helm values for Kubernetes, or the few lines of Python for Pipecat.
• The connector's page shows What has arrived and turns active with the first record.
• It then offers the matching dashboard, parser (NGINX, Apache, PostgreSQL, MySQL and Redis logs need theirs for status and severity; MongoDB, Elasticsearch, Kafka and RabbitMQ logs are parsed on the host) and alerts. Each alert opens previewed on your data before you save it.

Removing the connector revokes its key.

Docker hosts#

To collect every container's logs on a host, use the Docker fluentd log driver into a local OpenTelemetry Collector, which buffers to disk and sends to Logthread. Follow Docker hosts.

No-code auto-instrumentation#

• OpenTelemetry's zero-code agents (Java, .NET, Node.js, Python) send traces and metrics with only the two OTEL_ variables above.
• On Linux, eBPF auto-instrumentation traces services without touching them. See No-code tracing.

From a browser#

Browsers cannot send to Logthread directly: the ingest endpoints do not accept cross-origin requests, and an ingest key in a web page would be public. Send browser events to your own backend and forward them from there with an SDK or the JSON lines endpoint.

Limits and response codes#

Limits per request:

• 5 MiB after decompression;
• 10,000 records (log records or spans);
• 50,000 metric data points;
• 10 MiB for a Prometheus remote write.

Syslog and Fluent limits:

• A syslog message can be up to 64 KiB. Longer messages are split into parts, which stay linked.
• A Fluent chunk can be up to 10 MiB.

Each workspace has an ingest rate and a daily volume set by its plan.

Error bodies are JSON: {"code": ..., "message": "..."}. Retry 429 and 503. Do not retry 400, 401, 403, 413 or 415 until you have fixed the cause.

Field limits#

• A record has at most 128 attributes. That includes the 4 Logthread adds, so a JSON line can carry up to 124 of its own.
• The source name is cut to 128 characters.
• Attribute names starting with logthread. are reserved and dropped from what you send.
• Logthread adds logthread.event_id, logthread.received_at, logthread.key_id and logthread.source to every log record.

To mask secrets or drop fields before storage, use pipelines and redaction.

Nothing arrived?#

1. Check the reply. Send one test line with the curl above. A 200 {"accepted":1} means the key and network are fine; any other code is in the table above.
2. Check the key. It must be an ingest key (not an API key), and Keys must list it as active, not revoked.
3. Check the workspace. It must be ready: the email is verified and the setup checklist is complete.
4. Check the time range. Open Explore with Last 15 minutes. Records with old timestamps are stored at those times.
5. Check the service name. Search for service:<name>; Connect a source waits for the first record and says when it arrives.
6. Collectors. Check the collector's own log for TLS errors. Use the port for the protocol: 443 for HTTP, 4317 for gRPC, 6514 for syslog and 24224 for Forward.
7. Over quota. A 429 with daily_quota_exceeded means today's volume is used. The Usage page shows it.

Next: find what you sent in Explore and live tail.