Threads

A thread is every log line that shares one id, across all your services, in time order. The id can be an order id, a user id, a job id, a request id or a trace id. Threads follow an order id, or any other id, across services, and answer "what happened to this order?" in one click. They then show the other ids on the same lines, such as the payment id, the retry id or the user, so you can follow the story wherever it goes.

Threads need no tracing and no setup. They work on:

• attribute values, such as order_id in JSON logs;
• fields your pipelines and parsers extract;
• trace and span ids;
• whole words in the message.

Follow an order id across services#

Open Threads in the sidebar and paste any id. You can also open a thread from these places:

• Type req:<id> in the Explore query box, or paste the id into Find request on Home.
• In any log's details (Explore, Live tail, a thread), click the thread icon next to a field value.
• Click a sample value under Linking fields on the Threads page.

The thread view shows:

• A summary: how many lines, which services in the order they first logged, how long it took, the number of errors, and a link to the first error.
• Across services: one lane per service, with a mark for each step at its time. A red mark is an error. Click a mark to open that step.
• Steps: lines from the same service with the same message pattern, one after another, collapsed into one step with a count, such as "×12". Click a step to see its lines, then a line to see all its fields. The step with the first error opens on its own.
• Connected values: the other ids on these lines, with how many lines and services carry them. reaches more means the id is on lines this thread does not have yet.
• Add pulls those lines into this thread. For example, add the payment id to an order's thread to see the card being declined, retried and timing out.
• The arrow opens the id's own thread.
• A thread follows up to 5 ids. Each added id shows in the title, and you can remove it there.
• Shared context: values that many other lines share too, such as a region, host or version. They are too common to follow, so clicking one filters Explore on it instead.
• Traces: any traces found among the lines, with one click to the trace view.

The range is the last 24 hours by default; the range menu offers 1 hour and 7 days. A thread shows its oldest 500 lines and 20 traces, and says when there are more. The address /threads/<id>?with=<id> is shareable. Old /explore/request/<id> links still work.

How an id matches#

A line belongs to the thread when the id is:

• the value of a log or resource attribute;
• the trace or span id;
• a whole word in the message.

A longer word that only contains the id does not match: ord_1 does not match ord_12. Matching is case-sensitive for attribute values and ids, and ignores case for words in the message.

Which values are connected#

Connected values come from the attributes on the thread's lines. Logthread leaves out:

• values with spaces;
• true, false, null, none and -;
• numbers of at most 3 digits;
• dates;
• values shorter than 3 characters.

A value can be followed when it is specific. It must be on at most 25 lines, or on at most 2,000 lines that are also at most 1% of the range's lines and at most 20 times the lines it has on this thread. Anything more common is shown as shared context. The counts are exact distinct lines over the whole range, not just the lines shown.

Flows across services#

The search covers All services. It finds any id, word or field in every service, whether the value is in a field or in the message. When lines share no request id, it shows the matching lines in the order they were logged. With no filter and no shared id, it shows a graph of how services connect in the range, then the busiest steps of up to the last 24 hours. Choose a service to focus on it. Worth a look in the last hour lists firing incidents, error bursts and failing steps that are growing, whether or not flows are set up.

One search for everything#

Threads has one search box for any id, user, address, server, account, words or field. With nothing typed, it shows Top values by failures or activity, how services connect, and the busiest steps. Search a value for its full picture:

• how many attempts failed and succeeded, and when
• which statuses and codes they ended in
• where they came from: services, addresses, hosts and any other field
• the lines themselves, failed ones first

It reads fields and key=value text in messages, so it works without a pipeline and on old logs. Searching a value reads up to 7 days of lines; past that, it shows what was recorded for that value and what it connected to. On very busy ranges Logthread reads every failed line and an even sample of the rest, and the page says so: failures are exact and other counts are estimates.

Values connect across your logs, not only on one line. Lines join when they share a request id, or the same address within 5 minutes, even in different services. An address with more than 12 values in 5 minutes is treated as many parties behind one address, such as a proxy, and joins nothing. The graph runs from who to what: address, user, key, account, then the service and server. Search a value and the graph centres on it.

With many values, Show keeps the graph readable: Busiest and failing keeps each value's two strongest connections and says how many are hidden, Failing only does the same for connections that carried failures, and Everything shows them all.

New in this range lists connections the previous period of the same length did not have. When it happened shows the same values on a timeline: one row per value, dots when it was active, red when lines failed, and lines between rows when two values connected at that time.

Logthread also records these connections every 5 minutes as logs arrive. Ranges up to 7 days read raw lines. Longer ranges use what was recorded, so the picture and Top values grow to reach up to 366 days, within your retention. A value's picture shows its last 90 days. Recording stores up to 500 values and 500 connections per 5 minutes and is not billed.

What counts as failed#

A line fails when it has error severity or an HTTP status of 500 or above. The status comes from a field such as status or http.status_code, or from status=503 in the message. The map and Worth a look use this, and list 4xx responses separately as denied requests.

What's different counts 4xx too by default. Change it with Count as failed. Lines with no request id are compared one by one, so a one-line-per-request service still shows what its failures share. With Slow, it compares the slowest 5% of requests, and only when they take 100 ms or more; below that it says nothing here is slow.

Fields still inside your messages#

Threads and Pipelines list services whose lines hold key=value text that was never made into fields, with the keys found in your own newest lines. Choose Extract these fields to create a pipeline, preview it on real lines, and activate it. It applies to new logs.

Linking fields#

Fields that hold ids but link no lines yet are listed under Id fields not linking lines yet. This is usual when each id is on one line. Follow a value to check it, or choose the field if other services will log the same id.

Type a field name in Follow an id to see its most common values. A field=value such as status=200 is a filter, not an id, so the thread offers it in Explore and Flows.

The Threads page lists the fields whose values tie lines together, found automatically from your newest 50,000 lines in the last 24 hours. A field counts when all of these hold:

• it has at least 20 distinct id-like values;
• at least 5 of those values appear on more than one line;
• no single value is on half of its lines or more.

Values with spaces, short numbers, dates, JSON and file or object paths (.env, backups/db.dump) are not id-like, so log levels, HTTP methods, status codes, messages, timestamps and requested paths never qualify.

Each field shows:

• how many of its values appear in several services;
• how many appear on several lines;
• which services log it;
• up to 3 sample values, each a ready thread.

When a field's values are seen in 5 or more services, Use for stories (members and up) makes it the workspace's correlation key in one click. New logs are then joined by it in stories, Flows, What's different and the service map. Built-in keys such as request_id show Built in. The same suggestions, with this evidence, are under Settings, Correlation.

The service map uses these fields right away, even before you choose one: lines without a request id are joined by the strongest linking fields, and the map says "also joined by order_id".

If nothing is listed yet, send logs with an id as a field (for example order_id in JSON lines), or extract one with a pipeline.

Flows#

Flows show how requests actually move, from one log message to the next, within and across services, and what is changing right now. Open Threads → Flows, then pick:

• a service;
• a range, from 30 minutes to 30 days;
• Graph or Steps.

The graph#

The graph reads left to right, as a story:

• The main path: the busiest route from where requests enter, with every branch that leads to an error. A summary above the graph says it in words, for example: "Main path (86% of requests entering here): charging card → charge succeeded → to orders. 13% end in payment provider timeout after card declined, retrying."
• Other services: they are folded into small "from orders" and "to orders" nodes beside the step they connect to, so every line is short and nothing tangles.
• Loops: retries share a column and are drawn as arcs at the side.
• Quieter branches: they wait behind a +N badge on their step. Select the step and choose Show N more paths, or switch to All paths.
• What changes are shown:
• Size: a step is drawn a little larger the more traffic it carries.
• Glowing red: an error step that is growing against the previous period. It glows faster the faster it grows. A new error path pulses amber.
• Dimmed: a healthy step that dropped by 30% or more.
• Animation: dashes flow along every path in the direction requests go: faster on busy paths, red on paths to errors. Turn on Live and the counts tick up every minute as new data arrives.

Exploring from the graph:

• Hover a step to light up only the paths through it.
• Select a step to open its panel:
• this period's count against the previous one;
• Who and what (below);
• recent lines (click one for all its fields);
• Open in Explore;
• Only paths through it.
• Select a path to see its count and share, open up to 3 requests that took it as threads, or open the next step's lines in Explore.

On phones, Flows opens as a list of steps. The graph is one tap away.

Filter flows with a query#

Type an Explore query in Only requests matching to draw the flows of just those requests. For example:

• username:user042
• tenant:acme "timeout"
• app_version:2.4.1

It uses the same language as Explore. Fields your pipelines and parsers extract work too, and requests are joined by the correlation or request ids from your SDKs or Settings, Correlation. A request is included when any of its lines matches.

Filtered flows are computed exactly from the raw lines, so they cover up to 7 days. One value can be searched further back, from what was recorded. The graph glows and dims against the previous period of the same requests.

In a step's Who and what, the thread icon next to a value adds it as the filter, so you can go from "this IP is behind 70 usernames" straight to the path its requests take.

With Live on:

• a selected step's recent lines refresh every 10 seconds;
• Live tail opens the live stream of that step's lines with the same filter.

Who and what: links a log list cannot show#

The panel of a selected step shows two things about its lines:

• Concentrated values: for each field (attributes and resource attributes, such as username, tenant, version, host or region), the top values and their share. A field is marked concentrated when one value is on a fifth of the lines or more. For example, one username on a third of failed logins points to one person locked out.
• Unusual links: one value tied to many different values of another field, far more than is typical, for example "client_ip 10.0.0.9 → 73 different usernames, typical 1". That points to credential stuffing; you would not see it in a log list or a time chart.
• A link counts when it reaches at least 5 different values and 3 times the median for that pair of fields.
• Ordinary fan-out, such as a load-balanced host serving many users, never shows.

Every value opens its lines in Explore. Fields with a different value on almost every line, such as request ids, are left out: they are ids, not entities.

Three ways it helps#

• Users are locked out. In the auth service, "login failed: invalid credentials" glows red and grows, "account locked" pulses, and "login succeeded" fades. Select the failure:
• a concentrated username means one person is locked out;
• an unusual link from one client IP to many usernames means an attack;
• otherwise, check whether one identity provider is on 100% of the lines.
• Payments fail. "charging card" splits into "charge succeeded" and "card declined, retrying → payment provider timeout". The timeout glows. Select it, and Who and what shows app_version 2.4.1 concentrated, which points to the cause. Open a path to read one failing request end to end.
• After a deploy. Set the range to the 30 minutes since the deploy. A new amber path, or an error step glowing that was quiet before, points to what the deploy changed. What's different confirms the version.

On a dashboard#

Choose Add to dashboard on the Flows tab to put the graph of a service next to your other charts. The panel:

• follows the dashboard's time range;
• refreshes every minute;
• shows the busiest failure in words;
• opens the full Flows view on a click.

How flows are counted#

Flows need logs that carry a request, correlation or trace id, the same ids that build stories in Explore (see Settings, Correlation):

• Within each id, lines are put in time order.
• Each line whose previous line was a different message pattern counts as one transition. Repeats of the same message, such as a retry loop logging the same line, are one step.
• Logthread counts transitions every 5 minutes, about 10 minutes after the fact so late lines are included, and keeps the 500 busiest per 5 minutes. The rest are shown as Other transitions.

Flows are stored as the metric logthread.flow.transitions, with the attributes from_service, from_pattern, to_service and to_pattern. That means you can:

• chart it in Metrics or on a dashboard;
• alert on it with a metric threshold, for example when the path to a timeout rises.

You are not charged for this metric. It is kept for your workspace's retention.

What's different: what failing requests have in common#

Threads → What's different compares the requests that failed with the ones that did not, and shows what the failing ones have in common. Pick a service (or all services) and a range (1 hour to 7 days).

A request is a request, correlation or trace id. It is failing when any of its lines is an error or fatal; otherwise it is healthy. The tab shows:

• how many requests failed;
• Values: attribute values, resource attributes (such as service.version or host.name) and services that are much more common in failing requests. For example, app_version = 2.4.1 might be in 71% of failing requests and 4% of healthy ones, so "19× more often".
• Paths: moves from one message to the next that failing requests take much more often, such as "card declined, retrying → payment provider timeout".

Each item shows its failing and healthy bars and how many times more often it appears. It has an example failing request to open as a thread, and values also have a filter for Explore.

How it is decided#

• Excluded: fields that only restate the severity (level, severity and the like), and Logthread's own fields.
• Shown: a value or path must be in at least 3 failing requests and at least 5% of them. It must also be at least 5 percentage points and 1.5 times more common in failing requests than in healthy ones. The biggest differences come first.
• Sampling: with more than 20,000 requests in the range, the newest 20,000 are compared, and the tab says so.
• Several causes: when nothing stands out, the failures likely have several unrelated causes.

For AI agents#

Over MCP:

• find_request returns an id's thread: lines, steps, connected values and traces. Pass more ids in also to follow connected values.
• list_linking_fields returns the linking fields with sample ids.
• get_flows lists what is changing (growing errors, falling steps), then a service's transitions with counts, shares, what is new and what leads to errors.
• explain_step returns who and what is on a step's lines: concentrated values and unusual links.
• compare_failing returns the values and paths much more common in failing requests, each with a failing example id. One that never appears in healthy requests is marked only_in_failing and described as "only in failing requests" instead of a lift.

Pari, the in-app assistant, can point you to both.

API#

They use your session and read only your workspace's data. Threads and linking fields cover at most 7 days.

Next: link logs to traces in Correlate logs and traces.