Sentry error triage
By Toolspoke
Ranks yesterday's new and regressed Sentry errors by users affected, posts them to Slack and files Linear issues for the worst.
Needs a Toolspoke account. You choose which project it installs into.
- Sentry + Linear + Slack
- Required integrations
- Runs on a schedule
- Weekdays at 9am by default
- Post one message
- What its last step does
- Nothing destructive
- 0 of 36 actions in its tools
On this page
The short version
- What it is
- A ready-made Toolspoke agent that triages yesterday's Sentry errors and hands the worst to Linear.
- What it reads
- Sentry issues that are new or regressed in the last day, with users affected, events, releases and the top stack frame.
- What it writes
- Up to three Linear issues a run, by default, for top errors nobody has filed yet.
- Where it posts
- One Slack channel, chosen when you set it up.
- How often
- Weekdays at 9am once you switch it on. Monday's run covers the weekend.
- What it never does
- Resolve, ignore, reopen or assign a Sentry issue, or change an existing Linear issue.
- What you need
- A Toolspoke account, a Sentry auth token, a Linear workspace and a Slack workspace.
What Sentry error triage does
Sentry error triage is an AI agent that reads the last day's new and regressed Sentry issues each weekday, ranks them by the people they hurt and files the worst in Linear.
Sentry is good at catching errors and bad at telling you which ones matter. A busy project sees dozens of new issues a day, most of them noise, and the one that broke checkout for four hundred people looks the same in the list as a browser extension's warning.
So someone has to triage each morning, and on a busy day nobody does. The bad error gets found when a customer writes in.
Sentry error triage does that pass for you. It ranks by users affected, then by events, reads the top stack frame and the release for each, checks Linear for an issue that already exists and files the rest, up to a limit you choose. Then it posts one message so the team sees the whole morning at once.
In practice, each weekday you get:
- A ranked list of new and regressed errors, with users affected and release
- Linear issues filed for the worst few, with stack frame and a link back
- Links to existing Linear issues instead of duplicates
- A warning when many errors rose together after one release
How it works
Sentry error triage works in four steps each weekday morning: it finds new and regressed issues, reads each one, checks Linear and files what is missing, then posts one message. All of it comes from the agent's written instructions, which you can read and edit after you install it.
Step 1: Find the issues
It lists new and regressed issues in your projects, sorted by users affected.
Step 2: Read each one
It reads counts, first and last seen, the release and the top stack frame.
Step 3: Check, then file
It searches Linear first and files only what is not there yet.
Step 4: Post one message
It posts the ranked list with the Linear link beside each error.
It ranks by people, not noise
Each issue is ranked by users affected, then by events in the window. An error that fires ten thousand times for one test account sits below one that fires fifty times for fifty customers. Issues with no user data are ranked by events and marked as such.
It searches before it files
Before creating anything it calls search_issues in Linear for the Sentry short id and the error title. If a match exists in any state, the post links to it and nothing new is filed.
It files only a few
It files at most three issues a run unless you choose another number, and leaves assignee, priority and state for your team's own triage. A flood of auto-filed tickets is a different kind of noise.
It spots one cause behind many errors
It follows the Investigate an error spike skill. If many issues rose together or one release carries most of them, it says so and files one issue for the group. The limits are under what it will never do.
What a morning message looks like
A message from Sentry error triage is one Slack post. Issues it filed come first, then the ones already tracked, then the rest of the list.
Larkspur
Channels
- # eng-errors
- # engineering
- # releases
- # general
# eng-errors
Tuesday, last 24 hours. 9 new, 2 regressed in production. Filing limit 3, set at install.
Filed in Linear (3)
- TypeError in checkout/summary.tsx:88, 412 users, 1,903 events, first seen 07:55 in release 4.18.0. Filed as WEB-512.
- Timeout in invoices.export, regressed, 61 users, 240 events, back since 4.18.0. Filed as API-207.
- KeyError 'locale' in mailer/render.py:41, 38 users, 52 events. Filed as API-208.
Already tracked (2)
- Stripe webhook signature mismatch, 12 users. Already WEB-498, in progress.
- N+1 query on /projects, 9 users. Already API-190, in backlog.
Lower impact (6)
6 issues under 5 users each, listed at the end of the message. Release 4.18.0 carries 7 of today's 11 issues.
- Filed
The top errors nobody had filed, each now a Linear issue with the stack frame, release and a link to Sentry.
- Already tracked
Errors that matched an existing Linear issue, linked rather than filed again.
- Lower impact
Everything else, ranked, with a note when one release accounts for most of the day.
Why it sometimes says very little
If no issue was new or regressed, Sentry error triage posts one line saying so. A quiet morning is worth knowing about, and a single line is easy to skip.
A failed run is different. If Sentry cannot be read, it posts that once. If Linear cannot be read, it posts the list and files nothing, because filing without the duplicate check could double up.
Who uses Sentry error triage
Sentry error triage suits engineering teams that use Sentry and Linear and have no one whose job is to read the error feed. The examples use made-up names.
A small team with no on-call rota
Priya Raman leads four engineers. Nobody owned the Sentry feed, so errors were found by customers. Now the worst ones arrive in Linear before standup.
An engineering manager after a release
Daniel Osei wants to know when a release made things worse. The post says when one release carries most of the day's new errors.
A tech lead tired of duplicates
Hana Kim used to find three tickets for one error. The agent searches Linear first and links what already exists.
A founder who codes on Mondays
Luca Bianchi reads one message on Monday that covers the whole weekend, ranked by the people affected.
What Sentry error triage connects to
Sentry error triage connects to three tools: it reads Sentry, searches and files in Linear and posts to Slack. You connect each one yourself, and Toolspoke stores the connection encrypted, so the agent never sees the credential.
Sentry
RequiredFind errors, inspect traces, and triage production issues.
Authentication: Auth token
Linear
RequiredCreate, update, and search issues across your teams.
Authentication: API key
Slack
RequiredSearch conversations and send messages to your workspace.
Authentication: Bot token
Skills it follows
| Tool | What Sentry error triage does | What it cannot do |
|---|---|---|
| Sentry | Reads issues, events, tag breakdowns and releases (list_issues, get_issue, get_issue_event, list_issue_tag_values, list_releases). | Delete an issue: the connector has no delete action. It can resolve, ignore, reopen and assign issues, which this agent's instructions forbid. |
| Linear | Searches issues with search_issues and files new ones with create_issue. | Delete an issue: the connector has no delete action. It can update and comment on issues, which this agent is told not to do. |
| Slack | Posts one message with send_message, to a channel. | Delete messages: the connector has no delete action. |
The Sentry connection Toolspoke asks for can change an issue's status and assignee. Sentry error triage is told never to. That is an instruction, not a lock. To make it one, switch resolve_issue, resolve_issue_in_next_release, ignore_issue, reopen_issue and assign_issue off in your project. The access control guide explains how.
In Linear the agent needs create_issue, so the connection must allow writes. If you want only that, switch off update_issue and create_comment for this agent.
What Sentry error triage will never do
Sentry error triage is told never to resolve, ignore, reopen or assign a Sentry issue, and never to change an existing Linear issue. It decides what to show people, not what is fixed.
- Resolve, ignore, reopen or assign a Sentry issue, or change its status in any way.
- Update, comment on or close an existing Linear issue, or set an assignee or priority on one you create.
- File more Linear issues than the chosen number, or a second issue for something already filed.
- Paste user email addresses, IP addresses or request bodies from events into Slack or Linear.
Why it does not touch Sentry
Resolving or ignoring an error hides it from everyone. That is a judgement about the product, and a wrong one means the next occurrence alerts nobody. The agent reads Sentry and leaves its state to engineers.
Why it files so little
It files only the top few errors that have no Linear issue, and never sets an owner, priority or state. Your team's own triage decides who works on what.
What it keeps out of Slack and Linear
It counts users and does not name them. It does not copy email addresses, IP addresses or request bodies from events.
If something goes wrong
If Sentry cannot be read, post that once and stop. If Linear cannot be read or written, post the ranked list anyway and say no issues were filed, because filing without the duplicate check could double up. If an event has no stack trace, say so beside the issue rather than guessing a location.
These are written into Sentry error triage's instructions. They guide the agent, but they are not a permission. What it is allowed to touch is set by the access you grant.
Sentry error triage reads Sentry and files what is missing. Whether an error is fixed, ignored or assigned stays with a person.
When Sentry error triage runs
Sentry error triage runs on weekdays at 9am, but only after you switch its schedule on. It arrives switched off, so nothing runs and nothing is charged until you decide.
- Each run reads the last 24 hours, and Monday's covers the weekend.
- It files at most three Linear issues a run unless you choose another number.
- When nothing is new or regressed, it posts one line.
- If Sentry cannot be read, it posts that once.
Each run uses model credit from your workspace balance, so it makes five runs a week. The Sentry, Linear and Slack calls themselves are free.
Your copy is an ordinary agent in your project. You can pause its schedule there, run it once on demand, change the days and time and edit its instructions. The agent's page lists recent runs and their cost.
How to set up Sentry error triage
To triage errors each morning, add Sentry error triage to a Toolspoke project, connect Sentry, Linear and Slack, choose your projects and team, and switch on the schedule. Nothing runs until the last step.
Step 1: Create a Toolspoke account or sign in
Choose Use this agent. If you are new, you will be asked to sign up first, and you come straight back to this agent afterwards.
Step 2: Add it to a project
Pick the project it should live in. You get your own editable copy, and you can see the three tools it brings before anything connects.
Step 3: Connect Sentry
Create a personal auth token in Sentry and paste it with your organisation slug. It is stored encrypted and the agent never sees it.
Step 4: Connect Linear
Connect the Linear workspace issues should be filed in. The agent needs to create issues, so the connection allows writes.
Step 5: Connect Slack
Choose Sign in with Slack and approve the Toolspoke app. For a private channel, add the Toolspoke bot to it first.
Step 6: Choose projects, team and limit
Name the Sentry projects and environment, the Linear team key, the channel and how many issues to file a run. If you leave the limit out, it files three.
Step 7: Switch on the schedule
The schedule arrives switched off. Run it once to see a message, then switch it on.
Make Sentry error triage fit your team
Sentry error triage's instructions are plain English and you can edit them after you install it. Installing copies the agent, so your edits stay with your copy. Open the agent in Toolspoke and tell the builder what to change, and it rewrites the instructions.
Paste any sentence below into the builder chat, or say the same thing in your own words.
File at most 5 Linear issues a run.
Add the label sentry to every Linear issue you create.Leave out any issue whose title contains ResizeObserver.
Only file issues that affected at least 20 users.
Run on weekdays at 9am and 3pm.
This changes when the agent wakes up, so the builder edits its schedule, not the instructions.
None of these edits needs more access or touches the Never rules. They change what Sentry error triage is told, not what the connections allow.
How it compares to other options
Sentry error triage beats the alternatives when you want one ranked view each morning with the worst errors already in Linear, and loses to them when you need an alert the moment an error starts.
| Compared on | Sentry error triage | Sentry alert rules | Sentry's Linear integration | A person triaging each morning |
|---|---|---|---|---|
| Ranks by users affected | Yes | Per rule thresholds | No | If they look |
| Files issues without duplicates | Searches Linear first | No | Links when you click | Yes |
| One summary instead of many alerts | Yes, one per run | One per alert | No | Yes |
| Spots one release behind many errors | Yes | No | No | With time |
| Changes Sentry issue state | Told not to | No | Can sync status | Up to them |
| Arrives as it happens | No, each weekday morning | Yes | When you file | No |
| Setup effort | Connect three accounts | Write rules per project | Install the integration | Someone's morning |
Sentry's alert rules are the better choice for an outage: they fire in seconds. They alert one issue at a time and do not rank the day or check Linear.
Sentry's own Linear integration is better when an engineer is already looking at an issue and wants to file it in one click. It does not choose which issues deserve filing.
Frequently asked questions
Can Sentry error triage resolve or ignore an error?
The Sentry connection can, and the agent is told never to. That is an instruction, not a lock. To enforce it, switch the resolve, ignore, reopen and assign actions off in your project (how access works).
Will it flood Linear with tickets?
No. It files at most three a run unless you choose another number, and searches Linear for the Sentry id and title first, so it does not file a second issue for one error.
Does it assign the Linear issues?
No. It leaves assignee, priority and state empty for your team's own triage.
What counts as regressed?
An issue Sentry has marked as regressed: one that was resolved and has happened again.
Does it post user details to Slack?
No. It counts users affected and does not copy email addresses, IP addresses or request bodies from events.
Does it work with self-hosted Sentry?
Yes. The Sentry connection takes an instance URL for self-hosted installs.
Is Sentry error triage free?
It has no price of its own, and every Toolspoke plan includes agents. Tool calls cost nothing, but each run draws on your usage balance. Pricing has the details.
Find the bad error before a customer does
Connect Sentry, Linear and Slack, choose your projects and switch on the schedule. Your first morning list lands at 9am on the next weekday.