HomeFind your caseOur team's knowledge lives in people's heads
Case · AI
A common situation · not a client storyOur team's knowledge lives in people's heads.
Growing teams where answers live in Slack threads, stand-ups and a few long-serving people, and new people ask the same questions again.
What you're probably seeing
If this sounds familiar, you're in the right place.
- New people ask the same questions, and the same few people answer them.
- Answers exist, buried in Slack threads nobody can find.
- The wiki is out of date, so nobody trusts it.
- Managers can't tell who is blocked without asking everyone.
- When someone leaves, what they knew leaves with them.
The usual ways out
What teams usually try. Each works somewhere. None works everywhere.
| The way | Works when | The catch |
|---|---|---|
| Write a better wiki | Someone owns it and keeps it current | It goes stale within months |
| Make search work across tools | The answers are written down somewhere | Finds documents, not answers |
| An AI assistant over your documents | The documents are mostly right | Confident answers from stale pages |
| Collect short daily updates, and answer from them | You need to know what is happening now | Only works if people reply, so it must be quick |
| Pair new people with experts | The team is small | Takes the experts' time |
| Record the experts explaining things | A few topics come up again and again | Recordings are hard to search |
What other companies did. Their published results, not ours.
We read each company's own write-up and asked one question: would our diagnostic's tests have led to the same call? We judge the approach, not the result.
| Company | What they did | What they published | Source |
|---|---|---|---|
| UberInternal Slack help channels getting about 45,000 questions a month. Answers were scattered across an internal wiki, an internal Stack Overflow and other docs, so the same questions came up again across hundreds of channels. | Built Genie, a Slack bot that answers from the wiki, the internal Stack Overflow and engineering docs, and cites its sources. They only used sources most engineers could already see. They added feedback buttons, human escalation, and an app that scores each document's quality. | 154 Slack channels. 70,000+ questions answered since launch in September 2023. 48.9% helpfulness rate. An estimated 13,000 engineering hours saved.They noted: Answers are only as good as the source documents. Many internal sources could not be shown in Slack and had to be left out. Some questions, such as code reviews, still need a person. | Genie: Uber's Gen AI On-Call CopilotOct 2024 |
| GrabSlack channels for internal engineering tools, where on-call engineers kept answering the same 'how do I' and 'is this broken' questions. | First measured the load in a few teams. Then they bought a Slack FAQ bot (OneBar), chosen by a blind user vote and a proof of concept. Their requirements included saving Slack discussions into its knowledge base straight from Slack and auto-answering in chosen channels. They also started a glossary channel for internal terms. | The first contract limited the bot to 20 channels. It took about one quarter of steady feeding before the bot answered accurately and often. The glossary effort grew users from a couple of hundred to about 3,000 in weeks. No figures for reduced on-call load.They noted: With little data at the start, the bot gave wrong answers. Adoption needed tech talks, training and rewards for contributors. | How we automated FAQ responses at GrabJul 2022 |
| SpotifyIn a company-wide survey, engineers ranked not finding technical information as their third-biggest problem. Docs were spread across Confluence, Google Docs, READMEs and custom sites, and people could not tell whether a doc was current. | Set one standard way to write docs. Engineers keep Markdown next to their code, it is built automatically, and it is shown in one place (Backstage TechDocs) with owner, open issues and support channel. | TechDocs became the most used Backstage plugin at Spotify, with about 20% of Backstage traffic among 130+ plugins. No figures on time saved.They noted: Trust is hard to measure. They show signals like last-updated date and owner but had not built a trust score. Docs need to be both good and findable, and readers need an easy way to flag problems. | Announcing TechDocs: Spotify's docs-like-code plugin for BackstageSep 2020 |
| ConfluentGrew from about 250 to over 2,000 employees, with knowledge spread across 20+ systems. An internal survey showed people struggled to find information, especially in Support, Customer Success and Sales Engineering. | Adopted Glean's search across 20+ tools (Slack, Salesforce, Confluence and others) instead of reorganising the wiki. They are expanding into answers and collections and rolling out go links. | As reported by the vendor: 15,000+ hours saved per month, 1.5+ hours of search saved per employee per week, 70%+ of employees active. One support engineer reported 5–10 minutes less investigation per ticket. The method behind these figures is not stated.They noted: Their chief of staff said the problem was knowledge spread across good tools, not too many tools. Their VP of Business Technology & Data estimated search solved at least 80% of it, which might otherwise have taken months or years of wiki work. | How Confluent saves 15,000+ hours a month with GleanUndated · vendor story |
Our read
Reasonable call. One thing showed up late. Each approach was reasonable, but the write-ups name the quality and currency of the sources as the limit, which our diagnostic checks before anything is built.
Uber's own lesson is that answers depend on the clarity and accuracy of the source documents, and it reports a 48.9% helpfulness rate. Grab's bot gave wrong answers at first, for lack of data, and took about one quarter (three months) of steady feeding to answer accurately. Spotify found people could not tell whether a doc was current and had no way yet to score trust. We would have tested how current the written sources are, and where answers to the top questions live today, before choosing a wiki, search or an assistant. Uber's choice to use only widely visible sources matches our 'who may ask what' test. Grab measuring the question load first matches the 'top twenty questions' test.
What we would test first: List the twenty questions people ask most, and find where each answer lives today and how current it is.
What the diagnostic would test
The tests that rule the wrong ways out.
On your own systems and data, read-only first. The number of tests sets the diagnostic's price, quoted before it starts.
- 1
The twenty questions people ask most, and where the answers live today.
- 2
How current your written sources are.
- 3
Whether an assistant can answer from your real sources, and say who said it and when.
- 4
Who may ask what, and what must stay private.
- 5
Where people already work: Slack, email or a browser.
This week, without us
Three things you can do yourself.
- 1
For one week, note every question asked in your main channel, and who answered it.
- 2
Check the last update date on your ten most-viewed wiki pages.
- 3
Ask three people what they had to learn by asking someone, in their first month.
RAG and retrieval
Sound like you? Bring it to a 30-minute call. We'll tell you which tests your case needs, and what they cost.
This page describes what teams in this situation commonly face. It is not a client story.