A text-summarization agent built on Google's Agent Development Kit, with an evaluation loop, an Obsidian second brain that answers "what did I learn on Tuesday?", clickable provenance on every answer, and optional read-only Gmail and web access.
Every turn is summarised into your vault as a linked note, and the note graph is browsable in Obsidian. Ask the same thing twice and the second time is answered from the vault with no model call at all — measured at 23.5 s for the first ask and 0.04 s for the repeat.
Every answer ends with a Sources block, and each cited note is a link
that opens the actual file. The block is written in code, after the model
answers: the model is asked only for the shape of a source line, and
the vault name, the served model and the /vault/… href are all
substituted afterwards. A note title that does not resolve to a real file is
dropped rather than linked, because a link asserts the note exists.
Ask "What did you learn on Tuesday?" and it reads a day out of the vault instead of the conversation — so it covers notes written in sessions you never saw, and it costs no model call. That reader is also a command:
cd text_summarizer && uv run python -m text_summarizer.digest --date 2026-09-30
--json pipes cleanly into jq: every diagnostic goes to
stderr, so stdout is nothing but the document.
Ask "me fala a temperatura para osasco amanhã" and it reports every daily metric for that place and day — humidity, rain chance, wind and gusts, UV, sunrise, evapotranspiration — from a keyless API. A name that is genuinely ambiguous is asked about rather than guessed: Springfield matches eight US cities, so the agent offers them instead of picking one, and the city, state and population it did resolve travel with the numbers so a wrong match is visible rather than confident.
The part most people find interesting is the evaluation loop: edit the
agent's instructions in agent.py, run ADK evals, and watch
response_match_score move. That is the actual point of the repo —
everything else exists to make the loop measurable.
Two criteria, doing different jobs. tool_trajectory_avg_score is
the hard gate: it is 1.0 or nothing, and it fails the moment the agent stops
writing to the vault. response_match_score is ROUGE-1 word overlap
— a way to catch wording regressions, not a measure of whether an answer
got better. The auto-optimizer decides on the second one, which means a
rewrite that deletes the persistence rules would score
higher and get kept, so the loop refuses any rewrite that drops a
rule before writing it.
Three recordings of the agent actually running, in the order the work
happens. Same agent, same rules, same vault in all three — only the entry
point differs. The screenshots are the dev UI at :8001 taken
while a real turn was in flight; the panels are read back out of the live
vault or transcribed from the code, never invented.
save_summary_to_second_brain never asks the model to
repeat the text back.
Open the GIF on its own.
tools/capture_ui.py drives the real dev UI in
Firefox and screenshots it mid-turn, and tools/build_gifs.py assembles the
frames — reading the notes, topics and links it shows straight out of the live vault.
Re-record with sh tools/check_stack.sh first; that script exists because a
half-restarted stack produces a plausible recording of a broken run.
Docker and the Git CLI are the only prerequisites.
./run.sh creates .env from .env.example
on first run. Put your key in it — this is the only thing you actually need.
GEMINI_API_KEY requiredOPENROUTER_API_KEY optional; give several times the daily budget. Create a key on OpenRouter.OPENCODE_API_KEY optionalMODEL_PROVIDER optionalOpenCode → free OpenRouter → Gemini — Gemini is last because it needs no key, so it is the floor. openrouter builds an OpenRouter-led chain with no Gemini at all. See Models.RENDERER_URL optionalRENDERER_DISMISS_HOSTS / RENDERER_DISMISS_TEXT optionalThis picks your vault, builds both containers, and starts them.
./run.sh
If you already have Obsidian vaults, it finds them — including the ones in
your Obsidian config — and asks which to use. With nothing configured it
auto-creates agent-vault in a local vaults/ folder.
OBSIDIAN_VAULT_PARENT_HOST optional./vaults. Usually written for you.OBSIDIAN_VAULT_NAME optional./run.sh prompts and saves it.
The web UI runs on your own machine at
http://localhost:8001 —
open that after ./run.sh is up.
Paste any text and ask for a summary:
Summarize: Dogs are domesticated mammals known for their loyalty and
companionship. They are often called man's best friend. Dogs come in many
breeds, varying in size, color, and temperament.Every answer is written into your vault as a linked note, with topic notes and an index graph around it. Ask the same question again and the answer comes straight from the vault, with no model call.
If you configured Gmail in step 4, the agent can read your inbox too:
What are my latest 5 emails? Summarize them.Summarize all my e-mails related to Newsletters/Dzone label to this week.What did my manager ask me to do this week? Summarize the thread.You can also ask what you already know, which reads the vault rather than new text:
What do I already know about deployment pipelines? Cite the notes.Weather needs no key at all. It answers from the vault-free tier, and it draws the day — this is a real answer, captured from a running agent, with the graph rendered in code from the tool's own numbers:
Qual a temperatura agora em Osasco?- **Agora em Osasco (SP), às 17:11 (hora local):** 18,5 °C, com sensação térmica de 19,1 °C.
- **Hoje (segunda, 05/10/2026):** máxima de 22,9 °C e mínima de 16,1 °C (média de 18,3 °C).
```adk-chart
Hourly - Osasco, 2026-10-05, local time
°C 16.1..22.9 · rain % 0..44
hora 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
°C ▂ ▁ ▁ ▁ ▁ ▁ ▁ ▂ ▃ ▃ ▅ ▆ ▇ █ ▇ ▅ ▄ ▃ ▂ ▂ ▂ ▁ ▁ ▁
rain % ▃ ▂ ▁ ▁ ▁ ▁ ▁ ▁ ▁ ▁ ▁ ▁ ▁ ▂ ▃ ▄ ▄ ▄ ▄ ▄ ▅ ▅ ▆ █
now 17:00 18.5 °C rain 24%
```The headline number is the hour containing now, not the day's average — the average was what the agent used to answer this question, and it is the wrong number for "agora". Each row is scaled to its own range, and the model never plots a value itself: it writes one placeholder and the curve is drawn from the payload the tool returned.
More examples, including the ones that do not work and why, are in Testing in the web UI.
Each of these loads on its own; none of them affect the steps above.
GOOGLE_CLIENT_IDGOOGLE_CLIENT_SECRETGOOGLE_REFRESH_TOKENtext_summarizer/gmail_oauth.py once to mint the refresh token.
Full walkthrough in Integrations.
SEARXNG_URLWEB_SEARCH_ENABLEDagent-net bridge — find it with
docker network inspect agent-net.
WEB_SEARCH_ENABLED=false turns the tools off while leaving
the URL alone, which is what adk eval wants: live results
change daily and would make the score measure the day rather than your
edit.
WEATHER_ENABLEDWEATHER_LANGUAGEWEATHER_ENABLED=false to remove the tool. The agent asks
when a name is genuinely ambiguous ("Springfield" matches eight US
cities) instead of guessing, and reports the city, state and population
it resolved so a wrong match is visible. Weather is never written to the
vault: a forecast goes stale and the cache would replay it tomorrow.
BOT_NAMELANGFUSE_PUBLIC_KEYLANGFUSE_SECRET_KEYLANGFUSE_BASE_URLCACHE_ENABLED optionaltrue for normal use. Set it to false when running
adk eval, or the agent replays cached notes and the loop measures nothing.
./run.sh finds the vaults you already
have — including the ones in your Obsidian config — and asks which one to use.
One vault needs no configuration; several are never guessed between.
Everything except the model key is optional, and each one loads automatically once configured.
| Integration | Adds | Configure |
|---|---|---|
| OpenRouter | Free models as a fallback when Gemini rate-limits you | MODEL_PROVIDER=openrouter |
| Langfuse | A trace per turn: latency, cache hit/miss, and three quality scores on a 0–1 scale | LANGFUSE_* |
| Gmail | Read-only inbox tools | GOOGLE_CLIENT_ID,GOOGLE_CLIENT_SECRET,GOOGLE_REFRESH_TOKEN |
| SearXNG | Web search and a guarded page fetch, behind the vault | SEARXNG_URL |
| Open-Meteo | Every daily weather metric for a named place, plus the hour by hour and a curve drawn in the answer. Keyless. On by default | nothing — WEATHER_ENABLED=false to remove |
| Obsidian | Retrieval tools over your vault | set by ./run.sh |
Setup steps, including the one-time Google OAuth dance, are in Integrations.
Summaries, chat logs, topic stubs, and an index graph land in a real Obsidian vault, so the notes form a graph you can browse.
Choosing a vault
obsidian-mcp serves exactly one vault, so with several on disk
the choice is explicit and is never guessed — the agent and the MCP server
refuse and list the candidate names. ./run.sh turns that into a
numbered menu and remembers your answer.
The cache
Every note records a fingerprint of the request, and a repeat of that request is answered from the vault with no model call. Both sides compute the key in code from the same field, because a model-supplied key is paraphrased per run and would never match. A prompt naming a relative time is keyed per day: "summarize today's news" is the same sentence every morning and a different answer every morning, so folding them together would replay the first day's answer forever. A prompt with no relative time is byte-identical to before, so nothing already in your vault is orphaned.
Provenance
Each note records the model that served the turn and the vault it landed in,
read from the live invocation rather than passed in by the model. Unresolvable
provenance is omitted rather than guessed. Titles resolve against filenames
and frontmatter aliases, so the readable name the agent cites and
the name on disk are different strings and both still link. A note that
cannot be found stays a plain [[wikilink]] — no dead link is
ever emitted.
Architecture covers vault selection, the cache, and the Docker topology in detail.
These are Markdown and render on GitHub, so each link opens the source there.
Keys, running it, first prompts, troubleshooting.
Criteria, running evals, hands-on experiments, auto-optimization.
Read-only Gmail, using an Obsidian vault you already have, and a keyless weather tier.
Vault, web and renderer tiers, **Sources** provenance, and the model chain.
The chain: OpenCode, then free OpenRouter, then Gemini as the floor.
Prompt examples for the web UI — including the booking flow, which is the one that finds real defects.
The failures you are most likely to hit, and what they mean.
Gemini API key not valid
GEMINI_API_KEY is unset or wrong in .env.
'/vaults' holds N vaults; refusing to guess
Several vaults in the mount with no OBSIDIAN_VAULT_NAME set.
Re-run ./run.sh and pick one — it writes the answer to .env.
'x' is not a directory under '/vaults'
A typo in OBSIDIAN_VAULT_NAME. The error lists the valid names.
obsidian-mcp keeps restarting
The MCP server exits rather than serve a vault nobody chose. Read its error — it names the candidates.
No Gmail tools in the UI
All three GOOGLE_* vars must be set, and the token valid.
Eval scores look great but nothing is being tested
Run evals with CACHE_ENABLED=false. Otherwise the agent replays
a cached note and the loop measures nothing. WEB_SEARCH_ENABLED=false
for the same reason: live results change daily.
cannot reach the configured SearXNG instance
The host in SEARXNG_URL has to be your SearXNG container's
service name on agent-net, not a hostname you invented —
run docker network inspect agent-net and copy what it lists. A
self-hosted instance sits on a private Docker address, which is expected and
allowed here; only link-local, the cloud-metadata range, is refused.
The web tier looks configured but nothing changes
This is the silent failure. A search or fetch that errors is returned
to the model as data rather than raised, so the agent falls back to its own
knowledge, the answer still looks fine, and the trace shows a healthy turn.
Check the two calls really happened — web_search and
web_fetch in the session's event list — rather than trusting the
answer.
An answer has no Sources block
The block is written in code from source lines the model emitted, so a turn
that found nothing has nothing to render. If the answer clearly came from
your vault, the likely cause is a stale container:
docker compose up -d --build, and confirm the deployed copy
matches your tree. Gotcha 17 in AGENTS.md is this exact trap.
A turn ends with no answer at all
Usually the free fallback models: Gemini rate-limits, and the OpenRouter
:free tier often does too, so the turn dies mid-way. Read the
container log — docker compose logs agent-testing — and look for
failed with status 429. A paid OpenRouter key is the fix; no
amount of prompt editing is.
The answer is a Sources block with nothing above it
The reply exists — it is just not the last thing the turn said. When a tool call lands between the bullets and the citations, the model can end the turn on the citations alone, and the empty half is what the trace, the score and a UI that collapses tool-call turns all report. Look for the answer on an earlier event in the session, not only the last one. Rule 13 now requires the source lines to sit in the same message as the bullets.
The agent gave a page URL where a booking link was wanted
It read the page and cited the page, which is the correct answer when the
page's own links are opaque — a bare
checkout.ingresso.com/?sessionId=… cannot be tied to a time, so
there is nothing to attribute. A venue publishing schema.org
ScreeningEvent gets each session listed with its time, cinema and
booking URL together; look for a [sessions on this page] block in
the tool result. No block means the venue does not publish them, and the
per-film page is the route.
A note says the user confirmed something
A summary of the conversation rather than of the world — “the user confirmed the session”, “the conversation is finished”. Those are events in the chat, not things to remember, and each carries a valid fingerprint, so it can be replayed from the cache as though it were a fact. Delete them; rule 8 now names the subject explicitly, because both were formatted exactly like real summaries.
A page behind an age gate reads as empty
web_fetch returns a refusal naming the gate rather than a page of
nothing, and the agent is supposed to stop and ask you. If it reports the page
as unreadable instead, the gate markers did not match — they are checked
against the visible text with accents folded out, so a site that phrases its
notice differently from the built-in list needs its wording added to
AGE_GATE_MARKERS.
The agent asked you to confirm an age, then refused to go on
The second call needs attested=true, and that flag is refused unless
ask_user recorded a real answer in this session. So a page that stays
behind its gate usually means the agent asked in prose, or asked and then ignored
what came back — the question has to be the ask_user tool, and
the answer has to say answered_by: user. Declining, or answering with
something that was not on the list, deliberately does not count.
A turn died with "litellm.Timeout" and nothing retried
The fallback chain only advances on a failure that carries an HTTP status
— 429, 5xx. A client-side timeout carries none, so it ends the turn
with every remaining model untried. This is deliberate on ADK's part: a timed-out
request may have been processed, and failing over risks paying for and acting on
one prompt twice. It bites hardest now that the chain leads with a free endpoint.
Read docs/MODELS.md for the open fix.
The agent answered instead of asking which one
Rule 15 says to ask_user when a choice is a genuine fork, and names
screenings, seats, dates and times — the things a note goes stale on. Listing
them is right; choosing one for you is not, and needs a question. If it never
asks, the turn is not broken so much as unanswered: check the session for a
ask_user call. Nothing in the eval loop can see this either way, so
it will not show up as a score change. Measured on 2026-10-03: on the same
prompt with the same instructions, Gemini calls ask_user and the free
OpenCode model never does — so this is the model, not the wording.
The gate cleared but the page is still thin
The dismiss primitive is off unless you turn it on, and it is configured on
the renderer, not on the agent — it is the process that runs a
stranger's JavaScript, so what it may click is your decision rather than the
model's. Set RENDERER_DISMISS_HOSTS and
RENDERER_DISMISS_TEXT in the renderer project and restart it. Both
blank is the intended default, not a bug.