Post

A Gate in Front of the Agent's Context: pi-key-protector

A Gate in Front of the Agent's Context: pi-key-protector

Introduction

After writing code with agents for a while, it is easy to overlook one fact: anything a tool prints becomes part of the conversation. It is written to the session transcript, handed to the model provider, and quite often re-sent on every later turn.

So when a tool runs env, cat .env, docker inspect, or a curl with a token in it, the credentials travel along. And the agent has no idea it has just leaked them:

1
2
3
$ env | grep -Ei 'key|token|password'
OPENAI_API_KEY=sk-proj-...
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE

That result goes straight into the transcript and then to the provider’s servers, which means the value has stepped over the boundary of “my machine”.

What a permission gate can’t cover

Pi ships without any permission system: the thing that intercepts dangerous commands is a third-party extension, pi-permission-system (published as @gotgenes/pi-permission-system), which enforces allow/ask/deny policies before a command runs. That is the “upstream” line of defence. But echo $OPENAI_API_KEY is not dangerous at all — its output is. And most leaks are not requested by a human:

  • npm run build fails and dumps the contents of .npmrc, registry credentials included, into the log;
  • the agent runs kubectl get secret -o yaml while debugging a connection problem;
  • the agent wants to check the port configuration, so it runs cat .env and then pastes the whole file into its summary.

All of that is ordinary operation. A permission gate should not block it, and blocking it would make the agent useless. So a second line of defence is needed: mask credential fragments on their way back from the tool to the model.

Blocking env also blocks the safe way to do it

The obvious way to stop credential leaks is to block everything that touches the environment: env, printenv, printenv *KEY*, cat .env*. The problem is that a good share of those commands are the secure practice, and they share one property: the key only ever travels through a pipe, a redirect or a child process’s environment. It never appears in the tool result, so it never enters the conversation.

1
2
3
4
5
6
7
8
9
10
11
12
# 1. Hand the key to another program's own config: stdout is empty after the redirect
printf '%s\n' "$OPENAI_API_KEY" > ~/.config/mytool/api_key

# 2. Give a CLI its token on stdin — not as an argument, and not on screen
printf '%s' "$GH_TOKEN" | gh auth login --with-token
docker login -u "$USER" --password-stdin < ~/.config/docker/token

# 3. Inject the variable into one child process; the tool only reports success or failure
env OPENAI_API_KEY="$OPENAI_API_KEY" ./mytool --self-check

# 4. Take only the status code; the key stays in the request header
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $TOKEN" https://api.example.com/v1/models

None of these results contains a single credential character: the first prints nothing, the second reports a successful login, the third prints ok, the fourth prints 200. These are exactly the patterns that deserve to be encouraged — far safer than asking the agent to read a key into its context and paste it into a command.

Block such commands, though, and the user is left with two options: turn the gate off entirely, or work around it by pasting the key into the conversation or into a file the model reads. The second one is precisely the action that puts a credential in the transcript — over-blocking pushes the key back onto the path it was supposed to avoid.

So the criterion is “does the result contain credential characters”, not “does the command mention env”. That is also why the project’s second design goal is stated as a rule about false positives: their cost is not just annoyance, it is the less-safe workarounds they provoke.

That is what pi-key-protector does.

Three design goals

In priority order:

  1. No misses for the ordinary mistake. echo $API_KEY, env, cat .env — the most common operations must always be caught.
  2. Almost no false positives. Most of a tool result is not a secret: source code, build output, hashes, timestamps, tables of numbers. The moment ordinary text starts getting painted over, the agent can no longer work, and the user turns the feature off.
  3. Speed. Every tool result pays this cost, on the critical path of the agent loop.

These three conflict with each other, and nearly every technical choice in the project is a balance between them.

What counts as a credential

There are three sources, read once at session start and kept in memory only.

One: the variable name looks like a credential carrier. The name matches (KEY|TOKEN|SECRET|PASSWORD|PASSWD|CREDENTIAL) (case-insensitive), and the value survives a shape filter: at least 8 characters, no whitespace, not path-like, not one of the well-known configuration literals (true, production, localhost, …), and not a low-diversity string (aaaaaaaaaa).

That filter looks fussy but it earns its place: without it, SOMETHING_SECRET=production would turn the word production into a redaction everywhere it appears.

Two: the value matches a known format, whatever the variable is called. The list currently covers OpenAI (sk-, sk-proj-), Anthropic (sk-ant-), OpenRouter, AWS access keys (AKIA…), GitHub (ghp_…, github_pat_…), GitLab, npm, PyPI, Slack, Google API keys, Stripe, Hugging Face, JWTs, PEM private-key headers, and scheme://user:password@host connection strings. Format-matched values skip the shape filter above.

Three: dotenv files in the project. .env and its variants (.env.local, .env.production, …) are read at startup. This one came later: without it, cat .env hands the model a value that never existed in the process environment, and no matcher can protect a string it has never seen.

Dotenv discovery is deliberately bounded — it starts at the session directory, walks at most 3 levels down, skips hidden and build/vendor directories (node_modules, dist, vendor, …), does not follow symlinks, reads at most 50 files, and gives up after 256 KB. It runs on the session-start path and reads files the user did not point at.

Everything else in the environment — PATH, HOME, LANG and friends — is ignored.

Matching: find every fragment in one pass

This is the core problem of the whole project. Given:

  • a small credential set S (a few dozen, typically 20–200 characters each), and
  • an arbitrary text T (from a few hundred bytes to several megabytes of build log),

find every region of T that exposes enough of some s ∈ S, while keeping the rest of T byte-identical.

The naive implementation is for each secret: indexOf(secret). It fails all three goals at once:

  • it only catches contiguous full occurrences — a key split by a space (sk-123 456), a prefix-only leak, and a suffix-only leak are all missed;
  • it rebuilds the search for every result;
  • it is O(|S| · |T|).

What about Aho–Corasick? It finds exact occurrences of a pattern set in linear time, and solves the “full key printed” case perfectly. But it does not do partial matching: a 13-character prefix of a 39-character key is not an occurrence of any pattern. The usual patch — inserting every prefix and suffix of every credential — explodes to O(Σ|s|²) patterns and still misses interior fragments (say api03-abcdefghij from the middle of a key) and fragments that only appear after a separator split.

So the project uses a suffix automaton built from the credential set. It accepts exactly the substrings of those strings, with O(Σ|s|) states. Walking T through it yields, for every position i, the length best[i] of the longest substring ending at i that occurs in some credential — amortized O(1) per character.

That single array answers everything the policy asks: which fragments exist, how long each is, where it sits in the text, and — after attribution — where it sits inside the credential.

Cost per tool result: O(|S| + |T|) time, O(|S|) memory, plus one Int32Array of |T| entries. The automaton is built once per credential-set change (session start or /key-protector reload), not per result.

Masking: how much to keep so it is still useful

Masking everything is the safest option and the most useless one: whoever reads the transcript can no longer tell which key was involved.

The compromise is a bounded, structural reveal. Take ANTHROPIC_API_KEY=sk-ant-api03-abcdefghijklmnopqrstuvwxyz:

What the model would have receivedWhat the model actually receives
sk-ant-api03-abcdefghijklmnopqrstuvwxyzsk-ant-api**************************xyz
sk-ant-api03-abcdefgh (prefix leak)sk-ant-api***********
ghijklmnopqrstuvwxyz (suffix leak)*****************xyz
api03-abcdefghijklmn (starts at the substantive part)api*****************
ant-api03-abcdefghijklmn (starts inside the public prefix)ant-api*****************
sk -ant -api03 -abcdefghijklmnopqrstuvwxyz (split by whitespace)sk -ant -api** ************************xyz

A few rules deserve explanation.

The public prefix is free. sk-ant- is not a secret: it is published in Anthropic’s documentation and is identical for every customer. Spending 6 of the 13 revealed characters on it would be a waste, so known formats keep the prefix for free and reveal only 3 characters at each end of the substantive part. Credentials known only by variable name (SERVICE_PASSWORD=hunter2-S3cr3t-Pa55w0rd) have no such structure, so they keep the wider 6 + 3 window — those leading characters are the only thing that tells an operator which value is in play.

The reveal windows belong to the credential, not to the match. This is the part that took a second pass to get right. The windows are defined in credential coordinates: the head window ends at public prefix length + 3, the tail window starts at credential length - 3. A match only gets the intersection of its characters with those windows. Every shape of leak therefore behaves the same way: a complete value shows sk-ant-api, a leak starting at the substantive part shows api, a leak starting inside the public prefix shows the part it carries (ant-api). Nothing outside the leak is ever invented.

At least 4 characters are masked. That is a floor, guaranteeing a short credential can never be fully exposed by trimming the reveal budget. Trimming goes tail-first, because the prefix is the more informative hint for a human.

Masks preserve length and formatting: whitespace, quotes and punctuation stay as they are, and only credential characters become *.

Two appended notices

Besides the masked text, notices are appended to the result, because the model has to know it is not looking at the original output — otherwise it may draw wrong conclusions from edited content.

  • Something was masked → tell the model the output was modified, which variable names were involved (values never appear in a notice), and what the replacement rule was.
  • A single result exposes ≥ 90% of one credential → escalate to a security warning. Coverage here is “the union of all matched offsets inside the credential ÷ credential length”, so two halves in one output add up to 100%. The union never crosses result boundaries, so the notice can escalate while the masking policy stays stateless.

Deliberately not defended

The project gives this its own section, because stating the boundary clearly is more useful than claiming to defend everything.

Cross-result character-by-character leaks. A model that prints one character per tool result can walk a credential out one character at a time with no match at all — every result is far below the threshold. Catching it would need cross-result state. That design was considered and rejected:

  • it cannot distinguish an adversary dribbling characters out from an ordinary long log that happens to mention pieces of a credential across hundreds of results (a grep for a common substring, a build log echoing fragments). Once the ledger says “40% of this key is out”, legitimate text starts disappearing, and the user has no way to explain why;
  • it trades a sharp guarantee (“no single result exposes ≥ 10 consecutive credential characters”) for a fuzzy one (“eventually, maybe, if the model is malicious”) — and the fuzzy rule is exactly the one that produces the false positives this project most wants to avoid;
  • the failure it addresses is a targeted exfiltration attack, which belongs to the permission extension’s gating policy, plus the rule “rotate a credential that has demonstrably left the boundary”.

Other deliberate gaps: case and encoding variants (SK-PROJ-…, base64, hex dumps) are not normalized; image blocks and the structured details field are untouched; ! shell commands are not protected — Pi hands extensions the command rather than its output, so covering it would mean reimplementing command execution, and it would also mask what you are reading on your own screen. Ask the agent to run the command instead: that goes through the tool-result path and gets redacted.

One more thing that has to be said clearly: a value that has already been sent is sent. Masking happens on the way back from the tool; anything printed before the extension loaded is in the transcript. If a key appeared in full, the right response is to rotate it, not to hope for a retroactive fix.

Install and use

1
pi install git:github.com/Davidasx/pi-key-protector

No build step, no runtime dependency — Pi loads the TypeScript directly.

Check that it works:

1
2
3
/key-protector status     # loaded credentials, active policy, session stats
/key-protector scan sk-ant-api03-abcdefghijklmnopqrstuvwxyz
/key-protector reload     # after exporting a new key in Pi's environment

scan is a dry run: it shows exactly what would be masked without touching any model context. It is also the fastest self-check — if you wonder whether a value counts as a credential at all, just try it.

To turn it off for one run, use pi --no-key-protector. Configuration lives in ~/.pi/agent/key-protector.json, and a project can add <project>/.pi/key-protector.json (read only in trusted projects). Arrays union and scalars override, so a project file can add credentials but cannot silently stop protecting yours.

Closing thoughts

The problem this solves is narrow and concrete: make “accidentally printed a credential” a visible, handled event instead of a silent boundary crossing. It does not try to be a sandbox against a hostile model — that would need entirely different assumptions and a different price.

The interesting part to implement was the suffix automaton. I assumed indexOf plus a few special cases would be enough; only after listing the four leak shapes — prefix, suffix, interior, split by whitespace — did it become clear that the needed capability is “match all substrings”, which happens to have a linear-time standard solution.

The code and the full algorithm write-up are in Davidasx/pi-key-protector. docs/ALGORITHM.md records where every threshold came from and which alternatives were rejected — including the analysis above. If you plan to change the policy, read it first.

This post is licensed under CC BY 4.0 by the author.