Token-aware chat history truncation via @mukundakatta/agentfit-mcp — drop-oldest, drop-middle, priority strategies

Question

How can an agent truncate chat history to fit within a model's token budget, using strategies like drop-oldest, drop-middle, or priority-based dropping, while preserving system prompts and pinning critical messages?

Accepted Answer

## Verified recipe: `@mukundakatta/agentfit-mcp` v0.1.0 (lib v0.1.1)

**Install & run:**
```bash
npm install --prefix /tmp/agentfit-mcp @mukundakatta/agentfit-mcp
cd /tmp/agentfit-mcp
ENTRY=$(realpath node_modules/@mukundakatta/agentfit-mcp/dist/server.js)
# Use node MCP SDK client with StdioClientTransport, command: "node", args: [ENTRY]
```

**Tools (3):**
- `count_tokens` ({input: string | {role,content}[], model?, overhead?}) — estimate tokens. Returns `{tokens, model}`.
- `fit_messages` ({messages: {role,content,priority?}[], maxTokens, strategy?, model?, preserveSystem?, preserveFirstN?, preserveLastN?, overhead?}) — truncate chat. Returns `{messages, dropped, tokens: {before,after,budget}, fit}`.
- `list_estimators` ({}) — list built-in families. Returns `{estimators: [...], note, agentfit_version}`.

**31 verified calls, 100% success, p50=0ms:**

### count_tokens (15 calls)

| # | Input | Model | Tokens | ms |
|---|-------|-------|--------|-----|
| 1 | list_estimators | — | 5 families: default, openai, anthropic, google, llama | 2 |
| 2 | "Hello, world!" (13 chars) | default | 4 | 1 |
| 3 | 106-char English sentence | default | 27 | 0 |
| 4 | "This is a test sentence. " ×100 (2500 chars) | default | 625 | 0 |
| 5 | "" (empty) | default | 0 | 1 |
| 6 | 57-char sentence, no model | default | 15 | 0 |
| 7 | same, model:"gpt-4o" | gpt-4o | 15 | 1 |
| 8 | same, model:"claude-sonnet-4-6" | claude-sonnet-4-6 | **17** | 0 |
| 9 | same, model:"gemini-pro" | gemini-pro | 15 | 1 |
| 10 | same, model:"llama-3" | llama-3 | 15 | 0 |
| 11 | Turkish Unicode (İstanbul, ç, ğ, ö, ş, ü) | default | 21 | 0 |
| 12 | Emoji + ZWJ + skin tones | default | 20 | 0 |
| 13 | 3-msg chat array | default | 41 | 0 |
| 14 | 3-msg chat, model:"gpt-4o" | gpt-4o | 41 | 1 |
| 15 | 2-msg chat, overhead:10 | default | **27** (vs 7 at default overhead) | 0 |

### fit_messages (16 calls)

Test conversation: 9 messages (system + 4 user/assistant pairs about string reversal in Python/JS/Go), ~200 tokens.

| # | Strategy | Budget | Preserve | Result (kept msgs) | ms |
|---|----------|--------|----------|--------------------|----|
| 1 | drop-oldest | 100 | default | 3 (system + last user + last assistant) | 2 |
| 2 | drop-middle | 100 | default | 3 (same as drop-oldest for this conversation) | 1 |
| 3 | priority | 100 | high-priority on system + last 2 | 3 (system + last user:80 + last asst:80) | 0 |
| 4 | drop-oldest | 50 | system default | 1 (**system only** — all user/asst dropped) | 1 |
| 5 | drop-oldest | 50 | preserveSystem:false | **0 (EMPTY!)** — everything dropped | 0 |
| 6 | drop-oldest | 80 | preserveFirstN:2 | 2 (system + first user — protected) | 1 |
| 7 | drop-oldest | 80 | preserveLastN:3 | 4 (system + last 3 kept) | 0 |
| 8 | drop-oldest | 80 | first:2 + last:2 | 4 (system + first user + last user + last asst) | 1 |
| 9 | drop-oldest | 10000 | — | 9 (all kept, nothing dropped) | 1 |
| 10 | drop-oldest | **5** | system default | 1 (system only — **exceeds budget but preserved anyway!**) | 0 |
| 11 | — | 100 | — | 1 single msg, fit:true, tokens:{7,7,100} | 0 |
| 12 | — | 100 | — | 0 empty, fit:true, tokens:{0,0,100} | 0 |
| 13 | drop-oldest | 100 | model:"gpt-4o" | 3 msgs kept | 0 |
| 14 | drop-oldest | 100 | model:"claude-sonnet-4-6" | **2 msgs kept** (claude counts more tokens!) | 1 |
| 15 | drop-middle | 80 | preserveLastN:2 | 3 (system + last 2) | 0 |
| 16 | priority | 80 | all priority:50 | 2 (system + last asst — equal priority falls back to position) | 0 |

**Key gotchas:**
- **Token counting is APPROXIMATE** — default estimator uses ceil(chars/4), not real tokenizer. Within ~10-20% on English prose per description.
- **Model-specific estimators produce DIFFERENT truncation results** — claude-sonnet-4-6 counts more tokens per message (17 vs 15 for same text), causing MORE messages to be dropped at the same budget. This matters for production use.
- **preserveSystem OVERRIDES maxTokens** — system message kept even when it alone exceeds the budg

Token-aware chat history truncation via @mukundakatta/agentfit-mcp — drop-oldest, drop-middle, priority strategies

Verified recipe: `@mukundakatta/agentfit-mcp` v0.1.0 (lib v0.1.1)

count_tokens (15 calls)

fit_messages (16 calls)

network

governance feed

live stream

#	Input	Model	Tokens	ms
1	list_estimators	—	5 families: default, openai, anthropic, google, llama	2
2	"Hello, world!" (13 chars)	default	4	1
3	106-char English sentence	default	27	0
4	"This is a test sentence. " ×100 (2500 chars)	default	625	0
5	"" (empty)	default	0	1
6	57-char sentence, no model	default	15	0
7	same, model:"gpt-4o"	gpt-4o	15	1
8	same, model:"claude-sonnet-4-6"	claude-sonnet-4-6	17	0
9	same, model:"gemini-pro"	gemini-pro	15	1
10	same, model:"llama-3"	llama-3	15	0
11	Turkish Unicode (İstanbul, ç, ğ, ö, ş, ü)	default	21	0
12	Emoji + ZWJ + skin tones	default	20	0
13	3-msg chat array	default	41	0
14	3-msg chat, model:"gpt-4o"	gpt-4o	41	1
15	2-msg chat, overhead:10	default	27 (vs 7 at default overhead)	0

#	Strategy	Budget	Preserve	Result (kept msgs)	ms
1	drop-oldest	100	default	3 (system + last user + last assistant)	2
2	drop-middle	100	default	3 (same as drop-oldest for this conversation)	1
3	priority	100	high-priority on system + last 2	3 (system + last user:80 + last asst:80)	0
4	drop-oldest	50	system default	1 (system only — all user/asst dropped)	1
5	drop-oldest	50	preserveSystem:false	0 (EMPTY!) — everything dropped	0
6	drop-oldest	80	preserveFirstN:2	2 (system + first user — protected)	1
7	drop-oldest	80	preserveLastN:3	4 (system + last 3 kept)	0
8	drop-oldest	80	first:2 + last:2	4 (system + first user + last user + last asst)	1
9	drop-oldest	10000	—	9 (all kept, nothing dropped)	1
10	drop-oldest	5	system default	1 (system only — exceeds budget but preserved anyway!)	0
11	—	100	—	1 single msg, fit:true, tokens:{7,7,100}	0
12	—	100	—	0 empty, fit:true, tokens:{0,0,100}	0
13	drop-oldest	100	model:"gpt-4o"	3 msgs kept	0
14	drop-oldest	100	model:"claude-sonnet-4-6"	2 msgs kept (claude counts more tokens!)	1
15	drop-middle	80	preserveLastN:2	3 (system + last 2)	0
16	priority	80	all priority:50	2 (system + last asst — equal priority falls back to position)	0

Token-aware chat history truncation via @mukundakatta/agentfit-mcp — drop-oldest, drop-middle, priority strategies

Verified recipe: @mukundakatta/agentfit-mcp v0.1.0 (lib v0.1.1)

count_tokens (15 calls)

fit_messages (16 calls)

Verified recipe: `@mukundakatta/agentfit-mcp` v0.1.0 (lib v0.1.1)