Stop Paying AI to Plan the Same Task Twice

An agent updates the same sales spreadsheet every Monday. It finds new accounts, opens the same sites, copies the same fields, fixes missed rows, checks the totals, and saves the file.

Next Monday, it plans the job again.

We pay for that planning again: more input tokens, more model calls, more browser actions, more retries, another bill.

Most agents keep a transcript of the last run. That is not the same as knowing how to do the task again. A transcript records everything that happened. A useful skill keeps the steps that mattered, states when they apply, and includes a way to check the result.

A new paper called WikiSkill shows how much those tested instructions can change performance.

Skills helped a 9B model beat a 27B model

WikiSkill tested five models on math, question answering, spreadsheets, office tasks, and household simulations.

Across those benchmarks, a 27B model without skills scored 39.4. A 9B model with skills learned from earlier runs scored 47.4.

The smaller model did not receive new weights. It received better instructions.

That does not prove that skills always beat larger models. It shows something narrower and more useful: for repeated tasks, a tested procedure can matter as much as model size.

The effect appeared across the model set. WikiSkill raised the five models' average scores by 12.3 to 23.9 percentage points over using no skills. A 27B model gained the most, moving from 39.4 to 63.3.

The paper measures task accuracy. It does not report token savings, wall-clock speed, or dollars saved. Those are the next measurements that matter in production.

Skills improve instructions without retraining the model

Fine-tuning and reinforcement learning change the model itself. A skill leaves the model alone and changes the instructions it receives for a particular kind of task.

That makes a skill easier to inspect, test, update, limit to one company, or remove when it stops working. A team does not need to wait for a training run to record that an invoice form requires a project code, or that a report must be checked against the saved browser state before it is marked complete.

Skills do not replace training. Fine-tuning can teach behavior that should apply broadly. Reinforcement learning can improve a general policy. Skills fit knowledge that is local, procedural, or likely to change.

The useful split is simple:

• Put broad behavior in the model.
• Put changing procedures in skills.
• Put exact, stable repetitions in code.

That last step matters. If a browser task follows the same verified path every time, rtrvr can turn it into an AI Subroutine and remove the model from the repeated path. Skills are most useful when the task repeats but still needs judgment.

WikiSkill turns past runs into tested steps

WikiSkill keeps three records.

1. Runs: the complete reasoning, actions, tool results, and final answer.
2. Notes: repeated failures, successful repairs, and evidence that may matter later.
3. Skills: short instructions the worker can use on the next task.

The task-running agent cannot read the notes. A separate agent studies a small sample of runs, updates the notes, and proposes one skill change. The new skill is tested on held-out tasks. It stays only when the validation score beats the previous best.

Keeping the notes away from the worker produced the best result. In the paper's ablation, the default setup scored 63.7 across four benchmarks. Giving the worker access to the notes reduced that score to 60.9.

The reason is practical. If the worker can solve a task by rummaging through all the notes, a weak skill can appear to work. The next run then hides the same defect again.

The worker needs the tested instruction, not the research notebook behind it.

Loading every skill creates another token bill

WikiSkill gives the worker every active skill. That is a sensible research choice because it removes retrieval from the experiment. It is not a design for a large browser agent.

A browser agent may have personal preferences, company procedures, site instructions, recovery steps, verification checks, and safety rules. Loading all of them would spend tokens before the task begins. Conflicting instructions could also make the model worse.

Research on real skill libraries already shows the problem. SkillsBench found that curated skills improved results on average, but some tasks became worse. Focused skills also beat large documentation bundles. SkillRouter found that names and short descriptions are not enough for reliable retrieval. The body of the skill often contains the detail that decides whether it fits.

For a browser task, retrieval should use more than the user's sentence. It should consider:

• The requested result.
• The open site and current page.
• The user's role and permissions.
• The fields and controls currently available.
• The browser tools the agent may use.
• The model and runtime that tested the skill.
• The final state that will prove the task worked.

The agent should load the smallest set that covers the task.

The wrong skill can be worse than no skill

WikiSkill also found that skills do not transfer cleanly between models.

On its spreadsheet benchmark, Gemini scored 50.5 without skills. Skills written by the 4B model cut the score to 18.1. Skills written by the 27B model raised it to 63.4.

The weaker model had written detailed workarounds for its own limits. Those instructions forced the stronger model through unnecessary steps and used up its action budget.

So a saved skill needs more than a title and success count. It needs to say which model, browser runtime, site state, and tool set produced the evidence. A procedure that works in one setup is a candidate in another, not a fact.

This also changes how skills should be shared. Personal skills should stay personal. Company skills should come from work the company approved. Shared site skills should contain sanitized page structure, checks, and repairs, never credentials or private page content.

Browser agents need to choose, test, and remove skills

Browser work makes skill management harder because the environment changes. Buttons move. Forms gain required fields. Users have different roles. A page may contain hostile instructions. An action can send a message, place an order, publish content, or delete a record.

A production system needs four separate jobs:

1. Compile: turn verified runs into a proposed skill.
2. Choose: retrieve the few skills that fit the task and browser state.
3. Check: verify the saved record, submitted form, downloaded file, or other promised result.
4. Clean: merge duplicates, expire stale steps, and remove instructions that no longer help.

The cleanup step is missing from WikiSkill. Its notes keep growing even when a skill proposal is rejected. That preserves useful evidence, but it can also preserve a bad explanation.

We would keep the raw runs and lineage while reducing the active library:

• Merge two skills only when the same situation needs the same repair.
• Keep separate versions when models or sites need different steps.
• Remove one instruction and test whether completion, safety, cost, or speed gets worse.
• Expire selectors, endpoints, and layout-specific advice.
• Recheck a skill when the page structure changes.
• Revoke a skill that asks for capabilities the task does not need.

Browser content must remain untrusted. A page can provide evidence about its current state. It cannot write a new instruction for the agent or grant itself more permission. Research on multi-step prompt injection shows why a harmful instruction may be split across several pages instead of appearing in one obvious block.

Use the cheapest form that still works

Not every task needs the same amount of intelligence.

A new task needs a model to plan. A similar task can give the model a tested skill. A stable task can become code and run without a model in the repeated path. If the site changes, the system returns to planning, repairs the procedure, and tests it again.

The cheapest path is useful only when it still completes the task and stays within the user's permissions.

The next run should use fewer tokens

The next experiment is straightforward. Choose repeated browser jobs such as researching companies into a spreadsheet, updating authenticated records, and gathering information across sites for a document.

Run each job with and without learned skills. Measure tokens, model cost, browser actions, time, retries, verified completion, and performance after a site changes. Then test whether a smaller model with the right skill can match or beat a larger model starting from scratch.

rtrvr already records browser trajectories and is building the verification and promotion path described in our self-improving action layer roadmap. WikiSkill adds a useful middle layer: keep the full runs, turn their repeated lessons into short instructions, and test those instructions before the next agent sees them.

Models will improve. We should still stop paying them to plan the same task every time.

When an agent finishes a job, the useful steps should survive. The next run should load only the steps that fit. Old instructions should disappear when they stop working.

A completed task should make the next one cheaper.