Procedural memory: learnt skills

Episodic memory saves what happened. Semantic memory saves what is true. Procedural memory saves how something is done — a skill.

For an agent that means: when a task has been solved successfully, save the approach as a reusable procedure. Next time a similar task turns up the procedure is fetched and followed, instead of the agent groping its way to the same solution again.

The gain is double: fewer steps (cheaper, faster) and higher reliability (a tried recipe beats improvisation).

Voyager (Wang et al. 2023) showed the principle concretely in Minecraft: the agent wrote reusable skills as code, saved them in a library, and gradually built more advanced skills on top of simpler ones.

from dataclasses import dataclass, field

@dataclass
class Procedure:
    name: str
    when: str                     # a description of when it applies — what is matched semantically
    steps: list[dict]             # tool calls in order, with parameters as templates
    succeeded: int = 0
    failed: int = 0
    @property
    def reliability(self):
        n = self.succeeded + self.failed
        return (self.succeeded + 1) / (n + 2)     # Laplace smoothing: new procedures get ~0.5

class ProcedureLibrary:
    def __init__(self, embed, min_reliability=0.6):
        self.embed, self.procedures, self.min_r = embed, [], min_reliability

    def save(self, task, trace, llm):
        """Only called after a VERIFIABLY successful run."""
        p = llm.generalise(task, trace)          # abstract the concrete values into parameters
        if any(similarity(p.name, q.name) > 0.9 for q in self.procedures):
            return None                           # it already exists
        self.procedures.append(Procedure(**p, succeeded=1))
        return p["name"]

    def fetch(self, task, k=2):
        candidates = [p for p in self.procedures if p.reliability >= self.min_r]
        return sorted(candidates, key=lambda p: -cos(self.embed(task), self.embed(p.when)))[:k]

    def feedback(self, name, succeeded):
        for p in self.procedures:
            if p.name == name:
                p.succeeded += succeeded; p.failed += (not succeeded)

Three rules that make the difference between benefit and harm:

1. Only save verifiably successful solutions — otherwise a library of mistakes is built.
2. Follow the reliability up. A procedure that stops working (an API changed) should fall and drop out automatically.
3. The procedure is a suggestion, not a compulsion. The agent should be able to deviate when the situation differs — otherwise it becomes rigid.