For agents, on top of README.md, which they read first.

Notes for agents

  • No legacy migration code here. This repository is open source and the project rules out reading the old JSON and JSONL files in it. The one-off tool that did lives outside the repository. A schema change is a new numbered migration; never edit 0001_init.sql once a build that has it is released.
  • The merge rules stay in the core. Do not rewrite MemoryDoc::merge, Household::merge or ChatState::merge in SQL. The schema holds the rules about what must not exist (tombstone deletes, forgotten cannot be re-inserted, cascade, a picture goes with its last use): put a new rule of that kind in the migration, as a constraint or a trigger, so the wrong state cannot be written.
  • A turn has an identity; anything that holds what was said in one names it. Entry.turn is on every line of a turn and Mention.turn on every memory learned or said again in it; forgetting writes the identities (tombstone_turn) and the schema empties the turns. A new event that carries her words or the model's must be listed in tombstone_turn_empties_the_turn and journal_line_not_of_the_forgotten (the next migration, never an edit), and a new table that holds her words must be dropped by those triggers, as pending_reflection is.
  • The chat is a document, not rows. Emptying it for a forgetting is ChatState::scrub and its merge rule (see SCHEMA.md); do not add a trigger that deletes chat_turn, because the copy a holder keeps in memory would write it back.
  • The household document never holds a zero count. A device with nothing counted has no entry; the store drops zeros, so a zero in the document would make every merge of it a change.
  • Clearing the log by age never removes a line, it empties it into Expired: sync's counts and cursors are made of the lines' numbers. Say it in the log (Cleared) every time.
  • Forgetting is INSERT INTO tombstone, nothing else. Do not add a second path that deletes a fact, and do not add a table that holds a memory's words, embedding or picture without a gid that cascades from fact, and do not add a log event that carries a memory's words without a gid and a statement for it in the tombstone_forgets and journal_line_not_of_the_forgotten triggers (the next migration, never an edit). tests/forgetting.rs searches the file for the bytes of a forgotten memory: a new place that keeps them must be added to that test and must cascade.
  • A soft delete writes its soft_action row in the same transaction as the change, and the row must cascade from the thing it was done to (ON DELETE CASCADE), so forgetting leaves nothing to undo. The window is the core's rule, never SQL's.
  • Keep secure_delete on and scrub after a forgetting. They are set in Store::prepare (every connection) and SqliteMemory::keep. A connection that does not set secure_delete writes deleted bytes into the file.
  • Never log content. Words, facts, summaries, embeddings and pictures are not logged here: lengths, counts and ids are. An error from SQLite is logged as it comes; do not build one from her words.
  • One connection behind one lock is the atomicity the hubs' contracts ask for (two merges behave as if one ran first). Do not open a second connection to a database a Store has open, except to read it in a test.
  • Ids are never reused: fact.id is AUTOINCREMENT, and load_doc reads the next one from sqlite_sequence, which remembers the largest after the newest memory is forgotten.
  • A device's time_spent rows are written only with MAX: a save from a copy that is behind must not take time back.