lmjtfy.git / apps / lmjtfy / src / README.md
README.mdpreviewREADME.mdsource165 lines · 9.4 KB · raw
1# Chapter 3: the source, read in the order it runs
2
3A dozen or so files. They are easiest read in the order a question meets them: in
4through the router, out to the archive, back as HTML. You do not need to
5read Rust fluently to follow along. Every file opens with a comment (`//!`)
6that says what it is for, and those are worth reading first.
7
8```mermaid
9flowchart TD
10  L["lib.rs: routes, and the loop that answers"] --> AR["archive.rs: the archive object"]
11  L --> M["meter.rs: the budget object"]
12  AR --> AI["ai.rs: Workers AI"]
13  AR --> O["object.rs: talking to an object"]
14  M --> O
15  L --> V["view.rs: the page as HTML"]
16  V --> D["diagram.rs: the rules as SVG"]
17  V --> VC["view/: code pages, toasts"]
18  L --> P["playground.rs: /rules"]
19  L --> C["clone.rs: git, and the code page"]
20  C --> B["browse.rs: folders and files"]
21```
22
23## 1. lib.rs: the front door
24
25The routes are at the top (`fetch`): one line per address, each sent to a
26function. The interesting one is `ask`, which starts the loop that answers a
27question. It does not decide anything itself. It asks the rules engine
28(`Network::next`, chapter 6) what to do, does that, records what it learned,
29and asks again, until the rules say it is done. Each turn, the whole
30transcript so far is sent to the browser as one Datastar event.
31
32> **Aside.** Why re-send the whole transcript every time instead of just the
33> new bit? Because then the browser has no state to get wrong. It throws
34> away what it had and shows what it was sent. A reconnect, a missed event,
35> a second tab: none of them can leave the page in a state the server does
36> not know about.
37
38The calls themselves are made by `learn` (the facts request), `drafted`
39(the LLM) and `judge` (Jev judging what the LLM wrote), and every one of them
40goes through the archive. `glance` is the as-you-type request behind
41`/gate`. `answer` records what was asked so it can join the feed.
42
43## 2. archive.rs: the memory, and the only door out
44
45The archive is a Durable Object, and it is the only code that sends a
46request to Jev or the LLM. Ask it for a call and it either hands back the
47response it kept from last time, or makes the call once, keeps it, and hands
48that back. Two visitors asking the same new thing at once share one call
49(`once`). Chapter 9 is about why.
50
51It also keeps the feed of questions (`asked`), the tallies (`counts`), and
52the `/live` sockets of every open page, and it pushes the toasts.
53
54## 3. meter.rs: the budget object
55
56The other Durable Object. Before a call goes out, the archive asks it to
57*hold* the call's worst-case cost; after, to *settle* it at the real cost.
58It also reads the whole Cloudflare account's AI usage, so a dev server or
59the eval spending the same free allocation is counted too, and it keeps
60each visitor's per-minute count, in memory only. Chapter 10 has the
61arithmetic.
62
63## 4. object.rs and ai.rs: the plumbing
64
65`object.rs` posts a JSON message to a Durable Object and reads one back.
66`ai.rs` calls Workers AI through the `AI` binding, with JSON text in and
67JSON text out, so the reply can be shown exactly as it came.
68
69## 5. view.rs and view/: everything you see
70
71`view.rs` turns a `View` (what has happened so far) into HTML with maud. It
72is pure: no network, no clock, so its tests run on your laptop. The answer
73stamps, the bars, the panels that show every request and response
74pretty-printed, the home page's feeds: all here. Chapter 4 covers `view/`.
75
76`diagram.rs` draws the rules engine as an SVG, marked with what is known
77about the current question: what held, what failed, which rules fired and in
78what order.
79
80When the ending is one Jev cannot take (the gate said it is not a question,
81or the LLM wrote nothing Jev can answer), the explanation gets a block: "You
82would have Please choose an LLM instead:", three large coloured pixel logos (shuqikhor's, MIT; see chapter 15⅞), Claude,
83ChatGPT and Gemini, each a link to "Let Me MeatProxy That For You"
84(<https://lmmptfy.com/>) carrying the visitor's own words in the link's
85fragment, which no server sees; and under "Provided by:" a preview card of
86that site, as Discord would draw it. The Worker builds the card itself: it
87fetches `https://lmmptfy.com/` (the only address it ever fetches, with a
882.5 second limit and 64 KB read), reads the `<head>`'s `og:` tags and keeps
89them for an hour per isolate. The card image is served from our own
90`/meatproxy/card`, not linked, so the visitor's browser never asks the other
91site for anything; the route takes nothing from the request and re-derives
92the address from the kept head. If the site cannot be read, the icons show
93without the card. `meatproxy.rs` holds the format, the reading and the
94route. An outage (`Failed`) does not get any of it, and a question too long
95for the site's limit gets none rather than a cut one. The link preview and
96the feed read the archive, not this block, so they are unchanged.
97
98## 6. playground.rs: `/rules`
99
100The rules with facts you set by clicking: every test in the diagram is a
101link that sets its fact. It runs the same engine and asks nobody anything.
102
103## 7. clone.rs and browse.rs: the code, shared
104
105`clone.rs` is a tiny git server, or rather a forwarder: it passes a clone's
106two requests to GitHub with a read-only token and refuses everything else.
107The same address in a browser is the code's front page. `browse.rs` reads
108folders and files from GitHub's API so they can be shown as pages, like this
109one.
110
111`release.rs` is the same forwarding for a repository's GitHub release files:
112`code.lmjtfy.fun/whiskers/latest/arm64.apk` streams the latest release's
113arm64 build, from a private repository, to anyone. The route parsing, the
114file kinds and the headers are pure and tested; the chapter on the Worker has
115the rules.
116
117## 8. page.css and page.js
118
119The look, copied on purpose from typesafe.ai: near-black on white, one pink
120band, dithered dot fields, old-computer windows. And the only JavaScript the
121site has: typing a `?q=` question in, the copy buttons, the `/live` socket
122(online count, toasts, live feeds, the reload offer), and enlarging a
123diagram on a click, and telling `/seen` how long a page was in view and how
124far down it was read as it is put away. `live.js` is the SharedWorker that holds that socket
125once for all of a browser's tabs.
126
127## 9. events.rs: what happened, kept
128
129As the Worker serves a request it makes an event of it: what it was, what
130came of it, and everything the request and Cloudflare said about where it
131came from. The archive keeps each one as a row. Nothing on the site shows
132them; the owner reads them from a separate admin Worker. Chapter 2 says
133what is in a row, and chapter 9 has the event itself.
134
135> **Aside.** Keeping the row must not slow the page. So the Worker answers
136> first and writes after: `wait_until` lets a Worker finish a job once the
137> response has left.
138
139## For the people who maintain it
140
141| File | What |
142| --- | --- |
143| [lib.rs](lib.rs) | The routes, and the loop that does what the rules say next, streamed as the transcript. |
144| [archive.rs](archive.rs) | The `Archive` Durable Object: every answered request keyed by the exact body, the calls themselves, the feed, the tallies, the `/live` sockets. |
145| [meter.rs](meter.rs) | The `Budget` Durable Object: hold and settle, the account's usage, per-visitor counts. |
146| [object.rs](object.rs) | Posting a JSON message to a Durable Object. |
147| [ai.rs](ai.rs) | The Workers AI binding, JSON text in and out. |
148| [past.rs](past.rs) | What an object's storage can do that workers-rs has no binding for: name a moment in its last thirty days, go back to it, and run a migration step as all or nothing. |
149| [view.rs](view.rs) | The page and the transcript as HTML. A `View` in, markup out. |
150| [view/](view/) | The code pages, the toasts, and `view.rs`'s tests. |
151| [diagram.rs](diagram.rs) | The rules as an SVG of the Rete network, and `/rules.svg`. |
152| [playground.rs](playground.rs) | `/rules`: facts set by the link. |
153| [meatproxy.rs](meatproxy.rs) | The links to lmmptfy.com for a question Jev cannot take: the base address, the providers and the exact encoding, plus the preview card read from its `<head>` and the `/meatproxy/card` route. The parsing and markup are pure; only the two fetches use `worker`. |
154| [host.rs](host.rs) | Which address a request came to (`lmjtfy.fun`, `code.lmjtfy.fun`, the old ones) and what each does: the redirects, and which routes each has, under the `GIT_REDIRECT` switch (`GitRedirect`, on or off). Pure. |
155| [clone.rs](clone.rs) | `git clone`: smart HTTP forwarded read-only; the code page; the latest commit; clone and pull counting. |
156| [release.rs](release.rs) | `/<name>/latest/<kind>` and `/<name>/releases/<tag>/<asset>`: a repository's GitHub release files, looked up and streamed with the read-only token (never sent to the signed address), plus the two listing pages' data. Routes, kinds, content types and ranges are pure. |
157| [browse.rs](browse.rs) | `/lmjtfy.git/<path>`: folders and files from GitHub's contents API, kept a minute per isolate; jevcrates at its pin. |
158| [page.css](page.css), [page.js](page.js) | The look, and the browser code. |
159| [events.rs](events.rs) | A request as an `archive::Event`, with Cloudflare's account of where it came from, and the sending of it to the archive. |
160| [live.js](live.js) | `/live.js`: the SharedWorker that holds one `/live` socket for every tab of a browser. |
161
162The invariants for all of these (what must never await, what must stay
163pure, where calls may be made) are in the Worker's CLAUDE.md, one folder up.
164
165← Previous: [Chapter 2, the Worker](../) · Up: [apps/lmjtfy](../) · Next: [Chapter 4, view/](view/) →