The Movie Recommendation Bot lives in the launcher in the bottom corner of the movie pages on this site. It runs a short interview: rate five films you have seen, react to the words I used about them in my reviews, and settle a few head-to-head matchups. Then it picks one film you have not seen from my four-star-and-up shelf and pitches it in my voice, quoting a real sentence from my review.

The engineering brief was strict: no outside model service. The part that reads a review and writes a pitch is Qwen2.5-0.5B, a half-billion-parameter open model served by Ollama on the same box as the API. Nothing goes to a model provider and there is no per-token bill. A model this small does not do this job on its own. Most of the engineering is in shaping the job until it cannot miss. This is the writeup of how it fits together.

The site is the database

There is no separate database of movies. The reviews are the database. Each review is a Markdown file with front matter and a body:

+++
title = "..."
date = ...
rating = "★★★★☆"
categories = ["movies"]
+++

★★★★☆
Runtime | Rating | Release | Distributor
*A one-line tagline.*




The full review text.

When Hugo builds the site, it renders every review twice. Once as the HTML page you read, and once as JSON. One catalog file lists every film, and each review also gets its own JSON file carrying the raw text:

// /movies/index.json   (the catalog)
[ { "slug": "...", "title": "...", "year": 2025, "rating": 5, "tagline": "..." }, ... ]

// /movies/<slug>/index.json   (one review)
{ "slug": "...", "title": "...", "rating": 5, "review": "the full review text ..." }

No scraping, no HTML parsing, no export step. The JSON is a first-class output of the same build that publishes the site. Post a review and its machine-readable version exists the moment the page does.

The publishing pipeline

A review is pushed to GitHub, built by Hugo in Actions, and served as HTML and JSON from GitHub Pages and a CDN.
Publishing is a git push. The same build that renders the site emits the JSON the recommender reads.

Publishing is a git push. GitHub Actions builds the site with Hugo and deploys it to GitHub Pages, and the pages and the JSON go out together. The recommender reads the published JSON directly, so posting a review never redeploys anything. The API re-fetches the catalog on a ten-minute ETag loop, and a new film is recommendable as soon as that refresh lands.

One box

The recommender is one always-on Fly machine: one shared CPU, 1 GB of RAM, about six dollars a month. Two processes run on it side by side. uvicorn serves the API, and Ollama serves Qwen2.5-0.5B, pulled into the machine image at build time so the model is ready the moment the box boots. When the API needs the model, it calls localhost. Nothing leaves the machine.

Memory is the binding constraint. Qwen peaks around 700 MB with its weights loaded, which is most of the box. That number decided which models were even candidates.

How a recommendation gets made

The widget sends the interview to the API on the Fly box. The pick and the pitch go to the qwen model on the same machine, and the finished pitch is replayed to the browser over SSE.
Every model call terminates on this box.

The widget runs the interview and the model is called only where language is involved. The full accounting, per visitor:

  • One adjectives call per film you rate. Rate a film and the API fetches that review’s JSON and asks the model for exactly eight lowercase adjectives describing how the review characterizes it. They come back as chips you agree or disagree with. My words, your verdict.
  • Zero calls for my reactions. The line I answer each rating with is code. Your stars are compared against mine, a stance falls out (agreement, you above me, me above you), and a line is drawn from a pool with repeats excluded. It does not take a language model to disagree with you.
  • Zero calls for the matchups. The head-to-heads pair one of my three-star films against one of my four-stars, recent and recognizable, with the higher-rated side alternated so position never leaks my rating. Pure code.
  • One pick call per proposal. The API flattens your ratings, reactions, and matchups into a plain-text taste profile. The model gets that plus thirty candidates from the four-star shelf, one per line as “slug :: Title (Year)”, and returns five slugs as JSON. The first valid slug wins. Films you rated, both films from every matchup, and anything already proposed are excluded before the model ever sees the list. If the model returns garbage, the fallback sorts the shelf by rating and recency and takes the top. It never dead-ends.
  • A proposal costs nothing. The bot names its pick and asks whether you have seen it before writing a word. “I’ve seen it” adds the film to the exclusion list and re-picks. Only “not yet” spends a pitch.
  • One pitch call. The model gets the review text and one pre-selected sentence from it, and writes a short first-person pitch built around that exact quote.

The pitch does not stream to your screen live, and that is deliberate. The output has to be checked before anyone sees it, so the API collects the model’s complete text, repairs it, then replays it over a server-sent-events connection at one word every 20 milliseconds while the widget types it out. What you see is a verified draft played back, not a live wire to the model.

Teaching a very small model to behave

A half-billion-parameter model is tiny. Left alone with a big prompt it wanders, invents quotes, and ignores instructions. The fix is never to make the model smarter. It is to make every task small enough that the model cannot miss.

  • Give it less to read. The pick call sends thirty titles, not the whole shelf. A short list keeps the pick fast and keeps the model from losing the thread partway down a long catalog.
  • Never let it choose the quote. A small model cannot reliably copy a sentence out of a long review. So the code chooses. It splits the review into sentences, keeps the ones between 40 and 160 characters, and takes the first that hits a list of praise keywords, falling back to the longest. The model gets that exact line and writes two sentences of framing around it.
  • Ask for structure, then forgive it. Picks come back as JSON with format enforcement on. The parser still strips code fences, and when the model wraps the object in chatter, it digs the first balanced JSON object back out of the noise.
  • Repair, do not trust. If the quoted line in the output drifts from the one the model was handed, the code compares it against every sentence of the review with difflib and snaps it to the closest real one above a 0.6 similarity ratio. Markdown gets stripped and stray dashes become sentence breaks.
  • Keep it on a short leash. Temperature 0.2 on the structured calls, 0.3 on the pitch, and a hard 220-token output cap. Terse and predictable beats creative and wrong.

None of this makes the model smarter. It makes each task small enough that a small model does it well. The quote in every pitch is real because the model never picks it and never has to remember it.

The widget

The chat is a React 18 app in TypeScript, built with Vite in library mode. The build emits one self-contained ES module and one stylesheet into the site’s static files, and Hugo ships them like any other asset. Cloudscape supplies the buttons and the dark-mode theming, synced to the site’s theme with a MutationObserver. The transcript, the bubbles, and the chips are custom.

Nothing loads until it is asked for. The launcher appears only on the movie pages, and the first click dynamic-imports the module with a SHA-256 content hash appended to the URL at build time, so a new deploy invalidates caches by itself. The interview is a reducer state machine with named phases (gate, rate, stars, react, pairs, propose, recommend, followup), which is why the bot always knows what to ask next and nothing depends on the model behaving. Titles to rate are dealt four at a time, newest first, from the same /movies/index.json the backend reads. Progress persists in sessionStorage, so closing and reopening the panel resumes mid-interview and closing the tab forgets you.

The API

The service is one FastAPI file run by uvicorn, with httpx doing the fetching and Pydantic checking every request. Five endpoints, one job each: extract a review’s adjectives, react to a rating, pick a candidate, stream the pitch, report health.

There is no database and nothing touches a disk. The catalog refreshes from the site on the ten-minute ETag loop, review text is cached for fifteen minutes, and your interview lives in your browser. The only thing the box stores is a rate limit: a per-IP hourly window, a global daily ceiling, and a concurrency gate ahead of the model. The exact numbers are not published.

The input surface is narrow on purpose. The widget has no text box, so there is no prose for a visitor to send. Every field the API accepts is a slug matched against ^[a-z0-9-]+$, a bounded integer, or a capped list of single lowercase words, re-validated server-side. CORS is pinned to this site, and the client address is taken from Fly’s proxy header, because the first hop of X-Forwarded-For is whatever the client claims it is. You cannot type at the model, from the widget or from curl.

The whole stack

Static files on a CDN, one Python process, one 398 MB model, one box. It reads my reviews, matches your taste against mine, and pitches one film in my voice, quoting a line I actually wrote. The easy version of this project hands the reviews to a frontier model API and works on the first try. This version is harder in exactly one place, and that place is the reason it exists: making a model small enough to run on a phone do the work reliably.