The Movie Recommendation Bot lives in the launcher in the bottom corner of this site. Rate a handful of movies you have seen, react to a few of my words about them, and it picks one film you have not seen and pitches it in my voice, quoting a real line from my review. Everything it says is grounded in a review I actually wrote.

The recommendations were never really the point. The point was to see whether I could self-host the entire thing. Not just the site and the API, but the model too. The interesting part, the piece that reads a review and writes a pitch, runs on a small open model on a machine I control, instead of being handed off to a giant model running on someone else’s servers. This is the writeup of how it fits together.

The site is the database

There is no separate database of movies. The reviews are the database. Each review is a Markdown file with a little front matter and a body:

+++
title = "..."
date = ...
rating = "★★★★☆"
categories = ["movies"]
+++

★★★★☆
Runtime | Rating | Release | Distributor
*A one-line tagline.*




The full review text.

When Hugo builds the site, it renders every review twice. Once as the HTML page you read, and once as JSON. One catalog file lists every film, and each review also gets its own JSON file carrying the raw text:

// /movies/index.json   (the catalog)
[ { "slug": "...", "title": "...", "year": 2025, "rating": 5, "tagline": "..." }, ... ]

// /movies/<slug>/index.json   (one review)
{ "slug": "...", "title": "...", "rating": 5, "review": "the full review text ..." }

No scraping, no HTML parsing, no export step. The JSON is a first-class output of the same build that publishes the site. Post a review and its machine-readable version exists the moment the page does.

The publishing pipeline

A review is pushed to GitHub, built by Hugo in Actions, and served as HTML and JSON from GitHub Pages and a CDN.
Publishing is a git push. The same build that renders the site emits the JSON the recommender reads.

Publishing is a git push. I write a review, commit it, and push. GitHub Actions builds the site with Hugo and deploys it to GitHub Pages, which serves everything from a CDN. The whole thing is live in a minute or two.

The recommender never redeploys when I post a review, because it reads the published JSON directly. New films become recommendable on their own, with no extra step and nothing to keep in sync.

Everything runs on infrastructure I control

The recommender is one always-on machine. Two things run on that single box, side by side: a small API, and the model itself.

The API is a thin service that reads the site’s JSON, assembles what the model needs, and makes the calls. The model is Qwen2.5-0.5B, served locally by Ollama on the same box and baked into the machine image so it is ready the moment the box boots. When the API needs the model, it calls localhost. Nothing leaves the machine.

That is the whole idea. The site runs on infrastructure I control, the API runs on infrastructure I control, and the model runs on infrastructure I control. There is no outside model service anywhere in the loop.

How a recommendation gets made

The widget calls the API on the Fly box, which makes two calls to the local qwen model and streams the pitch back.
Two model calls, both to the model running on the same box.

You rate about five films and react to a few of the adjectives I used on them. The widget sends that to the API, which does the work in two model calls.

The first call builds a short profile of your taste from what you rated, then hands the model a shortlist of my highly rated films and asks for the best match. The API takes the film it names and fetches the full text of that one review.

The second call gives the model that review and asks for a short pitch in my voice, then streams the answer back to you token by token as it is written. Both calls hit the local model. The only thing the box ever stores is a rate limit.

Teaching a very small model to behave

A half-billion-parameter model is tiny. Left alone with a big prompt it wanders, invents quotes, and ignores instructions. Most of the work was shaping the job so a small model could not get it wrong.

  • Give it less to read. The candidate list is trimmed to a few dozen titles before the model ever sees it. A short list keeps the pick fast and keeps the model from losing the thread partway down a long catalog.
  • Do not ask it to quote. A small model cannot reliably copy a sentence out of a long review. So the code finds a strong, quotable line itself and hands that exact line to the model. The model only writes the framing around a quote it was given, never one it has to remember.
  • Ask for structure, then forgive it. The pick comes back as JSON. The parser strips stray formatting, and if the model wraps or trails the object with chatter, it digs the JSON back out of the noise.
  • Repair the output. If the model still drifts off the line it was handed, the code snaps the quote back to the closest real sentence in the review. Leftover formatting and dashes get cleaned before anything reaches you.
  • Keep it on a short leash. Low temperature and a hard cap on length keep it terse and predictable instead of rambling.

None of these make the model smarter. They make the task small enough that a small model does it well. What comes out is a pitch that sounds like me and never cites a line I did not write.

The widget

The chat box is a small React app written in TypeScript and built with Vite. The interface uses the Cloudscape design system, so the conversation, the buttons, and the streaming bubbles behave like a real product rather than a toy. The build compiles everything down to a single self-contained JavaScript module and one stylesheet, which is all the site needs to run it.

It loads lazily. A tiny snippet in the site footer pulls the widget in only when it is wanted, with a content hash in the filename so a new build refreshes on its own and nobody has to clear a cache. Once it is running it reads the movie catalog straight from the site’s JSON to show you films to rate, sends your ratings and reactions to the API, and renders the pitch as it streams back one word at a time. The whole interview is a small state machine, so it always knows where you are and what to ask next.

The API

The service behind it is deliberately thin. It is a FastAPI app in Python, run by uvicorn, using httpx to fetch and Pydantic to check every request. There are only a handful of endpoints, each with one job: read a review’s adjectives, pick a film, write the pitch, note whether we agree, report health. It runs on the same always-on box as the model, reads the site’s published JSON live, and calls the model on localhost. The pitch comes back over a streamed connection, so you see it as it is written.

Its limits are the honest kind. One modest CPU and one tiny model mean it is not fast and it is not clever, and it only ever recommends films I rated highly. It also looks after itself: it validates what comes in, throttles how often any one visitor can call it, and caps how much it will do, so it cannot be hammered or run up a bill. The exact knobs there I keep to myself.

The fun of it

That is the whole thing. A static site that doubles as a database, a thin API, and a half-billion-parameter model, all running on infrastructure I control, and none of it leaning on a giant model hosted somewhere else. It reads my actual reviews, matches your taste to mine, and writes a recommendation in my voice, on a model small enough to fit in the memory of a phone.

None of this needed to be self-hosted. It would have been easier to hand the hard part to a frontier model and let someone else’s servers do the thinking. But there is something satisfying about making a small, cheap, fully-owned stack do real work, and about doing the extra work to make a tiny model behave instead of throwing a big one at the problem. It is a small, opinionated thing that recommends movies, and it runs entirely on my own terms. That was always the point.