Assert the artifact, not the exit code

I wanted the bestseller chart for audiobooks to show up in my library without me doing anything. Scrape the chart, diff it against what I already own, hand the gaps to the thing that does the downloading. An hour of work, in theory.

The scraping was the easy part. The interesting failures were all in the two places nobody writes tests for: what happens when the page changes shape, and what happens when the downstream API tells you something that isn't true.

Rule one: a partial parse is a failure, not a small success

The chart is five pages of twenty. The naive version stores whatever it parsed and moves on. That's how you end up with a table that says the chart had four entries today, and a monitor that's perfectly happy because the job exited 0.

rows = scrape_chart()
if len(rows) < 90:
    print(f"only parsed {len(rows)} rows — refusing to store a partial chart",
          file=sys.stderr)
    return 1

Nine lines. The point is that a markup change can now never overwrite a good snapshot with a stub. The job goes red and yesterday's data is still correct, which is exactly the trade you want from a cache of something you don't own.

Same idea one layer down: the writer deletes and re-inserts today's rows inside a transaction, so a crash halfway through leaves the previous snapshot intact rather than half of two different days.

Rule two: monitor the table, not the cron

The healthcheck I registered doesn't look at exit codes, log lines, or whether systemd ran the unit. It asks the database:

SELECT count(*) FROM charts.chart_top100
WHERE captured_on >= current_date - 1

...and expects at least 100. That assertion survives every way this job can fail while appearing to run: the scraper 403s and catches its own exception, cron fires but the wrapper can't find its venv, the parse silently degrades to zero items, someone comments the line out "temporarily." A liveness probe catches none of those. I've been burned by exactly this before — a poller that logged "poll ok: 0 new" every sixty seconds for weeks while the feed behind it was stone dead. Every probe was green. Nothing had arrived since the first week of the month.

If your monitor can't tell the difference between working and running, it isn't monitoring the thing you care about.

The two lies the downstream API told

Lie one: the top search result is the book. Searching a title plus its author returned, at rank one, a forty-page cash-grab called SUMMARY AND ANALYSIS OF <the real book> by a publisher that exists only to make those. Searching the bare title returned the real thing first. Adding the author to the query made the results worse, which is not how anyone expects search to behave. The fix is a filter that would be insulting if it weren't necessary:

SUMMARY_JUNK = re.compile(
    r"\b(summary|analysis|workbook|study guide|conversation starters|"
    r"key takeaways|sidekick|trivia|companion)\b", re.I)

...plus asserting that the returned author's surname actually matches the one I asked for. Two of the first five things I added by hand were parasite listings. I only noticed because the titles were shouting in caps.

Lie two: HTTP 500 means it failed. Adding a book from a new author makes that app import the author's entire bibliography as unmonitored rows. Add a second book by the same author ten seconds later and you get:

HTTP 500: 23505 duplicate key value violates unique constraint
          "IX_Editions_ForeignEditionId"

Which reads like a crash and means "it's already here." The recovery is to believe the error instead of retrying it: look the book up by its foreign ID under that author, flip monitored to true, and fire the search command yourself.

except urllib.error.HTTPError as e:
    if "IX_Editions_ForeignEditionId" in msg or "23505" in msg:
        got = adopt_existing(book)   # find by foreignBookId, monitor, search

Six of the first batch of thirty went down this path. Treated as failures they'd have been retried every night forever, each retry producing the same 500.

The bit I'd keep

Statuses are a small closed set — added, owned, no_match, retry, failed — and the distinction between no_match and retry is the one that earns its keep. The metadata provider behind that API times out for minutes at a stretch. A timeout recorded as no_match is a permanent lie: the row gets written off and never looked at again. Recorded as retry, it's picked up on the next run and usually resolves on the first attempt.

A timeout is not an answer. Don't let your schema pretend it was one.