The probe that was never allowed to fail
A monitoring flag meant to hide noisy checks was quietly hiding the most important one. Six automations were green because nothing they ran could ever be reported.
Read →Chester Frazier · Gulfport, MS · b. 1988
Notes from the homelab,
the studio, and the day job.
Debugging stories, architecture notes, and the occasional thing I built because nobody else had. New posts when I have something worth writing down.
Posts
14 on this page · full archive (298)
The keepalive nobody could see
A wall-mounted dashboard froze on a job tile for eleven hours. The SSE socket was dead, EventSource said it was open, and the server's keepalive was a comment JavaScript never sees.
Read →The fix shipped three weeks ago. It never arrived.
A bug report I had already fixed — except the build carrying the fix never reached the phone. How to find the real client from an access log, and why a version number should be read out of the artifact.
Read →The database nobody wanted backed up took the other 41 with it
A nightly Postgres backup shipped nothing to four destinations because one frozen snapshot database had a connection limit of zero. The fix is a batch-job design rule, not a Postgres trick.
Read →The alert that fired because the system was working
A monitor paged every day for a week while the automation it watched was deliberately waiting. The bug wasn't in either one — it was that they shared a threshold.
Read →dig writes its errors to stdout, and my monitor believed them
A DNS probe paged claiming my records were wrong. They weren't. dig prints its own failure text on stdout, and my parser kept it as an answer.
Read →Lag is not drift: my replication monitor was green while the replica was wrong
A replication healthcheck that watched lag, cadence and error counts stayed green for months while the standby held stale text, missing rows and mojibake. Here's why bookkeeping can't see it.
Read →The screen that watches you back
My TV dashboard shows my agent's live activity whenever the agent is running — which makes it impossible for the agent to screenshot the dashboard. The fix was to move the shutter.
Read →The prompt parameter that defeated single sign-on
Users kept asking why a portal made them sign in when Microsoft's own apps never do. The answer was one OAuth parameter overriding the thing that would have signed them in silently.
Read →The radio knows something GPS doesn't
Building a two-person finder for a crowded venue: why a Bluetooth contact beats a GPS fix at close range, and why the app still refuses to print metres from an RSSI.
Read →The boot loader that phoned home
A PXE server that broke every few weeks turned out to be iPXE falling back to a hardcoded HTTPS URL. The fix was to stop letting DHCP decide where the boot menu lives.
Read →Reminders that fire on relevance, not on a clock
Every reminder system I've used is time-anchored. 'Next time I hire a handyman' has no time. Here's the trigger-anchored half — keywords, geofences, and payday — wired into an assistant's turn.
Read →Every read worked. Every write failed.
An automation could read all of Active Directory and change none of it. The rights were fine — PowerShell had quietly connected to a different domain controller.
Read →Two ways a CloudWatch alarm quietly stops meaning anything
An alarm with no OK action sits red forever and becomes furniture. And a 24-hour Period is a fixed UTC bucket, not a rolling window — so it false-pages every morning.
Read →