When an emergency message hits your phone, it should answer five things: who's warning you, what's happening, where, what to do, and how long it lasts. We read every AlertSF message and check.
Each AlertSF message is checked against five questions. Here's how often each one gets a clear answer.
The biggest gap is time. Most alerts don't say how long a situation will last, so you can't tell if an area will be closed for an hour or the rest of the day. The second biggest gap is who sent it. Nearly every message only says "ALERTSF" — the name of the delivery system, not the agency behind the warning. You can't tell if it came from the Fire Department, SFPUC, the Weather Service, or somewhere else. Location is the thing AlertSF does well: when a message tells you to avoid somewhere, it almost always names the specific street, intersection, or neighborhood.
Decades of research on emergency warning messages has settled on the same five questions. When a warning answers all five, people act. When it doesn't, people hesitate — calling 911, checking Twitter, asking neighbors — which is exactly the delay a warning exists to prevent. Here's what each question means in plain English, and what "good" and "bad" look like in an AlertSF message.
A good answer names the agency behind the warning: "The National Weather Service has issued a Flash Flood Warning," "SFPUC is investigating a gas leak," "The San Francisco Fire Department." You can tell at a glance whose expertise is backing the claim. The bad answer is to just say "ALERTSF" — which is the name of the delivery system, not the agency issuing the warning. A reader seeing "ALERTSF" has no way to tell whether the message came from the Fire Department, the Police Department, SFPUC, the Weather Service, or an automated script. Nearly every AlertSF message falls into the bad category.
A good answer names a specific threat you can evaluate: "Fire," "Gas leak," "Flash flood," "Power outage," "Heat advisory." A bad answer uses a category word that technically describes something but tells you nothing — "Police activity," "An incident," "An obstruction." If you see "Police Activity" on your phone, it could be a traffic stop or an active-shooter response. The phrase is the same.
A good answer gives a specific street, intersection, or neighborhood — something you can match against where you actually are. "Geary Boulevard between Divisadero and Masonic" lets you know immediately whether this affects you. A bad answer uses a category of places ("citywide," "beaches and harbors," "low-lying areas") that's too broad to act on. Location is the one thing AlertSF consistently gets right: almost every message names a specific spot.
A good answer gives you a clear action: "Avoid the area," "Take shelter," "Use alternate routes and expect delays," "Treat impacted traffic lights as four-way stops." A bad answer describes a situation without telling you what to do about it — most often a "resolved" notice that simply says "the incident has been resolved" and leaves you wondering whether it's now safe to return.
A good answer gives you a time: "Until 3:45 PM," "Restoration estimated by 1:45 PM," "Expect delays through the morning commute." A bad answer says nothing about duration — so you don't know if the situation is a matter of minutes, hours, or the rest of the day. This is by far the most common failure. Most AlertSF messages give you no sense of time at all.
Every AlertSF message is in here. Search by keyword, filter by the question you care about, and click any result to see the full text.
Every six hours, a script reads AlertSF and saves every new message to a public archive. A second script reads each new message and checks it against the five questions above. Each check produces a clear/vague/missing answer, a quote from the message that supports the answer, and a pointer to the specific rule that was applied — so every score can be traced back to the exact text it came from.
The checking is automated, done by a large language model. The rules the model follows are published at alertsf.health/codebook — the same rules a human reviewer would follow, written in plain language. The model reads the rules and then reads the message; there's no hidden prompt or editorial layer between the two. Running the same message through the same rules produces the same score every time.
The five-question framework isn't new. It comes from Mileti & Sorensen (1990), a foundational Oak Ridge National Laboratory study on how emergency warning messages actually work. A 2023 follow-up by Olson and colleagues applied the same framework to more than 6,000 federal Wireless Emergency Alerts sent between 2012 and 2022, and found that fewer than 8.5% answered all five questions — a finding cited in Congressional testimony in December 2025. This site applies that same framework to San Francisco specifically.
What this site measures — and what it doesn't. We measure whether a message is clear enough to act on: does it say who, what, where, what to do, and for how long? We don't measure whether the warning was accurate (if an alert says "flash flood warning" and there's no flood, we still count the hazard as stated) or whether the message was sent on time. Those are important, but they're different questions and this site doesn't try to answer them. We also only check the English text. AlertSF publishes the same messages in Chinese, Spanish, and Tagalog; evaluating the quality of those translations is a separate study.
Everything on this page is available as a file. Pick whichever format works for you.
| Spreadsheet | SF_Alerts_Data.xlsx — every message, full text, all five scores, in Excel |
| Scoring rules | alertsf.health/codebook — the rules we apply to every message, in plain English |
| Raw archive | raw/archive.json — every AlertSF message as JSON, updated every 6 hours |
| Live scores | scored/live.json — scored messages as JSON, with per-score receipts |
| Historical snapshot | data.js — the frozen set of scored messages this page reads |
Looking for the City of San Francisco's official AlertSF sign-up? Text your ZIP code to 888-777 or visit alertsf.org.