Scraping the Surface: How Data Journalism Exposes Systemic Inequity

The mayor stood at the podium last spring and called it a "fair housing renaissance." New ribbon, new banner, the whole production. Then we pulled the eviction filings the city's own court system publishes every month, and the spreadsheet told a different story.

That gap — between the press release and the CSV — is where we live.

We're not in the room when officials write their talking points. We don't need to be. Every claim a city makes leaves a trail somewhere: a permit database, a budget line, a 311 log, a court docket exported to a file nobody expected an outsider to actually open. Our job is to open it.

The Messy Spreadsheet Is the Real Interview

Here's the part journalism schools are still catching up on. Knowing how to scrape a government registry is now as important as knowing how to ask a hard question on camera.

Public records arrive ugly. Column headers in ALL CAPS. Dates stored as text. The same neighborhood spelled four different ways. Whitespace hiding in cells like it's getting paid to.

So we clean it. We strip duplicates, standardize the ZIP codes, and reconcile the categories some clerk invented on a Tuesday in 2014. This is unglamorous work. It's also where most of the truth gets decided, because a sloppy join can turn a real pattern into noise — or noise into a fake scandal.

Math doesn't lie, but bad data does. So we check our work twice before we publish once.

When the rows are clean, the patterns surface on their own. Pull eviction filings, map them by census tract, then overlay median income, and you stop arguing about whether displacement is "concentrated." You can see it. The same handful of blocks, year after year, carrying a load the rest of the city never feels.

Following the Money, One Row at a Time

Budgets are confessions written in numbers.

A police department can issue a statement about "community-first reform" and mean it sincerely. Then we read the line items. We watch overtime balloon while youth programs flatline, and we ask the question the press conference skipped: what did you actually fund?

We scraped the procurement portal. We matched vendor payments to contract IDs. The dataset showed a pattern of no-bid renewals that no reporter would catch reading documents one at a time, because the story wasn't in any single document. It was in the aggregate.

That's the quiet superpower here. Systemic inequity is, by definition, systemic — it hides in the totals, not the anecdotes. One family's story moves people. Ten thousand rows proving that family wasn't an exception moves policy.

We're not anti-government. We're pro-evidence. When the data backs an official up, we say so plainly, and that happens more than cynics expect.

But when the numbers and the narrative split, we go with the numbers. Reproducibly. We publish the methodology, link the raw files, and invite anyone to rerun the query themselves.

Because accountability shouldn't require trust. It should require a download link and a free afternoon.

The powerful have always had better lawyers and louder microphones. They've rarely had to worry about a stranger with a laptop, a copy of pandas, and the patience to read all 40,000 lines.

That stranger is us now. And the spreadsheet is open.