v1 end of life: raw.datafornerds.io will be retired on November 1, 2026. Migration guide →

About DataForNerds

The problem

Useful reference data is scattered across the web, buried in HTML tables, navigation menus, and documentation pages that change without notice. Every time you need this data in a script, a report, or an application, you end up writing a one-off scraper, copy-pasting from a webpage, or maintaining a spreadsheet that’s perpetually out of date.

The solution

DataForNerds collects this data for you. Automated collectors run daily via GitHub Actions, fetch the relevant source pages, normalize the output into consistent JSON schemas, and upload the results to a Cloudflare R2 bucket. A Cloudflare Worker serves the data via a simple REST API.

The result: a single GET request returns structured, up-to-date data as clean JSON.

How it works

  1. Collect - Automated collectors run on a daily schedule in GitHub Actions. Each collector targets specific public data sources.
  2. Validate - Output is validated against JSON Schemas, checked for minimum record counts, and compared against the previous run to detect changes.
  3. Publish - Validated data is wrapped in a standard envelope (metadata + data) and uploaded to Cloudflare R2.
  4. Serve - A Cloudflare Worker serves the data from R2 with caching headers, ETag support, and CORS.

Data freshness

Collectors run daily. Each run includes retry logic (3 attempts with linear backoff) and automatic failure reporting via GitHub Issues.

The lastCollected field in each dataset’s metadata tells you when the collector last ran. The lastModified field tells you when the data actually changed.

About the author

DataForNerds is built and maintained by Chris Kibble, an IT professional who got tired of writing the same web-scraping scripts over and over. What started as a personal solution to a recurring annoyance turned into a project worth sharing.

The journey

The first version of DataForNerds was an exercise in learning GitHub Actions. The collectors, the workflows, the validation logic: all of it was built as a way to explore what GitHub Actions could do. The data was useful, but the real point was the automation.

Version 2 is a different kind of experiment. The infrastructure layer, the Cloudflare Worker, the API design, the v1 compatibility layer, the JSON Schemas, and this website were built in collaboration with Claude, Anthropic’s AI assistant. Claude wrote a significant amount of the v2 code, from OpenTofu configurations to TypeScript Workers to Astro components. The conversation-driven development process, where architecture decisions and implementation happen in the same thread, turned out to be a fun way to rewrite the solution.

Open source

The collector scripts, workflows, infrastructure code, and this website are open source. Contributions, bug reports, and feature requests are welcome on GitHub.