Skip to content

OX Feed HTTP API Downloader

The PowerDNS Platform Filter OX Feed HTTP API Downloader (pdns-pf-oxfeed-httpapi-downloader) pulls URL-categorization data from a remote HTTP API and republishes it locally as an OX Feed. A deployment typically pairs it with a producer implementing the defined http contract on the upstream side, and serves the resulting feed directory through a static HTTP server such as Caddy or nginx to filter-node oxfeed-client instances.

What it does

  • Polls three upstream endpoints on a configurable schedule: /full (snapshot), /incremental (delta from a known version), and /codes (category code map).
  • Tracks the upstream version returned alongside each update so the next delta request asks only for changes since that version.
  • Writes each snapshot and delta into the local feed directory (feed.json, snapshot file, delta files, feed-codes.json) using atomic file operations and SHA256 checksums.
  • Applies configurable expiry rules to keep the on-disk feed history bounded.
  • Exposes a status page, Prometheus metrics, and a healthz endpoint over HTTP. Optionally serves the feed itself over the same port for development.

Where to look next

  • Architecture: how the downloader fits into Platform Filter, the upstream HTTP contract, and the concepts used throughout the rest of this site.
  • Configuration: full YAML reference — every field, its type, default, and what it controls.
  • Operations: template systemd unit, CLI flags, status page, healthz checks, and the Prometheus metrics surface.