Configuration¶

The RPZ downloader reads a per-instance YAML config file. The default path on a deployed box is /etc/pdns-pf/oxfeed-rpz-downloader/<instance>.yml, where <instance> matches the systemd template instance name. A commented template with every option is shipped as /etc/pdns-pf/oxfeed-rpz-downloader/example.yml.

Environment variables in YAML values are only expanded when the service is started with --expand-env. Use this to inject the TSIG secret through the unit's environment file:

feed:
  rpz_downloader:
    tsig_secret: "${RPZ_TSIG_SECRET}"

The shared config struct also exposes feed.command_source and feed.http_upstream sections, but the rpz-downloader binary only ever instantiates the RPZ source. Setting feed.rpz_downloader is mandatory; the other two source blocks should be left out. Use the dedicated pdns-pf-oxfeed-commandsource and pdns-pf-oxfeed-httpapi-downloader services for those sources.

Top-level structure¶

Parameter	Type	Default	Description
`run_once`	`boolean`	`false`	Exit after a single successful update cycle.
`feed`	`Feed`	—	Where and how the feed is written, plus the RPZ source itself. Required.
`http`	`HTTP`	`{}`	Status / metrics HTTP server. Disabled if `address` is empty.
`log`	`Log`	see section	Log format and level.

Deprecation policy¶

No YAML fields have been deprecated or removed.

A handful of command-line flags from the retired Python version are still accepted as drop-in replacements; see Operations → Deprecated flags for the mapping. New deployments should configure everything through the YAML file.

Full example config¶

Click to expand the shipped example config

# One may want the client to exit immediately after a run.
#run_once: true

# Configures where and how the feed directory with feed.json is written
feed:
    # Path where the feed is saved to.
    # This is equivalent to `simpleblob://fs?root_path=/tmp/feed`
    # or `file:///tmp/feed`.
    # As an alternative, the following format is accepted:
    #root:
    #    # The kind of storage.
    #    # One of "fs", "memory", or "s3"
    #    type: fs
    #    # Options for storage, passed on to simpleblob.
    #    # Backend "memory" takes no option.
    #    # See the following links for "fs" and "s3" options:
    #    # https://pkg.go.dev/github.com/PowerDNS/simpleblob/backends/fs#Options
    #    # https://pkg.go.dev/github.com/PowerDNS/simpleblob/backends/s3#Options
    #    options:
    #        root_path: /tmp/feed
    root: /tmp/feed

    # Optional. Allows trying again storage operations.
    retry:
        # Duration for which consecutive network errors are tolerated.
        # After this, an error will be returned.
        #timeout: 5m
        # Time to wait in case an error occurred while reading/writing.
        #interval: 20s
        # Optional. Maximum random time to add to interval to avoid multiple retries at once.
        #max_jitter: 1s
        # Optional. Static time to add to the interval for each failure.
        #backoff: 2s

    # Make every code have its own namespace.
    # WARNING: Changing this later will trigger a full feed UUID reset!
    #namespace_per_code: false

    # Enable internal memdb stores pruning.
    #prune_enabled: false

    # Time between two memdb stores prunings.
    #prune_interval: 24h

    # RPZ-specific configuration values
    rpz_downloader:
        # Address of the DNS server to send *XFR queries to, including port.
        address: rpz-server-addr:53

        # DNS NS value, required for incremental transfers (IXFR).
        zone: localhost.

        # Time to wait after a fetch before another fetch is done.
        #interval: 30m

        # Minimum time between two AXFR queries.
        # `0` (zero value) means "never".
        #axfr_interval: 24h

        # Duration for which consecutive network errors are tolerated.
        #max_network_failure_duration: 24h

        # Time to wait in addition of `interval`, in case a network
        # error occurred.
        #backoff: 10m

        # Duration after which an attempt at opening a connection to `address`
        # is considered a failure. `0` means it will not time out.
        #dial_timeout: 2m

        # Duration after which reading from `address` fails.
        #read_timeout: 15m

        # Duration after which writing to `address` fails.
        #write_timeout: 2m

        # Base64 secret for signed transaction to `zone`.
        # If defined and not empty, TSIG will be used.
        #tsig_secret: ""

        # Supported message digest algorithms are: hmac-sha1, hmac-sha224,
        # hmac-sha256, hmac-sha384 and hmac-sha512. hmac-md5 is not accepted.
        #tsig_algo: hmac-sha256

        # Time margin accepted when checking signature time.
        #tsig_fudge: 300s

# The HTTP server exposes Prometheus /metrics and a status page
http:
    # Address to listen on. Disabled by default.
    #address: ":9605"
    # Serve feed under /feed. Not for production use!
    #serve_feed: false

log:
    level: info       # options: trace, debug, info, warning, error, fatal
    format: human     # options: human, logfmt, json
    timestamp: short  # options: short, full, disable

`feed`¶

The feed section configures both the destination of the produced feed and the RPZ source that drives it.

Parameter	Type	Default	Description
`root`	`string` / `Root`	—	Storage destination for the feed tree. Required.
`retry`	`retry.Config`	`{}`	Retry policy for storage operations.
`namespace_per_code`	`boolean`	`false`	Place each classification code in its own namespace. Changing this resets the feed.
`rpz_downloader`	`RPZDownloader`	—	DNS *XFR settings. Required.
`expiry`	`ExpiryConfig`	see defaults	Snapshot and delta retention rules.
`snap_interval`	`go: Duration`	`1h`	Force a fresh snapshot at this cadence.
`prune_enabled`	`boolean`	`false`	Compact the in-memory feed state periodically.
`prune_interval`	`go: Duration`	`24h`	Time-based pruning trigger. Only consulted when `prune_enabled` is `true`.

`feed.root`¶

The simpleblob storage where the feed tree is written. Accepts either a string URI / path or a struct.

String form (the recommended default):

feed:
  root: /var/lib/pdns-pf-oxfeed-rpz-downloader/feeds/my-zone

A bare filesystem path is equivalent to file:///path and to the simpleblob://fs?root_path=/path URI.

Struct form, for backends that need options:

feed:
  root:
    type: fs        # one of "fs", "memory", "s3"
    options:
      root_path: /var/lib/pdns-pf-oxfeed-rpz-downloader/feeds/my-zone

The --oxfeed-path CLI flag overrides this field at startup. See simpleblob/backends/fs and simpleblob/backends/s3 for backend-specific options.

`feed.retry`¶

Wraps every storage call (snapshot write, delta write, codes write, cleanup) in a retry loop. All durations are optional and an empty section disables retries entirely.

Parameter	Type	Default	Description
`timeout`	`go: Duration`	—	Total time window during which retries are attempted. Must be strictly greater than `2*interval + max_jitter + backoff`.
`interval`	`go: Duration`	—	Time to wait after a failed attempt before retrying. Minimum 10 ms.
`max_jitter`	`go: Duration`	`0`	Random extra delay added on top of `interval`. Minimum 2 ms when non-zero.
`backoff`	`go: Duration`	`0`	Static delay added to `interval` for every failure. Minimum 1 ms when non-zero.

`feed.namespace_per_code`¶

When true, every classification code gets its own LMDB namespace on downstream clients.

Warning

Changing namespace_per_code after a feed has been written resets the feed UUID, which forces every downstream client to re-snap from scratch. Decide this value once per feed and leave it alone.

`feed.rpz_downloader`¶

The DNS source. address and zone are required; the rest have sane defaults.

Warning

tsig_secret is a credential. Use --expand-env and reference it as ${RPZ_TSIG_SECRET} from a systemd drop-in or environment file instead of writing it into the YAML directly. The service masks the value to (omitted) when logging the resolved config.

Parameter	Type	Default	Description
`address`	`string`	—	Upstream DNS server in `host:port` form. Required.
`zone`	`string`	—	Zone NS (the value that appears in the SOA owner field). Required for IXFR continuation across restarts. Trailing dot recommended (`example.com.`).
`interval`	`go: Duration`	`30m`	Time between two XFR queries. Must be `> 0`.
`warn_no_xfr_interval`	`go: Duration`	`3h`	Trigger the `rpz_recent_xfr_success` healthz warning when the last successful XFR is older than this. Must be `> 0`.
`axfr_interval`	`go: Duration`	`24h`	See AXFR interval. `0` disables forced AXFR.
`max_network_failure_duration`	`go: Duration`	`24h`	After this long in a row of network errors, the service exits non-zero and is restarted by systemd.
`backoff`	`go: Duration`	`10m`	Extra delay applied on top of `interval` after a network error.
`dial_timeout`	`go: Duration`	`2m`	Timeout for opening the TCP connection. `0` means no timeout.
`read_timeout`	`go: Duration`	`15m`	Timeout for reading the XFR response. `0` means no timeout. A full AXFR over a slow link can take several minutes; leave headroom.
`write_timeout`	`go: Duration`	`2m`	Timeout for writing the XFR query. `0` means no timeout.
`tsig_secret`	`string`	—	Base64-encoded TSIG key. Empty disables TSIG. Masked in log output.
`tsig_algo`	`string`	`hmac-sha256`	TSIG algorithm. See TSIG algorithms.
`tsig_fudge`	`go: Duration`	`300s`	Signature-time error margin. Rounded to seconds; max `65535s`.

AXFR interval¶

The first XFR after startup is always AXFR (the service has no SOA serial to ask for changes against). After that, IXFR is the default and axfr_interval forces an AXFR whenever it has elapsed since the last successful XFR.

The clock does not override interval. With interval = 13h and axfr_interval = 24h, an AXFR fires every 26h (2 * interval), not every 24 h. Setting axfr_interval = 0 disables forced AXFR entirely — the only AXFR is the one at startup (and whenever the local serial is reset, e.g. by wiping the feed root).

TSIG algorithms¶

Value	Notes
`hmac-sha1`	Accepted.
`hmac-sha224`	Accepted.
`hmac-sha256`	Accepted. Default.
`hmac-sha384`	Accepted.
`hmac-sha512`	Accepted.

hmac-md5 is not accepted; the service refuses to start with an explicit error if it is configured. A trailing dot is allowed (hmac-sha256.), in line with DNS algorithm naming.

`feed.expiry`¶

Snapshot and delta retention. The default values are feed.expiry.snapshots = {must_keep_count: 3, must_keep_interval: 30m, max_keep_count: 3} and feed.expiry.deltas = {must_keep_count: 5, must_keep_interval: 6h, max_keep_count: 200, max_keep_interval: 7d}. Deltas on the upgrade path from the latest snapshot to the latest delta are always preserved, even when expiry would otherwise drop them.

Each of snapshots and deltas accepts the same four fields:

Parameter	Type	Default	Description
`must_keep_count`	`int`	`0`	Never drop the last N entries, regardless of `max_keep_*`.
`must_keep_interval`	`go: Duration`	`0`	Never drop entries newer than this age.
`max_keep_count`	`int`	`0`	Drop oldest entries beyond N newest, subject to must-keep rules. `0` means no count limit.
`max_keep_interval`	`go: Duration`	`0`	Drop entries older than this age, subject to must-keep rules. `0` means no age limit.

Example: keep the last three snapshots and a week of deltas, capped at 200 to bound disk usage.

feed:
  expiry:
    snapshots:
      must_keep_count: 3
      must_keep_interval: 30m
      max_keep_count: 3
    deltas:
      must_keep_count: 5
      must_keep_interval: 6h
      max_keep_count: 200
      max_keep_interval: 168h

`feed.snap_interval`¶

How often to force a fresh snapshot even when the content has not changed enough to demand one. Shorter intervals shrink the delta chain that newly connecting clients must replay but produce more snapshot churn on disk. Set to 0s to write a snapshot on every accepted change (rarely useful — clients usually only need one snapshot per day).

`feed.prune_enabled`¶

When true, the service periodically compacts the in-memory feed state to recover stringstore space taken by deleted domains. See Architecture → In-memory pruning for trigger conditions and the CPU cost.

`feed.prune_interval`¶

Time-based prune trigger. Only consulted when prune_enabled is true. 0s disables the time-based trigger; mutation-based pruning still applies.

`http`¶

Parameter	Type	Default	Description
`address`	`string`	—	Bind address in `host:port` form (for example `:9605`). Empty disables the HTTP server entirely.
`serve_feed`	`boolean`	`false`	Expose the feed tree under `/feed`. See warning below.

--http-address on the command line overrides http.address.

Warning

serve_feed is intended for development and one-shot testing. The shipped status server has no ETag support, no auth, and no rate limiting — it is not safe to expose to clients. In production, write the feed to a directory and front it with a proper static HTTP server (Caddy or nginx).

`log`¶

Parameter	Type	Default	Description
`level`	`string`	`info`	One of `trace`, `debug`, `info`, `warning`, `error`, `fatal`.
`format`	`string`	`human`	One of `human`, `logfmt`, `json`.
`timestamp`	`string`	`short`	One of `short`, `full`, `disable`.

Configuration¶

Top-level structure¶

Deprecation policy¶

Full example config¶

feed¶

feed.root¶

feed.retry¶

feed.namespace_per_code¶

feed.rpz_downloader¶

AXFR interval¶

TSIG algorithms¶

feed.expiry¶

feed.snap_interval¶

feed.prune_enabled¶

feed.prune_interval¶

http¶

log¶

`feed`¶

`feed.root`¶

`feed.retry`¶

`feed.namespace_per_code`¶

`feed.rpz_downloader`¶

`feed.expiry`¶

`feed.snap_interval`¶

`feed.prune_enabled`¶

`feed.prune_interval`¶

`http`¶

`log`¶