Configuration¶

The command source reads a per-instance YAML config file. The default path on a deployed box is /etc/pdns-pf/oxfeed-commandsource/<instance>.yml, where <instance> matches the systemd template instance name. A commented template with every option is shipped as /etc/pdns-pf/oxfeed-commandsource/example.yml.

Environment variables in YAML values are only expanded when the service is started with -expand-env. With expansion enabled you can write command: "some-tool --token ${SOURCE_TOKEN}" and supply the variable through the unit's environment file.

The shared config struct also exposes feed.http_upstream and feed.rpz_downloader sections, but the command source binary refuses to start without feed.command_source.command and never instantiates those sources. Use the dedicated pdns-pf-oxfeed-httpapi-downloader and pdns-pf-oxfeed-rpz-downloader services for those sources.

Top-level structure¶

Parameter	Type	Default	Description
`run_once`	`boolean`	depends	Exit after a single update cycle. Defaults to `true` when no `-config` is given and `false` when a config file is loaded.
`feed`	`Feed`	—	Where and how the feed is written, plus the command source itself. Required.
`http`	`HTTP`	`{}`	Status / metrics HTTP server. Disabled if `address` is empty.
`log`	`Log`	see section	Log format and level.

Deprecation policy¶

No fields have been deprecated or removed as of the current release.

Full example config¶

Click to expand the shipped example config

# Example config file for the pdns-pf-oxfeed-commandsource service
#
# To add a systemd service instance, you need to take these steps (replace SOMENAME):
#
#   cp /etc/pdns-pf/oxfeed-commandsource/example.yml \
#      /etc/pdns-pf/oxfeed-commandsource/SOMENAME.yml
#   ln -s /lib/systemd/system/pdns-pf-oxfeed-commandsource@.service \
#         /etc/systemd/system/pdns-pf-oxfeed-commandsource@SOMENAME.service
#

# By default we run as a service when a config file is used.
# Set this to true to only do a single run.
#run_once: false

# Configures where and how the feed directory with feed.json is written
feed:
    # Path where the feed is saved to.
    # This is equivalent to `simpleblob://fs?root_path=/tmp/feed`
    # or `file:///tmp/feed`.
    # As an alternative, the following format is accepted:
    #root:
    #    # The kind of storage.
    #    # One of "fs", "memory", or "s3"
    #    type: fs
    #    # Options for storage, passed on to simpleblob.
    #    # Backend "memory" takes no option.
    #    # See the following links for "fs" and "s3" options:
    #    # https://pkg.go.dev/github.com/PowerDNS/simpleblob/backends/fs#Options
    #    # https://pkg.go.dev/github.com/PowerDNS/simpleblob/backends/s3#Options
    #    options:
    #        root_path: /tmp/feed
    root: /tmp/feed

    # Optional. Allows trying again storage operations.
    retry:
        # Duration for which consecutive network errors are tolerated.
        # After this, an error will be returned.
        #timeout: 5m
        # Time to wait in case an error occurred while reading/writing.
        #interval: 20s
        # Optional. Maximum random time to add to interval to avoid multiple retries at once.
        #max_jitter: 1s
        # Optional. Static time to add to the interval for each failure.
        #backoff: 2s

    # Interval we want snaps to be written, 0s is on any incoming update
    #snap_interval: 6h

    # Make every code have its own namespace.
    # WARNING: Changing this later will trigger a full feed UUID reset!
    #namespace_per_code: false

    # Enable internal memdb stores pruning.
    #prune_enabled: false

    # Time between two memdb stores prunings.
    #prune_interval: 24h

    command_source:
       # The external command to provide the data for the feed (required)
       #command: some-command-source --wait
       # If the command supports waiting for changes, this interval effectively
       # rate-limits updates. If the command does not support --wait, it
       # also affects update latency and resource usage for polling.
       #interval: 5s
       # Timeout for the command run. If the command takes longer than this,
       # it will be killed.
       #timeout: 10m

    # Configure snapshot and delta expiry rules for automatic feed cleaning.
    #expiry:
    #    snapshots:
    #        must_keep_count: 3
    #        must_keep_interval: 30m
    #        max_keep_count: 3
    #        max_keep_interval: 0s
    #    deltas:
    #        must_keep_count: 5
    #        must_keep_interval: 6h
    #        max_keep_count: 200
    #        max_keep_interval: 168h

# The HTTP server exposes Prometheus /metrics and a status page
http:
    # Address to listen on. ":8910" listens on port 8910 on all interfaces.
    # Disabled by default.
    #address: ":9605"
    # Serve feed under /feed. Not for production use!
    #serve_feed: false

log:
    level: info       # options: trace, debug, info, warning, error, fatal
    format: human     # options: human, logfmt, json
    timestamp: short  # options: short, full, disable

`feed`¶

The feed section configures both the destination of the produced feed and the command source that drives it.

Parameter	Type	Default	Description
`root`	`string` / `Root`	—	Storage destination for the feed tree. Required.
`retry`	`retry.Config`	`{}`	Retry policy for storage operations.
`namespace_per_code`	`boolean`	`false`	Place each classification code in its own namespace. Changing this resets the feed.
`command_source`	`CommandSource`	—	The command to run and its scheduling. Required.
`expiry`	`ExpiryConfig`	see defaults	Snapshot and delta retention rules.
`snap_interval`	`go: Duration`	`1h`	Force a fresh snapshot at this cadence.
`prune_enabled`	`boolean`	`false`	Compact the in-memory feed state periodically.
`prune_interval`	`go: Duration`	`24h`	Time-based pruning trigger. Only consulted when `prune_enabled` is true.

`feed.root`¶

The simpleblob storage where the feed tree is written. Accepts either a string URI / path or a struct.

String form (the recommended default):

feed:
  root: /var/lib/pdns-pf-oxfeed-commandsource/feeds/my-feed

A bare filesystem path is equivalent to file:///path and to the simpleblob://fs?root_path=/path URI.

Struct form, for backends that need options:

feed:
  root:
    type: fs        # one of "fs", "memory", "s3"
    options:
      root_path: /var/lib/pdns-pf-oxfeed-commandsource/feeds/my-feed

The shipped example uses the bare-path string. The -o CLI flag overrides this field at startup. See simpleblob/backends/fs and simpleblob/backends/s3 for backend-specific options.

`feed.retry`¶

Wraps every storage call (snapshot write, delta write, codes write, cleanup) in a retry loop. All durations are optional and an empty section disables retries entirely.

Parameter	Type	Default	Description
`timeout`	`go: Duration`	—	Total time window during which retries are attempted. Must be strictly greater than `2*interval + max_jitter + backoff`.
`interval`	`go: Duration`	—	Time to wait after a failed attempt before retrying. Minimum 10 ms.
`max_jitter`	`go: Duration`	`0`	Random extra delay added on top of `interval`. Minimum 2 ms when non-zero.
`backoff`	`go: Duration`	`0`	Static delay added to `interval` for every failure. Minimum 1 ms when non-zero.

`feed.namespace_per_code`¶

When true, every classification code gets its own LMDB namespace on downstream clients. This is required for domain-list style feeds where the same domain may carry different code sets per tenant.

Warning

Changing namespace_per_code after a feed has been written resets the feed UUID, which forces every downstream client to re-snap from scratch. Decide this value once per feed and leave it alone.

`feed.command_source`¶

Parameter	Type	Default	Description
`command`	`string`	—	Shell command executed via `/bin/sh -c`. Required.
`interval`	`go: Duration`	`60s`	Time between command runs. Minimum 100 ms. See Interval sizing.
`timeout`	`go: Duration`	`10m`	Command runtime budget. The process is killed if it exceeds this. Minimum 1 s.

Interval sizing¶

If the command supports a --wait (or equivalent) mode that blocks until the source data changes, interval mainly acts as a rate limit on delta production during heavy update bursts and as a CPU guard if the command keeps exiting with an error. With --wait, update latency is dominated by how fast the source actually changes, not by this interval.

If the command always returns immediately, interval directly controls update latency and the polling load on the source.

The command contract — what it must print, the exit codes it can return, the environment it receives — is documented under Architecture → The command contract.

`feed.expiry`¶

Snapshot and delta retention. The default values are feed.expiry.snapshots = {must_keep_count: 3, must_keep_interval: 30m, max_keep_count: 3} and feed.expiry.deltas = {must_keep_count: 5, must_keep_interval: 6h, max_keep_count: 200, max_keep_interval: 7d}. Deltas on the upgrade path from the latest snapshot to the latest delta are always preserved, even when expiry would otherwise drop them.

Each of snapshots and deltas accepts the same four fields:

Parameter	Type	Default	Description
`must_keep_count`	`int`	`0`	Never drop the last N entries, regardless of `max_keep_*`.
`must_keep_interval`	`go: Duration`	`0`	Never drop entries newer than this age.
`max_keep_count`	`int`	`0`	Drop oldest entries beyond N newest, subject to must-keep rules. `0` means no count limit.
`max_keep_interval`	`go: Duration`	`0`	Drop entries older than this age, subject to must-keep rules. `0` means no age limit.

Example: keep the last three snapshots and a week of deltas, capped at 200 to bound disk usage.

feed:
  expiry:
    snapshots:
      must_keep_count: 3
      must_keep_interval: 30m
      max_keep_count: 3
    deltas:
      must_keep_count: 5
      must_keep_interval: 6h
      max_keep_count: 200
      max_keep_interval: 168h

`feed.snap_interval`¶

How often to force a fresh snapshot even when the content has not changed enough to demand one. Shorter intervals shrink the delta chain that newly connecting clients must replay but produce more snapshot churn on disk. Set to 0s to write a snapshot on every accepted change (rarely useful — clients usually only need one snapshot per day).

`feed.prune_enabled`¶

When true, the service periodically compacts the in-memory feed state to recover stringstore space taken by deleted domains. See Architecture → In-memory pruning for trigger conditions and the CPU cost.

`feed.prune_interval`¶

Time-based prune trigger. Only consulted when prune_enabled is true. 0s disables the time-based trigger; mutation-based pruning still applies.

`http`¶

Parameter	Type	Default	Description
`address`	`string`	—	Bind address in `host:port` form (for example `:9605`). Empty disables the HTTP server entirely.
`serve_feed`	`boolean`	`false`	Expose the feed tree under `/feed/`. See warning below.

-http-addr and -http-serve-feed on the command line override the matching fields.

Warning

serve_feed is intended for development and one-shot testing. The shipped status server has no ETag support, no auth, and no rate limiting — it is not safe to expose to clients. In production, write the feed to a directory and front it with a proper static HTTP server (Caddy or nginx).

`log`¶

Parameter	Type	Default	Description
`level`	`string`	`info`	One of `trace`, `debug`, `info`, `warning`, `error`, `fatal`.
`format`	`string`	`human`	One of `human`, `logfmt`, `json`.
`timestamp`	`string`	`short`	One of `short`, `full`, `disable`.

-log-level, -log-format, -log-timestamp, and -debug on the command line override the matching fields.

Configuration¶

Top-level structure¶

Deprecation policy¶

Full example config¶

feed¶

feed.root¶

feed.retry¶

feed.namespace_per_code¶

feed.command_source¶

Interval sizing¶

feed.expiry¶

feed.snap_interval¶

feed.prune_enabled¶

feed.prune_interval¶

http¶

log¶