Filtering

When enabled, filtering matches DNS traffic against policy rules to determine whether or not the filtering engine should intervene. More details about filtering can be found in the Filtering overview.

Configuration Reference

Sets of filter settings can be defined under the root node filterSettings. Each set of settings should be a key-value pair, with:

key: Name of the set
value: Dictionary holding the configuration of filter settings

filterSettings:
  mysettings:
    <Settings>

The configuration can be quite complex, as there are several concepts involved. Make sure you understand the concepts explained in this guide before attempting to configure them.

Parameters which can be used to configure filter settings are shown in the below table.

Parameter	Type	Default	Description
`debugTXT`	`boolean`	`false`	If true, answer DNS queries with filtering debug output if you request TXT records for a domain prefixed by `_filter.`. For example: `TXT _filter.some.domain.com`
`debugRules`	`boolean`	`false`	If true (and `logLevel` is set to `debug`), generate verbose rule engine logging in the dnsdist and/or recursor containers performing filtering.
`dynamic`	DynamicFiltering	`{}`	Dynamic filtering configuration
`feeds`	List of Feed	`[]`	Feeds to be consumed for use in filtering
`logFormat`	`string`	`"human"`	Format of logging. Available options: `"human"` `"json"`
`logLevel`	`string`	`"info"`	Level of logging. Available options: `"debug"` `"info"` `"warn"` `"error"`
`proxies`	List of `string`		IP addresses of proxies to be returned when traffic is filtered, unless overridden at a lower level (vHost for example). Can be a list of both IPv4 and IPv6 addresses
`static`	StaticFiltering	`{}`	Static filtering configuration
`vHosts`	Map of vHost	`{}`	vHosts configuration
`vHostService`	dnsdistService	`type: ClusterIP`	Default service object used as template for all vHost service objects. Can also be overridden on each vHost configuration

Feed configuration

When feeds are configured, the dnsdist or recursor pod which is performing filtering will have an extra container named oxfeedclient which will be responsible to download the feeds, process them and repeat when it identifies the upstream feeds have been updated. These feeds are provided by the Controlplane Feeds.

Example of a configuration with one feed:

filterSettings:
  mysettings:
    feeds:
      - url: https://commandsource-testcmdsource.controlplane.svc:8443/feed/
        namespace: testfeed
        username: myUsername
        password: myPassword
        tlsconfig:
          insecure_skip_verify: true

The above configuration will cause the oxfeedclient container to consume a feed from the url https://commandsource-testcmdsource.controlplane.svc:8443/feed/ and expose the learned categories via the naming convention [NAMESPACE NAME]/[CODE]. In this example, if the feed contains a category with code 1, it would be accessible as testfeed/1.

Additionally, it attempts to authenticate using myUsername and myPassword and will not attempt to perform any TLS verification (not recommended).

Parameters which can be used to configure feeds are shown in the below table.

Parameter	Type	Required	Default	Description
`loadFirst`	`boolean`		`false`	If true, prioritize this feed when the `oxfeedclient` container starts. If multiple feeds are configured this can be used to ensure a specific feed is loaded first.
`namespace`	`string`	`yes`		Namespace to prepend to the codes of the categories learned from this feed
`password`	`string`			Password to use for authentication with the feed. Using `secretName` is preferred
`secretName`	`string`			Name of a pre-existing Kubernetes Secret containing a username & password for authentication with the feed
`secretPasswordKey`	`string`		`"password"`	If `secretName` is specified: name of the item inside the Secret which holds the password
`secretUsernameKey`	`string`		`"username"`	If `secretName` is specified: name of the item inside the Secret which holds the username
`url`	`string`	`yes`		Full URL of the feed
`username`	`string`			Username to use for authentication with the feed. Using `secretName` is preferred
`tlsconfig`	TLSConfig		`{}`	TLS configuration options

Feed TLS config

Parameter	Type	Default	Description
`ca`	`string`		CA in PEM format to use for validation. Use of `caSecret` is recommended for security reasons
`caSecret`	`string`		Name of a pre-existing Kubernetes Secret with a data item named `ca.crt` containing the CA in PEM format to use for validation
`insecure_skip_verify`	`boolean`	`false`	Skip validation of the feed's certificate chain and hostname

Dynamic Filtering

Parameters which can be used to configure dynamic filtering settings. Important to note is that you must explicitly set enabled to true. In addition, it is highly recommended that you familiarize yourself with the Filtering Overview before proceeding with configuration.

NATS mesh

Dynamic filtering requires NATS in the Userplane to be configured to connect to the Controlplane. Information about adding this configuration can be found in the NATS Configuration Reference

Parameter	Type	Required	Default	Description
`categoryOverridesFeed`	Feed	`yes`	`{}`	Configuration of the connection towards the Controlplane `Category Overrides` feed
`deviceRegister`	Device Register		`{}`	Configuration of Device Register in `producer` mode
`domainListsFeed`	Feed	`yes`	`{}`	Configuration of the connection towards the Controlplane `Domain Lists` feed
`ecidmappings`	List of ECID Mapping		`- option_code: 65001 pre_encoding: 'binary' post_encoding: 'hex' post_prefix: 'mac:' - option_code: 65074 pre_encoding: 'string' post_encoding: 'string' name: true`	Configuration of EDNS Client ID (ECID) options parsed when Device Register is enabled. Note: When you provide any configuration under this node, you will overwrite the entire list of defaults to avoid unwanted conflicts between the defaults and your overrides.
`enabled`	`boolean`	`yes`	`false`	If `true`, enable dynamic filtering
`ipmapper`	IP Mapper		`{}`	Configuration of IP Mapper, if Radius Listener is enabled in the Userplane scope
`radiusListener`	Radius Listener		`{}`	Configuration of Radius Listener to consume RADIUS messages in the Userplane scope
`redis`	Redis Configuration	`yes`	`{}`	Configuration of the Redis read-only cluster (mirror) which replicates from the Controlplane Redis `replica` Service. Note: Since this is a `read-only` cluster it requires the `mirror` configuration towards the Controlplane with Dynamic Filtering blueprint deployed
`settingsSyncer`	Settings Syncer		`{}`	Configuration of Settings Syncer, responsible for synchronizing global settings, IP mappings and subscriber preferences to a local LMDB used by the filtering engine

ECID Mapping

You can configure an ECID mapping via the below configuration options:

Overriding ECID mapping defaults

When you provide any configuration under the ecidmappings node, you will replace all the defaults as those could conflict with your new configuration.

Parameter	Type	Required	Default	Description
`name`	`boolean`		`false`	If true, indicates this `option_code` contains a device name instead of an ID
`option_code`	`integer`	`yes`		Numerical ID of the EDNS option to look for in DNS queries
`post_encoding`	`string`	`yes`		Format to decode the contents of the parsed EDNS option to when formulating the ID or name. Available options: `"string"` `"binary"` `"hex"`
`post_prefix`	`string`		`""`	If configured, add this prefix to Device ID
`pre_encoding`	`string`	`yes`		Format to decode the contents of the EDNS option from. Available options: `"string"` `"binary"` `"ecid"` `"hex"`

Settings Syncer

You can configure the Settings Syncer via the below configuration options:

Parameter	Type	Default	Description
`lmdbLogStats`	`boolean`	`true`	If `true`, periodically log LMDB stats
`lmdbLogStatsInterval`	`go:`DurationString	`1m`	Interval between logging of LMDB stats if `lmdbLogStats` is `true`
`lmdbOptions`	`dictionary`	`{}`	Extra options to be set on the LMDB used by Settings Syncer
`lmdbScrapeSmaps`	`boolean`	`true`	Whether to scrape /proc/self/smaps to calculate LMDB memory statistics
`logFormat`	`string`	`"human"`	Format of logging. Available options: `"human"` `"json"`
`logLevel`	`string`	`"info"`	Level of logging. Available options: `"debug"` `"info"` `"warn"` `"error"`
`region`	`string`	`""`	Optional CG-NAT region for ip4 syncing
`restart`	`boolean`	`true`	If `true`, restart Settings Syncer after a Redis failure
`restartWait`	`go:`DurationString	`5s`	If `restart` is `true`, wait for this duration before restarting after a Redis failure

Static Filtering

Parameters which can be used to configure static filtering settings are shown in the below table.

Parameter	Type	Default	Description
`categories`	List of Category	`[]`	Categories to be calculated
`domainlists`	Map of DomainList	`{}`	Static domain lists
`filtersettings`	List of UserSettings	`[]`	Mapping of users, products, profiles, etc to filtering objects such as domainlists and categories
`ipmappings`	List of IPMapping	`[]`	Mapping of IP prefixes to users
`policyrules`	List of PolicyRules	`[]`	Filtering policies

Domain Lists

Domain lists allow for the configuration of static lists of domains to which filtering decisions can be applied. For example:

filterSettings:
  mysettings:
    static:
      domainlists:
        global-allow-list:
          - powerdns.com
        global-block-list:
          - evil.example
          - malware.example

The above will expose 2 static domain lists:

Domain list global-allow-list containing the powerdns.com domain
Domain list global-block-list containing 2 example domains that may be desirable to block

These domain lists can be referenced by Setting Mappings and Policy Rules using their names (ie: global-allow-list and global-block-list).

User Settings

User Settings allow for defining user settings and mapping them to filtering objects. For example:

filterSettings:
  mysettings:
    static:
      filtersettings:
        - username: user1
          filter_content: true
          named_category_lists:
            categories-denylist:
              - someCategory

Above mapping will assign the user user1 a user-scoped category list named categories-denylist which contains the category named someCategory. This user-scoped category list categories-denylist can then be referenced in policy rules.

filter_content

The filter_content parameter must be set to true to enable the filtering of defined content such as the members of the someCategory category in the above example.

Parameters which can be used to configure user settings are shown in the below table.

Parameter	Type	Required	Default	Description
`username`	`string`	`yes`		Username to match against
`timezone`	`string`			Timezone in the format `Europe/Amsterdam`, local time of the server if unspecified and `UTC` if configured with an unknown timezone
`usertags`	List of `string`		`[]`	Extra tags for the user which can be referenced in policy rules
`product`	`string`		`""`	Product used by user which can be referenced in policy rules
`filter_content`	`boolean`		`false`	If true, apply the filtering of `named_category_lists` and `named_domain_lists`
`named_category_lists`	Map of UserScopedCategoryList		`{}`	Category lists scoped to this user
`named_domain_lists`	Map of UserScopedDomainList		`{}`	Domain lists scoped to this user

A user scoped category list allows for configuring category lists which only apply to that specific user. For example:

filterSettings:
  mysettings:
    static:
      filtersettings:
        - username: user1
          filter_content: true
          named_category_lists:
            categories-allowlist:
              - socialmedia
            categories-denylist:
              - malware
              - botnets

In the above example, this user's scope has the following category lists defined:

Category list categories-allowlist references the category socialmedia with the intention of allowing this user access to social media
Category list categories-denylist references the categories malware and botnets with the intention of safeguarding the user against potentially bad actors

Policy Rules

These category lists have only been scoped to the user, they still need to be enforced by a policy rule to become active

User scoped domain list

A user scoped domain list allows for configuring domain lists which only apply to that specific user. For example:

filterSettings:
  mysettings:
    static:
      filtersettings:
        - username: user1
          filter_content: true
          named_domain_lists:
            domains-allowlist:
              - powerdns.com
            domains-denylist:
              - malware.example
              - botnet.example

In the above example, this user's scope has the following domain lists defined:

Domain list domains-allowlist references the domain powerdns.com with the intention of allowing this user access to this domain
Domain list domains-denylist references the categories malware.example and botnet.example with the intention of safeguarding the user against resolving these domains

Policy Rules

These domain lists have only been scoped to the user, they still need to be enforced by a policy rule to become active

IP Mapping

IP Mappings allow for mapping of IP prefixes to users. For example:

filterSettings:
  mysettings:
    static:
      ipmappings:
        - prefix: 10.0.0.0/8
          username: private
        - prefix: 172.16.0.0/12
          username: private
        - prefix: 192.168.0.0/16
          username: private

Above mapping will assign the user private to traffic originating from the 3 IPv4 address ranges allocated for private networks.

Parameters which can be used to configure ip mappings are shown in the below table.

Parameter	Type	Required	Default	Description
`prefix`	`string`	`yes`		Prefix to match against
`username`	`string`	`yes`		Username to assign to traffic matched against this prefix

Policy Rules

Policy rules tie together all the other objects which have been configured and are the glue which enables the actual filtering process. For example:

filterSettings:
  mysettings:
    static:
      domainlists:
        global-block-list:
          - evil.example
          - malware.example
      policyrules:
        - name: block-domain-list
          scope: all
          action: blacklist
          source: list
          source_list: global-block-list

In this example we define a domainlist named global-block-list containing 2 domains which we want to globally block for everybody.

We then have a policy rule named block-domain-list which will perform the action blacklist (ie: block) for this domainlist by specifying source to be a list and refer to the name by setting source_list to the name of our list: global-block-list.

Parameters which can be used to configure policy rules are shown in the below table.

Parameter	Type	Required	Description
`priority`	`integer`		Priority of this rule. If unset the priority is defined by the ordering of the list, where the first entry is first applied, then second, etc.
`name`	`string`	`yes`	Name of the policy rule
`action`	`string`	`yes`	Action to be taken when matched. Available options: `"whitelist"` `"blacklist"` `"monitor"`
`scope`	`string`	`yes`	Scope of the rule. Available options: `"all"` `"content"` `"security"` `"homework"` In the current release only `"all"` and `"content"` are applicable and have the same outcome
`source`	`string`	`yes`	Type of object to match against. Available options: `"category"` `"code"` `"list"`
`products`	List of `string`		Products to match against
`category_list`	`string`		If `source: category` and `source_category_mode: user`, set this to the name of the User scoped category list
`usertag`	`string`		If set, match against this usertag
`source_list`	`string`		If `source: list`, set this to the name of the domain list
`source_codes`	List of `string`		If `source: code`, set this to a list of codes to match against. These should be namespaced codes, ie: `testfeed/2` and `myfeed/3`
`source_category_mode`	`string`		If `source: category`, specify whether the category should be fetched from: A statically defined category under `categories`: set this to `static` A category list defined as User scoped category list: set this to `user`
`source_category_static`	List of `string`		If `source: category` and `source_category_mode: static`, set this to a list of names of categories defined under `categories`
`redirect_ips`	List of `string`		IP addresses of proxies to be returned when this policy rule is blocking. Can be a list of both IPv4 and IPv6 addresses
`redirect_rcode`	`integer`		Numerical representation of DNS RCode to be returned when this policy rule is blocking. Available options: NoError: `0` ServFail: `2` NXDomain: `3`

vHosts

When virtual hosts (vHosts) are defined in a filterSettings configuration and the settings are applied to a set of dnsdist instances, the following happens:

Service objects are created for each vHost to expose it users
vHosts can be referenced in rules to apply different policies to them

To define vHosts, you can use the vHosts parameter inside your filterSettings configuration, for example:

filterSettings:
  mysettings:
    vHosts:
      vhost1:
        <vHost configuration>
      vhost2:
        <vHost configuration>

Each defined vHost can be further configured using the following parameters:

Parameter	Type	Default	Description
`defaultproduct`	`string`		Default product to be assigned to traffic on this vHost which is not matched to a user
`defaultredirect`	`string`		This overrides the global 'defaultredirect' setting for this vhost if set. It only affects the HTTP proxy.
`defaultuser`	`string`		Default user to be assigned to traffic on this vHost when the source IP is not matched
`forceuser`	`string`		User to be forced on all traffic on this vHost
`metricsname`	`string`		Name to be used for this vHost in metrics, if unset this will be derived from the name of the vHost
`protobuftag`	`string`		Additional protobuf tag to set if traffic on this vHost is exported to dstoredist
`proxies`	List of `string`		IP addresses of proxies to be returned when traffic is filtered. Can be a list of both IPv4 and IPv6 addresses
`proxydeviceprofile`	`integer`		Set a fixed device profile number for traffic on this vHost
`service`	dnsdistService		Configuration of the dnsdist service object for this vHost
`unfiltered`	`boolean`	`false`	If true, no filtering is applied to traffic on this vHost

Example: Dynamic Filtering

Since the configuration becomes quite complex, below is an example of a basic Dynamic Filtering Userplane deployment.

This example assumes a Controlplane is deployed with the Dynamic Filtering blueprint included, in a namespace controlplane on the same Kubernetes cluster as the Userplane.

# Name of the cluster in the global NATS mesh (must be unique for each controlplane / userplane in the mesh)
name: userplane_demo

# Define filter settings
filterSettings:
  mydynamicset:
    dynamic:
      # Enable it
      enabled: true

      # Configure the domain lists feed endpoint from the controlplane
      domainListsFeed:
        url: http://httpapi-admin-domains.controlplane.svc:8080/feed/
        secretName: dom-lists-feeduser-secret

      # Configure the category overrides feed endpoint from the controlplane
      categoryOverridesFeed:
        url: http://httpapi-admin-overrides.controlplane.svc:8080/feed/
        secretName: cat-overrides-feeduser-secret

      # Configure the redis replica endpoint from the controlplane
      redis:
        mirror:
          host: "redis-dynamic-replica.controlplane.svc"
          passwordSecretName: "redis-credentials"
          passwordSecretKey: "password"

# NATS configuration
nats:
  # Make this NATS cluster join the larger mesh by connecting to the controlplane's NATS cluster
  hubAddress: "nats-hub.controlplane.svc:7422"
  hubPasswordSecretName: "hubuser-password-secret"

# Dnsdist with dynamic filtering
dnsdists:
  testdist:
    filtering:
      enabled: true
      settings: mydynamicset
    replicas: 2
    pools:
      default:
        serverGroups:
          - group: testrec
    service:
      type: NodePort

# Recursor
recursors:
  testrec:
    replicas: 2

Parameter	Type	Required	Description
`codes`	List of `string`	`yes`	List of codes learned from feed(s)
`name`	`string`	`yes`	Name of the category
`title`	`string`		Title of the category. Only used for presentation purposes

Filtering

Configuration Reference

Feed configuration

Feed TLS config

Dynamic Filtering

ECID Mapping

Settings Syncer

Static Filtering

Categories

Domain Lists

User Settings

User scoped category list

User scoped domain list

IP Mapping

Policy Rules

vHosts

Example: Dynamic Filtering