Filtering
When enabled, filtering matches DNS traffic against policy rules to determine whether or not the filtering engine should intervene. More details about filtering can be found in the Filtering overview.
Configuration Reference
Sets of filter settings can be defined under the root node filterSettings
. Each set of settings should be a key-value pair, with:
- key: Name of the set
- value: Dictionary holding the configuration of filter settings
The configuration can be quite complex, as there are several concepts involved. Make sure you understand the concepts explained in this guide before attempting to configure them.
Parameters which can be used to configure filter settings are shown in the below table.
Parameter | Type | Default | Description |
---|---|---|---|
debugTXT |
boolean |
false |
If true, answer DNS queries with filtering debug output if you request TXT records for a domain prefixed by _filter. . For example: TXT _filter.some.domain.com |
debugRules |
boolean |
false |
If true (and logLevel is set to debug ), generate verbose rule engine logging in the dnsdist and/or recursor containers performing filtering. |
dynamic |
DynamicFiltering | {} |
Dynamic filtering configuration |
feeds |
List of Feed | [] |
Feeds to be consumed for use in filtering |
logFormat |
string |
"human" |
Format of logging. Available options: "human" "json" |
logLevel |
string |
"info" |
Level of logging. Available options: "debug" "info" "warn" "error" |
proxies |
List of string |
IP addresses of proxies to be returned when traffic is filtered, unless overridden at a lower level (vHost for example). Can be a list of both IPv4 and IPv6 addresses |
|
static |
StaticFiltering | {} |
Static filtering configuration |
vHosts |
Map of vHost | {} |
vHosts configuration |
vHostService |
dnsdistService | type: ClusterIP |
Default service object used as template for all vHost service objects. Can also be overridden on each vHost configuration |
Feed configuration
When feeds are configured, the dnsdist or recursor pod which is performing filtering will have an extra container named oxfeedclient
which will be responsible to download the feeds, process them and repeat when it identifies the upstream feeds have been updated. These feeds are provided by the Controlplane Feeds.
Example of a configuration with one feed:
filterSettings:
mysettings:
feeds:
- url: https://commandsource-testcmdsource.controlplane.svc:8443/feed/
namespace: testfeed
username: myUsername
password: myPassword
tlsconfig:
insecure_skip_verify: true
The above configuration will cause the oxfeedclient
container to consume a feed from the url https://commandsource-testcmdsource.controlplane.svc:8443/feed/
and expose the learned categories via the naming convention [NAMESPACE NAME]/[CODE]
. In this example, if the feed contains a category with code 1
, it would be accessible as testfeed/1
.
Additionally, it attempts to authenticate using myUsername
and myPassword
and will not attempt to perform any TLS verification (not recommended).
Parameters which can be used to configure feeds are shown in the below table.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
loadFirst |
boolean |
false |
If true, prioritize this feed when the oxfeedclient container starts. If multiple feeds are configured this can be used to ensure a specific feed is loaded first. |
|
namespace |
string |
yes |
Namespace to prepend to the codes of the categories learned from this feed | |
password |
string |
Password to use for authentication with the feed. Using secretName is preferred |
||
secretName |
string |
Name of a pre-existing Kubernetes Secret containing a username & password for authentication with the feed | ||
secretPasswordKey |
string |
"password" |
If secretName is specified: name of the item inside the Secret which holds the password |
|
secretUsernameKey |
string |
"username" |
If secretName is specified: name of the item inside the Secret which holds the username |
|
url |
string |
yes |
Full URL of the feed | |
username |
string |
Username to use for authentication with the feed. Using secretName is preferred |
||
tlsconfig |
TLSConfig | {} |
TLS configuration options |
Feed TLS config
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
ca |
string |
CA in PEM format to use for validation. Use of caSecret is recommended for security reasons |
||
caSecret |
string |
Name of a pre-existing Kubernetes Secret with a data item named ca.crt containing the CA in PEM format to use for validation |
||
insecure_skip_verify |
boolean |
false |
Skip validation of the feed's certificate chain and hostname |
Dynamic Filtering
Parameters which can be used to configure dynamic filtering settings. Important to note is that you must explicitly set enabled
to true
. In addition, it is highly recommended that you familiarize yourself with the Filtering Overview before proceeding with configuration.
NATS mesh
Dynamic filtering requires NATS in the Userplane to be configured to connect to the Controlplane. Information about adding this configuration can be found in the NATS Configuration Reference
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
categoryOverridesFeed |
Feed | yes |
{} |
Configuration of the connection towards the Controlplane Category Overrides feed |
deviceRegister |
Device Register | {} |
Configuration of Device Register in producer mode |
|
domainListsFeed |
Feed | yes |
{} |
Configuration of the connection towards the Controlplane Domain Lists feed |
ecidmappings |
List of ECID Mapping |
|
Configuration of EDNS Client ID (ECID) options parsed when Device Register is enabled. Note: When you provide any configuration under this node, you will overwrite the entire list of defaults to avoid unwanted conflicts between the defaults and your overrides. |
|
enabled |
boolean |
yes |
false |
If true , enable dynamic filtering |
ipmapper |
IP Mapper | {} |
Configuration of IP Mapper, if Radius Listener is enabled in the Userplane scope | |
radiusListener |
Radius Listener | {} |
Configuration of Radius Listener to consume RADIUS messages in the Userplane scope | |
redis |
Redis Configuration | yes |
{} |
Configuration of the Redis read-only cluster (mirror) which replicates from the Controlplane Redis replica Service.Note: Since this is a read-only cluster it requires the mirror configuration towards the Controlplane with Dynamic Filtering blueprint deployed |
settingsSyncer |
Settings Syncer | {} |
Configuration of Settings Syncer, responsible for synchronizing global settings, IP mappings and subscriber preferences to a local LMDB used by the filtering engine |
ECID Mapping
You can configure an ECID mapping via the below configuration options:
Overriding ECID mapping defaults
When you provide any configuration under the ecidmappings
node, you will replace all the defaults as those could conflict with your new configuration.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
name |
boolean |
false |
If true, indicates this option_code contains a device name instead of an ID |
|
option_code |
integer |
yes |
Numerical ID of the EDNS option to look for in DNS queries | |
post_encoding |
string |
yes |
Format to decode the contents of the parsed EDNS option to when formulating the ID or name. Available options: "string" "binary" "hex" |
|
post_prefix |
string |
"" |
If configured, add this prefix to Device ID | |
pre_encoding |
string |
yes |
Format to decode the contents of the EDNS option from. Available options: "string" "binary" "ecid" "hex" |
Settings Syncer
You can configure the Settings Syncer via the below configuration options:
Parameter | Type | Default | Description |
---|---|---|---|
lmdbLogStats |
boolean |
true |
If true , periodically log LMDB stats |
lmdbLogStatsInterval |
go: DurationString |
1m |
Interval between logging of LMDB stats if lmdbLogStats is true |
lmdbOptions |
dictionary |
{} |
Extra options to be set on the LMDB used by Settings Syncer |
lmdbScrapeSmaps |
boolean |
true |
Whether to scrape /proc/self/smaps to calculate LMDB memory statistics |
logFormat |
string |
"human" |
Format of logging. Available options: "human" "json" |
logLevel |
string |
"info" |
Level of logging. Available options: "debug" "info" "warn" "error" |
region |
string |
"" |
Optional CG-NAT region for ip4 syncing |
restart |
boolean |
true |
If true , restart Settings Syncer after a Redis failure |
restartWait |
go: DurationString |
5s |
If restart is true , wait for this duration before restarting after a Redis failure |
Static Filtering
Parameters which can be used to configure static filtering settings are shown in the below table.
Parameter | Type | Default | Description |
---|---|---|---|
categories |
List of Category | [] |
Categories to be calculated |
domainlists |
Map of DomainList | {} |
Static domain lists |
filtersettings |
List of UserSettings | [] |
Mapping of users, products, profiles, etc to filtering objects such as domainlists and categories |
ipmappings |
List of IPMapping | [] |
Mapping of IP prefixes to users |
policyrules |
List of PolicyRules | [] |
Filtering policies |
Categories
These calculated categories allow for the combination of categories learned from feeds. For example:
filterSettings:
mysettings:
static:
categories:
- name: calccategory
title: My calculated category
codes:
- myfeed/1
- otherfeed/1
- otherfeed/2
The above will expose a category named calccategory
which will contain the sum of DomainPaths from:
- Category with code
1
in the feed configured withnamespace: myfeed
- Categories with codes
1
and2
in the feed configured withnamespace: otherfeed
Parameters which can be used to configure categories are shown in the below table.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
codes |
List of string |
yes |
List of codes learned from feed(s) | |
name |
string |
yes |
Name of the category | |
title |
string |
Title of the category. Only used for presentation purposes |
Domain Lists
Domain lists allow for the configuration of static lists of domains to which filtering decisions can be applied. For example:
filterSettings:
mysettings:
static:
domainlists:
global-allow-list:
- powerdns.com
global-block-list:
- evil.example
- malware.example
The above will expose 2 static domain lists:
- Domain list
global-allow-list
containing thepowerdns.com
domain - Domain list
global-block-list
containing 2 example domains that may be desirable to block
These domain lists can be referenced by Setting Mappings and Policy Rules using their names (ie: global-allow-list
and global-block-list
).
User Settings
User Settings allow for defining user settings and mapping them to filtering objects. For example:
filterSettings:
mysettings:
static:
filtersettings:
- username: user1
filter_content: true
named_category_lists:
categories-denylist:
- someCategory
Above mapping will assign the user user1
a user-scoped category list named categories-denylist
which contains the category named someCategory
. This user-scoped category list categories-denylist
can then be referenced in policy rules.
filter_content
The filter_content
parameter must be set to true
to enable the filtering of defined content
such as the members of the someCategory
category in the above example.
Parameters which can be used to configure user settings are shown in the below table.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
username |
string |
yes |
Username to match against | |
timezone |
string |
Timezone in the format Europe/Amsterdam , local time of the server if unspecified and UTC if configured with an unknown timezone |
||
usertags |
List of string |
[] |
Extra tags for the user which can be referenced in policy rules | |
product |
string |
"" |
Product used by user which can be referenced in policy rules | |
filter_content |
boolean |
false |
If true, apply the filtering of named_category_lists and named_domain_lists |
|
named_category_lists |
Map of UserScopedCategoryList | {} |
Category lists scoped to this user | |
named_domain_lists |
Map of UserScopedDomainList | {} |
Domain lists scoped to this user |
User scoped category list
A user scoped category list allows for configuring category lists which only apply to that specific user. For example:
filterSettings:
mysettings:
static:
filtersettings:
- username: user1
filter_content: true
named_category_lists:
categories-allowlist:
- socialmedia
categories-denylist:
- malware
- botnets
In the above example, this user's scope has the following category lists defined:
- Category list
categories-allowlist
references the categorysocialmedia
with the intention of allowing this user access to social media - Category list
categories-denylist
references the categoriesmalware
andbotnets
with the intention of safeguarding the user against potentially bad actors
Policy Rules
These category lists have only been scoped to the user, they still need to be enforced by a policy rule to become active
User scoped domain list
A user scoped domain list allows for configuring domain lists which only apply to that specific user. For example:
filterSettings:
mysettings:
static:
filtersettings:
- username: user1
filter_content: true
named_domain_lists:
domains-allowlist:
- powerdns.com
domains-denylist:
- malware.example
- botnet.example
In the above example, this user's scope has the following domain lists defined:
- Domain list
domains-allowlist
references the domainpowerdns.com
with the intention of allowing this user access to this domain - Domain list
domains-denylist
references the categoriesmalware.example
andbotnet.example
with the intention of safeguarding the user against resolving these domains
Policy Rules
These domain lists have only been scoped to the user, they still need to be enforced by a policy rule to become active
IP Mapping
IP Mappings allow for mapping of IP prefixes to users. For example:
filterSettings:
mysettings:
static:
ipmappings:
- prefix: 10.0.0.0/8
username: private
- prefix: 172.16.0.0/12
username: private
- prefix: 192.168.0.0/16
username: private
Above mapping will assign the user private
to traffic originating from the 3 IPv4 address ranges allocated for private networks.
Parameters which can be used to configure ip mappings are shown in the below table.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
prefix |
string |
yes |
Prefix to match against | |
username |
string |
yes |
Username to assign to traffic matched against this prefix |
Policy Rules
Policy rules tie together all the other objects which have been configured and are the glue which enables the actual filtering process. For example:
filterSettings:
mysettings:
static:
domainlists:
global-block-list:
- evil.example
- malware.example
policyrules:
- name: block-domain-list
scope: all
action: blacklist
source: list
source_list: global-block-list
In this example we define a domainlist
named global-block-list
containing 2 domains which we want to globally block for everybody.
We then have a policy rule named block-domain-list
which will perform the action blacklist
(ie: block) for this domainlist by specifying source
to be a list
and refer to the name by setting source_list
to the name of our list: global-block-list
.
Parameters which can be used to configure policy rules are shown in the below table.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
priority |
integer |
Priority of this rule. If unset the priority is defined by the ordering of the list, where the first entry is first applied, then second, etc. | ||
name |
string |
yes |
Name of the policy rule | |
action |
string |
yes |
Action to be taken when matched. Available options: "whitelist" "blacklist" "monitor" |
|
scope |
string |
yes |
Scope of the rule. Available options: "all" "content" "security" "homework" In the current release only "all" and "content" are applicable and have the same outcome |
|
source |
string |
yes |
Type of object to match against. Available options: "category" "code" "list" |
|
products |
List of string |
Products to match against | ||
category_list |
string |
If source: category and source_category_mode: user , set this to the name of the User scoped category list |
||
usertag |
string |
If set, match against this usertag | ||
source_list |
string |
If source: list , set this to the name of the domain list |
||
source_codes |
List of string |
If source: code , set this to a list of codes to match against.These should be namespaced codes, ie: testfeed/2 and myfeed/3 |
||
source_category_mode |
string |
If source: category , specify whether the category should be fetched from: A statically defined category under categories : set this to static A category list defined as User scoped category list: set this to user |
||
source_category_static |
List of string |
If source: category and source_category_mode: static , set this to a list of names of categories defined under categories |
||
redirect_ips |
List of string |
IP addresses of proxies to be returned when this policy rule is blocking. Can be a list of both IPv4 and IPv6 addresses |
||
redirect_rcode |
integer |
Numerical representation of DNS RCode to be returned when this policy rule is blocking. Available options: NoError: 0 ServFail: 2 NXDomain: 3 |
vHosts
When virtual hosts (vHosts) are defined in a filterSettings
configuration and the settings are applied to a set of dnsdist instances, the following happens:
- Service objects are created for each vHost to expose it users
- vHosts can be referenced in rules to apply different policies to them
To define vHosts, you can use the vHosts
parameter inside your filterSettings
configuration, for example:
Each defined vHost can be further configured using the following parameters:
Parameter | Type | Default | Description |
---|---|---|---|
defaultproduct |
string |
Default product to be assigned to traffic on this vHost which is not matched to a user | |
defaultredirect |
string |
This overrides the global 'defaultredirect' setting for this vhost if set. It only affects the HTTP proxy. | |
defaultuser |
string |
Default user to be assigned to traffic on this vHost when the source IP is not matched | |
forceuser |
string |
User to be forced on all traffic on this vHost | |
metricsname |
string |
Name to be used for this vHost in metrics, if unset this will be derived from the name of the vHost | |
protobuftag |
string |
Additional protobuf tag to set if traffic on this vHost is exported to dstoredist | |
proxies |
List of string |
IP addresses of proxies to be returned when traffic is filtered. Can be a list of both IPv4 and IPv6 addresses |
|
proxydeviceprofile |
integer |
Set a fixed device profile number for traffic on this vHost | |
service |
dnsdistService | Configuration of the dnsdist service object for this vHost | |
unfiltered |
boolean |
false |
If true, no filtering is applied to traffic on this vHost |
Example: Dynamic Filtering
Since the configuration becomes quite complex, below is an example of a basic Dynamic Filtering Userplane deployment.
This example assumes a Controlplane is deployed with the Dynamic Filtering blueprint included, in a namespace controlplane
on the same Kubernetes cluster as the Userplane.
# Name of the cluster in the global NATS mesh (must be unique for each controlplane / userplane in the mesh)
name: userplane_demo
# Define filter settings
filterSettings:
mydynamicset:
dynamic:
# Enable it
enabled: true
# Configure the domain lists feed endpoint from the controlplane
domainListsFeed:
url: http://httpapi-admin-domains.controlplane.svc:8080/feed/
secretName: dom-lists-feeduser-secret
# Configure the category overrides feed endpoint from the controlplane
categoryOverridesFeed:
url: http://httpapi-admin-overrides.controlplane.svc:8080/feed/
secretName: cat-overrides-feeduser-secret
# Configure the redis replica endpoint from the controlplane
redis:
mirror:
host: "redis-dynamic-replica.controlplane.svc"
passwordSecretName: "redis-credentials"
passwordSecretKey: "password"
# NATS configuration
nats:
# Make this NATS cluster join the larger mesh by connecting to the controlplane's NATS cluster
hubAddress: "nats-hub.controlplane.svc:7422"
hubPasswordSecretName: "hubuser-password-secret"
# Dnsdist with dynamic filtering
dnsdists:
testdist:
filtering:
enabled: true
settings: mydynamicset
replicas: 2
pools:
default:
serverGroups:
- group: testrec
service:
type: NodePort
# Recursor
recursors:
testrec:
replicas: 2