How it works

dstore accepts events over the network and subsequently stores them. Each event contains a DNS message (source address, query name, query type, etc.) augmented by performance counters all of which is committed to storage.

dstore receives events wrapped in Protocol Buffers (a.k.a. protobuf). These are a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

The events destined for dstore can be emitted by a variety of sources:

• PowerDNS Recursor
• PowerDNS dnsdist
• PowerDNS Authoritative Server
• dnspcap2protobuf from the PowerDNS distribution

Components

dstore consists of a number of components which interact with each other. Depending on the functionality desired, some of these components are optional. For example, if you are interested in storage of DNS messages and retrieval via the command-line only, you run tcpdistro and dgrep, omitting the UI server and egateway.

We next describe the individual components which make up dstore.

tcpdistro

This is the primary daemon which listens for events on TCP port 2000. It is responsible for listening for events, decoding protobufs, and managing the local storage of events in the data store.

Tightly coupled to tcpdistro is the dstore utility which is used by system administrators to clean up (or rotate) the storage facility.

dgrep

You use the dgrep utility to query, from the command line, the dstore storage facility. It’s a simple, but very fast tool, which emits JSON objects containing responses of previously stored DNS messages.

{
  "items": [
    {
      "answer": "",
      "origRequestor": "192.0.2.213",
      "preason": "",
      "qid": "278387a5-3080-4111-b8e0-1a25c1fb2340",
      "qr": false,
      "question": "www.powerdns.com.",
      "rcode": 0,
      "requestor": "192.168.1.130",
      "responder": "192.168.1.206",
      "tags": [],
      "timeSec": 1466489029,
      "timeUsec": 712294,
      "type": "AAAA"
    }
  ],
  "luaresult": {},
  "msec": 1.111,
  "number": 1
}

dcat

You can use the dcat utility to concatenate dstore files and print them on the standard output.

Examples :

To read the raw dstore file and print its content as JSON :

./src/dcat --quiet <path-to-raw-dstore-file> | jq

You can also print them as raw protobuf messages using the –raw switch :

./src/dcat --raw --quiet <path-to-raw-dstore-file> | ./go/src/dnspb2json/dnspb2json | jq

egateway

The optional egateway is a HTTP gateway to the DNS messages in dstore. It is intended to run on the machine with the storage, and it listens on a configurable TCP port, which defaults to 8081.

UI server (dstore-web)

dstore-web is a Python Django-based application which serves as a Web back-end for a Web browser on the one hand, and talks to egateway on the other. The UI server can run on a separate machine from the egateway, providing it can communicate with egateway.

The result provides an attractive and fast Web-based user-interface to dstore.

dnspcap2protobuf

The dnspcap2protobuf utility from the PowerDNS distribution reads a pcap file and converts it to protobuf format which you can subsequently feed to dstore:

tcpdump -i enp0s25 udp port 53 -w example.pcap
dnspcap2protobuf example.pcap /dev/stdout | nc 127.0.0.1 2000