How it works
============

All the software components in :program:`dstore` accept *events* over the network and process them in some way;
for example distributing them to different destinations, converting them to different formats, aggregating them into reports etc.
Each event contains a DNS message (source address, query name, query type, etc.) augmented by performance counters.

:program:`dstore` receives events wrapped in `Protocol Buffers <https://en.m.wikipedia.org/wiki/Protocol_Buffersprotobuf>`__ (a.k.a. protobuf).
These are a language-neutral, platform-neutral, extensible mechanism for serializing structured data.

The events destined for :program:`dstore` can be emitted by a variety of sources:

-  PowerDNS Recursor
-  PowerDNS dnsdist
-  PowerDNS Authoritative Server
-  :program:`dnspcap2protobuf` from the PowerDNS distribution


Components
----------

:program:`dstore` consists of a number of components which interact with each other.

We next describe the individual components which make up :program:`dstore`.

``dstore-dist``
~~~~~~~~~~~~~

The :program:`dstore-dist` is the primary daemon which listens for events on TCP port 2000, and performs a variety of processing
on those events, including filtering, distribution to different destinations, and conversion to new protocols such as Kafka.
It is responsible for listening for events, decoding protobufs, and the onwards forwarding of events.

``dstore-dist-topn-reporter``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The :program:`dstore-dist-topn-reporter` is used to generate reports from protobuf messages. These reports are usually
based on sampled data (`dstore-dist` can be used to perform the data sampling before passing the events to `topn-reporter`).
Reports are stored in a backend, which can either be HTTP or Elasticsearch.

dnspcap2protobuf
~~~~~~~~~~~~~~~~

The :program:`dnspcap2protobuf` utility from the PowerDNS distribution reads a pcap file and converts it to *protobuf* format which you can subsequently feed to :program:`dstore`:

.. code:: bash

   tcpdump -i enp0s25 udp port 53 -w example.pcap
   dnspcap2protobuf example.pcap /dev/stdout | nc 127.0.0.1 2000