Previous topic

Performance Guide

Next topic

DNSSEC in the PowerDNS Recursor

This Page

Metrics and Statistics

The PowerDNS Recursor collects many statistics about itself.

Regular Statistics Log

Every half hour or so (configurable with statistics-interval, the recursor outputs a line with statistics. To force the output of statistics, send the process a SIGUSR1. A line of statistics looks like this:

Feb 10 14:16:03 stats: 125784 questions, 13971 cache entries, 309 negative entries, 84% cache hits, outpacket/query ratio 37%, 12% throttled

This means that there are 13791 different names cached, which each may have multiple records attached to them. There are 309 items in the negative cache, items of which it is known that don’t exist and won’t do so for the near future. 84% of incoming questions could be answered without any additional queries going out to the net.

The outpacket/query ratio means that on average, 0.37 packets were needed to answer a question. Initially this ratio may be well over 100% as additional queries may be needed to actually recurse the DNS and figure out the addresses of nameservers.

Finally, 12% of queries were not performed because identical queries had gone out previously and failed, saving load on servers worldwide.

Sending metrics to Graphite/Metronome over Carbon

For carbon/graphite/metronome, we use the following namespace. Everything starts with ‘pdns.’, which is then followed by the local hostname. Thirdly, we add ‘recursor’ to signify the daemon generating the metrics. This is then rounded off with the actual name of the metric. As an example: ‘pdns.ns1.recursor.questions’.

Care has been taken to make the sending of statistics as unobtrusive as possible, the daemons will not be hindered by an unreachable carbon server, timeouts or connection refused situations.

To benefit from our carbon/graphite support, either install Graphite, or use our own lightweight statistics daemon, Metronome, currently available on GitHub.

To enable sending metrics, set carbon-server, possibly carbon-interval and possibly carbon-ourname in the configuration.

Warning

If your hostname includes dots, they will be replaced by underscores so as not to confuse the namespace.

If you include dots in carbon-ourname, they will not be replaced by underscores. As PowerDNS assumes you know what you are doing if you override your hostname.

Sending metrics over SNMP

New in version 4.1.0.

The recursor can export statistics over SNMP and send traps from Lua, provided support is compiled into the Recursor and snmp-agent set.

MIB

-- -*- snmpv2 -*-
-- ----------------------------------------------------------------------
-- MIB file for PowerDNS Recursor
-- ----------------------------------------------------------------------

PDNSRECURSOR-MIB DEFINITIONS ::= BEGIN

IMPORTS
    OBJECT-TYPE, MODULE-IDENTITY, enterprises,
    Counter64, NOTIFICATION-TYPE
        FROM SNMPv2-SMI
    CounterBasedGauge64
        FROM HCNUM-TC
    OBJECT-GROUP, MODULE-COMPLIANCE, NOTIFICATION-GROUP
        FROM SNMPv2-CONF;

rec MODULE-IDENTITY
    LAST-UPDATED "201812240000Z"
    ORGANIZATION "PowerDNS BV"
    CONTACT-INFO "support@powerdns.com"
    DESCRIPTION
       "This MIB module describes information gathered through PowerDNS Recursor."

    REVISION "201611290000Z"
    DESCRIPTION "Initial revision."

    REVISION "201812240000Z"
    DESCRIPTION "Added the dnssecAuthenticDataQueries and dnssecCheckDisabledQueries stats."

    ::= { powerdns 2 }

powerdns		OBJECT IDENTIFIER ::= { enterprises 43315 }

stats OBJECT IDENTIFIER ::= { rec 1 }

questions OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of questions"
    ::= { stats 1 }

ipv6Questions OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 questions"
    ::= { stats 2 }

tcpQuestions OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of TCP questions"
    ::= { stats 3 }

cacheHits OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of cache hits"
    ::= { stats 4 }

cacheMisses OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of cache misses"
    ::= { stats 5 }

cacheEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of cache entries"
    ::= { stats 6 }

cacheBytes OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Size of the cache in bytes"
    ::= { stats 7 }

packetcacheHits OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of packetcache hits"
    ::= { stats 8 }

packetcacheMisses OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of packetcache misses"
    ::= { stats 9 }

packetcacheEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of packetcache entries"
    ::= { stats 10 }

packetcacheBytes OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Size of the packetcache in bytes"
    ::= { stats 11 }

mallocBytes OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of bytes allocated by malloc"
    ::= { stats 12 }

servfailAnswers OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of servfail answers"
    ::= { stats 13 }

nxdomainAnswers OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of nxdomain answers"
    ::= { stats 14 }

noerrorAnswers OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of noerror answers"
    ::= { stats 15 }

unauthorizedUdp OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of unauthorized UDP queries"
    ::= { stats 16 }

unauthorizedTcp OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of unauthorized TCP queries"
    ::= { stats 17 }

tcpClientOverflow OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of TCP client connections refused because of too many connections"
    ::= { stats 18 }

clientParseErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of client parse errors"
    ::= { stats 19 }

serverParseErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of server parse errors"
    ::= { stats 20 }

tooOldDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because of a timeout"
    ::= { stats 21 }

answers01 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries answered in less than 1 ms"
    ::= { stats 22 }

answers110 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries answered in 1-10 ms"
    ::= { stats 23 }

answers10100 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries answered in 10-100 ms"
    ::= { stats 24 }

answers1001000 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries answered in 100-1000 ms"
    ::= { stats 25 }

answersSlow OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries answered in more than 1000 ms"
    ::= { stats 26 }

auth4Answers01 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 queries answered in less than 1 ms"
    ::= { stats 27 }

auth4Answers110 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 queries answered in 1-10 ms"
    ::= { stats 28 }

auth4Answers10100 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 queries answered in 10-100 ms"
    ::= { stats 29 }

auth4Answers1001000 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 queries answered in 100-1000 ms"
    ::= { stats 30 }

auth4Answersslow OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 queries answered in more than 1000 ms"
    ::= { stats 31 }

auth6Answers01 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 queries answered in less than 1 ms"
    ::= { stats 32 }

auth6Answers110 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 queries answered in 1-10 ms"
    ::= { stats 33 }

auth6Answers10100 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 queries answered in 10-100 ms"
    ::= { stats 34 }

auth6Answers1001000 OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 queries answered in 100-1000 ms"
    ::= { stats 35 }

auth6AnswersSlow OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 queries answered in more than 1000 ms"
    ::= { stats 36 }

qaLatency OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Average latency in microseconds"
    ::= { stats 37 }

unexpectedPackets OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of unexpected packets"
    ::= { stats 38 }

caseMismatches OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of case mismatches"
    ::= { stats 39 }

spoofPrevents OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of spoof prevents"
    ::= { stats 40 }

nssetInvalidations OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of nsset invalidations"
    ::= { stats 41 }

resourceLimits OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of resolution aborted because of a local resource limit"
    ::= { stats 42 }

overCapacityDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because the threads limit was reached"
    ::= { stats 43 }

policyDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because of a policy"
    ::= { stats 44 }

noPacketError OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of calls to recvmsg() that returned no packet even though the socket was ready"
    ::= { stats 45 }

dlgOnlyDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of records dropped because of they belonged to a delegation-only zone"
    ::= { stats 46 }

ignoredPackets OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of ignored packets"
    ::= { stats 47 }

maxMthreadStack OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Maximum size of the Mthread stack"
    ::= { stats 48 }

negcacheEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of negcache entries"
    ::= { stats 49 }

throttleEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of throttle entries"
    ::= { stats 50 }

nsspeedsEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of nsspeeds entries"
    ::= { stats 51 }

failedHostEntries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of failed host entries"
    ::= { stats 52 }

concurrentQueries OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of concurrent queries"
    ::= { stats 53 }

securityStatus OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Current security status"
    ::= { stats 54 }

outgoingTimeouts OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing timeouts"
    ::= { stats 55 }

outgoing4Timeouts OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv4 outgoing timeouts"
    ::= { stats 56 }

outgoing6Timeouts OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 outgoing timeouts"
    ::= { stats 57 }

tcpOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing TCP queries sent"
    ::= { stats 58 }

allOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing queries sent"
    ::= { stats 59 }

ipv6Outqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of IPv6 outgoing queries sent"
    ::= { stats 60 }

throttledOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of throttled outgoing queries"
    ::= { stats 61 }

dontOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing queries not sent because of a 'dont-query' setting"
    ::= { stats 62 }

unreachables OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of errors due to an unreachable server"
    ::= { stats 63 }

chainResends OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of chain resends"
    ::= { stats 64 }

tcpClients OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of TCP clients"
    ::= { stats 65 }

udpRecvbufErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of UDP recvbuf errors (Linux only)"
    ::= { stats 66 }

udpSndbufErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of UDP sndbuf errors (Linux only)"
    ::= { stats 67 }

udpNoportErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of UDP noport errors (Linux only)"
    ::= { stats 68 }

udpinErrors OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of UDP in errors (Linux only)"
    ::= { stats 69 }

ednsPingMatches OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of EDNS Ping matches"
    ::= { stats 70 }

ednsPingMismatches OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of EDNS Ping mismatches"
    ::= { stats 71 }

dnssecQueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC queries"
    ::= { stats 72 }

nopingOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing queries w/o ping"
    ::= { stats 73 }

noednsOutqueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of outgoing queries w/o EDNS"
    ::= { stats 74 }

uptime OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Process uptime in seconds"
    ::= { stats 75 }

realMemoryUsage OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Memory usage"
    ::= { stats 76 }

fdUsage OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "File descriptors usage"
    ::= { stats 77 }

userMsec OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "CPU usage (user) in ms"
    ::= { stats 78 }

sysMsec OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "CPU usage (system) in ms"
    ::= { stats 79 }

dnssecValidations OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC validations"
    ::= { stats 80 }

dnssecResultInsecure OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC insecure results"
    ::= { stats 81 }

dnssecResultSecure OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC secure results"
    ::= { stats 82 }

dnssecResultBogus OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC bogus results"
    ::= { stats 83 }

dnssecResultIndeterminate OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC indeterminate results"
    ::= { stats 84 }

dnssecResultNta OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of DNSSEC NTA results"
    ::= { stats 85 }

policyResultNoaction OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated no-action results"
    ::= { stats 86 }

policyResultDrop OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated drops"
    ::= { stats 87 }

policyResultNxdomain OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated NXdomain results"
    ::= { stats 88 }

policyResultNodata OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated nodata results"
    ::= { stats 89 }

policyResultTruncate OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated truncate results"
    ::= { stats 90 }

policyResultCustom OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of policy-mandated custom results"
    ::= { stats 91 }

queryPipeFullDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because the query distribution pipe was full"
    ::= { stats 92 }

truncatedDrops OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because they were larger than 512 bytes"
    ::= { stats 93 }

emptyQueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries dropped because they had a QD count of 0"
    ::= { stats 94 }

dnssecAuthenticDataQueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries received with the AD bit set"
    ::= { stats 95 }

dnssecCheckDisabledQueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries received with the CD bit set"
    ::= { stats 96 }

variableResponses OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of variable responses"
    ::= { stats 97 }

specialMemoryUsage OBJECT-TYPE
    SYNTAX CounterBasedGauge64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Memory usage (more precise but expensive to retrieve)"
    ::= { stats 98 }

rebalancedQueries OBJECT-TYPE
    SYNTAX Counter64
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Number of queries re-distributed because the first selected worker thread was above the target load"
    ::= { stats 99 }

---
--- Traps / Notifications
---

trap OBJECT IDENTIFIER ::= { rec 10 }
traps OBJECT IDENTIFIER ::= { trap 0 } --- reverse-mappable
trapObjects OBJECT IDENTIFIER ::= { rec 11 }

trapReason OBJECT-TYPE
    SYNTAX OCTET STRING
    MAX-ACCESS read-only
    STATUS current
    DESCRIPTION
        "Reason for this trap"
    ::= { trapObjects 1 }

customTrap NOTIFICATION-TYPE
    OBJECTS {
        trapReason
    }
    STATUS current
    DESCRIPTION "Trap sent by sendCustomTrap"
    ::= { traps 1 }

---
--- Conformance
---

recConformance OBJECT IDENTIFIER ::= { rec 100 }

recCompliances MODULE-COMPLIANCE
    STATUS current
    DESCRIPTION "PowerDNS Recursor compliance statement"
    MODULE
    MANDATORY-GROUPS {
        recGroup,
        recTrapsGroup
    }
    ::= { recConformance 1 }

recGroup OBJECT-GROUP
    OBJECTS {
        questions,
        ipv6Questions,
        tcpQuestions,
        cacheHits,
        cacheMisses,
        cacheEntries,
        cacheBytes,
        packetcacheHits,
        packetcacheMisses,
        packetcacheEntries,
        packetcacheBytes,
        mallocBytes,
        servfailAnswers,
        nxdomainAnswers,
        noerrorAnswers,
        unauthorizedUdp,
        unauthorizedTcp,
        tcpClientOverflow,
        clientParseErrors,
        serverParseErrors,
        tooOldDrops,
        answers01,
        answers110,
        answers10100,
        answers1001000,
        answersSlow,
        auth4Answers01,
        auth4Answers110,
        auth4Answers10100,
        auth4Answers1001000,
        auth4Answersslow,
        auth6Answers01,
        auth6Answers110,
        auth6Answers10100,
        auth6Answers1001000,
        auth6AnswersSlow,
        qaLatency,
        unexpectedPackets,
        caseMismatches,
        spoofPrevents,
        nssetInvalidations,
        resourceLimits,
        overCapacityDrops,
        policyDrops,
        noPacketError,
        dlgOnlyDrops,
        ignoredPackets,
        maxMthreadStack,
        negcacheEntries,
        throttleEntries,
        nsspeedsEntries,
        failedHostEntries,
        concurrentQueries,
        securityStatus,
        outgoingTimeouts,
        outgoing4Timeouts,
        outgoing6Timeouts,
        tcpOutqueries,
        allOutqueries,
        ipv6Outqueries,
        throttledOutqueries,
        dontOutqueries,
        unreachables,
        chainResends,
        tcpClients,
        udpRecvbufErrors,
        udpSndbufErrors,
        udpNoportErrors,
        udpinErrors,
        ednsPingMatches,
        ednsPingMismatches,
        dnssecQueries,
        nopingOutqueries,
        noednsOutqueries,
        uptime,
        realMemoryUsage,
        specialMemoryUsage,
        fdUsage,
        userMsec,
        sysMsec,
        dnssecValidations,
        dnssecResultInsecure,
        dnssecResultSecure,
        dnssecResultBogus,
        dnssecResultIndeterminate,
        dnssecResultNta,
        policyResultNoaction,
        policyResultDrop,
        policyResultNxdomain,
        policyResultNodata,
        policyResultTruncate,
        policyResultCustom,
        queryPipeFullDrops,
        truncatedDrops,
        emptyQueries,
        dnssecAuthenticDataQueries,
        dnssecCheckDisabledQueries,
        variableResponses,
        specialMemoryUsage,
        rebalancedQueries,
        trapReason
    }
    STATUS current
    DESCRIPTION "Objects conformance group for PowerDNS Recursor"
    ::= { recConformance 2 }

recTrapsGroup NOTIFICATION-GROUP
    NOTIFICATIONS {
        customTrap
    }
    STATUS current
    DESCRIPTION "Traps conformance group for PowerDNS Recursor"
    ::= { recConformance 3 }

END

Getting Metrics from the Recursor

Should Carbon not be the preferred way of receiving metric, several other techniques can be employed to retrieve metrics.

Using the Webserver

The API exposes a statistics endpoint at GET /api/v1/servers/:server_id/statistics. This endpoint exports all statistics in a single JSON document.

Using rec_control

Metrics can also be gathered on the system itself by invoking rec_control:

rec_control get-all

Single statistics can also be retrieved with the get command, e.g.:

rec_control get all-outqueries

External programs can use this technique to scrape metrics.

Gathered Information

These statistics are gathered.

It should be noted that answers0-1 + answers1-10 + answers10-100 + answers100-1000 + answers-slow + packetcache-hits + over-capacity-drops + policy-drops = questions.

Also note that unauthorized-tcp and unauthorized-udp packets do not end up in the ‘questions’ count.

all-outqueries

counts the number of outgoing UDP queries since starting

answers-slow

counts the number of queries answered after 1 second

answers0-1

counts the number of queries answered within 1 millisecond

answers1-10

counts the number of queries answered within 10 milliseconds

answers10-100

counts the number of queries answered within 100 milliseconds

answers100-1000

counts the number of queries answered within 1 second

auth4-answers-slow

counts the number of queries answered by auth4s after 1 second (4.0)

auth4-answers0-1

counts the number of queries answered by auth4s within 1 millisecond (4.0)

auth4-answers1-10

counts the number of queries answered by auth4s within 10 milliseconds (4.0)

auth4-answers10-100

counts the number of queries answered by auth4s within 100 milliseconds (4.0)

auth4-answers100-1000

counts the number of queries answered by auth4s within 1 second (4.0)

auth6-answers-slow

counts the number of queries answered by auth6s after 1 second (4.0)

auth6-answers0-1

counts the number of queries answered by auth6s within 1 millisecond (4.0)

auth6-answers1-10

counts the number of queries answered by auth6s within 10 milliseconds (4.0)

auth6-answers10-100

counts the number of queries answered by auth6s within 100 milliseconds (4.0)

auth6-answers100-1000

counts the number of queries answered by auth6s within 1 second (4.0)

auth-zone-queries

counts the number of queries to locally hosted authoritative zones (auth-zones) since starting

cache-bytes

size of the cache in bytes

cache-entries

shows the number of entries in the cache

cache-hits

counts the number of cache hits since starting, this does not include hits that got answered from the packet-cache

cache-misses

counts the number of cache misses since starting

case-mismatches

counts the number of mismatches in character case since starting

chain-resends

number of queries chained to existing outstanding query

client-parse-errors

counts number of client packets that could not be parsed

concurrent-queries

shows the number of MThreads currently running

cpu-msec-thread-n

shows the number of milliseconds spent in thread n. Available since 4.1.12.

dlg-only-drops

number of records dropped because of delegation-only setting

dnssec-authentic-data-queries

New in version 4.2.

number of queries received with the AD bit set

dnssec-check-disabled-queries

New in version 4.2.

number of queries received with the CD bit set

dnssec-queries

number of queries received with the DO bit set

dnssec-result-bogus

number of DNSSEC validations that had the Bogus state

dnssec-result-indeterminate

number of DNSSEC validations that had the Indeterminate state

dnssec-result-insecure

number of DNSSEC validations that had the Insecure state

dnssec-result-nta

number of DNSSEC validations that had the NTA (negative trust anchor) state

dnssec-result-secure

number of DNSSEC validations that had the Secure state

dnssec-validations

number of DNSSEC validations performed

dont-outqueries

number of outgoing queries dropped because of dont-query setting (since 3.3)

ecs-queries

number of outgoing queries adorned with an EDNS Client Subnet option (since 4.1)

ecs-responses

number of responses received from authoritative servers with an EDNS Client Subnet option we used (since 4.1)

ecs-v4-response-bits-*

New in version 4.2.0.

number of responses received from authoritative servers with an IPv4 EDNS Client Subnet option we used, of this subnet size (1 to 32).

ecs-v6-response-bits-*

New in version 4.2.0.

number of responses received from authoritative servers with an IPv6 EDNS Client Subnet option we used, of this subnet size (1 to 128).

edns-ping-matches

number of servers that sent a valid EDNS PING response

edns-ping-mismatches

number of servers that sent an invalid EDNS PING response

failed-host-entries

number of servers that failed to resolve

ignored-packets

counts the number of non-query packets received on server sockets that should only get query packets

ipv6-outqueries

number of outgoing queries over IPv6

ipv6-questions

counts all end-user initiated queries with the RD bit set, received over IPv6 UDP

malloc-bytes

returns the number of bytes allocated by the process (broken, always returns 0)

max-cache-entries

currently configured maximum number of cache entries

max-packetcache-entries

currently configured maximum number of packet cache entries

max-mthread-stack

maximum amount of thread stack ever used

negcache-entries

shows the number of entries in the negative answer cache

no-packet-error

number of erroneous received packets

noedns-outqueries

number of queries sent out without EDNS

noerror-answers

counts the number of times it answered NOERROR since starting

noping-outqueries

number of queries sent out without ENDS PING

nsset-invalidations

number of times an nsset was dropped because it no longer worked

nsspeeds-entries

shows the number of entries in the NS speeds map

nxdomain-answers

counts the number of times it answered NXDOMAIN since starting

outgoing-timeouts

counts the number of timeouts on outgoing UDP queries since starting

outgoing4-timeouts

counts the number of timeouts on outgoing UDP IPv4 queries since starting (since 4.0)

outgoing6-timeouts

counts the number of timeouts on outgoing UDP IPv6 queries since starting (since 4.0)

over-capacity-drops

questions dropped because over maximum concurrent query limit (since 3.2)

packetcache-bytes

size of the packet cache in bytes (since 3.3.1)

packetcache-entries

size of packet cache (since 3.2)

packetcache-hits

packet cache hits (since 3.2)

packetcache-misses

packet cache misses (since 3.2)

policy-drops

packets dropped because of (Lua) policy decision

policy-result-noaction

packets that were not actioned upon by the RPZ/filter engine

policy-result-drop

packets that were dropped by the RPZ/filter engine

policy-result-nxdomain

packets that were replied to with NXDOMAIN by the RPZ/filter engine

policy-result-nodata

packets that were replied to with no data by the RPZ/filter engine

policy-result-truncate

packets that were forced to TCP by the RPZ/filter engine

policy-result-custom

packets that were sent a custom answer by the RPZ/filter engine

qa-latency

shows the current latency average, in microseconds, exponentially weighted over past ‘latency-statistic-size’ packets

query-pipe-full-drops

New in version 4.2.

questions dropped because the query distribution pipe was full

questions

counts all end-user initiated queries with the RD bit set

rebalanced-queries

New in version 4.1.12.

number of queries balanced to a different worker thread because the first selected one was above the target load configured with ‘distribution-load-factor’

resource-limits

counts number of queries that could not be performed because of resource limits

security-status

security status based on Security Polling

server-parse-errors

counts number of server replied packets that could not be parsed

servfail-answers

counts the number of times it answered SERVFAIL since starting

spoof-prevents

number of times PowerDNS considered itself spoofed, and dropped the data

sys-msec

number of CPU milliseconds spent in ‘system’ mode

tcp-client-overflow

number of times an IP address was denied TCP access because it already had too many connections

tcp-clients

counts the number of currently active TCP/IP clients

tcp-outqueries

counts the number of outgoing TCP queries since starting

tcp-questions

counts all incoming TCP queries (since starting)

throttle-entries

shows the number of entries in the throttle map

throttled-out

counts the number of throttled outgoing UDP queries since starting

throttled-outqueries

idem to throttled-out

too-old-drops

questions dropped that were too old

truncated-drops

New in version 4.2.

questions dropped because they were larger than 512 bytes

empty-queries

New in version 4.2.

questions dropped because they had a QD count of 0

unauthorized-tcp

number of TCP questions denied because of allow-from restrictions

unauthorized-udp

number of UDP questions denied because of allow-from restrictions

unexpected-packets

number of answers from remote servers that were unexpected (might point to spoofing)

unreachables

number of times nameservers were unreachable since starting

uptime

number of seconds process has been running (since 3.1.5)

user-msec

number of CPU milliseconds spent in ‘user’ mode

variable-responses

New in version 4.2.

Responses that were marked as ‘variable’. This could be because of EDNS Client Subnet or Lua rules that indicate this variable status (dependent on time or who is asking, for example).

x-our-latency

New in version 4.1: Not yet proven to be reliable

PowerDNS measures per query how much time has been spent waiting on authoritative servers. In addition, the Recursor measures the total amount of time needed to answer a question. The difference between these two durations is a measure of how much time was spent within PowerDNS. This metric is the average of that difference, in microseconds.

x-ourtime0-1

New in version 4.1: Not yet proven to be reliable

Counts responses where between 0 and 1 milliseconds was spent within the Recursor. See variable-responses for further details.

x-ourtime1-2

New in version 4.1: Not yet proven to be reliable

Counts responses where between 1 and 2 milliseconds was spent within the Recursor. See variable-responses for further details.

x-ourtime2-4

New in version 4.1: Not yet proven to be reliable

Counts responses where between 2 and 4 milliseconds was spent within the Recursor. Since 4.1. See variable-responses for further details.

x-ourtime4-8

New in version 4.1: Not yet proven to be reliable

Counts responses where between 4 and 8 milliseconds was spent within the Recursor. See variable-responses for further details.

x-ourtime8-16

New in version 4.1: Not yet proven to be reliable

Counts responses where between 8 and 16 milliseconds was spent within the Recursor. See variable-responses for further details.

x-ourtime16-32

New in version 4.1: Not yet proven to be reliable

Counts responses where between 16 and 32 milliseconds was spent within the Recursor. See variable-responses for further details.

x-ourtime-slow

New in version 4.1: Not yet proven to be reliable

Counts responses where more than 32 milliseconds was spent within the Recursor. See variable-responses for further details.