Zone and web scanning

NZRS regularly conducts zone scans and web scans.  This page sets our why we do it, how we do it, when we do it and other details.  If you have any questions then please contact us.

!! You may have arrived here from visiting zonescan.nzrs.net.nz, which is the domain name that we scan from. !!

Quick links:

Practice Statement

1. Introduction

1.1. Purpose

NZRS periodically iterates through the .nz zones and for each domain within those zones conducts a data collection scan for the following purposes:

  • Research and analysis into DNS, how it works and how it is used.
    e.g. What is the usage of less common DNS Resource Records such as SRV and how that changes over time.
  • Gathering data that can help with the maintenance and development of the DNS infrastructure;
    e.g. How much of our traffic originates from different networks and does that match our server placement to provide the optimal response times.
  • Providing useful information for our customers, including registrars and DNS providers;
    e.g. Identifying lame delegations and reporting these to the registrar.
  • Providing public information on the use of the Internet in NZ.
    e.g. Reporting on the uptake of IPv6 as measured by the number of IPv6 enabled domain names.

This practice statement sets out how NZRS conducts this scanning.

1.2. Link to .nz policy

The use of zone data by NZRS in this way is explicitly authorised in the .nz Zone Transfer Policy. Additionally, DNCL may within the scope of the same policy, authorise access to zone data to third parties who intend to use it for scans.

2. Scanning

2.1. Types of scan

NZRS conducts two types of scans:

  • Zone scanning. The collector looks up public DNS records within the delegated zones. To be clear, this means that NZRS data collectors send DNS queries to the authoritative DNS servers for each delegated zone.
  •  Web scanning. The collector connects to publicly accessible web servers and retrieves the home page only, unless the home page has very little text on it in which case another page may be tried.

The two types of scan are conducted independently.

2.2. Query rates

For zone scanning, aggregate query rates are controlled so as not to exceed 50 queries per second to any individual nameserver. The determination of what is a single nameserver is made by using the domain name of the nameservers specified in the .nz zones and not the IP addresses.

For host scanning, aggregate connection rates are controlled so as to not exceed 40 simultaneous connections to any individual host. The determination of what is a single host is made by using the IP address of the host.

2.3. Scan frequency

The scan frequency is limited to a maximum of 3 scans per day. Ordinarily the number of scans is much lower than this but this figure is set sufficiently high to enable restarts of failed scans.

2.4. Notifications and audit trail

NZRS maintains on its website a schedule of planned scans, giving no less than 2 days notice of the scans. This list details:

  • The type of scan being conducted (zone or host);
  • The approximate time and anticipated duration of the scan;
  • The source addresses of the scan.

NZRS also maintains an audit trail of all historical and active scans with the same information as above.

2.5. IP addresses used

The source addresses used for the scans are published on the schedule.

Where possible, IP addresses used for scanning contain a correctly configured reverse entry that provides information about the role of this host in scanning, which is

zonescan.nzrs.net.nz

This domain has a web redirect to this page.

2.6. Blocking scans

It is recognised that some infrastructure operators may object to this activity for a variety of reasons and do not wish their infrastructure to be included in any scan. To address this concern the following safeguards are provided:

  1. The scans respect robots.txt and equivalents.
  2. The source addresses of the scanning hosts are published to enable a network operator to block access to the scans within their own infrastructure and with minimal disruption to other activity. Where possible these IP addresses are not used for any other operational purpose so that a network operator blocking these IP address should not experience any other operational impact.

If an infrastructure operator believes that our scanning is causing problems for their infrastructure then they should contact us and we will take appropriate action. If an operator believes that any such scanning as authorised by DNCL causes problems for their infrastructure then they should contact DNCL as well.

2.7. Coordinated scanning and third parties

We recognise the issues that could arise from multiple uncoordinated scans and are committed to cooperating with DNCL to coordinate with anyone else authorised by them to conduct scans.

3. Data

3.1. Collection

No data is collected just because it can. Data is only collected where it can be used to further the purposes above.

The following is a non-exhaustive list of data that is retrieved along with an example of use that illustrates the purpose behind the collection:

For zone scanning:

  • Authoritative NS records These are then compared against the delegations in the .nz zones and issues such as lame delegations detected.
  • Authoritative TTL. This supports analysis into how TTL affects traffic and how a .nz service failure is felt over time.
  • Record type survey. To identify the presence and number of novel DNS records as a signal of new technology adoption. This includes, for example, DNSSEC records, SPF, DKIM, SRV, LOC, etc.

We also test nameservers to see if they support the following undesirable and desirable service/behaviour:

  • Public zone transfer (if this is successful then we discard the data received)
  • Recursion
  • DNSSEC
  • NSID

For host scanning:

  • SSL certificates. To identify they number of X.509 certs in .nz and various features, such as how many are self-signed and so would benefit from DANE.
  • META statement keywords and outbound links. To identify semantic links between domain name labels.
  • Trust seals and SSL certificates. To help identify the overall security posture of .nz.
  • Text. To provide better understanding of the use of .nz.

3.2. Storage

All data collected is kept as retrieved indefinitely.

3.3. Sharing

It is extremely unlikely that any data collected through this scanning process contains personal information about an identifiable individual. While this means that no legal constraints exist, there are expectations of confidentiality and commercial integrity for which a voluntary commitment to privacy is appropriate. This means that:

  • Only anonymised and aggregated data is available as open data, where 'open' is taken to mean publicly available, freely licensed and free of charge.
  • Non-anonymised and non-aggregated data is only available under restrictions, generally limiting access to the registrants, registrars and DNS providers that the data applies to.

This data is published as described above on our Internet Data Portal.