Descriptive analysis

As of 1st August 2017, there are 89 active registrars in the .nz register. The prediction procedure is not feasible for all the registrars. Two features help us to determine whether a registrar’s data is statistically ready for prediction: (1) the age, i.e. how long a registrar’s data has existed in the register. The more data points we have for prediction modelling, the more accurate the prediction will be. (2) the size. The larger a registrar’s size is, the clearer the trend we may find underlying the historical change in its size.

As shown in the above figure, the distribution of a registrar’s age and size exhibits interesting points. Among these registrars, 40 of them have under 1000 active domains. The smallest registrar, although it has been recorded since June 2010, has only 7 domains. The age distribution shows the opposite. 76 registrars are older than 5 years old, among which 56 registrars are older than 10 years old (Note that there have been transfers of domains between registrars now and then for different reasons, so some of the information here might not be precise). The registrars that are at least 60 months old and have at least 5000 domains are selected for prediction. That leaves us 20 registrars.

The prediction

The prediction procedure follows the one described in the previous blog. Two assumptions are made so that the procedure can be applied: (1) although a domain might be transferred to another registrar, it is treated as a new create into that registrar. (2) different SLDs are assumed to behave similarly, so that we have enough data points for prediction.

Let’s first have a look at the retention behaviour. Take Registrar A as an example, the following two figures show the retention behaviour of domains registered at different periods (i.e. cohorts, the retention rate is estimated using multi-cohorts data). It can be seen that the drop out rate is high in the early years and then slows down as the domains stay longer. From the heat map we can observe, on average, the retention rate of relatively recent cohort is higher, which is a good thing to know.

The new creates forecast reveals some interesting findings as well. See the two registrars showing below, the historical new creates data shows a clear downward trend. This might indicate a change in the focus of the business. The forecast therefore could be negative for some periods and will be replaced by zero.

Some registrars have extremely stationary and low-quantity new creates over time, see the figure below. For such registrars, there are barely trend or cyclic fluctuations underlying the data points.

On the opposite, some registrars have new creates data that fluctuates greatly and shows no clear trend or seasonality. Accurate forecasts for such cases are hard. A further look at the reasons behind those fluctuations will be helpful for more reasonable forecast.

To test the performance of the prediction, historical data up to May 2017 is used to make prediction for June, July and August 2017. The table below shows the MAPE (mean absolute percent error) for each registrar in descending order by size. As mentioned before, smaller registrar size makes it harder for prediction. Hence it is not surprising to see some of the bottom 10 registrars have a MAPE greater than 10%. In general, our procedure generates comparatively accurate prediction for relatively big registrars.

Finally, let’s see the prediction results for the top 20 registrars. The total size of this group is increasing over time. Larger registrars also show an increasing trend. Some registrars’ size decrease slightly each month. This is due to the forecasted low new creates and/or comparatively larger number of drop outs in certain months.

Registrar size prediction is more challenging compared with register size prediction due to the data quality after segmentation. Nonetheless, some interesting findings surface in between. Since bulk transfer of domains between registrars happen for various reasons (e.g., movement of re-sellers or large portfolio holders between registrars), a further investigation on those cases will help improve the quality of data and prediction. For data that is reasonably stationary, using a naive or moving average forecasting technique might be a better choice. These could be directions for follow on work.

The collection process is straightforward:

• Extract the list of active .nz domains in the register
• For each domain, verify if there is an A record for the host www.domain. If the resolution process fails, the domain won't be included.
• Test each domain using sslyze. We have a script that will test for different versions of SSL and TLS, protocol features and will collect information about the certificate chain on the site.
• Once the collection is completed, produce aggregated counters and make them available on IDP, our Internet Data Portal.

We started in January 2017 and so far the process has completed five times, giving us some valuable datapoints.

HTTPS support

Let's start with the overall picture of how much of the .nz namespace has a secure website.

On our first collection, we didn't register how many domains failed the test due to incorrect DNS response, it was added afterwards. Despite that detail, there is a lot to notice here. Around 14% of the domains fail the DNS test, and around 0.7% have invalid certificates, for example, where the name in the certificate doesn't match the website name. The positive news is HTTPS support grew from 44% to 47% during this year.

Protocol Support

HTTPS relies on cryptographic protocols to ensure privacy and authenticity, such as SSL and TLS. A given webserver can support multiple crypto protocols at the same time. SSL v2.0 was released in February 1995, SSL v3.0 released in 1996, TLS v1.0 was defined in January 1999 as a replacement for SSL v3.0, TLS v1.1 was published in April 2006, TLS v1.2 was published in August 2008 and TLS v1.3 is currently being drafted in the IETF and we don't test for it.

As SSL v2.0 was deprecated and prohibited in 2011 in RFC 6176, and SSL v3.0 was deprecated in June 2015 by RFC 7568, there SHOULD NOT be any domains supporting it. Sites with those crypto protocols activated are a risk given the number of known exploitable vulnerabilities against SSL. If you are an administrator of a secure website, you can use the excellent Qualys SSL Site tester to verify for weaknesses.

Certificate Public Keys

Crypto protocols use public key cryptography to provide privacy and authenticity. To achieve this, SSL and TLS rely on certificates issued by Certificate Authorities (CA), that authenticate the identity of the website you are visiting, and contain a cryptographic key to encrypt traffic.

Cryptographic keys have two main properties: the algorithm used to generated them, where we can have RSA and ECC, and the key size measured in bits.

As part of the testing, we collect what kind of cryptographic keys the sites with HTTPS enabled are using.

You can see 3 different key sizes for RSA keys: 1024, 2048 and 4096. A 1024-bit RSA key is considered weak and should not be used. The fact keys of that size were visible at the beginning of this year but now have disappeared shows a healthy attitude towards security maintenance. The transition from 2048-bit keys to 4096-keys observed since August 2017 also is a great indication of the hygiene of the .nz namespace.

What about the 256 id-ecPublicKey cases? Those are sites using the new Eliptic Curve DSA cryptography, which uses smaller key sizes. There are also a few cases using 384-bit ECDSA keys.

To make the visualisation easy to read, we omit values with less than 1% of domains, but we can find some oddities such as sites with unusual key sizes. For example, there are still a few cases with weak 512-bit RSA keys, or 3000-bit keys, or 1536-bit keys (1536 is half-way between 1024 and 2048).

Certificate Signature Algorithms

Each SSL certificate contains a digital signature, a hash of the certificate content, signed with the issuing Certificate Authority private key. This digital signature allows the verification of the integrity of the certificate and enables browsers to verify the validity of the certificate.

There are a few hashing algorithms used for signatures, such as MD5, SHA-1 family and SHA-256 family, including SHA-256, SHA-384 and SHA-512.

Great news right? Most of the sites use the strong SHA-256 hashes, with RSA or ECDSA. As SHA-1 is subject to collision attacks, it's feasible to generate fraudulent certificates and trick users into using the wrong website. Firefox now warns users visiting secure websites using certificates signed with SHA-1, and NIST has recommended moving away from SHA-1 since 2014. So despite the fraction of sites with weak hashes moving from 7.1% to 4.5%, we still have reasons to be concerned.

As with the previous plot, for clarity we didn't include signature algorithms with less than 1% of the domains. There are a few domains using MD5 as hashing algorithms in their certificates, considering MD5 was deprecated back in 2013, that's certainly not good news.

Certificate Authorities

As we mentioned before, certificates are issued by Certificate Authorities, and browsers are configured to trust a set of CAs. An administrator can relatively easily set up their own CA and create certificates, but those won't be verifiable by a browser.

The plot below shows the top 10 issuing Certificate Authorities observed in the .nz namespace. For simplicity, we group the long tail of CAs including self-issued certificates into the 'Other' category.

The figure shows the biggest players in the certificate industry as observed in New Zealand: UserTrust Network, GoDaddy, GeoTrust, Comodo and Let's Encrypt. The interesting bit is the evolution across time: Let's Encrypt grew from 16.29% to 22.84% of the domains in 9 months, and that growth came at the cost of more traditional CAs such as GoDaddy, GeoTrust and Comodo. The other CA with a massive growth is UserTrust Network, that nearly tripled their market share since January 2017.

Let's Encrypt provides an interesting story: started back in December 2015 to issue certificates for free, with a validity of 90 days and a fully automated process. Before that, all CAs were for profit organisations, charging in some cases considerable fees to issue a certificate. Let's Encrypt is supported by the Internet Security Research Group (ISRG), which includes Facebook, the Mozilla Foundation, Google Chrome, EFF and many others.

Wrap up

We started this collection with the motivation of finding out more about trust and security in the .nz namespace and we've learned a lot in the process. The collection will carry on every other month and a future blog post could cover more details about the protocols, such as negotiation features, certificate chain length and intermediate CAs.

Prediction Procedure

Like any population size prediction problem, the key in register size prediction is to understand what the "births"(flows in) and "deaths"(flows out) are. The figure below conceptualizes register size changes for each month.

Each month some of the existing domains stop being active, which may then leave the register. Meanwhile, some new domains are created on the register. Hence, the calculation of register size for month t+1 can be summarized as:

Register size (t+1) = Register size (t) + New creates(t+1) - Dropping out (t+1)

Where the dropping outs are modelled by the domain retention prediction, and the number of new creates are modelled by the new creates forecasting. The shifted Beta Geometric model (as used in retention modelling) gives us year-to-year retention rates. The retention rates for the remaining 11 months of a year are assumed to be 100%. This assumption is based on our observation that most of the domains are registered or renewed for 1 year. Although this means our prediction procedure overestimates the register size, we will see in the results that the errors are reasonably small.

Inputs & Results

Retention Prediction

The input of this step is simple. For instance, to make predictions for Jan 2017 onwards, all we need is the domains that are active in Dec 2016 and their age (i.e., how long they have been active). A sample of the input is shown below:

A finding from the results in the domain retention prediction is that different SLDs have different retention behaviour. Hence retention prediction is done separately for each group - co.nz, org.nz, net.nz, and other SLDs. We fit the model with 12 year's historical data points of 12 cohorts (representing the domains created in each month of 2004); from which we calculate the 95% confidence interval retention rates shown below. As different SLDs are combined as one group, the variation in their retention behaviour is bigger, which can be seen from the larger spread in the 95% confidence interval.

In order to test the accuracy of retention prediction, the predicted retention rates are applied to domains that were active on 1st Dec 2016. The prediction of how many domains stay active in the register starts from Jan to Apr 2017. The 95% confidence interval in comparison with the actual values is shown in the following figures. Although predictions are slightly overestimated, the errors on average are around 1% which proves that the predictions are reasonably accurate.

New Creates

In new creates forecasting, I introduced how to do it using SARIMA model which needs parameter tuning. In Feb 2017, Facebook open sourced a Python/R package called Prophet to automate the time forecasting process. It used the additive model which makes it computationally efficient. For those interested in finding more about Prophet, I recommend reading Facebook’s white paper. It is used here to do new creates forecasting for different groups.

The following figure shows the new creates prediction for co.nz. The prediction captures the trend nicely. Looking at the test period starting from Jan 2016 to Apr 2017, we see most of the actual values (represented by red dots) are captured by the 95% confidence interval. A spike occurred in Apr 2017 which was caused by the end of reservation period for registration at the second level. Hence, it is relatively normal that the prediction didn’t capture that special event.

Register size

Now we are finally ready to predict the total number of active domains in the register! This is done by combining the number of domains that stay from last month plus the number of new creates in between. The figure below shows the predicted register size from Jan to Apr 2017 (at the beginning of each month) compared with the actual value. The results are fairly satisfying. The 95% confidence interval successfully covers actual register size of Feb and Apr. The absolute errors are all less than 1%. The underestimation in Apr was caused by the end of reservation period for registration at the second level.

Knowing that the procedure is working, let’s check out the predictions from May 2017 up to the end of this financial year (the figure below shows the register size prediction at the beginning of each month):

Final thoughts

Register size prediction is my first project after joining NZRS and I've learned a lot from it, e.g. working with Python and understanding the register behaviour. The prediction procedure can be further improved by investigating the minority domains that are registered / renewed on a monthly basis, and/ or revisiting the prediction if special events occur in the future. For practitioners who intend to implement this procedure, it is important to check the availability of input data since the models require specific details about the domains in the register.

Finally, I'd like to quote Nils Bohr who said "Prediction is very difficult, especially if it's about the future". Although I've been working to make the prediction accurate, I will be pleased to see faster growth in the register size. So, let's work hard to prove me wrong!

Resolvers are implemented differently to prioritise the NS RRset for a domain depending on where it came from, the parent or the child zone. These differences in implementation result in different traffic volumes to authoritative name servers when the NS RRset has different TTL (Time To Live) values configured in parent zone and child zone. DNS standards do not prescribe the behaviour of the resolver (its centricity).

In this post we outline an experiment created to detect centricity patterns of major local ISP resolvers in New Zealand and public DNS resolvers. The work is inspired by a presentation from Ólafur Guðmundsson available here.

DNS Resolution

The figure below shows the non-cached resolution process for the address of the domain name string.sub.experiment.nz.

The process involves the following steps (for a clean cache):

1. The client (end-user) sends a DNS query to the resolver configured in their network settings (normally the ISP's DNS resolver or a public DNS resolver).
2. The resolver doesn't know the address for string.sub.experiment.nz, but knows the IP addresses for the root name servers, so it asks one of them.
3. The root server also doesn't have the information about string.sub.experiment.nz, but it holds the locations of the name servers for each top-level domain (such as .nz, .au, .com, etc.), so it answers with the referral (NS RRset) to the .nz name servers, who might know the address of string.sub.experiment.nz.
4. The resolver then starts another iterative query against one of .nz name servers.
5. The .nz name server holds a list of name servers for each domain delegated in its zone rather than the address of string.sub.experiment.nz, so it responds with the referral to experiment.nz name servers.
6. The resolver does its third iterative query against one of experiment.nz name servers.
7. As per the preceding servers, it replies with the referral to sub.experiment.nz name servers, which are the name servers that hold the information about string.sub.experiment.nz.
8. The resolver sends the query to sub.experiment.nz name server.
9. The name server replies with the A or AAAA records (depending on the query type) containing IPv4 or IPv6 addresses for string.sub.experiment.nz.
10. The resolver sends back the final answer to client.

During the process, the resolver handles the recursive resolution task for the client by repeating the query and following successive referrals until gets to the final answer.

Resolver caching and time-to-live (TTL)

In normal operations, a resolver caches the answers it gets from name servers for future reference. This allows it to speed up DNS resolution and reduce the traffic to name servers. To keep updated with changes in zone data, every DNS record has a TTL. This TTL controls how long the record can be cached before it is no longer considered valid.

Besides caching answers from client's queries, a resolver also caches referrals (NS RRsets) to the name servers it gets in the process. These will be later used when an answer is not in its cache. Depending on its implementation, some resolvers only keep the NS RRsets from parent zone, which are called parent-centric resolvers. Other resolvers overwrite parent NS RRsets with child zone NS RRsets when receiving answers from child zone, which are called child-centric resolvers.

In real life, it's very common to have different TTL values configured for NS RRsets in parent and child zones, making caching time different between parent-centric and child-centric resolvers for the same query.

Parent and Child Zone Setup

To detect a resolver's centricity pattern, we set up a child zone and a parent zone and the delegation chain as shown in the figure blow.

The steps are outlined below:

1. We registered experiment.nz with two name servers, and for simplicity, pointed the two name servers to the same IP address. The following records are added to the .nz zone:

   experiment.nz. 86400 IN NS ns1.experiment.nz.
   experiment.nz. 86400 IN NS ns2.experiment.nz.
   ns1.experiment.nz. 86400 IN A 150.242.40.246
   ns2.experiment.nz. 86400 IN A 150.242.40.246
   
   (Note: 86400 is the default TTL for all delegations and cannot be altered)

2. We set up the experiment.nz zone:

   experiment.nz. 86400 IN NS ns1.experiment.nz.
   experiment.nz. 86400 IN NS ns2.experiment.nz.
   ns1.experiment.nz. 86400 IN A 150.242.40.246
   ns2.experiment.nz. 86400 IN A 150.242.40.246
   sub.experiment.nz. 10 IN NS ns.sub.experiment.nz.
   ns.sub.experiment.nz. 10 IN A 150.242.41.248

3. We set up sub.experiment.nz zone:

   sub.experiment.nz. 120 IN NS ns.sub.experiment.nz.
   ns.sub.experiment.nz. 120 IN A 150.242.41.248
   * 0 IN A 127.0.53.53

   We use the wildcard to match any query name in our experiment and set TTL of 0 so it will not be cached, which allows us to focus on the caching time of the referral.

RIPE Atlas

We used RIPE Atlas, an Internet measurement platform that provides thousands of active probes located around the world and a REST API to conduct measurements, and simulated DNS queries from end users. We selected 80 active probes located in New Zealand, all of which can be used to query public resolvers such as GoogleDNS and OpenDNS, and compiled a list of ISP resolvers' addresses for providers hosting probes.

Experiment

Each probe sent DNS queries to their resolver IP addresses at fixed interval for a period of time. Measurements for different resolvers were run in parallel to improve the efficiency, and measurements for the same resolver (multiple IPs) were run one by one to ensure they did not interfere with each other's data in the cache.

To help identify queries from different measurements we generated a unique query name every time by composing the query name using "resolver name" + "resolver ip" + "probe id" + timestamp of the run. For example, a query for name opendns208.67.222.222-10269-1480038557.sub.experiment.nz was sent by probe #10269 to OpenDNS's 208.67.222.222 ip address, and 1480038557 was the start timestamp of the program.

Result Analysis

From the query log we extract four fields to build query sequences at both parent and child name servers:

• timestamp (ts)
• resolver's service IP (probing target)
• probe ID
• resolver's source IP querying the name server

As the TTL for the queried record is set to 0, every query should be sent to the child name server. The query is sent to parent server as well only when the NS record is not found in the cache (first query or TTL expire) and the resolver has to ask parent name server for the referral. As each query name is unique, by checking the parent and child query logs, we are able to mark a query with 'P' if it is sent to both servers, or 'C' if it is sent only to the child server.

With a fixed query interval of 60s, we tested three different combinations of parent TTL and child TTL, and verified the expected patterns as below:

• parent-ttl=10s, child-ttl=120s

  PPPPPPPPPPPPPPPPP <- Parent Centric
  PCPCPCPCPCPCPCPCP <- Child Centric

• parent-ttl=5m, child-ttl=30s

  PCCCCPCCCCPCCCCPC <- Parent Centric
  PPPPPPPPPPPPPPPPP <- Child Centric

• parent-ttl=5m, child-ttl=10m

  PCCCCPCCCCPCCCCPC <- Parent Centric
  PCCCCCCCCCPCCCCCC <- Child Centric

There could be other patterns indicating minimum TTL enforcement, TTL stretching, etc.

We found for a single measurement (same target and same probe), the source IP address in the query log changes for each query. This may due to the architecture of a high performance resolver where a load balancer may be installed announcing the service IP address with two or more DNS servers behind doing the recursive resolution. The choice of server is unpredictable, and could be affected by many factors such as policy, server load, network performance or probe location.

We could also tell if the balanced servers were sharing caches (behaving as one server), by analysing the consecutive queries using the same service IP. If not, each source IP will be treated as an independent caching server, and be analysed separately.

For local ISP resolvers, we observed 33 servers (source IP addresses) in the query log with different patterns as below:

• parent-centric: 5 servers
• child-centric: 13 servers
• minimum (parent-ttl, child-ttl): 11 servers
• mixed pattern: 2 servers from the same resolver show a pattern mixed with child-centric and NS non-cached.
• NS non-cached: 2 servers from the same resolver send queries to parent and child with probing interval regardless of NS TTL value.

Among all the servers above, two from the same resolver were sending queries to child server with larger interval than probing interval, possibly because the 0 TTL of the queried record was replaced by a value of minimum TTL enforcement.

For public DNS resolvers, we get the following results:

• 6 addresses from OpenDNS, all of which behave as child-centric
• 73 addresses in four prefixes 173.194.171/24, 173.194.93/24, 74.125.41/24, 103.9.106/24 from GoogleDNS, which show a quite unique behaviour: most of queries use two different IPs to query the parent and child name servers. As GoogleDNS's addresses 8.8.8.8 and 8.8.4.4 are very popular probing targets on RIPE Atlas and easy to get conflicts, we only probed them with parent-ttl=10s and child-ttl=120s, in which a steady parent-centric pattern was shown, and could do the same for testing of other values in the future.

Future Work

In this post we detailed an experiment to detect the centricity of local ISP resolvers and public resolvers using RIPE Atlas with probes located in New Zealand. Interesting patterns and behaviours were found indicating different implementations and architectures of resolvers. Besides centricity, other factors such as TTL values and query intervals for different domain names also play a critical role in how much traffic volume to .nz name servers is reduced by caching, which we will explore in the future.

Several probabilistic models have been developed for retention prediction. In a survey paper written by Peter and Bruce, customer’s relationship with a company can be classified into four types (see Figure 1). In the domain name industry, there are two characteristics:

• Whether a domain name is renewed or not can be observed from the registry's database;
• A domain name is usually renewed for a certain length of period which can vary from 1 month to 120 months, 1 year is the most common.

Hence, the domain business belongs to the contractual and discrete type according to this two-dimensional classification. One commonly used retention prediction model for this type is the shifted-Beta-Geometric (sBG) model developed by Peter and Bruce in 2007. In this post, the objective is to predict domain retention probability of the .nz registry using the sBG model.

The sBG Model

The sBG model is based on two assumptions and I describe them in domain name service setting:

• A domain name remains active with constant retention probability $1-\theta$. From the definition of survivor function, we have:

  a. The probability of churn (the domain will not be renewed at $t$): $$P(T=t|\theta)=\theta(1-\theta)^{t-1}$$

  b. The survivor function (probability that the domain is still active at $t$): $$S(t|\theta)=(1-\theta)^t$$

• Heterogeneity in $\theta$ is modeled by Beta distribution with the pdf: $$f(\theta|\alpha, \beta)=\frac{\theta^{\alpha-1}(1-\theta)^{\beta-1}}{B(\alpha,\beta)}$$ where $B(\alpha,\beta)$ is the Beta function.

Individual domain's value of $\theta$ is unobserved (not measurable from the dataset), therefore the expectation over the Beta distribution (e.g., $E[P(T=t|\Theta=\theta)]$) is used to get a randomly chosen domain’s probability of churn and survivor function.

Compared with models that assume a constant churn rate for all customers, the advantage of the sBG model is that it takes customer heterogeneity into account. After some transformation, the retention rate is expressed in the following concise form:
$$r_t=\frac{\beta+t-1}{\alpha+\beta+t-1}$$

Domain Retention Prediction

The implementation of the sBG model lies in the estimation of the two parameters: $\alpha$ and $\beta$. In their paper (Appendix B), Peter and Bruce showed how to implement the model and compute the maximum likelihood estimates in Excel. We follow the same procedure and code it in Python (code can be found here) to find the $\hat{\alpha}$ and $\hat{\beta}$. The survival data shown in Table 1 are for the domains registered in April 2004 at three parent levels: co.nz, org.nz and net.nz.

We fit the sBG model to the first 6,7,8 years of the data for each parent level respectively to compare the accuracy of the estimation. The parameter estimation results are summarized in Table 2. Using these parameter estimates, the survivor function for each parent level is extrapolated out to year 12. The model-based results along with the actual numbers are plotted in Figure 1. The resulting predictions for the survival probability are quite accurate, especially when more data points are used to do estimation. It can be seen that the survival probability is decreasing and the decreasing rate gets smaller with time, meaning that less and less of domains stop being renewed. Another observation is that the survival probability of org.nz is higher than co.nz and net.nz. Hence, this model can help us to identify groups of domains with different retention behaviour so that we can analyse the characteristics shared within each group.

Another interesting plot is the retention rate. The model-based retention rates and the actual numbers are plotted in Figure 3. Although the model does not track the actual data perfectly, it fits the data on average and captures the trend. There are two reasons: firstly, our data points fluctuate a lot due to possible special events which makes the estimation harder; secondly, although the survival probability and the retention rate are closely related, the survival probability is easier to predict since it has a cumulative form which makes it less sensitive to period-to-period variations. It can be seen that the retention rates are increasing and the retention rate of org.nz is higher. In fact, this is similar to our observation found from the data. Figure 4 shows the retention rates (proportion of domains still active in the next year) for domains created at different periods. It can be seen that the mean retention rates are increasing with time and the domains with age of over 6 years have average retention rates over 0.90. On the other hand, retention rates of newly created domains have a much larger spread compared with older domains, indicating domains are unstable in early age.

Conclusion

In this post, the sBG model is implemented to fit the data of domains at different parent levels. The sBG model is a simple and powerful. It is interesting to see different retention behaviour among different parent levels. This model will be very helpful in identifying domains with different behaviours so that we can the factors/drivers behind.

The .nz Activity data set on Internet Data Portal keeps daily aggregated registration statistics for .nz domains. This article presents a time series analysis of .nz Activity data since 2012 using SARIMAX model in state space method of Python. While the .nz Activity data contains data of different kinds of measures, this article focuses on the monthly count of newly created domains. A few transformations are made to the data so that it can be read as a time series object. The Python code can be found here.

Visualize the data

The first step is to visualize the data to have a general understanding. It is important to discover the overall trend and/or seasonal trend so that we know which type of model to use. Note that we use data from January 2012 to December 2015 to build the model so that we can evaluate the performance of out-of-sample forecasting using data of 2016.

As Figure 1 indicates, there are some spikes in our data point. Particularly, the creations in September / October 2014 and March 2015 are higher than other months. Taking a further look at the data, it shows that: the jump in September 2012 was due to the launch of kiwi.nz; the spikes in September/October 2014 were due to registrations being allowed directly at the second level and at the end of March 2015 it was the end of the preferential registration and reservation period for second level .nz domains. Since these are all special events, they should be treated as outliers. Our observations are also supported by the boxplot of the data, see Figure 2.

The five outliers here are caused by special events and using a technique for dealing with outliers found here, we replace these outliers with the mean, which is 11396. The data after this treatment is shown in Figure 3.

As we visualize the number of creations per month, we can see that there is an upward trend and there is a seasonality to it. Figure 4 shows plots generated from the seasonal decompose function which deconstructs a time series into several components. The upward trend starts from March 2014 and the seasonality is every 12 months.

Make it stationary

The next step is to make the time series stationary, i.e., the mean and variance of our data points should be constant over time. Why is stationarity important? Most statistical models (e.g. linear regression model) are based on the assumption that the underlying data can be stationarized, so that the statistics (e.g. mean, variance, covariance) from the past can be used to predict the future. Otherwise, considering a data series with an upward trend as an example, its mean in the future will always be underestimated if the current mean is used as a statistical descriptor.

From the decomposition results, we already know there are a trend and seasonality. Another way to test it is to use rolling statistics and Dickey-Fuller test. It can be seen from Figure 5 since the trend is not very strong, the rolling mean and standard deviation does not vary too much over time. The p-value is $2.99e^{-3}$ which is not high.

Now let’s see if any improvement in the stationarity can be achieved. Discussions of several data transformation methods can be found here .

One commonly used method is differencing. The first difference of a time series is a series of changes from a period to the next. Hence, differencing can help us to eliminate the impact of the trend. By changing the number of periods, we can get the seasonal difference as well. Sometimes, both of the two transformations are used and we get the first difference of seasonal difference. For example, the first difference of seasonal difference in March 2014 is the equal to the March-February difference in 2014 minus March-February difference in 2013, given that the length of a season is 12 months. We take the first difference and the first difference of seasonal difference of the time series and compare their results.

It can be seen from Figure 6 that, there are still some obvious signs of seasonality (see data points around November 2012 and November 2014 as an example). The plot for the first difference of the seasonal difference is more stationary. Supporting evidence can be found by looking at the smaller p-value which is $1.48e^{-9}$. We now have a rough idea about how to set the parameters for the model. Next, we will plot some charts to further assist us to find optimal parameters.

The Seasonal ARIMA model

The Autoregressive Integrated Moving Averages (ARIMA) models are the most general class of models for forecasting a time series that can be stationarized. More information about ARIMA is available here. The forecasts are based on three parameters:

• $p$: the number of autoregressive (AR) terms, i.e., the number of lagged dependent variables.
• $d$: the number of non-seasonal differences needed to make the time series stationary.
• $q$: the number of lagged forecast errors for the moving average (MA) term.

A Seasonal ARIMA model is denoted by $(p,d,q)\times(P,D,Q)_s$, where $s$ is the number of periods per season and $(P,D,Q)$ are seasonal terms. As discussed above, we know $d=1$ since the impact of the upward trend is reduced after taking the first difference. Also, $D=1$ since the time series becomes further stationarized after taking the first difference of seasonal difference. Autocorrelation and partial autocorrelation graphs can help us with other parameters. Figure 8 shows the two plots for stationarized data.

Now we can identify our seasonal ARIMA model by observing the behaviour of the two plots, according to the rules for identifying ARIMA models . In the PACF, there’s a negative spike at lag 12 which may suggest an AR term for the seasonal part of the model. In the non-seasonal legs of PACF, there is a spike at lag 13 which suggest an AR term for the non-seasonal part of the model. Different combinations of these parameters are tried out to fit the model and the one with the lowest AIC value is selected, which is a (1,1,0)*(2,1,0,12) with AIC = 593.

Predictions and Forecasting

With the model built, we can make predictions and forecasting based on our data. Data observed from January - August 2016 is used to test the accuracy of our model. The predict command can generate the one-step-ahead prediction and dynamic prediction. One-step-ahead prediction always uses observed values to predict the next in-sample value, whereas Dynamic prediction is equal to one-step-ahead prediction only up to a certain data point and then the previous predicted value will be used to make predictions.

Here the dynamic prediction is performed starting in the October 2015. The mean value of predictions and their 95% confidence interval are shown in Figure 9. It can be seen that the one-step-ahead is slightly better. However, since it uses actual observed value for each subsequent prediction, the one-step-ahead prediction can only be used for in-sample testing. When we want to do out-of-sample forecasting, the dynamic prediction must be used since the actual value is not available. As shown in Figure 9, our model can predict pretty well. The prediction error is shown in Figure 10. The Mean Squared Error of the one-step-ahead and Dynamic prediction is 58112 and 143150 respectively.

Finally, we can make the out-of-sample forecast for September 2016 - July 2017, which is shown in Figure 11. It can be seen that the forecasted value captures both the seasonal and upward trends.

Conclusion

Through this blog we have explored the ARIMA model and the statsmodel package in Python by creating a time series forecast for .nz activity data. The principle advantage of ARIMA is the comprehensiveness of the family of models. Features behind the data can be nicely captured by identifying the model appropriately. This procedure can be easily applied to other time series data, e.g., a different ccTLD data, with similar​ data available.

With the recent rollout of new TLDs, the opportunity to create new, meaningful domains has presented a range of opportunities for the global internet community.

Almost all new TLDs have provided opportunities for registrations of domains exclusively at the second-level (i.e. ‘anything.something’), as having domains at the third level (i.e. ‘anything.anything.something’) is seen as cumbersome and unnecessary in newly created TLDs.

However this is not the case globally, as many TLDs (predominantly ccTLDs, or “country code TLDs”) offer domain registration only at the third level.

Forward thinking ccTLDs such as the .uk (United Kingdom) and .nz (New Zealand) extensions have recently undertaken extensive reviews, which ultimately led to the decision to change policy framework and allow registrations at the second-level.
I recently sat down for a frank discussion on the merits of and experiences in introducing ccTLD registrations at the second-level with Oliver Hope, Director of Registry Services for Nominet and David Morrison, Chief Marketing Officer at NZRS (the .nz Registry) to understand the methods used to gain grass roots approval of the change, the impact the change has had today and what may be expected as we move into the future.

TONY KIRSCH: Let me start off with a slightly historical perspective. How did opening up at the second-level even come about? What were the drivers behind it? I think it’s important for people to understand the philosophy, because it’s a big shift and I’m sure that for many you’ve spoken to about this, it could at face value appear like a bit of a scam.

OLIVER HOPE: A big part for us was just that the internet and the industry are changing. More and more people and other domain alternatives are direct. For the UK, one of the biggest ccTLDs there is, everyone knows us as predominately .co.uk, and that’s great. But if you’re a business trying to attract a customer in France, New Zealand or Australia, it might not necessarily be their natural instinct to go to .co.uk. They’re used to .fr in France, or .de in Germany, you know the list is endless and weighted that way. We have to accept how the industry is changing and what people are getting used to. There are a few other things as well- it appeals to new groups and opens some space up, because we’re quite saturated with 10.5 million domains, even though in comparison to .com, this isn’t many.

TONY KIRSCH: What about in terms of meaningful domains?

OLIVER HOPE: In terms of meaningful domains, .uk is saturated. The change also responds a bit to the challenges of new gTLD introduction. You know everything out there only really has one dot and we had two which is a little bit of a blocker. Additionally, one of the key things is it’s shorter. Certainly Twitter has proved that in recent years, shorter and sharper equals better. If you say to people “Do you want me to make your email or your website shorter by three characters?”, someone who knows what they’re talking about is all for that.

DAVID MORRISON: On the New Zealand side, our reasons are very similar. The evolution of domain names particularly in the country code space has gone from a ‘something.something.something’ to a two-level structure ‘something.something’. So for us, it’s very much about keeping it fresh and relevant. The other point also is that within the .nz space we have 14 second levels such as .org.nz, .net.nz, .kiwi.nz and.geek.nz. So we had this categorisation in place, and if you wanted to get a domain name you had to decide which category you were in. “Am I a company, an organisation? Well, no not really, I’m a sports club. So I don’t really fit into an .org.nz or a .net.nz or a .co.nz”. So it’s about giving people true choice and not forcing them to categorise themselves in a way that wasn’t appropriate for their organisation.

TONY KIRSCH: So the categories became a limiting factor?

DAVID MORRISON: Correct, and less meaningful over time as the internet has evolved and become more relevant for more people.

TONY KIRSCH: That’s interesting. So let’s play the role of the current registrant for a moment. Is the introduction at the second-level a challenge for the current registrant? For example, if I’ve registered my .co.nz or my .co.uk, wouldn’t I think that you’ve introduced a competitor for me?

DAVID MORRISON: I think both .uk and .nz have tried to treat that in similar ways. For New Zealand, we allowed domains that were registered prior to a certain date to have priority on second-level registrations. So if you had a third-level domain name that was unique across the zone, prior to a particular date, you had the right to reserve the shorter name at no cost for two years. Then at the end of those two years, there’s an expectation that you would register it.

TONY KIRSCH: And if you haven’t, it would go back into the pool?

DAVID MORRISON: That’s still to be decided. There’s policy review occurring later this year. But in terms of initial introduction people had a choice to say “I do want to protect it, but I don’t want to have to outlay out the money for it”. We had about 20,000 domain names take up that option and that has resulted in about 18,000 reservations currently in place which are expected to fall due around the 30th September 2016, so we might see some activity around that. Additionally, where there were multiple domains which matched in differing second levels we had a different process to .uk where we had deemed those domains to be in conflict. In fact, many of those names are still in conflict. We have about 17,000 or 18,000 conflict sets or approximately 40,000 domain names. Most conflicts were people who conflicted with themselves where they had for example a matching .co.nz and a .org.nz registration. Those two policy decisions helped alleviate a lot of the negative outcomes. TONY KIRSCH: What happened with the conflicts? Were they unavailable?

DAVID MORRISON: The conflicts stay locked until the parties reach a resolution and no one’s been forced to come to a resolution.

TONY KIRSCH: That’s interesting. Did you facilitate those introductions between conflicting registrants?

DAVID MORRISON: The Domain Name Commission, the regulator for .nz, set up a website to facilitate people being able to lodge their preference and encourage the use of the WHOIS to look up who the other parties were and get in contact directly themselves.

OLIVER HOPE: We’re similar at .uk but probably not as complex because we don’t have as many second-level zones. The main ones that we have are .org.uk, .me.uk and .co.uk. There are some others but they’re a bit more specialised and very low volume. We have a five-year right of refusal which ends in 2019. The pricing is not free but rather the standard registration fee. In terms of conflict, originally we were going to give priority to the earliest-registered name across the main three levels, but after consultation we changed that to .co.uk having preference if it’s before certain dates. Our difficulty is we have approximately 2,700 members, a huge channel and you’re never going to please everyone so you put it to consultation. The danger is ending up with a horse designed by a committee – in other words, a camel. We ended up with this five year right of refusal, which, while far from perfect, was less problematic than many of the alternatives.

TONY KIRSCH: Policy changes are never perfect and wonderful for everyone. So what happened?

OLIVER HOPE: There are some people who absolutely love it and said we should just give everyone their .co.uk match for free. But then there’s other people who are more commercial who would absolutely hate that. Then there are people who think we should just open it up as a new launch and sell to anyone. Finally, then there are people who think you should only ever be able to buy if you have prior use, so you try to weave your way through a very complex minefield and it’s good to see both .uk and .nz end up in similar places. We’ve both done the same exercise with very similar products, and ended up in roughly the same place.

DAVID MORRISON: We’ve seen some really interesting current activity in the registry. Registration at the second-level accounts for just under 16 percent of active registrations. That’s in the space of less than 18 months. Nearly a third to a quarter of all creates are at the second-level. So, we are going to see some change in registrations at the third-level. TONY KIRSCH: Over what time period do you expect this to occur?

DAVID MORRISON: It’ll be a long time. Some of our second levels such as .org.nz and .net.nz are declining whilst direct registrations under .nz are growing.

TONY KIRSCH: Were they declining already?

DAVID MORRISON: They were flat and growth in .co.nz was also starting to flatten. We’ve got approximately 480,000 .co.nz domains and 104,000 .nz domains.

TONY KIRSCH: That’s really impressive!

DAVID MORRISON: If you were to compare us to a gTLD launch for example, at the same time we would have been about number 10 or 11 on the new gTLDs table. No one’s heard of us or the results, so we just go about our business and do our thing at the bottom of the world. But I think we’re going to see a transition to .nz over time. We did some deeper analysis and found that in the last year about 60,000 .co.nz domains that were newly registered did not register the .nz version. And 35,000 of the .nz registrants did not register the corresponding .co.nz. So the behaviour we are seeing from the registrants is that they’re not registering other alternatives, possibly because they aren’t aware or not seeing the need to register both. They’re either making a conscious decision to go for .co.nz and not care about the shorter .nz version or vice versa. Within the general public it’s about 37 percent that have awareness of the shorter version of the name. And within registrants, we directly surveyed some of our Registrars’ customers last year with their permission, and 77 percent of people had awareness of the second-level.

TONY KIRSCH: Oliver, is that symptomatic of what you were saying in regards to .uk?

OLIVER HOPE: We’re doing things slightly differently and engaging in a lot of promotions. We’ve actually just finished one where we gave away right of refusal domains. You go to a Registrar and if you had the right of refusal to your .uk, you could register it for free. We’re doing things like that where you buy one and get one free. We’re really trying to market and push .uk because that’s the new product that we want everyone to be aware of. However, with regards the future, we’re flexible and open. We’re not 100 percent sure what we will do in 2019. We might have 8 million rights holders still or we might have only half a million. We see about a third of all the new direct .uk domains are not pre-existing rights holders, they’re brand new domains.

DAVID MORRISON: Yes, there’s been a similar experience for us.

OLIVER HOPE: And obviously you can’t register if someone owns the .co.uk already or if they have rights, you cannot register the .uk. So that’s what’s really quite interesting – that it’s quite a high percentage of people, they’re new customers buying new domains that don’t currently exist.

TONY KIRSCH: And they’re not trying to steal other people’s brands or digital identities?

OLIVER HOPE: No.

DAVID MORRISON: No. We really haven’t seen a lot of behaviour of people trying to steal brands. They’re coming up with the name and then going on to register it; the support of registrars is really important with this. About 26 to 30 percent of .nz domains are creates, so I can look at the Registrars where the proportion is either the same or greater than this. If Registrars are at that or above then it indicates they’re doing a reasonable job of promotion and making people aware of the shorter version. Where Registrars are not, it’s an indicator that they may not have a great awareness of the New Zealand market, or that they’ve got deep resale channels, so they’ve enabled their systems to support .nz, but their resellers downstream have not turned on similar functionality. We’re seeing some registrars with 97 percent of creates still at .co.nz.

TONY KIRSCH: Which lets you go and engage them on that then?

DAVID MORRISON: That’s right, but we aren’t about pushing, at the end of the day we’re all about the registrant being able to make a choice so we’re not going to force someone to choose .co.nz or .nz. We’re a not-for-profit entity so we’re not doing this for profit-driven reasons. It’s about ensuring that the registrant has the option of choice, and that at point of sale those options are available to them.

TONY KIRSCH: That’s a really important point, because I think there is a public perception of opening at the second-level as a bit of a money grabbing exercise. In particular from the larger corporates who are contacted every day with offers to register their name under different TLDs. Not only are you a not-for-profit, it almost to me sounds like you have a philosophy that there is a need to change but with adequate protections. You can’t please everyone but it sounds to me like both of you have done the best that you can to protect the people that are your current customers, but at the same time position yourselves for the future.

OLIVER HOPE: Yes, you have to protect your business for your own good and the good of your members. Of course, some people aren’t going to like the way that is, that’s not ideal but we will always do what we think is the absolute best thing for that whole organisation and the UK internet.

DAVID MORRISON: InternetNZ does a wide range of things that we make quite transparent in terms where money is spent, so when people call this a money-grabbing exercise we can talk quite transparently about what we do as a group of organisations, why we made this decision and what would happen if we just sat back and didn’t evolve with the domain industry. Over time there’s a possibility that growth and revenues may stagnate, which means the society can do less.

TONY KIRSCH: I think Oliver had a good point of it being about consistency. The truth is that over the next three to five years, if not already today, there are a lot of people in a lot of countries that think of that second-level structure ‘something.something’.

DAVID MORRISON: Yeah, that is where it’s going.

OLIVER HOPE: It needs to join up.

TONY KIRSCH: Five years from now, how do you see it working? Do you see the things working in parallel where each zone has their niche, or does one start to take over from the other?

OLIVER HOPE: I don’t actually have a clear answer because it really depends on adoption, and part of that will take time because if you’re a local building firm and you’ve got your web address on your van, you’re not going to get your van resprayed because you want to take three characters off it. When you buy your next van, it’s probably more likely because it’s shorter. But, that’s that slow burn of growing adoption. It’s only 20-odd years really since the internet kicked off. Five years down the line in our industry it could be so wide, the sphere of what it could be is hard to predict.

DAVID MORRISON: Within the New Zealand space, .co.nz is so ingrained into the psyche, so when you mention a website, the default is .co.nz, so I think we’re talking generations here. Five years from now, I think .nz will have a greater share of the pie and it will grow from 16 percent to maybe 20, 30 or 40 percent, but .co.nz will still be a dominant player for a long time to come. There are a lot of established brands and a lot of small organisations with limited funds that aren’t going to rebrand because of a domain name change. They may not see the value of it. With new businesses coming on board, I think they will adopt it but it’s going to be a long, slow change. Then again if it doesn’t happen, it’s actually not the end of the world. Because we’re there to provide choice to our customers and if they choose .co.nz as their preferred place to play, while I don’t think that will be the case, that’s okay but the choice is there.

TONY KIRSCH: So the idea is not necessarily to pit them against each other?

DAVID MORRISON: No, not at all.

OLIVER HOPE: No.

TONY KIRSCH: It’s all about choice and ensuring consistency with global standards and mainstream behaviours then?

DAVID MORRISON: Yes very much so. We have our own ‘micro-brands’ or options within our own space, it lets us say “there’s a range of options here; each hold their own separate values and identities and you choose the one that is right for you.”

TONY KIRSCH: Thank you both so much for your time. Opening up at the second-level for ccTLDs like yours sounds remarkably similar to the principles of new TLDs. I think it’s great that we realise that the change is probably not for our own immediate benefit, but rather it’s for our kids so when they establish their startups or any other online activity requiring a domain name, they have real choice available to them.

When kids grow up and they go to a Registrar they won’t have this hang up of what we have. We think a direct registration at the second-level is innovative but they won’t even know any different, it will just feel like it’s been like this forever as we do with colour TV for example.

All the best with your future endeavours in this space – this is really great information for both the DNA members and the wider community. We really appreciate you sharing your time and insights with us on this important topic.

Memoryless Beauty: Markov Chain

Markov process, named for Andrei Markov, is a stochastic process that has Markov property which states that the future state of the process is independent of the past, given the present. It has been regarded as a simple and classic way to analyse complicated stochastic processes because of its 'memoryless' property.

To better understand how it works, let us first look at a very interesting example which can be naturally modelled by a Discrete Time Markov Chain (DTMC).

Gambler ruin problem

Let's consider a gambler who has \$100. He bets \$1 each round of the game, and wins with probability of 0.5. He will stop playing when he either gets broke or wins \$1000. You may ask: what's the probability that he goes broke? or, in the long run, how many games will he play before he stops? Such questions can be answered by modelling it with DTMC.

The amount of money he has at the end of each game is called the state of the process. Here we have 1001 possible states that forms the state space. The transition from a state to another can be represented by a States transition diagram shown below. The nodes represents the states and the number over the arrows stands for the transition probability.

It can be seen that the states 1,2,...,999 can communicate with each other since one state is reachable from another. On the other hand, state 0 (broke) and state 1000 are absorbing since there's no escape from those states. They are also called the closed class. A markov chain is said to be irreducible if all the states communicate with each other. For the Gambler Ruin problem, it is not irreducible since we have two closed classes.

We can also list the transition probabilities in a transition probability matrix. Assuming we have a state space with states 1,2,...,r, then the transition matrix denoted by $P$ is determined by the following.

$$ P=
\begin{bmatrix} P_{11} & P_{12} & \cdots & P_{1r} \\ P_{21} & P_{22} & \cdots &P_{2r} \\ \vdots & \vdots & \ddots & \vdots \\ P_{r1} & P_{r2} & \cdots & P_{rr} \\ \end{bmatrix} $$

Given that we are in state $i$, the next step must be in one of the possible states. Hence, each row in the transition matrix must sum to one. That is:

$$\sum_{k=1}^{r} P_{ik} = \sum_{k=1}^{r}P(X_{m+1}=k|X_{m}=i)=1, P_{ij}\geq 0, \forall i$$

Domain Retention Prediction

Now we are ready to investigate the domain retention prediction problem using DTMC. There are four possible states of a domain name: Active, Pending Release, Expired and do-not- exist. We are interested in the probability of a domain to stay in the register. Since the do-not-exist state is not reachable from other states and the transit from the do-not-exist is beyond the scope of this study, it is not considered here. The transition diagram is shown below. A simplified lifecycle diagram can be found on GetYourselfOnline.

From the diagram it can be seen that all the states communicate with each other, therefore it is an irreducible chain. Moreover, note that there is a positive probability of remaining in a state if the chain starts with that state. A beautiful thing about an irreducible markov chain is that it has a stationary distribution, which is the long term behaviour of the chain. The stationary distribution $\pi$ is a vector obtained by solving a set of linear equations $\pi = \pi P$.

Given the sequence data set containing states of domain names at different time points, we can estimate the transition probability matrix by using the following equation:

$$ \hat{P} _{ij} = \frac{n_{ij}}{\sum_{k=1}^{r}n_{ik}} $$

Where $n_{ij}$ denotes the number of times the chain moved from state $i$ to $j$. The denominator, $\sum_{k=1}^{r}n_{ik}$ is the total number of transitions out of state $i$. Estimating the entries like this also corresponds to the maximum likelihood estimator of the transition probability matrix.

Having these in mind, we can do predictive analysis to answer questions like:
1. Given that a domain name is active today, what is the probability that it is active / pending release / expired after 3 months, 6 months, 9 months?
2. If a new domain name is registered today, what fraction of its life will it spend in each of the three states?

Next Steps

In this post, we have explored DTMC and its application on domain retention prediction. Some preliminary results have been obtained from a simulation using Python and a data sample from the .nz registry. Another interesting idea for predictive modelling is to use a Markov chain classifier. A portion of the factors considered included:

• Registration Information
• Status, purpose, and information of the website for the domain name
• Historical information, e.g., length of time being active in the register, website traffic, total number of renewals
• Setup indicators, e.g. the domain has a nameserver, web server or mail server in use

One main objective of the domain name retention study is to find factors mostly influencing the probability of renewal, i.e., the KPIs associated with domain names. With that knowledge, registrars can better understand their customers and provider better service to attract high-value domain registrations to increase income. A follow-up blog post will discuss results and insights in details.