Anti-Malware and Anti-phishing

Conservative abuse reporting throws new TLD program under the bus

ICANN has released a January 2019 domain abuse report generated from the Domain Abuse Activity Reporting system (DAAR). DAAR is a system for studying and reporting on domain name registration and security threat (domain abuse) behavior across top-level domain (TLD) registries and registrars. While at ICANN, I was actively involved with DAAR from inception to early production, and I’m pleased that ICANN has begun monthly reporting.  

According to ICANN’s DAAR project page, one purpose of the project is to “provide the ICANN community with a reliable, persistent, and reproducible set of data from which security threat (abuse) analyses could be performed.” The January 2019 report provides a distribution of domains identified as security threats and a breakdown of security threats by class: phishing, botnet command-control, spam, and malware hosting. It does so for all new and legacy registries for which the DAAR project can collect TLD zone data. 

The report is promising in the sense that ICANN has finally begun reporting abuse, so we should celebrate this landmark publication.

The report is disappointing for several reasons.

Top-level Domain Reporting Tells a Partial Story

The report provides only summary statistics for the new and legacy Top-level Domains (TLDs), in pie-chart format, and thus provide the operational community with “findings” that are not actionable. Most importantly, the data tell a largely misleading story. To explain further, I’ve reproduced two pie charts from the report, side by side:

Screenshot (84) Screenshot (85)

Together, these charts say the following:

Over one-half of the domains identified as security threats are
registered in one-eighth of the generic TLD name space.

This is an actionable conclusion.

Let’s assume that you wear a network or email administrator’s blocking policy hat. An important part of your role is to mitigate risk for your organization in the most straightforward and expeditious manner. Your pragmatic conclusion from this finding may well be…

Administrators can reduce an organization’s exposure to over one-half
of domain-related security threats by BLOCKING ALL new TLDs.

Administrators can implement this security policy at firewalls, mail servers, proxies, or DNS resolvers. It is an appropriate blocking rule based on the limited insights that ICANN’s report offers.

Conservative reporting in this case does more harm than good. By failing to be open and transparent about the high levels of abuse in new TLDs, ICANN actively frustrates efforts to promote Universal Acceptance of domain names and email addresses and calls future new TLD delegations into question.

An Opportunity Lost

Later in the report, ICANN could have helped administrators refine blocking policies, where it reports that

“of the 781,795 domains identified as security threats reported in
 341 new gTLDs:

• 35 percent were in the 5 most-exploited new gTLDs.
• 52 percent were in the 10 most-exploited new gTLDs.
• 88 percent were in the 25 most-exploited new gTLDs.
• 98 percent were in the 50 most-exploited new gTLDs.”

Administrators now know only 341 new TLDs have abuse domains reported. That’s less approximately one-third of the generic TLDs. They also now know that 98% of the new gTLD security threats are concentrated in 50 of new gTLDs, so they can refine the above rule:

Administrators can reduce an organization’s exposure to nearly one-half
of domain-related security threats by BLOCKING 50 new TLDs.

Except… ICANN does not publish the names of TLDs in the report.

With only the findings that ICANN’s report publishes, the most risk-averse blocking rule we can compose throws nearly the entire new TLD program under the bus.

The Truth is Out There

ICANN’s Context Document for the January 2019 report lists the reputation feeds that DAAR employs. The largest and most popularly used among these are the Spamhaus and SURBL data. With some work, you can make use of the statistics published at Spamhaus World’s Most Abused TLDs or SURBL’s Most Abused TLDs page to approximate the 10, 20, or 50 new TLDs that DAAR reports as concentrations of abuse domains. But you have to ask, “Why would an administrator go through this effort if the governance body for domain names is unwilling to commit itself to publishing actionable data or at least data that can inform its own policy community where the problems lie?”

Naming may be shaming, but it can also be enlightening. It’s been my experience that some TLD operators do not actively investigate abuse. Some TLDs have had high badness indexes for years.  Public disclosure might be the forcing function that instigates change in abuse mitigation.

To illustrate a use case for naming to enlighten, I’ve reproduced recent values for the Spamhaus “badness index” for the city TLDs delegated as new TLDs:

amsterdam = 0.0% bad
barcelona = 0.0% bad  
brussels = 0.0% bad  
capetown = 0.0% bad  
kyoto = 0.0% bad    
moscow = 0.0% bad
москва = 0.0% bad
stockholm = 0.0% bad
taipei = 0.0% bad
wien = 0.0% bad
佛山= 0.0% bad
广东  = 0.0% bad   
hamburg = 0.4% bad
nyc = 0.4% bad

berlin = 0.5% bad
sydney = 0.6% bad  
melbourne = 0.7% bad
paris = 0.7% bad
vegas = 0.9% bad
cologne = 1.2% bad  
miami = 2.0% bad
quebec = 3.0% bad
istanbul = 4.0% bad
osaka = 9.1% bad  
nagoya = 45.0% bad
yokohama = 44.0% bad
tokyo = 59.5% bad

Using TLD data only, several Japanese cities appears to be loci of abuse. Like or not, many organizations use Spamhaus or SURBL data and may choose to block all of the domain names delegated from these TLDs.

But again, you’re only seeing part of the story. Ask any operational security investigator or reputation feed operator and they will tell you that Attackers and criminals are TLD agnostic or opportunistic. Sponsoring registrar data often reveals more about the loci of abuse than TLD data.

No Data on Registrar Abuse

For reasons that only ICANN can provide, the January 2019 report does not contain registrar data. Again, the Truth is Out There. Spamhaus also publishes a Top 10 Most Abused Registrars List, where you can observe that Asia-Pacific is a geo-locus of registrars with extraordinarily high badness indexes. If ICANN were to publish registrar data, administrators might be able to learn more about the interrelationships between TLDs and registrar abuse, and refine blocking policies accordingly; more importantly, ICANN community would have data that could influence future registrar accreditation agreement deliberations.  

Urge ICANN to Meet Commitments

The ICANN community harps about the need for ICANN organization to operate transparenly and accountably. The community, especially the contracted TLD and registrar parties, should be held to the same standards. Urge ICANN to publish data that identify the outliers. Urge ICANN, too, to publish registrar data. These data are essential if ICANN organization and community intend to meet the commitment to “provide the community with a reliable, persistent, and reproducible set of data from which security threat (abuse) analyses could be performed.”

APWG and M3AAWG Survey Finds ICANN WHOIS Changes Impede Cyber Investigations

The Anti-Phishing Working Group (APWG) and the Messaging, Malware and Mobile Anti-Abuse Working Group (M3AAWG) have collaborated to conduct a survey of cyber investigators and anti-abuse service providers to understand how ICANN’s Temporary Specification for gTLD Registration Data has affected their access and usage of domain name registration information and their ability to mitigate abuse. I served as Principal Investigator for APWG and M3AAWG for this project. I received strong subject matter expertise support from both working groups.
From our analysis of 327 survey responses we find that the changes to WHOIS access following ICANN’s implementation of the Temp Spec is significantly impeding cyber applications and forensic investigations and allowing more harm to victims.
The "Temp Spec" has introduced delays to investigations and the reduced utility of public WHOIS data is a dire problem. The loss of timely and repeatable access to complete WHOIS data is impeding investigations of all kinds, from cybercrime activities such as phishing and ransomware, to the distribution of fake news and subversive political influence campaigns.
The report contains a detailed analysis of the sets of questions asked to an targeted audience of cyber security practitioners, anti-abuse service providers and law enforcement officers, who were contacted by primary using APWG and M3AAWG mailing lists, augmented with trust collaboration mailing lists used by operational security and law enforcement to share threat intelligence data. The analyses are complemented with comments submitted by survey respondents. Many of these are quite insightful.
From the analyses, the APWG and M3AAWG make the following findings:
  1. Cyber-investigations and mitigations are impeded because investigators are unable to access complete domain name registration data.
  2. The mitigation or triage of cyber incidents cannot be accomplished in a timely manner.
  3. WHOIS has become an unreliable or less meaningful source of threat intelligence.
  4. Requests to access non-public WHOIS by legitimate investigators for legitimate. purposes are routinely refused.
  5. Those who protect Internet resources are also making more coarse blocking or mitigation decisions in the absence of what was formerly reliable data. 
  6. The utility of WHOIS has been severely damaged.
  7. The redaction of WHOIS data is excessive.

APWG and M3AAWG make a number of recommendations as well:

  1. There must be an accredited access mechanism, providing tiered or gated access to qualified security actors.
  2. ICANN should not allow redaction of the contact data of legal entities.
  3. ICANN should adopt a contact data access request specification that will ensure consistency across all accredited registrars and gTLD registries.
  4. ICANN should ensure that the accredited access to redacted WHOIS data does not introduce delays in collecting or processing WHOIS data, and further, that the access not be encumbered by per request authorizations.
  5. ICANN should reconsider the current redaction policy.
  6. We ask that ICANN publish point of contact email addresses to provide investigators with an effective means of identifying domains associated with a victim or person of interest in an investigation.

In their final comments, the Working Groups encourage ICANN to improve the current, difficult condition, stating:

"We recognize that ICANN is likely aware of several of these issues. We also realize that ICANN organization and Board of Directors are awaiting the Expedited Policy Development Process for answers to many issues; however, we believe that the ICANN Board of Directors and ICANN organization have the ability to update the Temp Spec to fix the problems that this survey and others have identified as most pressing or egregious while the EPDP work continues."

It's essential that ICANN  implement recommendations 2, 4, and 6 and quickly. From a public safety perspective, these are necessary adjustments. These fall within ICANN's remit to ensure security and stability of the Internet's Identifier systems. ICANN organization should further ensure that the parties involved in consensus policy development for the remaining recommendations consider the findings and analyses in this survey. This would be consistent with the organization's expressed desire to apply data to ensure informed policy deliberation. 


Download FINAL ICANN GDPR and WHOIS Users Survey 20181018.pdf (1683.6K)