« Cyber Monday safety advice from a late 1960s Rhythm and Brass Band | Main | A practical resource for managing advanced persistent threats (APTs) »

Friday, 02 December 2011

In search of... Foreign Language Spam Filter for Apple Mail (Mail.app)

Foreign language spam is a problem everyone shares, not just folks who read English. If you don't understand a language, and can't read the characters of the script used, you probably won't get much value from messages you receive. Worse, you won't understand the "text" of the links embedded in the message.

Common sense would dictate that you don't click on something you can't read but decades of evidence proves otherwise.  A practical solution to foreign language spam is to prevent it from appearing in your In Box.

If you are an Apple Mail (Mail.app) user, and your organization isn't filtering foreign languages at a mail server or antispam gateway, you'll find that filtering foreign spam is problematic for several reasons. First, there are thousands of languages. Many of these can be written using characters from one or more scripts and many of these can be encoded using  one of (again) many character sets. I receive mail daily composed using Arabic, Chinese, Russian, Japanese, and Korean. Examining the raw source email  From: and Subject: headers of these messages reveals they are encoded using UTF-8, windows-1251, GB18030 iso-2022-jp, KOI8-R, respectively. Filtering based on  these encodings alone is a messy Mail Rule.  

Filtering based on the "language" found in email messages has similar scaling and messiness issues. The problem is compounded by the fact that email messages encoded to support languages that are not represented using 7 bit ASCII can also have multiple body "parts" (as well as attachments) . Body parts are typically distinguished and bounded by the use of Content-Type: headers. Apple's Mail.app seems to examine only the first encountered of this header type, which is often (perhaps intentionally by spammers) multipart/alternative or multipart/mixed so a rule that is supposed to filter  text/plain; charset="windows-1251" as junk will fail to match.

I found a solution at Buzzdroid that works on the principle that certain words (of, the, and, is, a) are found in nearly any rationally composed English message body. Buzzdroi'ds logic is straightforward: if these words are not present, the message is probably not "readable English". A Mail Rule of this kind looks like this:

Foreignspamfilter

My variation to this theme (below) is working well, as the Junk folder illustrates:

Foreignspam

This is an imperfect solution to be sure. For example, Exchange Calendar invitations match the rule. Tempting...  I created an exception rule "Calendar" and made certain the rule preceded the Foreign Language Spam rule.  Occasionally I receive email composed as if it were a text or Tweet and for these, I may whitelist the sender. After 10 days of testing, I'm happier with a 1 in 10 false positive in my Junk folder than the distraction of messages I cannot read in my In Box. 

As the Buzzdroid folks say, "Any Apple Mail Rule should only be the last line of defense against spam." This problem can be handled at mail servers or antispam gateways but there is an administrative cost for organizations with language diversity: while it might serve me to block all languages that do not use characters from Latin alphabets, many of my colleagues would be inconvenienced. 

I'm curious to hear from others whether a similar approach would work for other languages. If you come up with an Arabic, Chinese, Japanese, Korean, or Russian "only" Rule for Mail.app, let me know.

Posted at 10:37 AM in Anti-Malware and Anti-phishing, Stop Think Connect |

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

Find me on Mastodon and Facebook

Spotlight Posts

  • FTC's proposed Trade Regulation Rule on Impersonation of Government and Businesses is important for #infosec
    M3AAWG comments on the FTC's proposed rule Trade Regulation Rule on Impersonation of Government and Businesses is available. I was one of the contributors to the comment. In the comment, M3AAWG "suggests additional regulatory solutions and best practices to complement the goals of this rule, such as clarifying the scope of the rule to include the use of domain names in impersonation schemes and the use of technologies that enable impersonation" and the important role that Whois plays in investigating impersonation and fraud. Several reports that my Interisle colleagues and I published are cited in the comment, along with the...
  • Cybercrime Information Center reports sharp increases in phishing attacks, unique domains reported in August-October 2022 time period
    A recent ebb in domain names reported for phishing created some euphoria in the domain industry. With phishing activity, as with tides, ebbs are invariably followed by flows. In their August- October 2022 Quarterly Phishing Activity, the Cybercrime Information Center observed a 40% increase in phishing attacks and a nearly 20% increase in unique domain names reported for phishing.
  • Cybercrime Information Center's YouTube Channel
    My colleagues and I at Interisle have created a YouTube channel to complement the quarterly reporting and studies of cybercrime we host at the Cybercrime Information Center. Our goal is to share, in a few minutes, the findings and insights from our research in a format that's easily accessible and shareable, informative, and entertaining. The inaugural videos provide 2-5 minute summaries of our annual Phishing and Malware Landscape studies. We'll continue to produce videos of our annual studies. If you like what you see, leave us a comment.
  • Malware Landscape 2022: unabated malware growth, continued exploitation of IoT devices
    My colleagues at Interisle and I have published a study, Malware Landscape 2022: A Study of the Scope and Distribution of Malware. The study, which analyzed 2.5 million records of distinct malware events from May 2021 to April 2022 collected by the Cybercrime Information Center, explains what malware was most prevalent, where malware was served from, and what resources criminals used to pursue their attacks.
  • New TLDs are coming #Dangerclose.
    A Domain Name Wire post, Time to pay attention to the next round of new TLDs, begins with an ominous: They’re coming. Eventually. While not as dramatic or enduring as Arnold Schwarzenegger's "I'll be back", the reporter cites policy activity at ICANN as evidence that new TLDs are coming. Eventually. In a September 2019 post, and in response to the ICANN memorandum, Readiness to Support Future Rounds of New gTLDs, I asked, Has enough been done to study and rectify the concentration of security threats in the new TLD space? In that post, I quoted correspondence from ICANN's security advisory...

Search

My Photo

Categories