Gregory J. Cook, EA, CPA Accredited Tax Advisor

How we stopped 99.887% of the spam traffic to our mail boxes and 100% of the viruses! (Update 12-10-04) Successfully fighting spam is difficult, expensive and to be effective, we found that it requires multiple approaches. There are many purported "solutions" out there. In our search for a solution our requirements were as follows:

Guidelines we used for our requirements...

1. We operate and maintain our own mail server "in-house" for security and confidentiality. We do not allow our mail to pass through any third party for these reasons as well as potential liability.

2. We decided early on in the process that any solution(s) implemented should also fight and protect against viruses as well as spam.

3. We placed a high priority on insuring that legitimate mail would pass and flow through the system without being inadvertently blocked or lost.

4. Our second priority was to take as much of the time consuming day-to-day management out of the hands of the end users. Because we are a small firm and do not have a person on staff that acts as "full-time" Mail Administrator, this presented some unique obstacles. We found that a fully automated process controlled and run by computers 100%, was not realistic without some human intervention on a daily basis. Note: Simply "tagging" or labeling a message as spam and allowing it to be delivered to the user mail box, is not a solution by our standards.

5. Finally, the cost of our system must be reasonable, relative to our anticipated volume of legitimate mail and within our budget.

Solutions we put in place with 3 Lines of Defense...

1. We set up a dual processor, dual/mirrored hard drives, public and private nic's server at a cost of approximately $2,500. We chose Novell 6.0 for the OS and Novell GroupWise 6.5 for the mail software (approx. $2,000). We feel that using Novell greatly reduced our exposure to viruses by virtue of the fact that most viruses seem to be targeted to Microsoft products. The GroupWise Internet Agent (GWIA) allowed us to use real-time blacklists (RBL's) and our own in-house black-list to successfully block 4,204 junk mails of the 8,049 we were receiving each day! This represented a 52% reduction right away by simply denying connections. In addition, we set the software to not do anything with messages addressed to non-existent users (known as dictionary spam), by default it denies them and returns an "undeliverable" to the sender. This prevented our post office from processing and getting clogged-up with 3,159 undeliverable messages per day. So, only 686 messages out of 8,049 per day get past our first line of defense, the GWIA, reducing the workload on the server by 91%.

2. We chose McAfee Netshield for Netware version 4.6 anti-virus protection for the mail server and McAfee VirusScan Enterprise 7.0 for all workstations (cost approx. $800).

3. Incoming mail messages flowing from the GWIA are intercepted by a Windows 2000 workstation ($800) running Guinevere 2.15 ($500), SpamAssassin 2.63 (FREE) and McAfee 7.0 (above). Of the 686 messages per day that make it to this second line of defense, only 94 survive; 189 score so high as spam that they are just deleted forever, 394 score a high enough probability as spam to be archived but not delivered to the end user, 3 are automatically forwarded to SpamCop ($50 per year) for reporting and 6 are identified as viruses by McAfee (virus definitions updated automatically every 30 minutes) and stopped. Note: We only review what's being blocked here when reporting to SpamCop (at our leisure) or in the very rare event that someone says they didn't get a message that should have made it through. Thank you Michael J. Bell for Guinevere and integrating SpamAssassin!

4. The next stop for the surviving 94 mail messages per day is our GroupWise Message Transfer Agent (MTA) back on the mail server. GWAVA 3.0 ($975) intercepts the mail, McAfee 4.6 for Netware (automatically updated nightly) scans it a second time for viruses, 1 message per day is identified as a virus that made it to this third line of defense and 8 messages are identified by GWAVA as being potential spam. These 8 messages are forwarded to our Mail Administrator for review and usually 2 of them are released to the end user as legitimate mail.

5. Finally, the 87 legitimate email messages that are delivered to our end user mail boxes daily, cost us an estimated average of $0.11 each calculated as follows: $7,575 initial cost spread over a 3 year life expectancy = $2,525 plus $1,000 annual maintenance = $3,525 divided by 31,755 messages per year (87 x 365) = $0.11 each (per incoming message).

Prior to upgrading to GroupWise 6.5 and implementing the RBL blocks, our Guinevere and SpamAssassin workstation was processing as much mail in one day, as it does today in an entire month! These programs we use in our three pronged defense are all extremely fast and efficient. A legitimate incoming message will hit the users desktop notifier in well under 20 seconds.

Daily management of our system requires only 15 minutes of human intervention by the Mail Administrator

We have formatted our plan so that the only daily requirement of the Mail Admin is to review the few mails caught by GWAVA as spam.

I feel that we owe much of our success to these real-time black lists (RBL's)!

Additional time may be spent on an OFFENSE in addition to our 3 pronged DEFENSE at the discretion of the Mail Admin...
We found that by consistently taking an offense against spam, we further lowered the workload on our Guinevere/SpamAssassin workstation (by almost 50% over a thirty day period). I sometimes spend 15 minutes at home while having my morning coffee reporting spammers to SpamCop. I simply log in to the Guinevere and SpamAssassin workstation at the office and forward a group of the messages archived as spam to SpamCop. If the message has to do with online pharmacies or medications I copy webcomplaints@ora.fda.gov or if the mail has to do with investments I copy enforcement@sec.gov and usually always copy uce@ftc.gov (now spam@uce.gov) on all spam reports.

Additionally we maintain our own in-house "white-list" (duplicated in SpamAssassin and GWAVA), which requires very little maintenance...
The "white-list" is essential in our effort to obtain zero false positives. A false positive is a legitimate message mistaken for spam. We do not white list domains, only users. White listing an entire domain would certainly allow more spam through.

We also maintain our own in-house "black-list" at the GWIA...

For our "black-list" however, we do the exact opposite. We do not black list users. We black list domains. Black listing users would be futile. Consider these real life samples; A spam from INKDEALS.9869.3879600@MSMDEALS.COM gets the entire domain black listed as msmdeals.com and *.msmdeals.com. LRXAKCDSIXEBP@MYSTUPIDSCHOOL.COM gets mystupidschool.com and *.mystupidschool.com black listed (take a minute to visit these sites before black-listing them). Note the second listing adding the "*.", this is because many spammers use an upper-level domain that they change frequently, i.e., garbage@123.spamdomain.com tomorrow becomes moregarbage@456.spamdomain.com.

 BTUYKAPTFRRE@YAHOO.COM gets no action. We aren't blocking the yahoo.com domain, neither are we wasting our time blocking the user BTUYKAPTFRRE (probably a "throw away" or one time use account or just faked). Otherwise, our black-list would be infinitely too large to manage and it would only be a very small percentage as effective as it is. The three samples never made it to a user mail box, they were pulled from the Guinevere - SpamAssassin archive. So, black-listing is not required to stop the spam.

NOTE: We do not use our "In-House Black-List" to stop spam (per se), we use it to improve the over-all efficiency of our system. Why have Guinevere or GWAVA work future messages from a known spam domain again and again, much less us take the time to report them again and again through SpamCop?

We suggest that end users on the network utilize two tools available to them in their GroupWise mail clients. One is a built-in junk mail handler (not very useful in my opinion) and the other is a plug-in for spam/ham from GWAVA and Guinevere (very useful for Bayesian Learning). Today, the need for using these tools is very limited due to the success of our system.

To date SpamAssassin has learned 7,850 spam and 5,124 ham (as of June 23, 2004). We do utilize the auto-learning feature with very conservative settings. Although we strive to learn equal amounts of good (ham) and bad (spam) mail, it is extremely difficult due to our having abundantly more bad mail than good. NOTE: 8,959 SPAM and 5,707 HAM as of October 14, 2004. So, you can see just how conservative our auto-learn settings are.

I've seen recent statistics that suggest as much as one half of the mail today is spam. We have been using our email addresses since 1996. As you can see from my information presented above, we receive far more spam than legitimate mail. I vaguely remember a time when that was not the case, but not with clarity.

Prior to implementing this system, I estimated that spam and viruses cost my firm in excess of $20,000 per year! Do you need to ask why I dislike spammers so much, and why I am sharing our knowledge in the fight against spam?
My blood pressure actually goes up when I think about these people out there, that have nothing better to do than work at putting junk on my computers.  This battle that we are fighting is never-ending and the spammers are a continuously moving target.

As a small business owner, I have viewed all this spam we received much like a collect telephone solicitation call that by-passed our receptionist and went straight to the employees. I honestly believe that not one employee purchased one single item from these spammers in the last eight years, but the time consumed by my employees reading and deleting the "crap" they received cost our firm $160,000+ in lost productivity ($20,000+ per year). That money/time would have been better spent on almost anything. Especially since the spammers did not benefit, nor did anyone else. IT WAS ONE BIG WASTE!

Spam is a drag on our economy.

Why do we go to all this trouble to keep our mail system open to the general public?

The truth of the matter is, we primarily do it in hopes of gaining new clients/customers for our products and services. Otherwise, we would simply set our mail server to reject all mail except a white-list of existing customers, (we've actually considered this). In fact, we've made it part of our of our future plan!

If the future cost of our maintaining an open system becomes prohibitive, we will simply close it to everyone except our white-list.

Why take such drastic action?

It appears to be the lower cost alternative. Consider just 10 employees receiving 150 junk mails each per day. Scanning and deleting the junk could easily take 1 hour of productivity away from our business per employee daily! The math; 1 hr times 10 employees = 10 hrs per day, 10 hrs times $25 (average rate per hr) = $250 per day, $250 times 250 work days per year = $62,500 annually!

We actually have an employee that was receiving 150 junk mails per day, prior to our taking action.

I am certain that in two to three years our system of spam and virus prevention will look very different than it does today, June 23, 2004.

Never, ever think that the spammers are winning this war...

I assure you that they are not winning. They are losers. The Spamming business has already experienced it's "hey-day". From here on out, it is a down hill slide for the old spammers and new entries to the spamming business will wash out very quickly.

The spammer is in business just like us, and to be successful they must make a profit.

The best evidence that the spammers are losing the battle is the content of their messages today compared to two years ago!

Just look at some of the messages! They appear to have been written by some unintelligible babbling idiot. The useless content is so obfuscated in a desperate attempt to get past spam barriers, that you can't even discern what they are promoting!

I actually laugh with satisfaction when reporting these stupidly garbled ones to SpamCop, knowing that once the spammer has stooped to this, he isn't far from being put out of business altogether.

eMail is not FREE!

As you can see from my analysis, the true cost of email to my firm (on a per inbound message basis) is $0.11.

This estimate ignores the many hours my "computer guy" and I spent developing and configuring the system we have today. It also ignores our ongoing T-1 line charges, (which we had anyway, for internet access and our web servers).

I estimate that 90% of the incoming messages require a response. So, even if we divide the cost in half, all email traffic costs my firm 5.5 cents each (about $300 per month for us). Has our phone bill gone down by $300 per month? I don't think so.

However, we do business with clients in all fifty states and nineteen foreign countries. Just the time zone factor alone has made a difference in convenience and response time to the client.

Many spammers now include a list of dictionary words in their messages in another desperate attempt to get to us. Thus far, it's had no effect at our site, other than the fact that it makes me chuckle more when pulling some from the spam archive and reporting them. It let's me know that they are having a difficult time.

Prior to implementing this system, I estimated that spam and viruses cost my firm in excess of $20,000 per year! Do you need to ask why I dislike spammers so much, and why I am sharing our knowledge in the fight against spam?

To help stop spam, I vowed to never buy a product or service promoted through spam. If ever I saw something that I thought I couldn't live without, I would pay 1,000 times more to seek out and buy it from a non-spammer.

You're sharing your strategy with spammers too by publicly displaying it here...

I believe that by sharing what I've learned through research and testing, more can and will be achieved by persons fighting spam, than the spammers.

Although our system is only a 25 user system with two post offices, I believe similar results can be achieved in any business environment, even with a much larger system.

Initially we intended to devote only one page to the war on spam. With 54 requests for additional information in two weeks we decided to add more info. Click here for update.

We found many purported "solutions" out there to stop spam. We paid licensing fees to several companies for software that we found would cost us less to simply let them expire than to try to make them work. In our search for a solution we also found that if you're not careful, you can spend more than it's even worth! That is the harsh and sad reality...

Tools:  Attachment File Types  Bayesian Learning  Novell GroupWise  Novell OS  McAfee Security  Guinevere  SpamAssassin  GWAVA  SpamCop