I've put a copy of Sp@mX on the Web site that does the RIR lookups from your computer.

As you know APNIC cut off access to my server's IP address because we "were sending a large volume of abuse complaints to domains in their registry" (spammers).

You need this update to report Asian spam!

Huh?

Sorry, Regional Internet Registry - places around the world that store Internet domain information.

Thanks for the regular updates!!

Thought you might like to know that since I have been using your program, which is for every single spam email recieved, that 2 ISP's so far have been in contact, saying that there are too many abuse messages coming from my address!!. Apparently Sp@mX has sent 102 abuse messages to one of them, and 89 to the other!!

This apparently generated some sort of flag and that was why they got in contact, one even wanted me just to bin them in future "as we are now aware of the situation"

But, your program has reclaimed my email inbox, I'm now down from over 100 emails a day to a new low of 25 emails, I am continuing to use the program and it's own tally is now 660 since I registered it 8 days ago.

Keep up the great work

Thanks for this great feedback!

Just wanted to say I like the cute animated arrows

been under 100 spams per day now... Sunday even reached 64!

After 2 months it looks like spam has been cut by 50%, from between 180-200 the first time I started tracking, to 80-90 now and hopefully continues to go down even further.

I am a bit more unique since I have several email addresses, so I get several times the amount of random spammage

But Sp@mX has enabled me to fight back against the onslaught

Total spam reported after tonight's run will probably be around 18,800

Jeff when can we look forward to sp@mX keeping a daily/monthly record of spam reported? Keep up the great work. I'm finally noticing a difference!

That's great!

Absolutely! Stay tuned...

Here's a nice one to encourage us:



Trend still down - thanks, Jeff!

And this:


Two of them actually, this morning

Hi. Perhaps someone can tell me what this means? Processed a spam a few minutes ago and the log of the process came out like this:-
"validating user ron.whitehead@ntlworld.com ron.whitehead
white list loaded from hsc, Sp@mX ready

processing msg1.txt
found URL c.h2.seethesimple.com
resolving c.h2.seethesimple.com
evaluate URL answer
domain c.h2.seethesimple.com resolves to 222.47.62.179
resolving 212.250.162.8
evaluate DNS answer
resolving 220.170.168.29
evaluate DNS answer
claims to be #0 = ntl.com, actually is #0 = ntl.com, Equal
claims to be #1 = alabamabowling.org, actually is #1 = -- No DNS --, Not Equal
verified base #1 IP 220.170.168.29
building chain
received line #1 IP 220.170.168.29 type spam source
looking up abuse information
evaluate abuse answer
lookup 222.47.62.179
evaluate RIR answer
evaluate RIR answer
looking up abuse information
abuse type spamvertized URL
abuse type spam source
adding 222.47.62.179 to honeypot
spammer already exists in honeypot
adding 220.170.168.29 to honeypot
spammer added to honeypot
nothing to do reportspammer
error reportSpammer()
done
done, configuration reset after remote run, ready...
evaluate RIR answer
evaluate abuse answer
error DoNextRIRLookup()"

Does this mean that the process has been completed, or is this a refusal by an ISP to accept my email? I've had only one log like this before, am I right in suggesting that the last line makes it looks as though the process has not been properly completed?

Ron

Looks good in any language

Jeff, quick question: if, say, Mark reports a spam and it's in the honeypot, is it reported again when I get it or is it discarded silently?

I think it's the latter but if ISPs are complaining about the volume of complaints...

The second process would generate only one complaint per individual spam which ISPs might find more acceptable ...? Maybe v5 could have the SpamX server consolidate a single complaint per x-hours period from all subscribers? I'm just thinking of a way that might circumvent the argument that 120 complaints = spamming the ISP %-\ Even an hourly consolidated complaint would only creat 24 a day against the spammer's output of 1000s ...

I guess it comes back to that question: when does complaining too much = spamming / DoS on an ISP?

4.1.1 is working well and I appreciate the frequent updates and tweaks. There are a few "issues" that have been around since early 3.X versions that could use attention:

1. It appears that spam containing odd (non US character) still can't be processed. I don't know if this is anything you can handle or not. I've been sending them to SpamCop which seems to be able to report them. Worse, these emails still don't get placed into the notprocessed folders but instead seem to halt the reporting process.

2. Is there a way to auto empty the notprocess folder? Perhaps a setting that would delete them after a few days (or immediately if so desired). I think a default of keeping is useful for those that want to report them manually.

3. There appears to still be areas where you wish us to enter preference data and such that doesn't allow pasting. There's no Copy/Paste menus which would be nice. Everything under the "Configure" menu falls into this camp. I'm not sure this is even "allowed" under OS X guidelines.

4. Unlike all other dialogs, the new "Filter" dialog doesn't have a close (red) radio button under OS X which it needs. You might instead want to disable the yellow "grow" button since it brings nothing to the party (I think that's OK to gray out depending on the dialog structure).

Tifferg,



I'm tempted to say, "When they stop allowing/promoting/generating SPAM".

IOW aren't we really discussing the fine line between effective reporting to people genuinely concerned and actually 'on our side' on the one hand. And persistent offenders (who'd be unlikely to do much about our reports anyway) on the other?

I guess I'm still unclear of the long term purposes of Sp@mX: if so many of our reports are ineffective because they're always going to be blocked (received another one just now: "xxxx@xxxxxxxxx.com (my reporting email address) is blocked to this server", how can we move forward?

What evidence have we of where we're most effective?

Shouldn't we be maximising our efforts there? And in huge (co-ordinated on the hsc servers, as you're implying) quantities?

I remember Jeff mentioning that the strength of Sp@mX in comparison to SpamCop—based on his experience and conclusions—is that Sp@mX complaints are personalized and thus lead to the complainant’s email address being eliminated from future mailings even if the spammers don’t curtail further mail campaigns toward non-complaining recipients or anonymous abuse reporters when the ISPs fail to take proper action. Therefore, whatever measures are taken—even if an attempt to consolidate reports were to be made in the future—an important objective to maintain is ensuring that complaints continue to be personalized.

I like this latest version of Sp@mX. APNIC’s reaction was unbelievably ridiculous and obviously runs counter to its very purpose. Instead of pursuing the objections of their own organizational motto, they simply acquiesced to the complaints of ISP administrators who refuse to do their jobs, wishing instead to coddle spammers. Now that 4.1.1 can search the RIRs directly from our own computers, Sp@mX users can eventually form the vocal majority that refuses to be silenced, causing the minority Asian ISP admins to take proper action against spammers.

I sense that consolidating complaints into a single message once every few hours would just make it easier for the ISPs to ignore and thereby shirk their responsibilities. Such consolidated complaints could be more easily blocked, just as APNIC unreasonably blocked RIR searches from Jeff’s server. When action is taken by widely distributed individuals, it feels more like a broad groundswell against spam rather than one vocal individual who can easily be silenced. The more widespread that the direct anti-spam complaints becomes, the less tenable will be the position of spammers, and fewer ISPs will be willing to coddle them much longer.

If Asian abuse desk managers feel that their jobs are overwhelming, it is the responsibility of the ISPs to staff their abuse desks properly and adopt more efficient means of dealing with customers who abuse their services or Westerners who take advantage of the open proxy relay servers or of nonsecure computers on their networks. If the volume of complaints is overwhelming, it is because the complainants are being overwhelmed by Internet users who are taking advantage of lenient or unenforced policies. Now, Internet users are refusing to be overwhelmed by annoying spam, so the abuse desks have to take proper action rather than hoping that the problem will simply go away if it is ignored long enough.

I look forward to the day when email messages will be stamped with an unforgable ethernet address of the originating computer, making it impossible for spammers to hide their identities. Then, I hope that ISPs can automate their systems by immediately disconnecting users about whom twenty or more complaints have been received within a month. Until then, ISPs must find an effective and efficient way to deal with an increasing number of complaints now that we have found a means of lodging such complaints. Spammers should not continue to be coddled, and we have to make ISPs realize that the income they receive from spamming customers is not worth the burden of dealing with the complaints that come as a result.

Xiaopangzi:

Very well said.

Xiaopangzi, thanks for that thoughtful and informative post

I am still left with these two questions:

1) (How) is Sp@mX countering those spammers who have their own servers and are effectively their own host/ISP?

2) How can we make a real difference - I expect as you say - to those 'good' ISPs who are with us all the way and just need chapter and verse to know where to act next, but somehow seem still to have open relays?

IOW hasn't the time come for us to distinguish between at last four targets:

• self-contained pammers (my 1) above) who will operate regardless; I've always understood these are about 200 criminals operating mostly from the USA but using servers elsewhere
• poorly configured ISPs and hosts, whom we're really helping by reporting; they hear us and act
• the 'big' ISPs like Comcast and Verizon who somehow still manage to allow it to go on
• zombie and trojaned Windows machines

Doesn't this imply a potentially different tactic in each case?

Am I close, Jeff?

I'm not sure that by using Sp@mX we can make an impact regarding spammers who are their own ISPs or use spam-friendly ISPs. What we can do is use other tools in addition to Sp@mX. I still report spam using SpamCop, and from time to time still send reports to the FDA, FTC, etc.

Another option may be found HERE

Even spammers who are their own host/ISP are answerable to the next level up the domain chain. I think the NICs are the ultimate issuers of IP addresses and, as with recent results, they should be telling ISPs: clean up your act or you won't be an ISP - we're withdrawing your IP block. That is the ultimate sanction which I guess they are reluctant to wield which has let certain ISPs off the hook so far as failing to act against spammers.

I don't know the technical feasibility of it, but I expect that if, for arguments sake, chinanet.net utterly ignored every complaint made, APNIC could cut them off from the Internet. I think several million severely upset customers banging on the doors might concentrate their focus on a serious policy failure. That it should even have to be considered is a serious failure on their part in my opinion.

One benefit of SpamX is that reports are going to the higher level domain registrars so that they are aware of the problem. It is to be hoped that they do take responsible action to curb the excess of spammers abusing their domain.


I do too but they are filtering my main new account, my old spam bucket is my principle test bed for SpamX as that old account has a far higher volume of spam. Both have their place in the battle lines.

You are right Ken and I guess I'd forgotten that in my zeal The next best alternative, which I'm sure Jeff is working on, is to have some form of daily/weekly/monthly statistics that show just which ISPs are responsible for what proportion of spam being reported eg. a graph or chart showing: ISP | # of reports | # of active responses ie: user warned/account closed | # ignored reports - "mailbox full" type of response | percentage of overall volume per period.

That sort of analysis would go a long way in marketing SpamX as well as clearly displaying who the worst offenders are in propagating spam, ignoring complaints or actively working with everyone against spam. My only caveat is that the records used must be 100% accurate or I can envisage writs like confetti over Jeff's desk That said, as every complaint I, and seemingly most of you, send contains the 'discussion' clause, no ISP can say in their defence that they have not had a chance to put their point of view.

Yes, that’s one of my concerns. I sometimes wonder whether some users are being a little haphazard about their use or verification of filter results. I left Jeff’s notice of Sp@mX updates on the server for several days because my email application wasn’t deleting it from the server after email download sessions, and I noticed that every time that I ran Sp@mX, his update notices would invariably be labeled as spam each time, indicating that multiple people continued to report those notices as spam. Fortunately, with the little incremental update to 4.1.1, we can remove some unjustly blacklisted addresses. I had to do that several times over two days before it would finally remain labeled as legitimate email and then finally removed the update notice from my mail server via the Webmail interface.

That makes me wonder how many other incorrect entries there may be in the honeypot, which could really be detrimental to the reputation of Sp@mX. None of that is Jeff’s fault, and in fact, it is a problem with all other spam filtering systems on the Internet that pool blacklists instead of relying on users’ specific individual archive of messages (such as SpamSieve).

I guess there is no way to avoid inaccurate entries except to hope that people would be more careful when making their selections or applying labels. I know that it is tedious for those who have hundreds of spam messages each day, but improper selections that lead to inaccurate entries in the blacklist honeypot will come back to bite us in the end if ISPs begin blocking Sp@mX complaints due to mistrust of the accuracy. I don’t know whether our complaints are recognizable as being generated by Sp@mX anymore, though, except that the format would be the same.

A partial solution may be to require that any particular IP address be reported by five different users before it can be moved from a temporary probation(ary) pool to the official blacklist honeypot. That’s not a perfect approach, but it is a little safer than the present approach. Any individuals who don’t want to be bothered with individually clicking on messages to label them as spam themselves can benefit by waiting for a while before processing their messages if they can afford to do so. I know that I was adding a lot of IP addresses to the blacklist when I was running Sp@mX every few minutes, but on the days that I actually waited a few hours before processing messages, most of the offending IP addresses were already in the honeypot. Sp@mX becomes more effective or more automated the longer that a user can wait before processing their messages, because other users will have done most of the work of filtering the addresses beforehand if they are receiving messages from the same spammers.

I do definitely agree that statistics—both individual and pool-wide—would be helpful in discovering the most problematic domains and determining whether there are significant changes over time, as well as being useful in marketing Sp@mX.

I agree 100%
I suggested a similar approach in the past. There has to be a threshold. Just because we have a powerful tool against spam, that does not mean we can use it irresponsibly. In the long run, the strength of Sp@mX will depend on how we all use it. ISPs themselves have the problem(s) associated with dealing with spammers, and instead of helping them we may become an additional burden if we are not responsible.

I feel the revised message many are using directing recipients of complaints to this message board is I guess the only way to find out.

Jeff, can you lock a root thread for those querying the blacklist? The root message could explain again why they have been blacklisted and a direct connection to you for removal, perhaps you could create a dedicated e-mail address like blacklistquery at hendricom they could use or even <shock horror!> go the way of many and create a web form for them to paste the complaint mail and their reason for contesting the listing?

As it is, I believe the ISP would have to register on the board before they could post which may be just too tedious a process for a busy overworked Admin.

As has been said before, this is certainly an evolving product and the more input from all sources, the better it becomes