Spamhuntress

A customer contacted me, complaining that mail didn’t seem to reach one of their contacts. Mail had started flowing heavily between these two companies, but the mail only reached the recipients if they originated at the contact’s server, not the other way around.

The contact was pretty convinced the problem was on my end, after having done some troubleshooting and finding that the configuration of the server software was sound. I was equally convinced the problem was on their end.

A lot of troubleshooting later, we reached some conclusions:

Although the server at the other end appeared to work fine, it didn’t send the initial server greeting to some servers. This happened across linux distros and mailserver flavors. I couldn’t find a rhyme or reason why some didn’t get the server greeting, but it was consistent. Some did, others didn’t. When I telnettet to port 25 from the affected servers I had to press enter (got the server greeting and a 500 error) or send EHLO manually in order to get the server greeting.
They had Symantec Mail Security installed, and I tried to find any mention online about it having problems. Found nothing.

Finally I thought, OK, since there’s an Exchange server behind Symantec, maybe I should see if there are any problems there. I found two mentions of identical problems. In both cases, they had a Zywall5 router in front of and Exchange server. One person had updated the firmware on the router, and the problems had vanished. I called the company, and asked the IT person point blank: Do you by any chance have a Zywall router in front of the mail server? He immediately said yes. After updating the router firmware, the problems vanished right away.

I found a page with a list of servers from Norway caught comment spamming lately:

Norway’s comment spamming servers at Project Honeypot

I found it while searching for the IP of one server I knew was Norwegian. I contacted the owner (a webhost). He told me he knew about it, and it had been an open proxy server that had been used by spammers all over the world. He also told me he’d fixed it a couple of days ago. Problem is, the last entry in my logs from that server was today! Well, hopefully he’ll fix it.

I also tried contacting another company that’s not a webhost. My e-mail bounced.

It looks to me like there are a lot of compromised servers out there. Compromised in various fashions - from a glitch in configuration, to dishonest customers, to insecure scripts.

Those of us who are responsible for servers need to keep an eye on bandwidth usage as well as logs, and also keep an eye on Project Honeypot and similar services.

I occasionally find ways to monitor IP space for spam, viruses etc. Here’s one such new way to monitor your IP space:

Project Honeypot’s IP space monitoring

You know, I once notified my neighbor that his machine was compromised because of one of those services. Turns out he had a pirated version of windows (I believe it was windows 2000?). Because of that he didn’t get updates. Let’s just say that machine was doomed. Format c: /s - or something like that.

We’ve been fretting over what to do when an ISP or webhost is spam friendly or a spam supporter.

Here’s what the big guys do, such as Steve Lindford at Spamhaus.

He contacts the NOC, and gets the IP’s nullrouted there. So the downstream providers can promise bullet proof hosting all they want, the server will still go down the moment Steve gets on the horn…

Check out Steve’s post on NANAE

BTW, there’s a lot of hilarious stuff in that thread, so click on the link at the top as well: ironserver.com creamed (again)

That post concerns a spammer I’ve written about before:

Spamhaus and one angry spammer

That debacle with *** webhost got me thinking. What’s the best way of figuring out the compromised script on a shared virtual webhost?

Say the spamming is referrer spamming.

The spamming could be done in any of the following ways:

1) User uploads spam script to his user area
2) A compromised script is used with a functionality like a proxy server
3) The server is compromised, and the spammer has installed a script that’s spamming

What’s the one thing you can count on in that situation? The offending traffic is outbound. It’s coming from any port, but connecting to port 80 on the remote system (the website they’re connecting to, so it appears in access logs).

But the spamming could be controlled by inbound traffic, or by a script on the server.

What would be the best tools to narrow down to which script is doing the spamming, or matching inbound with outbound traffic?