Spamhuntress

80.237.140.233 gobbled up 60 megabytes from annelisabeth.com during December. He loaded specific pages over and over, then moved on to other pages. He shows up as proxy77.net in Awstats.

User agent:
Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.5) Gecko/20041108 Firefox/1.0

The IP number itself is a proxy from Germany.

My conjecture is that the spidering is done by a referrer spammer who’s particularly hungry. So hungry he’s stopped doing GET requests the last day or so. He’s switched to HEAD!

He’s using proxies for the actual referrer spamming, but keeps using that exact same user agent. I’ve seen him here too.

Another to watch out for is

216.220.192.132
Who identifies itself as
NutchCVS/0.7 (Experimental Nutch)

It gobbled up 30 MB of spamhuntress.com so far in December. That software is used for running search engine spiders. Problem is, this particular one isn’t identifying itself. If any of you knows anything about this one, please let me know.

I’ve seen some recent comments about Omni-Explorer, and one just a few days ago in particular said it had downloaded a gig of data off his website!

SiliconBeat

My wiki page on the bot is often referenced (referrers), so I’m kept up to date now and then.

I saw a spike in my bandwidth today.

IP:
68.14.199.27
wsip-68-14-199-27.no.no.cox.net

User agent:
Java/1.5.0_05

I found a guestbook entry on the net from that IP number. Absolutely no content. The obvious conclusion is that the bot just followed yet one more link and posted without realizing what it was doing.

151 requests in 2 minutes 46 seconds.

Ban with extreme prejudice!

As I explained on the wiki page about the Omni-Explorer, it’s still hungry.

Latest IP number was
65.19.150.249

It could be worse in terms of bandwidth this time around, but I really think there’s shoddy coding when it goes through over 700 files that fast!

Have a look:

Omni-Explorer

Analysis of Omni-Explorer

I had the Omni Explorer on one of my sites a few days ago, and thought it was aggressive.

Unfortunately I didn’t do anything about it.

Yesterday it hit NativeCelebs. That’s a HUGE site, and the bot proceeded to gobble up (According to Awstats) 304.24 MB.

I checked the raw log, and that IP number didn’t hit my site until
[22/May/2005:09:17:08 -0400]
and stopped
[22/May/2005:10:37:26 -0400]

IP number:
64.71.131.121 (nativecelebs)
64.71.131.120 (spamhuntress)
User agent:
OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Internet Categorizer

If you’ve got a large site, block it fast!

I had similar hits before (just a few)
64.62.175.131
OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Cars Crawler

I’ve had one access from that IP block before:
64.71.131.107
A normal browser UA, but didn’t load any extra files. Had a referrer from a site that links to me and went after the spampop page. Must have been a bot.

On NativeCelebs I’ve had a number of accesses from both Omni Explorer UA and normal browser UA from that IP block. I’ll find them and collate them here. All of the bots have full normal referrers. Wherever they came in from, that’s the referrer they leave. The same IP number can have the Omni UA one day and a normal browser UA another day. And apart from the gorge fest yesterday, I find the accesses one at a time, or a few at a time, starting May 16, 2005

64.71.131.107
64.71.131.108
64.71.131.109
64.71.131.110
64.71.131.111
64.71.131.112
64.71.131.114
64.71.131.115
64.71.131.120
64.71.131.121

In April I also had visits from this family of bots. And back then they came from a different IP block:
64.62.175.133-64.62.175.137

Earlier post about this bot

Analysis of Omni-Explorer

Had a really bad spike in bandwidth, and chased down the cause. Veerry aggressive spider. Here’s what Webmasterworld has on it.

Short version: Possible bot used for building scraper directories. In other words, search engine spam.

I had a hungry Japanese bot visiting my site during the night.

219.123.207.70
usen-219×123x207×70.ap-US.usen.ad.jp

Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)

It didn’t load supplemental files, and followed links indiscriminately.

It used only three minutes to suck down quite a number of files. I could see it on the bandwidth meter, even. It first came in a few minutes before the suckfest, and came in again a while later (from an outside link?).

I had a visitor from Missigua Locator 1.9 over the weekend.

Came from 67.159.3.190 on April 24. Kept going for 8 hours, without being too aggressive. It displayed typical bot behavior, including trying to load
my feed without removing feed: in front of the URL, triggering a 404.

I loaded the IP number as a website. Title is:
Top 10 Search engine placement _ Need Web Traffic?

But the only visible content is a green background and a simple form field/submit with this text above it:
To be removed, enter your email address below and click REMOVE

My guess would be - e-mail harvesters.

Someone else also had a visit from such a bot around that time, but from 69.115.135.243 (offline right now).

I would guess the bot has gobbled up all the spammer addresses by now?

I’ll put bots I find that I haven’t found an explanation for here. These are mostly recent hits to my sites:

IP address
203.144.160.242
caching1-true.asianet.co.th
User agents:
Shockwave Flash
Mozilla/4.0 (compatible;)

—————-

IP Address:
217.159.201.143
User agent:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; (R1 1.5))

Fetched 100 pages from a friend’s site in less than 2 minutes. Kept going after being banned.

The IP number only has one site on it, that’s got a password protected home page: bulletrehosting.com

The dns server is on Atrivo’s net, no known sites on the same IP number. The e-mail address in the whois info doesn’t work - domain name inactive. And finally, the address in the whois info suggests the company is incorporated in a tax haven. What are they doing? No clue.