Omni-explorer still hungry
As I explained on the wiki page about the Omni-Explorer, it’s still hungry.
Latest IP number was
65.19.150.249
It could be worse in terms of bandwidth this time around, but I really think there’s shoddy coding when it goes through over 700 files that fast!
Got hit with the same thing. It completely ignored the robots.txt file and crawled every page on the site (1096 pages at that moment). Probably a spam-harvester. I’ve complained to their upstream provider