chongqed
Wednesday, November 24, 2004
 
Two reasons why indexing kept pages is a bad idea

It is now common wisdom that wiki spammers are spamming wikis because they want to influence the results that search engines will return. That's why many wiki enignes now take care about the robots meta tag. That's why many wiki admins have hand-crafted robots.txt files, etc. If spam on wikis doesn't end up in a search engine, the reason for spamming is gone and wiki spamming should come to a halt a some later point.

Reason #1

If you clean your wiki on a regular basis, wiki spam should have a hard time being found by a search engine spider on the current revisions of your pages. It still could be found on your kept pages, though. Even though those kept pages will eventually expire, search engine robots might find them and it will raise the position of the spammer's site in the search results for the spammer's keyword. We don't want that, but it definitely makes the spammers happy. They don't have to care whether you clean the wiki or not. Two weeks till the kept (spammed) page expires should be good enough for them.

Reason #2

When looking at the server logs of the chongqed.org wiki after it was spammed a couple of times, I noticed a consistent pattern used at least by the Chinese wiki spammers: They ask Google for pages containing spammy URLs. In other words, they are trying to find spammed wiki pages, so they can spam them again, this time with different URLs or keywords or just to make sure that there spam stays fresh. This means that spammy pages (even if they are old revisions) on your wiki that are indexed by Google and friends, will attract more spammers!

Conclusion

That's why I'm suggesting that wiki admins allow only the most current revision of any page on their wiki to be indexed by search engine spiders. Even if you don't care about the spammer's page rank, you sure don't want to attract more spammers.

 
Comments:
The chongqed wiki really has turned out to be a great learning tool. We warn spammers they don't want to spam us, but if they don't speak English well or at all they won't understand what they are getting themselves into. Maybe that is why only Chinese spammers have hit us so far.

I agree completely with your conclusion. Before we believed spammers were just searching for any wikis to hit. Obviously some still are, but it seems the lazy ones, which are likely the majority, use the work of other spammers to find good places to spam. By searching for other spammy URLs they can find wikis that are not cleaned regularly or where previous revisions are allowed to be indexed. And of course once they spam they keep returning to freshen their links and make sure the competition didn't overwrite their spam. So it really is necessary for the health of your wiki to prevent the spam from being indexed. Once it is you are inviting more spammers.
 
Post a Comment

<< Home
This blog is a place for me to share my views on the wiki spam problem, the email spam problem, and life in general.

ARCHIVES
May 2004 / June 2004 / July 2004 / August 2004 / October 2004 / November 2004 / December 2004 / January 2005 / February 2005 / March 2005 / September 2005 / October 2005 / November 2005 / January 2006 / October 2006 / January 2007 / May 2007 /


LINKS