Dylan Greene dot com

May contain nuts.

My prediction: rel = "nofollow" will not prevent comment spam

January 19, 2005 2:19 PM

Google, MSN, Yahoo, and others have decided that the new best way to prevent comment spam is to use an HTML tag that will prevent Google (and other search engines) from following links that could be spam links.

That new tag works as so:

<a href="url" rel="nofollow">Link Text</a>

When indexing sites, this will prevent Google and others from "following" these links. Normally following those links helps a site's PageRank, which makes those sites appear higher in search results. The theory is that if spammers aren't getting their PageRank improved, then they will stop spamming blogs. Ha!

My prediction is that this change will not prevent comment spam because of the following issues:

  1. The text of the spam messages will still be indexed by search engines.
  2. Humans will still see the the spam messages.
  3. Humans will still be able to follow the spam links.
  4. Most importantly: Spammers can still post spam messages.

In short, all this does is make life a little easier for Google. Their PageRank system, which might just be flawed, has been abused and blamed for the onslaught of comment spam. Now they have an out - an excuse to to say it's not their fault anymore. I'm all for better search results in Google (ever try to search for a specific hotel?), but their solution simply does not prevent comment spam.

What do I recommend?

  1. For automated spam bots: To prevent bots from posting spam comments, I require JavaScript. When a human user clicks the Submit button, JavaScript to renames field names before the comment is submitted to the server. Fields are unique named every time the page loads, and the server will only post comments when it gets the field name it is expecting. This prevents the automated spam attacks because the software spammers are not able to predict the field names.
  2. For manually entered spam comments: I have basic spam filtering mechanism similar to many email spam filters. It looks for common spam words, URLs, and topics, and prevents those messages from being posted.
  3. Either way, I have a RSS feed which shows me whenever a spam comment is attempted, along with IP address and other information so I can track the progress, and watch for false positives (real comments that the system thought were spam), and easily ban IP's of known spammers.

Something important to me is that my solutions stop comment spam without requiring any extra effort from my users, Some sites now require registration or a CAPTCHA input to add a comment. I feel that this is just an unnecessary pain which prevents people many busy people adding their feedback. I also feel that the links in comments are often important enough that search engines should follow them, therefor always putting a nofollow tag will ultimately be unhelpful to those small sites that should get a higher PageRank.

And most importantly, unlike the nofollow "solution" from Google, my recommendations can actually prevent spam comments from appearing on sites, and that's what we all want, right?

Comments

In reply to your first suggestion, it's a sound suggestion. However JavaScript wouldnt work. Remember JavaScript is client side; if the JavaScript generates a random field ID the server will have no idea what it is and will not be able to validate.

Instead you could use a server side script on the "post comment" page that would generate the random id and store it in a session or cookie. Then the server would be able to find out what the random ids are supposed to be.

Actually a public key is also included as a hidden field. The JavaScript uses the public key to compute the field names. The server knows what fields to expect based on the public key.

I decided not to use cookies because some users disable them for privacy reasons, and I didn't use session variables in case you are browsing multiple pages at once.

The one problem with requiring JavaScript is that I can't post comments from my phone. :(

Well, my knowledge of crypto is very limited, but would that actually work? As I see it, the JavaScript would need two values, the key to encrypt it with, and some other piece of data. Now the other piece of data would have to be diferent everytime otherwise the same result would be generated and spammers would just compensate for this. You could use the date and time, but the time between the user submitting the form and the next page generating might be nominally diferent...

...unless I'm misunderstanding you?

If Google wanted to stop the spamming of blogs, they would have proposed a real solution that stopped spamming, (like yours). It seem that, from Google's perspective, reducing the search engine rankings of every blogger who has ever posted a comment or used trackback is either a good thing or at least not bad. That so many bloggers are applauding a move that will dramatically reduce their voice while doing very little to reduce spam is puzzling to me.

Nathan - I'm explaining it without giving away all the secrets. The public key and the public field name are the input for an algorithm that produces the field name that the server is looking for. That algorithm is easily viewable by viewing my source code, but my assumption is that the tools that comment spammers use do not and will not support JavaScript any time soon.

The Unofficially Yahoo Weblog agrees!

http://yahoo.weblogsinc.com/entry/1234000803028085/

Aha, I see. ;)

So in my opinion, the Blog software developers must include "random text image" validation in the forms. That will kill spam bots for 100%. I newer heard about workaround of this feature. For human it will increase the time for comment, but it will be ok I think.

I other hand javascript pop up pages will not be crawled in any way...

This 2 simple techniques will reduce spam comments a lot.

It seems like this person agrees too, but it's in German and automatic translation tools are only so good...

http://segert.net/weblog/item.php?i=109

Couldn't agree more. Infact, I blogged more or less an identical article to this yesterday: The Perils of rel='nofollow'

You offer some sane advise on cutting down comment spam. Although its now pretty trivial to develope a spam bot that is Javascript enabled.

I'm currently developing a PHP script that is pretty effective at filtering out referal/trackback spam: An Automatic Solution to Referral/Track-Back Spam. Any feedback is always apreciated :-)

hmmm... my links got lost somewhere in the post.

The perils of rel='nofollow' : http://richardathome.no-ip.com/index.php?article_id=416

An Automatic Solution to Referral/Track-Back Spam : http://richardathome.no-ip.com/index.php?article_id=415

>> rel = "nofollow" will not prevent comment spam

Correct!

nofollow - noLinks, noComments, noTrackbacks, noPageRank?

Hi Joerg,

usually I used nofollow - noLinks

I think there are better possibilities than nofollow. I have different directories without nofollow. Some of them are free for nearly all users. But I have written some strong rules for the users and I control everytime what they write in. I delete non- quality- comments and sites. O.K. that`s a lot of work. But in end I have a quality directory and good PR- Links for quality websites and descriptions inside.

Greets all,

MSN Bot is still crawling my site's bits which were rel="nofollow" :S
anybody know what the hell is going on?

<a href="index.php?do=disclaimer" rel="nofollow" target="_self">Disclaimer</a>

65.54.188.61 - - [21/Sep/2005:04:49:09 +0100] "GET /index.php?do=disclaimer HTTP/1.0" 200 9873 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

"PREDICTION: Dear Readers, Do you want that Palestine will be free? If your answer is yes, I recommend you the following home page: http://www.alah.hu Please send this letter to your friends! Sincerely, Zoltan Biro

No follow is to prevents search enhines from scrwling, however you can use robot.txt in your folder where prevented pages are.

Yes, I agree with Your prediction. In my opinion You can use this tag for sites like contact. So no pagerank goes to sites without real content.

Google PageRank carefully explained and what you can do with it - written by top SEO experts.

http://www.pagerank-prediction.com/

Google PageRank carefully explained and what you can do with it - written by top SEO experts.

http://www.pagerank-prediction.com/

check your site free seo tools

another site with good tools

I don't think it will stop spam either. What it may do is put off normal users from posting comments.

Any views on the fact that msn bot ignores rel="nofollow"

We dont use rel=nofollow, however there are times when it is handy and you always have the option, either to use it or not, depending on the situation required.

even rel=nofollow dosnt stops SEBots to visit spamming sites.

rel=nofollow can stop seBots to visit spamming sites, but it must be coinsided first by the programming.  Which depends on the language.

I have seen this explaination in other places and I have to say I agree, it won't prevent spam.

I think having this added to comments on your own blog will only put people off wanting to post on your blog in the first place, not just put off spammers.

156 SEO Tools

This SEO Tools page has links to the best SEO Tools on the internet and these tools will help you to optimize your website and move your search engine position higher.

http://www.seocompany.ca/tool/seo-tools.html

It is 22 months on, and your prediction was correct

What is worse, nofollow has unbalanced the natural linking of the net with many sites maintaining an unnatural amount of pagerank, especially when they benefit from unreciprocated trackbacks.

Comments on blogs have some extremely useful benefits, and this is partially how I actually arrived at this site.

A fresh comment on a blog for some reason makes it appear back on the radar for Google, and more specifically Google Alert. I have a Google alert setup for "nofollow" discussion, because I am in the middle of building a community site that supports people who don't use nofollow.
For some strange reason, today I received an alert because of the previous comment.

the nofollow tags are basically pointless, because outbound links from your site do not affect your sites pagerank or rating - so most sites are just paranoid and scared and use these tags thinking it will increase their pagerank in the long run. The rel no follow tags should only be used in dire cases, when your receiving spam site submissions.

Basically most blogs should not use nofollow tags, because it penalizes your commenter's - if  you let commenter's leave their link without the rel = nofollow, you reward them for commenting on your site and thus you'll actually get more users posting on your blog. People see it as a way to help the blogger, as well as be rewarded for commenting on a bloggers site. But using the nofollow tags is discouraging and some commenters may leave because it seems like a blog owner is being paranoid or too selfish. Remember some outbound links are needed to increase PR.  

I agree with Daringer, using the nofollow tag penalizes legitimate commentors. If you really want to eliminate comment spam, the best way to do it is through moderation. I have all my blogs set so that each comments must be approved before it is posted. I've also found that setting the blog so that people can post without signing up attracts more legit posters.

btw, I found this post while researching for a similar article regarding using the nofollow tag, but I was searching for a way to add a nofollow tag to certain links in my blogroll while leaving other links with natural flow. If you're using Wordpress and are interested in adding code to your blog enable this feature, I've posted a solution here..

http://www.affiliatebestprograms.com/2007/04/28/seo-tips-the-nofollow-attribute-and-your-blog/

Im debating on whether its worth it or not because how many people actually check before they comment if there is a nofollow or not? Its gotta be very low. However if you have a website that is only for SEO then I guess it is a wise move.

I think it still needs some tweeking according to our friends at <a href = "http://www.seegeorgiarealestate.com">Atlanta Real Estate</a>

Your right it won't increase blog spam. It will increase comments though on blog I think. Ill change it to dofollow on my site once I figure out how to do it.

I'm using nofollow solely because I hope my PR will increase and that my web site visitors will reciprocate by removing nofollow from their links as well.

I agree completely.  I've read an interview with a known spammer (as I'm sure at least a few of us have) who didn't seem phazed at all by this weak attempt at comment spam control.  He basically stated that as long as the text is still there and people can still click through, he is accomplishing his mission.  The best spam protection is human moderation.

Comment spamming in a blog about comment spamming :)

Your comment contains very useful information about all thank you

Ya thats true, but it will control spam upto some extent.Like if no tag then we get lot more spam comments.

<a href="http://www.somgraphic.com/">Somali Web Design</a>

Comments? what are they ;-)

Here is my share

I also dofollow on my site.

Your Comments
Name:
Email address (optional):
Home page, blog, or journal (optional):
Comments:

Related Posts

Category: Google
Category: My Blog
Category: My thoughts
Category: Ideas