As an effort to help search engines index my site better, I installed a robots.txt file. The reason why I did this will become clear in a minute. When you run a blog, search engines will parse your site and index it with a lot of duplicate data and posts. Duplicate data and posts can have a negative effect whereas the spider might think that your blog is a spam blog (splog). In the long run this can cause lower rankings.

Why the spiders think that way is pretty logically, at least how I think they work! If you write a blog post, and its current, it will be on your main index page (1 copy). At least two more copies will reside in your category and archive sections. Essentially you’ll have three copies of the same post if the spider parses your site that day!

As soon as your post drops off the front page it will end up in your category and archive sections; you’ll still have two copies of the same post! The best solution is to tell the spider NOT to index one of those sections! It’s that simple!

I found a version of an optimized WordPress robots.txt file and discussion here. I then modified it to my needs and installed it in the root directory. So far it seems to be working but only time will tell how effective it will be. From what I’ve read, using a robots.txt file is a long running experiment and you have to tweak it over time to maximize its effectiveness.

Ever since I analyzed my blog’s traffic using data mining and neural nets and began implementing a lot of the SEO tips and tricks I’ve read, I started to see a steady rise in my Alexa Rankings. I was tickled pink to see that my weekly average broke through their 100,000 barrier this weekend! I suspect this is because of the niche nature of my blog and my tutorials on building an AI financial model.

Thanks to all my readers and those your have subscribed to my feed! If you haven’t subscribed to my feed, now is a great time!

I posted about the power of Data Mining when analyzing your blog’s traffic and how to maximize your Google Adword advertising relative to your Adsense earnings, but I forgot to mention one critical thing! Search Engine Optimization (SEO)!

SEO is just a process to organize your blog, or website, in such a way that you’ll end up at the top when ever an Internet user searches for something that is relative to your site. If you advertise your blog using a Pay Per Click method, like Google Adwords, then being ranked at the top of searches is really important as Ms. Danielle points out!

It won’t come as a shock to readers of this blog that Data Mining can really help with your SEO! Techniques like associative analysis and cluster data mining are great ways to discover who’s clicking what on your site. Associative analysis is used to estimate the probability of whether a person will purchase a product given that they own a particular product or group of products.

Cluster data mining, on the other hand, can identify the profile or group of customers that are associated with a particular type of Web site [via Data Mining and Business Productivity, by Stephan Kudyba]. These two techniques are critical if you want to maximize any e-business!

Now here’s the caveat, before you can start data mining your site, you spend a few months gathering website statistics and data. However, this doesn’t preclude your ability to start optimizing your website for better web searching. Here are a 5 tips that I’ve been using that have had a great traffic impact in my blog’s short life.

5 SEO Tips:

1. Write valuable content or offer a valuable service. I can’t stress this enough;
2. If you run a blog, spend considerable time selecting the right categories, those help search engines effectively index your site. Over time I’ve modified my category list to create relevant descriptions for my blog posts;
3. Create a Crawl List and XML sitemap for Google. Doing this let’s the Google spider index your site easier and faster;
4. Use Google Webmaster tools to manage your sitemap and clean out old URLs;
5. Try to keep the size of your content on your site under 30k so your site can load in under 8 seconds for 56.6k modems. This helps your page load under 8 seconds.

Hat tip to Ms. Danielle for the photo!
[tags]Blogging, Tips, Howto, SEO, PPC, Adwords, Adsense[/tags]