Can Twitter Sentiment Analysis Predict the Stock Market

Ugly over at Uglychart.com just posted a link about research that sentiment mined over 10 million tweets from 2008 and was able to predict daily market behavior to an accuracy of 87.6%. While the post is vastly interesting from a text & sentiment mining perspective using social media, and the application of it to the stock market, I’m not 100% convinced its very viable.

Why? Well I tend to echo some of the comments left by readers at the bottom of the original post. For example, once this edge” is discovered by general market participants, it tends to get discounted and the edge goes away. So what we read here today is probably already discounted by the market and is just routine business as usual.”

Now, I certainly don’t mean we should abandon text & sentiment mining for the markets, but rather we should continue to use these tools to develop our own secret edges and evolve them as the market changes. Follow the advice of poker players and underarm deodorant manufacturers, never show your hand and never let them see you sweat.

Search Engine Optimization (SEO) and Data Mining

I posted about the power of Data Mining when analyzing your blog’s traffic and how to maximize your Google Adword advertising relative to your Adsense earnings, but I forgot to mention one critical thing! Search Engine Optimization (SEO)!

SEO is just a process to organize your blog, or website, in such a way that you’ll end up at the top when ever an Internet user searches for something that is relative to your site. If you advertise your blog using a Pay Per Click method, like Google Adwords, then being ranked at the top of searches is really important as Ms. Danielle points out!

It won’t come as a shock to readers of this blog that Data Mining can really help with your SEO! Techniques like associative analysis and cluster data mining are great ways to discover who’s clicking what on your site. Associative analysis is used to estimate the probability of whether a person will purchase a product given that they own a particular product or group of products.

Cluster data mining, on the other hand, can identify the profile or group of customers that are associated with a particular type of Web site [via Data Mining and Business Productivity, by Stephan Kudyba]. These two techniques are critical if you want to maximize any e-business!

Now here’s the caveat, before you can start data mining your site, you spend a few months gathering website statistics and data. However, this doesn’t preclude your ability to start optimizing your website for better web searching. Here are a 5 tips that I’ve been using that have had a great traffic impact in my blog’s short life.

5 SEO Tips:

  1. Write valuable content or offer a valuable service. I can’t stress this enough;
  2. If you run a blog, spend considerable time selecting the right categories, those help search engines effectively index your site. Over time I’ve modified my category list to create relevant descriptions for my blog posts;
  3. Create a Crawl List and XML sitemap for Google. Doing this let’s the Google spider index your site easier and faster;
  4. Use Google Webmaster tool to manage your sitemap and clean out old URLs;
  5. Try to keep the size of your content on your site under 30k so your site can load in under 8 seconds for 56.6k modems. This helps your page load under 8 seconds.

Update: I now use a great Python package called PySEOAnalyzer to review how the content on my blog is working. It’s open source and can be downloaded here.