April 2010 Archives

In several of my video tutorials I assign different parameters for my learning model "on the fly."  Of course the question any astute reader/viewer should ask is, "why did you choose those parameters instead of another combination?" That's a great question and the answer is, "well I just choose those parameters to illustrate my point for the video."  While this answer is not at all satisfying to the astute reader/viewer, it does lead us to ask the most important question of all, "what are the right parameters to choose?" This can be answered very well if you were to use Rapidminer's Parameter Optimization operator in your initial data discovery phase.  This operator allows you to choose some or all of the parameters in your experiment and iterate different values for them to meet some specific requirement on your part (i.e. performance). For example, if you were using the Neural Net operator and didn't know what to set your learning and momentum parameters to, to get the best classification accuracy, you would use the Parameter Optimization operator to interate different combinations of those parameters to find the best accuracy. Once the Parameter Optimization operator determines those values, you can input them into your experiment and truly optimize your model for performance! See below for an actual output from a parameter optimization model I'm working on.  You can see that Rapidminer indicated that  momentum of 0.3 and a learning rate of 0.6 was the best parameter settings to maximize the accuracy rate and minimize the classification error. While is operator is a fantastic feature (they got evolutionary optimizers too!) for us data modelers, its a massive computer resource suck.  I would advise anyone using this operator to have a very powerful server or computer, with oodles of memory, to run your iterations. FYI, I plan on going over a short parameter optimization example in upcoming webinar.

I have some great news!  If you've every wanted to ask me a direct question and/or get more information on my past video tutorials, this will be your chance! I plan on hosting my first Neural Market Trends webinar this May! I plan on going over a few of my video tutorials in more detail for the webinar course and will provide a long Q&A session to boot!  Of course I can answer any other general questions you might have about Rapidminer and applications for it. It should be noted that this will not be a free webinar!  I do plan on charging for it, but I will provide an "early sign up" incentive that will be reasonable.

My readers might be wondering why I have a beautiful Rapid-I logo so prominently featured at the top right side of my site. It's probably because the Rapid-I team has my logo so prominently featured on their partner page. Yes, you read right.  I've joined forces with the Rapid-I team and became a partner company with them to help spread the word and evangelize about Rapidminer, and their other exciting products too. What does this mean for you?  It means that I'll be focusing on using Rapidminer exclusively on this blog from now on and hopefully roll out some new and exciting features over the course of the year.  I have a lot of things cooking right now that I'm not a liberty to post about right now, but 2010 is shaping up to be a good year for my loyal readers!

Just wanted to pass this along, in case you haven't seen it already but  Rapidminer has released the Web Mining extension for version 5.0.  All you need to do to install it is to go to the pull down menu HELP > Update Rapidminer.  It should find all the new extensions and let you select them.

As I wrote before, I'm trying to make Neural Market Trends a better reader experience but stripping out a lot of things and keeping things simple.  As such, I've had a shiny new and unique branding logo created for my blog by Stefenie! Stef did a fantastic job and helped me translate my vision for the Neural Market Trends brand into an elegant logo.  She's professional, responsive, and her graphic design skills are great.  I highly recommend her for your future blog work or other graphic design needs!  Big Thanks Stefenie! Check out her blog to see some more of her talent!

For what its worth, there's an interesting post on Fast Foruier Transformation and how to build a Rapidminer model to find  stock market cycles in the forums going on.  Members Ronmac and AWC have been kind enough to share the Rapidminer XML code and data file for it. I fooled around with it and discovered some interesting patterns in the markets,  especially using S&P500 data. It would appear that there is a turning point in the market every 12 days or so.  Very interesting and this model definitely requires more analysis by yours truly...

In this video we continue building a financial time series model, using S&P500 daily OHLCV data, and the windowing, sliding validation, and forecasting performance operator.  We test the model with some out of sample S&P500 data. [flashvideo file=wp-content/uploads/2010/03/Rapidminer5-vid10.mp4 /] This video can be viewed in HQ by clicking this link here.  Please make sure you have Quicktime or another MP4 capable reader installed in your browser. Here are the XLS training and out of sample files.

1. S&P500 Training XLS
2. S&P500 Out of Sample XLS

March was my biggest month for web traffic ever!  This probably has nothing to do with my sense of wit or writing style, but rather due to my Rapidminer 5.0 Video Tutorials and a faster template.  The change in my blog template to a simplified version helped immensely with the load and response time. I noticed an immediate bump in keeping first time visitors on my site the very next day!

March was my worst month for web income because I cut out all advertisements on this blog (for now). With the exception of text link ads, which netted me about $36 for the last month, I had no other income.

On an interesting side note, I was able to max out my free Dropbox space from 2GB to almost 10 GB through the generosity of my readers.  All they did was signup for their own Dropbox account using my referral link and I ended getting nearly 8GB's of additional space.  Now I can move my large data around with ease.Thanks!

For the month of March 2010 I had the following  blog statistics:

• 5,425 visitors
• 14,144 page views
• 2.61 pages/visit
• 4:29 minutes/visit
• 56.65% bounce rate (this dropped significantly when I changed the template)
• 52.90% new visitors

I have a feeling that these numbers will drop after I post my last Rapidminer video and go on break.