In this video I highlight the data generation capabilities for Rapidminer 5.0 if you want to tinker around, and how to use a Genetic Optimization data pre-processor within a nested nested experiment. Yes, you read that correctly, a nested nested experiment.

Video download link (HQ):Rapidminer 5.0 Video Tutorial #4

I downloaded and started to fool around with another genetic algorithm addin for Excel.  This one is called xlbit and is created by Xlpert.  What I like about it is that it has a better interface than Gentik Solver, but its not free. It allows you do max and min for target cells, add constraints (a big plus), and lets you choose between three genetic algorithms: Elitism, Roulette, and Tournament.  It even creates a new worksheet with results and graph for your average and overall fitness.  Sweet!

If you like it, and I do, you can buy the addin for $15.  That's pretty reasonable considering the developer packages this addin with two more addins, a neural net and a type of value at risk routine.   Xlbit does come in a demo version, so you can try it out first.  

So what has your intrepid neural net / genetic algo techno trader been doing with this addin?  Writing new trading models? Analyzing the market? Finding hidden trading opportunities? No, not at all.  I've been genetic modeling the perfect measurements of women.

I’m recently came across some Python Code snippets for Machine Learning maintained by Stephen Marsland. I’m not a programmer or coder but I know several of my readers are.  Follow the link and enjoy.

Now that the new year is here and we went through the greatest meltdown since the Great Depression, what shall I do?  Well, max out my 401k for sure and update my neural net models!!!  I’ll be busy working on them but I’ll probably starting moving toward more genetic algorithmic models this year as they tend to adapt better to the shifting market place.

My market timing model seems to be holding up but I’m going to review it to make sure its still giving me the right signals.  The volatility indicator is showing a major pull back in market volatilty and if it continues to fall, I’ll say we’ve made a bottom.

On the flipside, Obama’s Infrastructure Stimulus plan seems to be going ahead as all my clients are scrambling (compressing schedules) to make the projects I’m working on meet the stimulus schedule.  That means I’ll be super busy from now till April.  If the stimulus really comes on full force, then I’ll be busy for years.

I believe that we are fast approaching a bottom based on my market timing model and simulations.  I know I’ve said that before but these levels of panic can’t sustain themselves for long without some sort of capitulation.  Below is a superimposed weekly chart (from 1990 through last week) of the my market timing indicator relative to the S&P500.

It’s a pretty good indication that some sort of top (not necessarily thee top) was made if a rapid rate of change in the indicator occurs while simultaneously making a new high.  Usually at these times I like to skim some profits and put them in bonds or cash.

Likewise, if the indicator experiences a massive rate of change while simultaneously making a new low, you have a pretty good idea that you’ve made a new bottom of sorts.  The timing model can’t tell me if I’ve made the BIG top or bottom, so I have to rely on other models to tell what that probability is and gauge how much money to take out or put into the markets.

Granted, we are seeing things in this market that are unprecedented but my model seems to be coping farily well.  The scary thing is I just watch my indicator blow through the 3.0, then 4.0, and now approach the 5.0 level without blinking.  These markets are in a free fall and I’m feeling the pain!

So how far will she go?

A hastily drawn yellow trend line on the chart shows the S&P500 finding support around the 1000 level first and the model issued a major BUY signal at the level.  Unfortunately we’ve already blown through 1000 so the next level of support, from inspecting the previous BIG bottom in 2003 is around the 800 level.  Then using my Genetic Algorithm model and Monte Carlo simulations, I come up with my most probable BIG bottom.  That’s the 700 level, more than 50% off the all time market high.

Could it go lower than 700, you bet!  But I’m willing to dump more money into my 401k at that level and then hold on tight.

The one nice thing about Genetic Algorithmic stock market models is that they evolve with new market data. The bad thing about them is that they’re a pain in the ass to create.  My current GA model is surprisingly simple and works in conjunction with my S&P500 Volatility Timing Model.

Although its highly secret, what I can tell you is that I use the Genetik Solver Excel Addin.  I specifically use it because its free and integrates with MS Excel.  Its a bit clunky to use but if you read my tutorial on it, you should be able to learn it in no time.

I take a snapshot of market data in real time using the input variables from my volatility model and use the Genetik Solver to analyze those data points.  I make sure to grab the max and min data points for each variable, load it into the Solver, enter my population size, cross over probability, mutation, and then run it.

The answers I get from running this analysis helps me understand at what levels my input variables (to the timing model) should rise or fall to for things to return to a state of calm.  From this I can better understand the current market internals and  what things to look for as the market evolves in real time.

Right now my Genetic Algorithmic stock market model tells me that in order for the markets to return to a calmer state,  Gold has to drop (no-brainer) and T-bills have to increase in yield to the 2% level.  What wasn’t so obvious was that fact the technology sector must remain fairly flat and the Nikkei 225 has to rally like crazy.  If Gold sits tight, T-Bill yields weaken, and you have a rally in the tech sector, watch out!  At least for for this week.

:)

My market timing model issued a BUY yesterday. The 1276 level in the S&P500 seems to be a very powerful area and I’ve added to my positions at 1276 the last time a BUY signal was flashed. Only time will tell if this was a wise move.

In other news, my new genetic algorithmic component to the timing model is showing that the market will firm up the further gold falls and as long as interest rates on the 13 week T-bills stabilize around here. Another component of the model looks at international markets and things seem to be firming up there as well.

Still though, my monte carlo model shows that volatility is expected to spike in the coming weeks, just in time for September.

In other non-market related news, I spend a lot of time in Twitter these days, you can follow me there if you like.

Ok, back to work.

It seems that every year in April and May I get extremely busy with work and my professional socieites! I guess everyone wants to get as much done before the summer break and the same goes for me!

This coming Thursday I’m having my second meeting with the ATS team to see 1) who’s on board, 2) what the end game is for the ATS, 3) answer any questions they have, and 4) come up with with a project plan with milestones. My main goal is to have to have prototype ATS up and running by end of summer that focuses on Stocks first. If its successful then we’ll be adding in a Forex option, Genetic Algorithms, and Neural Nets.

I ask my readers to please be patient as I expect posting to become intermittent as the ATS project kicks off. I suspect it will take up a lot of my free time as I make my dream become a reality.  The best way to be alerted to any new posts when I do check in is to subscribe to my RSS feed and/or follow my Twitter postings.

I will keep everyone apprised of its progress and if the team is willing, share a few “sneak peaks” on how its shaping up!

Oh, I’ll be hosting the next Rapidminer Meetup this June and if no one shows up this time, I’ll drop it completely! That will be your loss, not mine!

I had a great meeting with my IT contact today. He’s sold on the idea and has two developers interested to work on the ATS project. I’m really excited that my vision of an ATS is becoming reality!

Man do I have some wild ideas for the ATS using GA’s and Neural Nets but we’re first going to build a simple prototype to see if it works. We’ll probably focus on stocks for now but ultimately I’d like to enter the currency markets as well.

The next step for me is to write a presentation and technical requirements, have a meeting, and get everyone to sign a confidentiality agreement.

The markets will recoil in horror the day my ATS goes live, or maybe it wouldn’t give a shit. :)

Yes you read that right, I’m working on a new set of RapidMiner tutorial posts (I’m bagging the videos for now). I hope to share with my readers two new tutorials over the coming weeks/months and I expect to post the first installment this week.

The first tutorial will be about using RapidMiner’s Evolutionary Weighting and Genetic Algorithms to build a Market Timing Model. This tutorial will follow the same methodology I used to build my S&P500 Market Timing Model. If you ever wondered how to build a timing model using RapidMiner then make sure to check back often for these free and valuable lessons.

After I’m done with that tutorial, I’ll delve into the world of Web and Text Mining. I’ll show you how to web mine blogs and websites for interesting bits of data. I’ll make sure that all new tutorials will have downloadable templates and data for you to play with.

On top of this, I’ll be over the next few months I’ll be updating my original YALE tutorials to be compatible with the new RapidMiner format. The formats are close but not close enough. I’m sure that many readers downloaded the YALE templates and then opened with the new RapidMiner software and found some “run time errors.”

I’m curtailing the scope of my member section for the time being because I’m extremely busy at work and my mystery illness is still knocking me out. Right now I can only send out a weekly email with my S&P500 Market Timing Report, usually on Saturdays.

What you’ll get every weekend is an email with a few paragraphs of my read on the market based on my timing model and a PDF snapshot of where my proprietary volatility indicator is.

I’ve even simplified the signup process in the right sidebar, so all you need to do is click on the link and fill out some basic information. Just so you know, I’m doing this for free right now and I may change it at anytime.

The global market are getting hammered, things are looking ugly out there. I’m so glad that I’m mostly in cash and bonds right now, all thanks to my timing model. If you’ve been a reader of my newsletter (free to members) then you would’ve known that I got out of the market in mid December. The sell signals my model gave me in October and December have saved my ass from a whole world of hurt.

It looks like all the time I spent building neural net and genetic algorithm models has paid off.

I will be making changes to my membership and no longer toll gating any posts. From now this point forward, I will be charging for my market timing newsletter. The newsletter will be mailed out to paid members every week.