Survival Of The Fittest Trading Models

Posted on by

I found this link via Aleablog and found it be very interesting. These traders are using genetic programming to create several trading models, breed and mutate them for the fittest ones, and then trade them.

Given this complexity, Sokasian and his team spent several months creating a language that would ask the computer to think very abstractly about coming up with a viable trading model. The computer creates a population of some 200 or so models. “You’ve taught the computer how to generate a model on its own,” says Sokasian. It applies certain criteria to the models and trains them on historical data.

Models that performed well during the training period are allowed to go forward into the next gene pool of models. “You assess the performance of each one of the components of the model, then you either allow them to propagate or die off,” says Sokasian. Then begins a process called breeding and mutation. [Wall Street & Technology]

This gives me some ideas on how to use RapidMiner’s Evolutionary and Genetic Algorithm operators. Oh so much to do, so little time!

  • http://sexyhotbeauty.blogspot.com/ Sherry

    Exciting stuff!

    It’s about time Darwinism is programmed into trading software.

  • http://www.neuralmarkettrends.com Tom

    Poor Darwin, he probably never thought he’d be digitized in million years! :)

  • http://billakanodoodahs.com Bill aka NO DooDahs!

    Survivorship bias?

  • http://www.neuralmarkettrends.com Tom

    Bill, please clarify your comment.

  • dododu

    this doodah guy seems like one hell of a redneck….

  • http://billakanodoodahs.com Bill aka NO DooDahs!

    My neck’s as red as the next guy’s, I suppose. Here’s the punchline from my favorite redneck joke, why don’t you guess the question? The answer is “The Sabine River.”

    Roll 300,000 fair dice and discard any that didn’t roll a six. Repeat the process four more times. You now have about 39 dice (give or take a few) that are “lucky.” Does anyone think that, if these 39 dice were rolled 10 more times, that the total number of sixes from those 390 dice rolls would be statistically very far from 65?

    By taking only survivors from what could be random processes, you increase the odds of having models fitted to “luck” and not to reality.

    BTW I’m not a big fan of neural network and humanless algorithms, having seen many of them in the insurance world. I look at them more as “brainstorming idea factories” that need to have their output examined carefully.

  • http://www.neuralmarkettrends.com Tom

    Bill, I thought you were alluding to that. Your argument makes sense as you would be fitting models to the lucky survivor. Extending that logic than you can also say that homo sapiens are result of survivorship bias from evolution since we are lucky to have survived and thereby able to reproduce.

    To me that would discount our evolved “adaptability” and intelligence that helps us survive in the first place. Evolution favors the strongest for survival because, in my opinion, black swans can happen anywhere. How you deal with those swans will tell if you get to survive and pass on your genes. Cycle continues.

    I’ve checked out your site, do you use regression to predict markets? That in itself puts it in the “neural net” realm. You are correct, you should always question the output and ask “does this make sense?”

    Thanks for commenting, good ideas Bill!

  • http://billakanodoodahs.com Bill aka NO DooDahs!

    A long time ago, a meteor struck the Earth and created the Yucatan Peninsula, resulting in the die-off of many species. It so happened that, on the land, mostly small mammals survived and thrived. Minus that meteor, do you think it might be reptilian creatures with opposable digits discussing stocks on the internet? I think so. The land survivors were very far from the fittest before the meteor. Don’t discount the role of luck, even though other factors (intelligence, adaptability, culture) contributed, post-luck.

    The more complicated the model, the harder it is to figure out what is “luck” and what isn’t. If we don’t understand it, and can’t explain it to a layperson in two minutes or less, without using technical jargon, we probably shouldn’t use it. As I progress, I find myself pushing for simpler models.

  • http://www.neuralmarkettrends.com Tom

    Bill, I disagree with your assessment that the small animals (land survivors) weren’t the “fittest” at the time of the dinosaurs. If they weren’t then they’d all be extinct regardless of meteor or not. The meteor wiped out the dinosaurs, who were at the top of the food chain, which then created a void and let the mammals grow into the void that was left. Yes it was bad luck for the dinosaurs and good luck for the mammals.

    I think some of the cast on CNBC are surviving dinosaurs with opposable thumbs. :)

    Complexity breeds instability, nature likes simple.

  • http://www.meyersanalytics.com Dennis Meyers

    ahh..here we have curve fitting at it’s best!!

    All financial data is made up of noise(random processes) and perhaps a weak signal. whenever you perform any kind of optimization algorithm of a set of data on any fixed time frame you automatically fit the noise and the signal. It is usual for the noise in financial data to be much larger than the signal. While in out-of-sample data the signal will repeat, the noise will not and the curve fitted strategy will fail. that’s reality…get used to it! These ideas are covered in my working papers at http://www.meyersanalytics.com/articles.php

  • http://www.neuralmarkettrends.com Tom

    Dennis,
    Thanks for you link, I’ll dig deeper into your research as time permits. Thanks for stopping by and I hope you’ll return!