April 2007 Archives

Does anybody remember the Seinfeld episode where George Costanza started doing everything opposite? His life turned around and he women, fame, and fortune. Well I had George Costanza moment yesterday (ed. Repost from Digital Breakfast) . I'm trying a new currency trading strategy that does a few things opposite of what I did before. Here's the revised and stripped down Forex trading plan.

1. Concentrate on five major pairs as defined by Oanda.com. They are:
   • EUR/USD
   • AUD/USD
   • EUR/JPY
   • GBP/USD
   • GBP/JPY
   • USD/JPY
2. Watch the economic calendar for upcoming landmines! Fundamentals drive this market.
3. Position sizing has changed back to a % of NAV model. Use 15% of NAV as the initial position size. Scale the % down as the account gets larger. Ultimate % of Nav target is 3%.
4. Use only 10 DMA Bollinger Bands on the 1 hour charts.
5. Use my neural net models to confirm trends in the EUR, USD, JPY, and GBP.
6. Don’t use stop losses. Sounds scary doesn’t it?

Smaller multiple entries will help me “dollar cost” average out losses. The trick is to trade with the long term trend, buy on dips, let positions mature, and slowly scale out profitable positions over time. If fundamentals were to change (bank rate hikes, drops, etc) then re-evaluate trends.

[tags]Forex, Trading, Plan, Currencies[/tags]

Although this lesson isn’t hard, you have to logically build your YALE experiment and put all their “operators” in the right order. If you don’t then your experiment will crash, or worse, give you weird results. As I said in Lesson I, YALE has a very steep learning curve and I spent many hours pulling out my hair trying to figure out how to make it work.

A basic YALE experiment consists of the following operators:

1. The root
2. A data loader,
3. A data visualizer
4. A data validator
5. A model creator
6. A model writer
7. A model applier
8. A performance evaluator
9. Final Experiment

The Root

Every YALE experiment has a Root operator. The Root operator is just where your experiments starts out. Your entire YALE experiment will “roll up” to this operator in the end.

Now, click on the YALE icon and load up the program and select “Blank Experiment.” Once you did that your screen should look like this:

Data Loader

A blank YALE experiment won’t do us any good, we have to fill it up with operators to help us analyze data and build a model! Before we can build a model, we need to load in our data. Without the data we can’t analyze it and build a model, duh!

YALE requires you to have a data loading operator in your experiment “tree”. Since we are using an Excel spreadsheet we need to use an Excel compatible data loader.

Do the following steps: Right click the Root Operator > Select New Operator > IO > Examples > ExcelExampleSource.

You should then see the ExcelExampleSource operator directly below your Root operator

Data Visualizer

Once you’ve selected your data loader, you want to add another operator that will let you manipulate and visualize your data.

Tip: This operator is not necessary but I find it useful to see patterns in the data. Skip this step if you want too.

Once again do the following steps: Right click the Root Operator > Select New Operator > Visualization > ExampleVisualizer.

Once you click the ExampleVisualizer you’ll see it directly below your ExcelExampleSource operator. When you run the YALE experiment, the data gets loaded in and the ExampleVisualizer lets you click on any data visualizations YALE creates for you for more information. I like this operator a lot because it lets me find specific dates in the data charts when I see anything out of the ordinary.

Data Validator

The next step, and very important one, is to transform your data into something called training and validation sets. When you build a model, you build it from training data. You essentially train the model to learn the relationships in your input data to explain the output data. The validation set is used to test the trained data to make sure it makes sense. YALE has an operator that automatically splits up your data randomly into training and validation sets and then feeds those sets into the learning algorithm.

To find the Data Validation operator, do the following steps: Right click the Root operator, select New Operator > Validation > XValidation.

Model Creator

Next we’ll add in the model creator, also known as the learning algorithm. Before we can place the learning algorithm, we have to create a “split” in the experiment. We’re doing this because we’ll use the same XValidation data set (training and validation) to build the model and test its performance later.

To do this we have to use something called an Operator Chain. An Operator Chain, in my dictionary, is nothing more than a fancy name for a branch in your experiment tree.

To find the Operator Chain do the following steps: First, Right click on the XValidation operator > Select New Operator > Operator Chain.

Now, the step we’ve all been waiting for! We’re going to place the learning algorithm into our experiment. YALE has several different algorithms available for you to use. Some are regression based, others use machine learning, and the one we’ll use is a classification algorithm.

A classification algorithm takes your data and classifies your output data into categories based on your input. Huh? This simply means that the algorithm takes your data and groups it into similar categories (this very handy when modeling trends because you want to find emerging trends before everyone else does).

When you use prediction data (when you want to see if the trend has changed) against your model, YALE will look for similar patterns and then give you the output signal (UP, DOWN, RANGE, etc) based on the categories your model learned.

The learning algorithm we’ll use is a classifier called “IBk” and to install it into your experiment you have to Right click the Operator Chain > Select New Operator > Learner > Lazy > IBk.

Model Writer

Now, after your model is learned, you want to write it to a file, this way you can load it anytime you want and test new data against it.

To install the Model Writer, right click your Operator Chain > New Operator > IO > Models > ModelWriter

Great! The model portion of the experiment is now done! If you really wanted to, you could learn a model right now but how would you know if its any good? We need some performance measures so we can determine if our model is good enough to make some predictions! That’s handled in the Model Applier and Performance Evaluator section, see below!

Model Applier

Once again we need to create a branch in our experiment to handle the performance evaluation. To do that, we’ll need to split off in the same place as before; we’ll split off from the XValidation operator. Follow the steps in the Data Validation section to install a new Operator Chain.

Right click on the XValidation operator > Select New Operator > Operator Chain.

Once you did that, we have to install an operator that will take your newly learned model and apply it to the XValidation data sets. This operator is called, surprisingly a Model Applier.

Right click on the second Operator Chain > Select New Operator > ModelApplier

Performance Evaluator

The last operator you’ll need is the Performance Evaluator. This operator gives you the option to see the model’s prediction accuracy, correlation, squared correlation, and a host of other performance measures. This let’s you know, right off the bat, if your model is any good. I would never build a YALE experiment without some sort of performance evaluation measures!

Final Experiment

If you’ve followed along and did everything correctly, your final experiment framework should look like the image below. If it doesn’t, then please go back and make fixes. Any deviation from this framework could produce experiment errors which causes baldness.

There you have it, you’ve built your first YALE experiment! Of course you can cheat and just download the XML file here (in zip format): Gold XML

In Lesson IV we’ll cover setting your preferences and running the model for the first time. In Lesson V, we’ll cover how to interpret the results and check out some of the data visualization capabilities YALE has.

Thanks so much for attending the Neural Market Trends University! Please feel free to leave me a comment if you have any questions.

I was inspired to generate three data graphs from my currency models for my readers after reading Maoxian’s post on the British Pound. These graphs show price relationships between the Euro and the US Dollar Index, the Euro and the British Pound, and finally the Euro and the Swiss Franc.

Euro/USD Index

Euro/British Pound

Euro/Swiss Franc

With the exception of the Euro/British Pound, the price relationship between the Euro, USD Index, and the Swiss Franc is pretty much linearly correlated. :)

[tags]Currencies, Trends, FOREX, Franc, Euro, Pound[/tags]

Alright class, let’s get started! Before we begin with today's lesson, we should review the four key questions we asked about building an AI financial market model in Lesson I, they were:

1. Do I have the right software?
2. What do I want to model? What are my input and output variables?
3. Do I have the financial data to model with?
4. Is the data in the right format?

In this lesson we'll do two important things:

1. Review the input variable spreadsheet
2. Build the output variable (trend)

Please feel free to email me or post a comment if you have questions or need clarification on anything covered in today's lesson.

Review the input variable spreadsheet

For those who have decided to use our lesson example (Gold trend), and have downloaded the GA-Gold.xls, please open up the spreadsheet. Once you’ve opened the file, you should see something like this.

The GA-Gold.xls spreadsheet should contain 8 columns, 1 for the date, and 7 others for the input variables we identified in Lesson I. If you remember back to Lesson I, we discussed the relationship between input and output variables. The input variables is the data that drives your output variable; in this example the Gold trend. You'll notice that our spreadsheet only contains input variables and no output variables. How are we going to get our output? What do we do now? The simple answer is, we roll up our sleeves and build the trend!

Build the output variable (trend)

For this portion of the lesson you’ll need to find a chart of your asset or security and identify the UP, DOWN, or RANGE trend area. For the lesson example, just download a chart of Gold prices from 1997 through today and look at the areas where Gold is in a DOWN, UP, or RANGE trend.

Tip: Make sure that the chart you use covers the data range you plan model. It’s best to use a slightly larger range to help you inspect the chart visually and not miss any key information at the edges.

Now comes the hardest part of this exercise, add a column called “GC Trend” to the GA-Gold.xls and enter the words UP, DOWN, or RANGE in the GC column as it relates to the date shown on the Gold chart and on the spreadsheet. What you are doing is building the output variable of the model and when we run the AI analysis, the software will find relationships between your inputs and the output to generate UP, DOWN, or RANGE signals when you test the model later with new data.

Tip: You can add other words like STRONG UP or BUY if you like. It depends on what words you want to use to describe the trend.

When you're finished, your spreadsheet should look something like this:

Now, save your spreadsheet under another file name. This new spreadsheet becomes the input data you will build your trend model from. Of course you can skip all this hard work and download the lesson example spreadsheet: Gold Final Input.

Tip: Save this spreadsheet! Every 3 months or so, you should more input data to this spreadsheet and re-optimize your model. This is important because the market changes all the time and new patterns and relationships emerge. All you have to do is add UP, DOWN, or RANGE to GC Trend column.

Congratulations! Your hard work is now complete! The work gets easier from here on but we still have to cover a lot of ground! The next step will be to build the AI experiment, load in your spreadsheet, and create a model! We'll tackle this in Lesson III and Lesson IV.

• Aussie D - RANGE
• British Pound- STRONG UP
• USD - DOWN
• Euro - UP
• Yen - DOWN
• Swiss Franc - UP

I have a lot of people and readers ask me how can they build their own neural net or AI learned model using the open source (free) Y.A.L.E. software. YALE is a very powerful data mining and machine learning program but its learning curve is like jumping off a cliff and into an underwater cave; it’s deep and scary!

Have no fear, you too can build a financial model to help you identify trends in different equity, currency, or asset markets, in a few easy lesson. Follow along with Tom! I make modeling FUN!

First off, building a learned and predictive model requires a lot of “up front work” so in the first few lessons we’ll concentrate on the setup. Once the setup is complete, all you need to do in the future is update some data and then optimize the model again with a few mouse clicks.

Before we get started you’ll need to ask yourself the follow questions:

1. Do I have the right software?
2. What do I want to model? What are my input and output variables?
3. Do I have the financial data to model with?
4. Is the data in the right format?

Do I have the right software?

Visit YALE website and download the free software. Just follow the on screen instructions and install the system to your hard drive. If you’re running windows, like me, you’ll have to download the entire window’s executable file.

What do I want to model? What are my input and output variables?

This is the biggest question you’ll have to ask yourself. What exactly do you want to know? In the example I’ll use for this lesson, I posed the following question: “What is the current trend of Gold?” Once you know what you want to model, identifying your input and output variables becomes easier.

Whoa! Hold on Jack! What are input and output variables? Your output variable is nothing more than what you want to know. For example, if you want to know what the current trend of Gold is, your output variable might be a simple word as UP, DOWN, RANGE or BUY, SELL, and HOLD. In the end you want to know if the data you analyze gives you an output of BUY, HOLD, UP, DOWN, etc.

Next, we have the more difficult input variables. These variables are what “drives” the price and trend of your output variable. For example, we are interested in Gold; what might drive the price of Gold? You might say some factors are bond yields, the price of oil, the level of S&P500 index, or even Silver prices. You’re probably correct in these assumptions and all of these prices, yields, or index levels might very well become part of your input variables as you try to find relationships between them and the trend on Gold.

In our lesson example we'll use seven input drivers to build the Gold trend model. They are:

1. 1-Month Libor
2. 10 Year Treasury Note
3. 30 Year Treasury Note
4. Canadian Dollar (currency)
5. Russell 2000 index
6. German DAX index
7. Philadelphia Gold and Silver Index

Do I have the financial data to model with?

Next, we need to find data to model with. I use a data downloading program called TraderXL Pro which let’s me download daily and weekly closing prices of currencies, stocks, ETF’s, and futures. I plug in the symbols for my seven input variables and download them into an Excel spreadsheet.

Tip: Yahoo allows you to download free historical data so if you’re looking for data to play with, check out the historical prices link for a specific asset symbol.

What I typically do is download two sets of data, one data set for model building and a second set to test your model against. The minimum thing to do is to download a data range starting 2 years and 3 months ago and ending 3 months ago for your model building data set and a second data range starting 3 months ago through yesterday (or whatever last market close day) to test your model against.

The reason why you want to use at least 2 years of data is that you can capture rising bond yields, market sell offs, and other financial events that will influence the current trend you are looking for. For our lesson example I downloaded nearly 7 years of closing data.

Tip: I download weekly closing data to build my trend model and then daily data for my test data set. This way I can see on a daily basis if my input variable’s daily closing prices could change any trends quickly!

Is the data in the right format?

YALE allows you read Excel, CSV, and a host of other file types, so finding a format that’s compatible with YALE is easy. For this lesson we’ll stick to using Excel because its easy for everyone to understand. I’ve uploaded the example spreadsheet we’ll be using to build our Gold trend model: GA-Gold.xls

Stay tuned for Lesson II where I discuss the first step in building the trend model.

[tags] FXI, Trends, NeuralNet, ETF [/tags]