Building an AI Financial Market Model - Lesson III
In Lesson 2, I went over the concept of MultiObjective Feature Selection (MOFS). In this lesson, we’ll build on MOFS for our model but forecast the trend and measure its accuracy.
Revisiting MOFS
We learned in lesson 2 that RapidMiner can simultaneously select the best features in your data set while maximizing the performance. We ran the process and the best features were selected below.
From here we want to feed the data into three new operators that are part of the Series Extension. We will be using the Windowing, Sliding Window Validation, and the Forecasting Performance operator.
These operators are key to measuring the performance of your time series model. RapidMiner is good and determining the directional accuracy of time series and is a bit rough when it comes to point forecasts. My observation is that it’s futile to get a point forecast for an asset price, you have better luck with direction and volatility.
Our forecasting model will use a Support Vector Machine and an RBF kernel. Time series appear to benefit from this combination and you can always check out these links for more info.
Windowing the Data
RapidMiner allows you to do multivariate time series analysis also known as a model-driven approach to analysis. This is different than a data-driven approach, such as ARIMA, and allows you to use many different inputs to make a forecast. Of course, this means that point forecasting becomes very difficult when you have multiple inputs, but makes directional forecasting more robust.
The model-driven approach in RapidMiner requires you to Window your Data. To do that you’ll need to use the Window operator. This operator is often misunderstood, so I suggest you read my post in the community on how it works.
There are key parameters that you should be aware of especially the window size, the step size, whether or not you create a label, and the horizon.
When it comes to time series for the stock market, I usually choose a value of 5 for my window. This can be for 5 days, if your data is daily, or 5 weeks if it’s weekly. You can choose what you think is best.
The Step Size parameter tells the Windowing operator to create a new window with the next example row it encounters. If it was set to two, then it will move two examples ahead and make a new window.
Sliding Validation
The Sliding Window Validation operator is what is used to backtest your time series, it operates differently than a Cross Validation because it creates a “time window” on your data, builds a model, and tests it’s performance before sliding to another time point in your time series.
In our example we create a training and testing window width of 10 example rows, our step size is -1 (which is the size of the last testing window), and our horizon is 1. The horizon is how far into the future we want to predict, in this case it’s 1 example row.
There are some other interesting toggle parameters to choose from. The default is average performances only, so your Forecast Trend Accuracy will be your average performance. If you toggle on “cumulative training” then the Sliding Window Validation operator will keep adding the previous window to the training set. This is handy if you want see if the past time series data might affect your performance going forward BUT it makes training and testing very memory intensive.
Double clicking on the Sliding Window Validation operator we see a typical RapidMiner Validation training and testing sides where we can embed our SVM, Apply Model, and Forecasting Performance operators. The Forecasting Performance operator is a special Series Extension operator. You need to use this to forecast the trend on any time series problem.
Forecast the Trend
Once we run the process and the analysis completes, we see that we have a 55.5% average accuracy to predict the direction of the trend. Not great, but we can see if we can optimize the SVM parameters of C and gamma to get better performance out of the model.
In my next lesson I’ll go over how to do Optimization in RapidMiner to better forecast the trend.
That’s the end of Lesson 3 for your first AI financial market model. You can download the above sample process here. To install it, just go to File > Import Process. Lesson 4 will be updated shortly.
Read more RapidMiner tutorials
Thomas Ott
Member discussion