Comments on: Understanding Fuzzy Trend Following in Excel

By: Tom

Tom — Sun, 08 Jul 2007 20:30:07 +0000

You do point out a limitation, yes a trending stock is always outside the training range but your model can be optimized with new data.

The true value to trend following using Knn, in my opinion, is being able to identify if a sell off is the beginning of a trend change or just another buying opportunity.

Thanks for reading and commenting. Please feel free to comment anytime! :)

By: Richard

Richard — Sun, 08 Jul 2007 19:50:58 +0000

I find this very interesting. I’m not a practitioner in this area, so I don’t have any experience using these techniques on financial data. Maybe I will come to see this issue your way one day. But, today, I don’t.

I guess I just have a hard time believing there’s predictive value in isolated closing prices (or yields, or volumes), even when there’s a strong statistical correlation with the trend for all available historical data. To me, it doesn’t pass the “common sense” test.

A truly trending stock will be perpetually out of the training set range, and therefore the Knn will often either be completely in the dark (making decisions based on far-away data points) or data-poor (having only seen one or two passes through a price range). In the data-poor case, I would actually expect the Knn to be wrong more often than right, since when it finally sees a price it trained on again, it will be because the stock is coming back _down_ (or vice versa).

Regardless this was fun to think about, and I appreciate the replies.

By: Tom

Tom — Sun, 08 Jul 2007 18:38:26 +0000

Richard: What you’re asking now is where is the validation data set to test against the training set? Ultimately that’s how the system learns by minimizing its errors between the two. This spreadsheet example only uses training data.

Well yes, anyone can choose inputs they want too but there are statistical measures to determine the validity of your inputs, things like t-stats for each input and coefficient of determinations (R2). If you can determine statistical significance for the inputs driving your output, then a Knn approach to trend following is very robust indeed, even if you’re just using closing prices.

In this case, the closing prices of EWS and EWJ explain about 28% of the trend’s variance. Was this the best example to use? Probably not, but the intent was to show how Knn can be used to build a trend following system in Excel.

By: Richard

Richard — Sun, 08 Jul 2007 16:00:59 +0000

Without a feedback loop in a network, I don’t see how it can “learn” any kind of temporal relationship between isolated input closing prices. So, I just don’t see how a Knn approach on closing values can even yield a “rudimentary” trend determination. Obviously anyone can choose different inputs as they see fit; I was just trying to verify that I wasn’t missing something about the given example.

By: Tom

Tom — Sun, 08 Jul 2007 11:23:13 +0000

Richard: As I mentioned in the post, this is rudimentary way at creating a trend following system in Excel. The ETF’s were chosen to illustrate the good Dr’s spreadsheet example. Perhaps using the yield on a T-note with EWS would’ve made better sense but feel free to add your own inputs, and expand the model, as you see fit.

My personal models, using YALE, are way more sophisticated than this example and I use several inputs (closing price, volume, yields, etc) to help determine a trend. However in the end, its really the direction of the closing price that tells you the trend based on your timeframe.

By: Richard

Richard — Sun, 08 Jul 2007 02:49:56 +0000

I think I understand the KNN, as it’s like the Case-Based reasoning I used to play with in the natural language understanding arena. Makes sense. But, I was surprised to see the inputs you are using for trend evaluation, and am hoping you can shed some light on it.

It appears you are using closing values of the two ETFs as inputs? I don’t immediately see how a trend can be determined from this information. I would have expected high-dimensional inputs like X1..Xi and Y1..Yi being the last i days of % change in the ETFs, maybe, with Z == +/- EWS trend. That seems like a much more straightforward way to train a system about trends, to me.

Am I just not thinking straight? I don’t see how a human could take single-point data on two ETFs and decide what the trend is for new points. Using the actual values in $ rather than % differences makes the system brittle against values outside the range it was trained on, no?