Latest Writing Elsewhere December 2016

It’s hard to believe but 2016 is over. Here’s a list of my writings elsewhere in December 2016. I’m also including some RapidMiner community solutions that I had a hand in solving.

The Many Tools of Data Prep: Feature Generation & Selection

Upgrading RapidMiner: Where did my Processes Go?!

The Many Tools of Data Prep: Data Types and Conversions

The Many Tools of Data Prep: Data Quality

Operationalizing Analytics – It Shouldn’t Have to Hurt!

Using formulas inside RapidMiner Studio:

http://community.rapidminer.com/t5/RapidMiner-Studio/Use-a-formular-on-Results/m-p/35619#M24800

Do you really need to do a full re-install of RapidMiner Server?

http://community.rapidminer.com/t5/RapidMiner-Studio/Rapidminer-Studio-Cron-Trigger-Editor-Text-Problem/m-p/35449#M24712

Excute R vs R Scripting extension. Which is the right one?

http://community.rapidminer.com/t5/RapidMiner-Studio/Enabling-R-Scripting/m-p/35399#M24689

Missing Values or Filtering? Which is best?

http://community.rapidminer.com/t5/RapidMiner-Studio/Replace-Missing-Values-or-Filter-by-function/m-p/33870#M23903

 

 

Stock Trend Following Model in RapidMiner

Recently I discovered in my old archives a stock trend following model I started in RapidMiner. I never finished it back then but now I cleaned it up. I’m releasing it to the world as beta (use at your own risk) and will probably have some bugs” in it. The trend following model is based on Blackstar Funds, LLC paper Does Trend Following Work on Stocks?

The process is not production ready and there are some clean up things I need to do. I need to write a function that will cleanly output the ATR stops so I can chart them like in the paper.

Of course, the ultimate goal is to loop over a daily list of 52 week new highs and spit out recommendations, but that’s for another post and another process altogether.

Here’s a snapshot of the first part of the process.

stock-trend-following-process

Get the Stock Trend Following Model in RapidMiner

I have lots of RapidMiner and Finance/Trading related processes and posts in my Tutorial section, make sure to check those out too!

To get this Stock Trend Following Model, just download the zipped file here. You’ll need to install the Finance/Economics and Series extensions for the RapidMiner marketplace first to get this to work.

Plugging in Plot.ly, Python, and RapidMiner

I’ve struggled for a while trying to build an embedded visualization for my auto-generated blog posts. I tried D3js (javascript), Bokeh (python), and Plot.ly (various languages) and got frustrated quickly for something that will allow me to easily create and auto embed a chart.

In general, D3js has no barriers for embedding but it’s a pain in the butt to code javascript for a non-coder like me. Bokeh uses python and is kinda of nice since I know python but it’s very hard to auto embed a visualization on the fly. Plus the generated visualization is 1,000′s of lines of autogenerated code and clipping and pasting the code into a markdown post is a no-no for me.

Next I investigated Plotly. While not 100% perfect, I liked it from the get go. It’s syntax is very easy to learn and you can code it using javascript, python, pandas, and R. Since I tend to avoid R, I tried coding in their javascript and python/pandas API. The same frustrations I had in coding D3js came back for their javascript API, so I focused completely on their python/pandas API.

That was a success. When I wrote out the python/pandas code and them embedded it my RapidMiner process (see below), I successfully generated a static PNG image from my RapidMiner process and auto embedded it into my markdown post.

The only snag I ran into is that I needed to get an API token from Plot.ly to autogetnerate the static image. You can see in the code below that I X’d” it out but it was pretty easy to get it once you create an account with Plot.ly.

If you check out the python code I put into the RapidMiner Execute Python operator, you’ll notice that I use macros to alter the name of the autogenerated files. This is crucial if I want to set it and forget it” autoposting in a production sense (like using the RapidMiner Server), but that’s a post for another day.

Here’s the python code in RapidMiner: