Tag Tensorflow

Posts: 5

Introduction to Keras

This is a really great introductory video on Keras and how simple it makes calling complex deep learning libraries like Tensorflow. The 13 year old author builds a great deep learning model in under 100 lines of code.

I do some questions w.r.t. to the AUC score being 1 (that always raised red flags in my mind) but he he does share the code on Github so everyone can follow along.

This is why I like Keras a lot, it's like H2O.ai and makes the complex work of coding very simple and accessible.


TensorFlow and High Level APIs

I got a chance to watch this great presentation on the upcoming release of TensorFlow v2 by Martin Wicke. He goes over the big changes - and there's a lot - plus how you can upgrade your earlier versions of TensorFlow to the new one. Let's hope the new version is faster than before! My video notes are below:


  • Since it's release, TensorFlow (TF) has grown into a vibrant community
  • Learned a lot on how people used TF
  • Realized using TF can be painful
  • You can do everything in TF but what is the best way
  • TF 2.0 alpha is just been released
  • Do 'pip install -U --pre tensorflow'
  • Adopted tf.keras as high-level API (SWEET!)
  • Includes eager execution by default
  • TF 2 is a major release that removes duplicate functionality, makes the APIs consistent, and makes it compatible in different TF versions
  • New flexibilities: full low-level API, internal operations are accessible now (tf.raw_ops), and inheritable interfaces for variables, checkpoints, and layers
  • How do I upgrade to TensorFlow 2
  • Google is starting the process of converting the largest codebase ever
  • Will provide migration guides and best practices
  • Two scripts will be shipped: backward compatibility and a conversion script
  • The reorganization of API causes a lot of function name changes

TensorFlow v2

TensorFlow v2

  • Release candidate in 'Spring 2019' < might be a bit flexible in the timeline
  • All on GitHub and project tracker
  • Needs user testing, please go download it
  • Karmel Allison is an Engineering manager for TF and will show off high-level APIs
  • TF adopted Keras
  • Implemented Keras and optimized in TF as tf.keras
  • Keras built from the ground up to be pythonic and simple
  • Tf.keras was built for small models, whereas in Google they need HUGE model building
  • Focused on production-ready estimators
  • How do you bridge the gap from simple vs scalable API
  • Debug with Eager, easy to review Numpy array
  • TF also consolidated many APIs into Keras
  • There's one set of Optimizers now, fully scalable
  • One set of Metrics and Losses now
  • One set of Layers
  • Took care of RNN layers in TF, there is one version of GRE and LSTM layers and selects the right CPU/GPU at runtime
  • Easier configurable data parsing now (WOW, I have to check this out)
  • TensorBoard is now integrated into Keras
  • TF distribute strategy for distributing work with Keras
  • Can add or change distribution strategy with a few lines of code
  • TF Models can be exported to SavedModel using the Keras function (and reloaded too)
  • Coming soon: multi-node sync
  • Coming soon: TPU's

There's a lot in this 22 minute video about TensorFlow v2. Must watch.


Deep Learning with TensorFlow and Keras

A really good technical introduction to deep learning from @AndrewYNg's Coursera that touches on stocastic gradient descent for optimization, which is really cool in my book. Sample processes are provided on the original article but I think you should just take the Coursera. :)

The following problems are taken from a few assignments from the coursera courses Introduction to Deep Learning (by Higher School of Economics) and Neural Networks and Deep Learning (by Prof Andrew Ng, deeplearning.ai). The problem descriptions are taken straightaway from the assignments Read More.


Why is Tensorflow so slow?

This article cracks me up.

Let's be real: tensorflow is more like 5 frameworks awkwardly fused together than a single framework - there are so many ways to do 2d convolutions, in tf.layers, the "Dense" layer is different than the "dense" one - how are they different? only one way to find out!

There are a ton of tutorials on the internet for doing the same thing in tensorflow but pretty much every one is different because there are so many damn APIs and they're all incompatible with each other.

Sorry for being so biased - but after a long time of trying to learn to love it, I still hate tensorflow. I've found pytorch to be a lot quicker to write stuff in, even though it does take some getting used to (it's basically numpy on your GPU).

and the best one:

Just like EA with games, almost anything involved with Google will have something like this.

Android: fragmentation

Angular 2: complex huge mess

Tensorflow: Tensorslow (emphasis mine)

The reasons are different though but similarly all have management problems

H/t to @mschmitz_


Data Science and Machine Learning Education

When I first self-taught myself 'data science,' there wasn't a lot on the Internet to help me. I spent years cobbling information together reading what I could find about it. Now, there's a plethora of Data Science and Machine Learning education available. There's forums, open source libraries and much much more. Most of it is free and damn good. There's no better time for a non data scientist or machine learning wannabe to learn about it, if you want to put in the time in.

I just stumbled across Jason Maye's presentation on Machine Learning 101. Jason is from Google and he does a bang up job of explaining what features (attributes) are, the basics of machine learning, what is AI vs Machine Learning vs Deep Learning, and much more.

AI vs Machine Learning vs Deep Learning

He touches on many commonly used algorithms like multilayer preceptrons, k-nn, decisions, trees reinforcement learning, and even good old linear regression. He even embeds some great videos on how all this works and recommends that you set aside 2 hours in a quiet room to listen/read/watch his presentation.

How Google Uses Machine Learning

Bonus: There's quite a bit of discussion on TensorFlow too.


Neural Market Trends is the online home of Thomas Ott.