The definitive weekly newsletter on A.I. and Deep Learning, published by Waikit Lau and Arthur Chan. Our background spans MIT, CMU, Bessemer Venture Partners, Nuance, BBN, etc. Every week, we curate and analyze the most relevant and impactful developments in A.I.
We also run Facebook’s most active A.I. group with 191,000+ members and host a weekly “office hour” on YouTube.
Editorial
Thoughts From Your Humble Curators
This week we cover Adrian Rosebrock’s interview with François Chollet. We also discuss Ben Evans’ article on “Ways of Thinking About Machine Learning” and how DeepMind now can play Quake III’s “Capture The Flag” at human level.
As always, if you like our newsletter, feel free to subscribe and forward it to your colleagues.
This newsletter is published by Waikit Lau and Arthur Chan. We also run Facebook’s most active A.I. group with 155,000+ members and host an occasional “office hour” on YouTube. To help defray our publishing costs, you may donate via link. Or you can donate by sending Eth to this address: 0xEB44F762c58Da2200957b5cc2C04473F609eAA65. Join our community for real-time discussions with this iOS app here: https://itunes.apple.com/us/app/expertify/id969850760
News
Blog Posts
An interview with François Chollet by Adrian Rosebrock
François Chollet is the lead developer of Keras, a popular package that interfaces with Tensorflow, Theano and Pytorch. A little about the interviewer too: Adrian Rosebrock has been making a name for his many hand-ons tutorial of using OpenCV, Nvidia DIGITS for deep learning applications. His questions are sharp and touch on many issues in AI today.
Capture The Flag
After AlphaGo, we are all looking for the next breakthrough in AI. So this piece on how DeepMind cracks Quake III Capture the Flag is interesting.
To understand this piece, you should understand that the agent is learning only from pixels and scores, they weren’t told about the rules. This is quite unlike OpenAI’s work on DotAI which quite a few of assumptions were made.
More interestingly, after training with human players and self-play, the agent shows cooperative behavior. That makes sense: if you put an RL-agent in an environment which cooperation is rewarded, then obviously the agent will cooperate more.
As Jack Clark mentioned in his Import AI newsletter, this result doesn’t seem to get as much press as other DeepMind’s result. It’s a pity. Large scale RL-training can now competitive and cooperative players, we think it’s a big deal.
You can also find the paper version at here.
Ways to Think about Machine Learning
This is written by Ben Evans who works at a16z, trending around two weeks ago. Evans suggest several helpful ways to think of ML. For example, he believes thinking of ML as AI is really unhelpful. It’s much more useful to think of ML as automation. In a way, that follows the view from industrial revolution. i.e. machines are slowly replacing humans. Evans works in a16, so his view certainly comes from practice.
Another thought, which is intertwined with his view that “ML as automation”, is the idea that ML is a foundational technology similar to relational database in 80s. This is an interesting view. In fact, say if you just think about technology such as image classification, it is really the foundation of other relevant technologies such detection, segmentation. And of course, most medical image classification is still, in a nutshell, image classification based on Convnet. And who knows, what will people dream up in the next 10 years?
We think Evans’ article is a useful view to look at AI, his view is more helpful than say thinking AI is a super-intelligence, or AI will steal our jobs. Those are wishful thinkings mixed up with fears and silly imaginations.
Open Source
ModaNet
Here is a clothing dataset which is annotated in polygons. ModaNet is based on the PaperDoll dataset with 50k+ images. It’s useful for such as commercial clothing classification and understanding.
Contemporary Classic
Understanding LSTMs
LSTM is a tough subject. When you first see a LSTM, you will mostly be mesmerized by its complicated structure as compared to a simple vanilla DNN. There seems to be few keys for understanding:
- It’s helpful to think of each gate as a neural network. So an LSTM-unit is just a composition of multiple networks where each has its unique function.
- Forget about the diagram. This is a rare case that understanding the equations first is more important.
- Start from a simpler architecture such as GRU. Update and reset gates are more intuitive to understand than the standard 3-gate form of LSTM.
That said, if you do want to learn LSTM better, we highly recommend you to read this article from Chris Olah. In the article, Olah dissect the an LSTM, more importantly, map different gates to its corresponding equations. It was a very helpful tool for one of us (Arthur) to understand the subject.
About Us
This newsletter is published by Waikit Lau and Arthur Chan. We also run Facebook’s most active A.I. group with 155,000+ members and host an occasional “office hour” on YouTube. To help defray our publishing costs, you may donate via link. Or you can donate by sending Eth to this address: 0xEB44F762c58Da2200957b5cc2C04473F609eAA65. Join our community for real-time discussions with this iOS app here: https://itunes.apple.com/us/app/expertify/id969850760