The definitive weekly newsletter on A.I. and Deep Learning, published by Waikit Lau and Arthur Chan. Our background spans MIT, CMU, Bessemer Venture Partners, Nuance, BBN, etc. Every week, we curate and analyze the most relevant and impactful developments in A.I.
In a major shuffle of talents, Apple hired Google AI's chief, John Giannandrea, and Jeff Dean became the new AI Chief of Google. We cover both pieces in our News section.
We also look deeper into Google's MobileNetV2 and summarized its various components.
As always, if you like our newsletter, feel free to forward it to your friends/colleagues!
This newsletter is a labor of love from us. All publishing costs and operating expenses are paid out of our pockets. If you like what we do, you can help defray our costs by sending a donation via link. For crypto enthusiasts, you can donate by sending Eth to this address: 0xEB44F762c58Da2200957b5cc2C04473F609eAA65.
Comet.ml allows tracking of machine learning experiments with an emphasis on collaboration and knowledge sharing.
It allows you to keep your current workflow and report your result from any machine - whether it’s your laptop or a GPU cluster.
It will automatically track your hyperparameters, metrics, code, model definition and allows you to easily compare and analyze models.
Just add a single line of code to your training script:
Apple's hiring John Giannandrea, the AI Chief of Google, is an important move. It shows Apple's determination to be a major presence in AI - During John Gianndrea's tenure, AI has been gaining presence inside Google. From what we see, this is big win for Apple.
As John Giannandrea is gone, who would be the new AI Chief in Google? You guess it right. Jeff Dean!
No doubt Dr. Dean has a lot of achievements in AI - He found the Google Brain team, involved in developing the currently most popular toolkit, Tensorflow. So he being the Google Chief makes a lot of sense.
Of course, talking about Jeff Dean, we got to bring up one of Jeff Dean facts, right? Oh well, from what we heard: Jeff Dean doesn't exist, he's actually an advanced AI created by Jeff Dean.
"The key driver should not only be technological progress, but human progress." Macron said. It's interesting to learn from a world leader on how AI should be adopted in the national level. The French government is planning to spend ~$1.85 billion on developing A.I..
Model compression in general, is seen as speed-up or memory reduction technique. And unlike research which optimizes performance, some techniques floating around are ad-hoc. This makes Google's MobileNet particularly interesting because they come up with all kind of non-trivial methods to optimize speed/space.
Reading the original paper, there are three major ingredients in Mobilenet V2:
Depthwise Separable Convolutions: this actually the major innovation in V1. The idea is that convolution layer can be factored into two convolutions. This was how V1 was able to reduce some models for a factor of 6-8.
Linear bottleneck layer: this is new, the team found that they could insert linear bottleneck layer to summarize information of a convolution layer. There is an interesting discussion on Manifold Hypothesis. Yet, the author stated that the theory only provided them an "empirical hint".
Inverted Residue: this is also new, which means that MobileNet V2 uses an architecture which has opposite direction from the usual Resnet. According to the authors, it's motivated by linear bottleneck layer. Looking at Section 5, it's probably because this new architecture gave rise to an efficient implementation.
Overall, MobileNetV2 is impressive and gives further 30% reduction in computation and memory. Sounds like something you want to check out.
The new program released by Microsoft, it's certainly interesting in terms of both breadth and depth. In this 10-class course, MS provides a comprehensive coverage of basic machine learning and AI. Each class requires 8-16 hours classwork which we feel is fairly substantial. We think MS' courses may be comparable to Google's earlier release of on-line "Crash course".
Looking at the original one-page letter Also, "This plan will irreparably damage Google’s brand and its ability to compete for talent." The letter also names couple of defense contractors. Indeed, many U.S. talents head to Google, mainly to avoid involvement with the military. So perhaps it is where the statement comes from.
This newsletter is published by Waikit Lau and Arthur Chan. We also run Facebook's most active A.I. group with 125,000+ members and host an occasional "office hour" on YouTube. To help defray our publishing costs, you may donate via link. Or you can donate by sending Eth to this address: 0xEB44F762c58Da2200957b5cc2C04473F609eAA65. Join our community for real-time discussions with this iOS app here: https://itunes.apple.com/us/app/expertify/id969850760
Artificial Intelligence and Deep Learning Weekly
Speech Recognition, Machine Learning, and Random Musing of Arthur Chan