Elon vs Mark

Ah…… Elon and Mark. One is the real-life Ironman. One wants to build a J. A. R. V. I. S.. Isn’t it a surprise that they argue? But it’s happening!

If there is anything like trash talk in A.I., Elon Musk and Mark Zuckerberg are doing it. How should we view their fight?

Besides the obvious entertainment value of the fight. (You know this piece has ~370 likes at AIDL?) It’s hard to dismiss the danger of artificial intelligence. But we want to qualify we meant by “danger”. For example, immediate danger of A.I. based weapon is real. Some members quickly point us to this piece on automatic combat module. And you bet that all your familiar deep-learning technologies are all being used for military purposes. For example, speech recognition can be easily used as a means to automatically collect data from phone messages.

It makes sense to ask whether this kind of existing A.I. applications are causing danger of humanity. But this is not the A.I. what Elon Musk is cautioning people! The A.I. he was talking about is more what Prof. Nick Bostrom called Superintelligence, which is more a God-like being which has intelligence human cannot fathom. In this tenet of thinking, or shall we call “alarmist”, A.I. is a super scary subject. For example, just right now, alarmist could tell you “Well, a super-intelligent A.I. must already exist, because the best strategy to hurt human being is to hide itself”. If you say, “How about the current AI development just doesn’t seem to be this scary?”, then Alarmists would retort that there might quote prominent statisticians I. J. Good‘s concept of “intelligence explosion”, claiming you just can’t predict how intelligence of machine would develop.

So how should we view the matter? So far, it doesn’t seem like there are any real proof on the alarmists’ side of thinking. More importantly, the theory of alarmists such as Musk’s doesn’t allow any one of us to experiment and falsify [1]. Is it really a good scientific theory? We doubt. On the other hand, people who criticize Musk are usually practitioners, actual ML researchers, and those who actually build an AI system before. They support their arguments with facts and experience. We found their views make more sense.

That’s why, between Elon and Mark, we chose Mark. We are not alone, apparently Prof. Andrew Ng seems to agree with us. He suggested when we talk about the danger of A.I., rather than talking about Musk’s killer robots, the core issues should be poverty, inequality and job loss. We cannot agree more.

[1] We use the term “falsify” similar to Thomas Kuhn’s “The Structure of Scientific Revolution”.

techcrunch.com

The Origin of The Imagenet Database

We seem to forget that modern deep learning is indebted to Stanford Professor, Fei-Fei Li who propose, design and collect the Imagenet database.

As for those who protest and say “Imagenet is just a database! The fascinating part is XNet, YNet and ZNet!”[1] Remember the adage “90% of data science is data preparation”? In fact, as the Quartz story being told, Prof. Li’s journey of creating a 1-million annotated image database is yet another technical breakthrough in the field. e.g. If not because Amazon mechanical turk was there, it was close to impossible for her to organize so many workers to do such annotation.

So check out the story of the origin of Imagenet. Perhaps more importantly, check out the story of Prof. Fei-Fei Li, the mother of Imagenet.

[1] Make-up neural networks names, but ZFNet is real.

qz.com

Apple Machine Learning Blog

Apple starts a new ML blog! It is surprising because Apple is known for its defense-level of secrecy. We just couldn’t help to feel…. “Ah, Apple really wants to get into the A.I. bandwagon.”

But the hints of secrecy is still there. For example, whereas Google is very open on core technologies such as speech recognition or machine translation, Apple chose research subjects such as GAN-based synthetic images. In that way, nothing relevant to its core IP would be disclosed.

Looking at the technical content, which is mostly on GAN and how to use it well. No surprise that Apple has its own captain of deep learning like Soumith Chintala. But who is he/she? No one knows! Once again, it is Apple’s secrecy at work – all the posts are written by “Apple engineers”. In that way, Apple leave its competitors no way to poach any of its very competent employees.

All-in-all, we learn that Apple is taking steps on A.I. and deep learning, but we couldn’t help to wonder – In the future, would Apple’s culture of secrecy works well with the current more open culture of AI/DL? Would this clash impede AI development within Apple?

apple.com

The Last Imagenet Results

Imagenet 2017 results came out last week. In terms of classification, it went from 2.94% last year to 2.25% this year. Talking about stars in last two years of Imagenet, we have to talk about teams from China. e.g. The winner this year is a team from Beijing’s Momenta.ai and a postdoc researcher from U. Oxford. While we don’t see the dramatic improvement back in the first few Imagenets, the improvement is still fairly impressive (*).

At first we thought it is just another ensemble type of work. Yet the Momenta.ai+Oxford team did design a new learning block called “squeeze-and-excite”, they also improve the GPU memory pipeline. The team also promise to release a report later on.

It’s unfortunate that this is the last Imagenet, and as we know major sites haven’t participate since last year. But we are always grateful with the pioneering effort of collection of the database as it has resulted in many breakthroughs in deep learning and computer vision.

Footnote: Normally, we measure the relative improvement when working on a task. So with 0.75% absolute improvement and we start from 2.94%, you can think the improvement is ~25%.

image-net.org

World’s First Self-Driving Car with L3 Autonomy

For a long time, SDC we know are at L2 autonomy, that is so-called “hands-off”, which is the driver can take their hands off the wheel, but only monitor and engage in driving if necessary. As we know, Google was more the champion in L2 – it only requires engagement for every 5000 miles. (Uber? Every Mile.) But L2 was more known to be the status quo.

Then Audi comes in and claim that it has L3 on its Cardiallac CT3, that is “eyes-off” which allows users to take their eyes off and only require to attend driving in special limited time. More surprising here is that Audi is a large auto, and it has the reputation of strong quality control.

One thing for sure, Germany’ autos companies are among the first which researched on SDC. Of course, it was much more popularized as illuminaries such as Prof. Sebastian Thrun has competed in DARPA Grand Challenge. And eventually brought SDC technology to many research institutions of the States. So we shouldn’t feel too surprised that Audi has also researched on SDC, and apparently it is for sale now.

CT3 costs 98k euros, and it’s certainly for the die-hard fan of SDC, it will first be on-sale in selected are in Europe.

ieee.org

DeepMind’s Founder on AI and Neuroscience

Here is an interview from Verge with DeepMind Founder David Hassabis on his views about the current separation of two fields: artificial intelligence and neuroscience. Indeed, while neuroscience class these days would still concern on how computation can emerge from human biology, AI nowadays usually focus on practical applications. Should it be that way? We share Hassabis’ view – it shouldn’t be. There are many directions of AI research which can be inspired by neuroscience. And modern AI techniques which process large sets of data, can also shed light on unsolved problems in neuroscience. We can see much of these fusions in both our main FB group AIDL and our satellite group Computational Neuroscience and Neurobiology.

Hassabis’s paper was published in the prestigious Cell. Here is the link.

theverge.com

Google’s Gradient Ventures

Google started a new VC firm, called Gradient Ventures. So what’s interesting about the firm (beside the relevant, great name) ?

If you take a look at the list of Founding Partners and Advisors, most of them are engineers. On the advisor front, you find Prof. Daphne Koller and Dr. Peter Norvig, both of whom are famous in teaching AI/ML classes. So Gradient’s structure is very different from traditional fund which mostly run by career VCs.

The goal of Gradient is also fairly interesting. As reported by CNBS news, sometimes Google would put Google engineers into the startup to help out a project, and there are explicit rule to disallow them to bring back any IP back to Google. Another perk is that startup could access the vast amount of data accumulated by Google if they got invested by Gradient. That gives an extra edge for both the startups and Google.

We think this could be a boon to the A.I. startup ecosystem as there are few companies in the world who know the space better. That said, we can’t help but be skeptical about Google’s motive here – how this can be anything but a funnel for Google acquisitions or a funnel for figuring out where Google to apply its A.I. prowess next. Recall that Google invested in Uber too before deciding to compete with it. So, startups, do your due diligence.

blog.google

Microsoft Research AI

As we learned this week, MS is pushing forward Microsoft Research AI (MSRAI) This is clearly a measure to counter research labs such as Google’s Brain and DeepMind, Facebook’s FAIR and OpenAI. Microsoft already owns one research company with similar style – Maluuba. So, what does MRAI offer that’s different?

The first thing to notice is Microsoft Research (MSR) has always been a strong contender in AI and ML in the first place. Most notable is that they are one of the few groups come up with a deep-learning-based speech recognition system. MS’ aim perhaps is to create a specialized task force, with the scale larger than a single research department, on the field of AI/ML/DL. MSRAI seems to be the case.

Perhaps another differentiator of MSRAI: Once you look at the other five companies we mentioned – all of them focused on artificial special intelligence. Whereas MSRAI’s effort seems to focus on the general intelligence, the holy grail of our field. So the type of collaboration MSRAI is involved in is also different from other sites. e.g. She is working with MIT’s Center for Brains, Minds and Machines.

techcrunch.com

Baidu-Nvidia Alliance

On the news of AI alliance, perhaps we should talk about the Baidu-Nvidia Alliance happened around a week ago. As we knew, both companies are important players in deep learning. Nvidia dominates in terms of deep learning hardware, and we all know Baidu is the “Google of China”. So their alliance is very significant.

Three important gists about the deal.

• Baidu will adopt Nvidia Volta in large scale on its cloud platform.
• Baidu will pick up Nvidia PX on its self-driving car initiative. In China, Baidu certainly has advantage.
• Nvidia, on the other hand, will help optimize Baidu’s deep learning framework, Paddle-Paddle.

So far, it seems to indicate the alliance is more beneficial to Nvidia than to Baidu. One can imagine the Baidu’s Nvidia adoption would expand Nvidia dominance even towards China. Yet Nvidia’s effort seems to focus on merely helping making Peddle-peddle to be faster. Perhaps, this event signifies the power of Nvidia’s leverage in the AI space again, just like we reported on Issue 13 and Issue 14.

insidehpc.com

ICO to DeepMind : “Just That You Can, Doesn’t Mean That You Should.”

This week, UK’s Independent Commissioner’s Office (ICO) said that DeepMind and UK’s National Health Service (NHS) “failed to comply the protection law”.

Back in September 2015, DeepMind and The Royal Free London NHS Foundation Trust (“Royal Free”) entered into an agreement where Royal Free transferred 1.6 Million records of patient data to DeepMind. In February 2016, such data enabled DeepMind to launch an application called Stream. One objective of Stream is that it can identify and treat acute kidney injury (AKI).

In April 2016, New Scientist get hold of the agreement and reported the event. They found that the data included sensitive patient information, such as if the patient HIV-positive or drug overdose.

As a consequence, not only ICO was formed to investigate if DeepMind violates any laws. Another watchdog group, National Data Guard (NDG) was also investigating the matter, and it concluded in May that DeepMind’s handling of 1.6 Million record data has “inappropriate legal basis”.

That brings us to ICO’s decision this Monday. But that’s not the end of the story! A turn of event happened two days later – an independent review panel decides that DeepMind didn’t breach the Data Protection Act. They found “that DMH had acted only as a data processor on behalf of the Royal Free, which has remained the data controller.” (Source) Yet ICO thinks DeepMind is less absolved.. (quoted from ICO letter)

The processing of patient records by DeepMind significantly differs from what data subjects might reasonably have expected to happen to their data when presenting at the Royal Free for treatment.

For example, patients present due to accidents or receiving radiology, had no prior agreement with Royal Free to share their data. In the view of ICO, it violated UK Data Protection Act‘s Principle One: Personal data shall be processed fairly and lawfully. Similarly, the ICO found that there are 3 more principles from the Act were violated by DeepMind.

While some experts commended the effort of DeepMind setting up an independent panel, the ICO’s investigation gave an impression that the DeepMind-Royal Free deal was hastily done and were in violation.

Given the size of data and the prominence of the group, one would think the brilliant DeepMind decision makers would be more careful here e.g. Shouldn’t they check if the patient has consent of their data being used? Shouldn’t the patient be informed if their record is shared? It didn’t seem to be the case.

Perhaps more frustrating is that many DeepMind’s development on patient privacy seems to all happen after the fact. e.g. the confidential patient record based on blockchain happened only this year after the probing.

When it comes to applying machine learning on healthcare, the whole DeepMind-Royal Free incident reminds us that patient data is very different than other types of data. We share the view of the Information Commissioner, Elizabeth Denham. Really, “It’s not a choice between privacy or innovation”, if DeepMind didn’t rush, they could have come up with a fool-proof solution.

As a final word and rephrasing Denham’s another lesson – Just that you can do Deep Learning, doesn’t mean you should.

Acknowledgement:
We thank AIDL member Stuart Gray for bringing up the issue to the forum.

Reference:

• New Scientist’s Original Article Back in April 2016
• UK Data Protection Principle
• National Data Guard’s take of the matter
• ICO decision
• ICO letter
• ICO undertaking
• ICO Forth Lessons Learn
• DeepMind Response to ICO
• DeepMind Independent Reviewers
• DeepMind Independent Reviewers’ Annual Report
• DeepMind’s Verifiable Data Audit

deepmind.com

Closer Look at Amazon Skills Growth Number.

Amazon has grown its Skills from 10000 to 15000 in the past few short months.

What do the numbers mean? We will use this Wired piece as the basis of our analysis.

• First of all, skills were first reported around the end of 2015 with 135 skills, but at the end of 2016, the number grew to ~7000 skills. So we are talking about ~600 skills/month growth.
• Yet we also know that in January 2017, 3000 more skills appeared. That brought the skills number to 10000 in the Wired piece.
• Now there are 4 calendar months from February to June. So we know that the growth in last 4 months is ~1250 skills/month.

So the first conclusion you may draw is that “Wow! Skill’s growth in the last 4 months is faster than the average growth last year!” That’s very tempting. But then smart AIDL members remind us that in June, Amazon was running a promotion to give away 2500 Echoes for developers who publish a new Skill. So let’s assume each developer publish 1 skill and we discount exactly 2500 skills from the Feb-Jun numbers, we are really talking about ~600 skills/month, which doesn’t seem to be too different from the numbers from 2016.

Now one question you should also ask is did Amazon run any other promotions before June? Yes there was, from our discussion with members who are developers, Amazon gave out Echo Dot to hackathons if you publish several Skills. There are also incentives for top-performing Skill developers. But it’s hard to imagine their effect is bigger than the 2500-Echo give away.

Btw, Amazon is giving out 5000 Echoes in July too. They also provide templates for developers to speed up the process. So perhaps, next month we will see the number of Skills swell to …… 15000+5000=20000.

In terms of numbers, Amazon has seemingly huge advantage of number of Skills compare to Google’s and Microsoft’s. Such number is boosted by on-going marketing campaign. In our view, this is classical vanity metric marketing, like when iOS reported 1M apps in the App Store but it works as it gets coverage. Coverage begets legitimacy in the eyes of consumers.

Acknowledgement: We thank AIDL member Stuart Gray for mentioning Amazon’s June promotion to us.

techcrunch.com

Prof. Andrew Ng is Joining the Board of Drive.ai

This is probably the first official role of Prof. Ng after his departure from Baidu. What do we think about Prof. Ng joining Drive.ai’s board? Are there any technical reasons why Ng join the board?

We think one important reason is that Drive.ai adopts an all-in-one approach in their steering and navigation system. In long run, such approach is likely to outperform a multi-stage approach. That’s perhaps why Ng says Drive.ai is “one horse worth betting on”.

Of course, what everyone is more interested now is deeplearning.ai. So far no one knows what it is, but this TC piece probably did the best job in digging.

cnn.com

No Investors Are Going To Look For AI Startups In Few Years

Frank Chen shows himself not just a partner of a16z, but an advocate of AI. His A16Z AI Playbook is a highly readable guide for general tech audience on what AI is.

This time, Frank opines on the prospect of AI startup. He believe that in few years, AI would be like other technologies, such as Web and Mobile, become so ubiquitous that every one just expect your company has the best solution available.

Where did we hear this view before? Oh, Bonsai’s Mark Hammond just told us the same in our last Office Hour! Indeed, we agree with both Frank and Mark. There are just too much influx of talents into the AI/ML field. This would probably increase competition and commoditize many AI SaaS APIs, and eventually lower their price. Eventually, it becomes everybody game to use state-of-the-art AI system. Just look at our coverage of the “Not Hotdog” system by Tim Anglade, you will notice that using deep learning is no longer just the game for elite researchers.

machinelearnings.co

NYT On Prof. Geoffrey Hinton

We don’t need to tell you too much about Prof. Hinton. This is a great piece from NYT about the good professor. (You know he wanted to be a carpenter when he was young?)

nytimes.com

Prof. Yoshua Bengio named “Officer of the Order of Canada”

Congrat Prof. Bengio!

theglobeandmail.com

Tesla hires Andrej Karpathy

Tesla is hiring our beloved teacher of cs231n, Dr. Andrej Karpathy, away from Open.ai. Karpathy would become the Director of AI and Autopilot Vision at Tesla. And according to piece from TC, he will closely work with Jim Keller, who now overseeing both software and hardware division. How should we see the whole event?

• If you really think about it, Dr. K only work in an industrial research setting for about a year. Yet he is now overseeing a major A.I. functionality of Tesla. Normally you would expect such position to be filled by a Professor-level personal. But for a freshly PhD? This event is simply extraordinary.
• The first thing to mention is perhaps Dr. K himself, a young scholar in the field of deep learning, his skill is known to be proficient in multiple subfields of deep learning – albeit computer vision, his unreasonably popular article on “Unreasonable Effectiveness of RNN/LSTM”, his foray into topics such as reinforcement learning and generative models. Even his software such as convnet.js and arxiv-sanity are super well-received. Of course, he is also an early pioneer of image captioning. We know that this guy is a real deal, has the right stuffs and has it all in deep learning.
• But the whole event also shows certain desperation of Tesla : wouldn’t a professor level of expertise make more sense? Would Dr. K has enough industrial experience to tackle the AI challenges of Self Driving Car?
• Despite his skill, we believe Dr. K is facing a tremendous challenge – there is certainly a huge technical problems of how to really get Tesla to go beyond Level 2 of autonomy. Tesla is also facing fierce competitions from Waymo and 10+ car vendors.
• But then, Tesla’s gamble does make sense – Dr. K is not only a deep learning researcher, he is also a beloved teacher of many AI/DLers. His star power not only gain respect for Tesla, but will help to attract more talents in the future.

In any case, we congratulate Dr. K for joining Tesla. We only hope that he can still lecture on deep learning from time to time.

techcrunch.com

Yoshua Bengio Now Advises Microsoft (But He is Still Neutral!)

Wow, this widely-circulated Wired piece was saying Prof. Bengio is working for Microsoft now. But the Good Professor quickly correct the rumor. (Thanks to Zubair Ahmed who informs us!) But then the article does explains why he chose Microsoft. In a way, it is to balance the other two powers in deep learning – Google already has Prof. Hinton and Facebook has Prof. LeCun, and it’s fair there is duopoly of the two houses in deep learning.

Just to be clear. Microsoft, in particular MSR ,has never been weak in deep learning in the first place. e.g. The Cognitive Toolkit stands out among deep learning toolkits. And we all know that MSR is one of the sites which achieve human parity in the notoriously difficult Switchboard task.

Yet you may say Microsoft lacks of a strategic theme like Google and Facebook on deep learning. As you might perceive from Google I/O 2017 and F8, both sites give you a sense strong presence in the DL space. Nowadays developers know about Tensorflow and TPU from Google, or PyTorch from Facebook, how many people know about DL works done by Microsoft?

Would Prof. Bengio’s advisory role change MS’ role in deep learning? Only time will tell. The Wired piece trace several personals from Microsoft and it’s worthwhile for your time.

wired.com

Does Uber Knows Its Acquisition Has Taken Alphabet Files?

It sounds like the case. From Uber’s filing at Jun 8:

On or about March 11, 2016, Mr. Levandowski reported to Mr. Kalanick, Nina Qi and Cameron Poetzscher at Uber as well as Lior Ron that he had identified five discs in his possession containing Google information. Mr. Kalanick conveyed to Mr. Levandowski in response that Mr. Levandowski should not bring any Google information into Uber and that Uber did not want any Google information. Shortly thereafter, Mr. Levandowski communicated to Uber that he had destroyed the discs.

It’s one thing an ex-Waymo employee stealing IP, but it’s another that Uber knows it. We will wait to see the consequence for the now beleaguered company.

recode.net

Andrew Ng’s DeepLearing.ai

No details yet, but we are all excited!

deeplearning.ai

Apple Core ML

As we learn from WWDC 2017, Apple shows its resolve to be competitive in machine learning. Lots of press releases there – macOS HighSierra; the term “machine learning” appears 6x.

Of course, the main dish of Apple’s deep learning effort is Core ML. We are still getting up to speed on the library, but it allows iOS developers to interface apps with different machine learning libraries. While this might be slightly late to the game, this is a great step and Apple’s size might allow it to catch up very quickly. In a way, it is matching Apple’s ML software capability with the likes of TensorflowLite.

apple.com

WWDC 2017 – Machine Learning Features from Apple

So other than CoreML, what machine learning features excite us about WWDC2017? There are two:

• Sirikit This is the first time Apple allow developers to program their speech recognizer/dialogue system. From an ML-standpoint, it means that Apple is confident enough that speech recognition accuracy would not be affected even with a untuned vocabulary set. That’s a big deal.
• Handwriting and search – This is a long overdue but it isn’t easy stuff. Handwriting recognition (HWR) shares theoretical formulation with speech recognition, yet fewer research groups are working on it.

recode.net

Forbes’ Interview with Andrew Ng

This is perhaps the first interview of Prof. Andrew Ng after his departure of Baidu. What are the takeaways?

• While it is still unclear what he tries to do onward, it seems he is trying to form yet another initiative to support global AI development, as he stated in the interview:

One thing that excites me is finding ways to support the global AI community so that people everywhere can access the knowledge and tools that they need to make AI transformations.

• There are several inspiring quotes from the interview, here is the one we like:

In addition to work ethic, learning continuously and working very hard to keep on learning is essential. One of the challenges of learning is that it has almost no short-term rewards. You can spend all weekend studying, and then on Monday your boss does not know you worked so hard. Also, you are not that much better at your job because you only studied hard for one or two days. The secret to learning is to not do it only for a weekend, but week after week for a year, or week after week for a decade. The time scale is measured in months or years, not in weeks. I believe in building organizations that invest in every employee.

It echoes what we see on the AIDL forum, because too many of our members come to us and ask how you can learn quickly and get benefits from machine/deep learning. Unfortunately, there is no short-cut. You want to keep on learning, then slowly you would start to master the material and apply it real-life, albeit at work, for fun or start a business.

Last thing to note: As many people point out: in the earlier version of the Forbes’ interview, Peter High has mistaken NLP as neurolinguistic programming. It does affect the technical credential of Mr. High, but we still find the Prof. Ng’s view interesting and inspiring.

forbes.com

Meredith Whittaker’s Gender Analysis of ICML

Founder of Google Open Research, Meredith Whittaker analyzes the gender of authors from ICML. It’s disturbing to see women is underrepresented in a major conference of machine learning.

Courtesy from Jack Clark’s Import AI.

twitter.com

Kaggle now reaches 1 Million Developers Signed Up for Competition

Kaggle, in many ways, is a legendary archetype. We witness the “epitome of competitions”, Netflix Kaggle Challenge which offer $1 million dollars to developers. We know about xgboost, its popularity were in essence fanned by Kaggle. Now this is the 7th year of Kaggle, just acquired by Google and reached their 1 Millionth user. We expect nothing will change – Kaggle will still be the training ground of learners of machine learning, and we are grateful for the contributions of Kaggle and all kagglers.

venturebeat.com

Coursera raised $64M in Series D

The home of our most favorite on-line classes, including Andrew Ng’s Machine Learning and Hinton’s Neural Network Machine Learning, is raising $64M in Series D.

bizjournals.com

SDC Timeline for 11 Automakers

Ever feel confused about development of self-driving car? I know we do. This VB’s article summarizes the promise and progress of 11 top automakers in the planet.

venturebeat.com

Apple Neural Engine

Bloomberg reported that Apple is working on a chip for AI-specific tasks, known as Apple Neural Engine (ANE). The chip would mainly used for facial recognition and speech recognition.

Apple hadn’t responded to the report yet, but analysts are whispering what the ANE is going to be. Our guess is that unlike Google’s TPU, ANE is more a chip for embedded device like iPhone and IPad. Apple treats privacy matter seriously so it’s hard to imagine they come up with a TPU-like chip which focus on processing in data centers. But it’s thinkable that Apple would design Neural Engine by putting an optimized matrix computations into a separate unit, much like TPU.

bloomberg.com

AlphaGo vs Ke Jie – Game 3

Once again, Ke Jie resigned.

Quote from the Game

“This Summit is one of the greatest matches that I’ve had. I believe, it’s actually one of the greatest matches in history.” – Ke Jie, 9 Dan Professional, during the post match wrap up

We held this event aiming to discover new insights into this ancient, beautiful game. I can safely say that what has taken place since Tuesday has exceeded our highest hopes. We have seen many new and exciting moves, and we also saw AlphaGo truly pushed to its limits by the great genius Ke Jie, particularly in the amazing game two. It was thrilling to be in the match room and the control room watching that game unfold, and it has been a truly huge honour to play with such a great master.” – Demis Hassabis, CEO of Deepmind, during the post match wrap up

youtube.com

Prospect of Neuromorphic Chips.

Is there any design better than GPU/TPU for deep learning? And what if it is like deep learning, which is also biologically-inspired? Then you are thinking about neuromorphics chips, which loosely speaking is to build circuitry based on knowledge of human neurons.

Just a note for our readers: biological and artificial neurons operate in very different principles. For example, biological neurons communicate based on spikes and it’s time varying. Whereas, ANN is more a static approximation of biological neural networks, with different activation functions. (*)

This piece from IEEE gives a great high-level review on the advantages and disadvantages of neuromorphics design, and give you a clear picture what neuromorphic chips and its challenge against systems such as Tensor Process Unit. If you want a more technical version, check out Prof. Eugenio Culurciello’s article, which we also included in this issue.

(*) University of Washington’s on-line course on Computational Neuroscience would give you many valuable insights.

ieee.org

AlphaGo Defeats Team of Five Top Players

After two wins against Ke Jie, AlphaGo was able to defeat a committee of five top players, or “Team Go”. Is it a significant event?

Let’s think in this way: In terms of Elo points, given AlphaGo records (50 wins 1 lose), we may safely say it is around 400 Elo points stronger than the strongest human player.

The question is would forming a committee of five players overcome this 400 points difference? We doubt, because humans also have to communicate when they decide a move, and it seems to be a much slower channel. In a nutshell, it doesn’t seem to be a surprise AlphaGo also beat “Team Go”.

techcrunch.com

Google Launches AI Investment Platform.

We will just quote the Axios’ article here:

Why it matters: Google has never before had an investment effort aimed at a specific type of technology. Plus, this will be led by engineers, rather than by professional venture capitalists.

axios.com

AlphaGo’s Retirement

After defeating the best Go player, DeepMind announces that they are retiring AlphaGo. CEO Dennis Hassabis states:

The research team behind AlphaGo will now throw their considerable energy into the next set of grand challenges, developing advanced general algorithms that could one day help scientists as they tackle some of our most complex problems, such as finding new cures for diseases, dramatically reducing energy consumption, or inventing revolutionary new materials.

theverge.com

AlphaGo vs Ke Jie – Game 1

It’s happening! The first game between AlphaGo and the strongest Go player in the planet, Ke Jie. According to DeepMind’s coverage, Ke Jie took an active stance and use the most favorite opening (3-3 corner invasion) of AlphaGo to start with. The game was very tight until move 50.

The game seems to be very close, AlphaGo only won by half a point. The Twitterverse shows a different view though. For example, Andrey Karpathy (@karpathy) and Denny Britz (@dennybritz) both believe this is perhaps more of a feature of optimization – AlphaGo is optimized to win more, rather than to win more stones. Still, a win is a win. It doesn’t matter how many stones we are talking about.

Remark of the game:

Ahead of today, I studied and prepared quite a lot. At the very beginning, I made fierce, targeted moves, two 3-3 moves. I copied some moves that AlphaGo liked to use in past games. AlphaGo also made some unexpected moves as well. I was deeply impressed. Also, there was a cut that quite shocked me, because it was a move that would never happen in a human-to-human Go match. But, afterwards I analyzed the move and I found that it was very good. It is one move with two or even more purposes. We call it one stone, two birds. I am quite convinced by this loss that AlphaGo is really strong. From AlphaGo there are lots of things that are worthwhile learning and exploring. The influence of AlphaGo has been widespread. We should explore our minds and expand our thinking. – Ke Jie, 9 Dan Professional, in the post-match press conference

AlphaGo’s way is not to claim territory here or there, but to place every stone in a position where it will be most useful. This is the true theory of Go: not ‘what do I want to build?’ – Fan Hui, The European Champion

deepmind.com

AlphaGo vs Tencent Jueyi (FineArt) ?

Just like computer chess continues to advanced after Kasparov is beaten so is computer Go. While the main event is still AlphaGo vs the top human, more or more the focus turns into whether there are other Go machines which can compete with AlphaGo. Then the name of Tencent comes up, because as a computer Go machine goes, its career is no less impressive than AlphaGo.

Tencent Jueyi, or in Chinese 絕藝 (literally means “extraordinary skills”, translated as FineArt), won the championship title (link in Chinese), and it has 11 consecutive wins against other opponents, and 13 consecutive wins against Ke Jie. The amazing thing, is it is only being developed for around a year.

Perhaps AlphaGo should go against Jueyi, it would be a truly interesting competition to showcase which company has the best hardware plus software stacks.

qz.com

Softbank is now the 4-th Biggest Shareholder of Nvidia

This adds to its ARM portfolio, which is rumored to be working on their own low-powered A.I. chip for the edge. Our view is that in 10-20 years, Masa will be known as the Warren Buffet of tech. You heard it hear first.

bloomberg.com

Softbank Vision Fund

$95 billion dollars. Boom!
Go big or go home, right?

reuters.com

Alpha Go vs Ke Jie Game 2

The second game. Once again, AlphaGo is undefeated. And officially AlphaGo becomes the strongest Go player in the world.

Several notable quotes:

AlphaGo made some moves which were opposite from my vision of how to maximize the possibility of winning. I also thought I was very close to winning the game in the middle but maybe that’s not what AlphaGo was thinking. I’m a little bit sad, it’s a bit of a regret because I think I played pretty well. – Ke Jie

AlphaGo wins game 2. What an amazing and complex game! Ke Jie pushed AlphaGo right to the limit.
Demis Hassabis, at Twitter

engadget.com

Women in A. I.

A piece by Mariya Yao, summarized achievements by women in the field of A.I. Again, a topic that we here at AIDL always feel strongly about. Women’s contribution to CS has historically been minimized (intentionally or unintentionally). Keep these comin’…

forbes.com

Second Generation of TPU

Imagine this, Nvidia P100 is probably the best GPU you can buy last year, you bought one, but realize there is yet another device which is as fast as 32 P100 combined! So here is TPU v2, a monster device quietly developed last 2 years and it’s only released after v1’s specification is opened. We chose the article from The Next Platform, because it has the best writeup.

nextplatform.com

Free Tensorflow Research Cloud with 1000 TPUs

If you were Google, perhaps an important question you would ask is how to sell the powerful TPU. Google’s natural strategy is cloud. The way they market and test the water is through opening 1000 TPUs to public. So here’s the free sign-up links for everyone: not only researchers, but anyone who has computational need.

It says a lot of Google’s cloud strategy – TPU is an important differentiator between Google and its competitors such as Amazon and lately Nvidia. Google tends to think that inference will be where the cost/compute/power bottlenecks will be.

googleblog.com

Tensorflowlite

While Tensorflow is very versatile in building AI-based tools, it is also notoriously difficult to port its code to embedded platform. Tensorflowlite is going to change that. We don’t have too much detail yet, but we expect convenience features, or utilities, which can quickly transform models onto Android.

theverge.com

AMD Radeon RX Vega

In the last issue (issue 13) , we argued that Nvidia has incredible moat on deep learning hardware. Here’s AMD’s counterstrike! And so far it looks respectable. The most noticeable feature is the 16G on-board memory. That’s a new in the industry.

Would AMD makes researchers switch? It’s still hard to say, as we argued last issue, Nvidia has both hardware and software advantage. It’s hard to move away from CUDA from existing development. Since many CUDA core libraries are close source, it’s hard for AMD to come up with an equivalent architecture using say OpenCL.

hothardware.com