Category Archives: Uncategorized

AIDL Weekly Issue #19 - Fact-Checking - Does Facebook AI Create Its Own Language?

Editorial

Thoughts From Your Humble Curators

This weekend is the Forth of July long weekend. So news are getting lighter. Yet development of AI and deep learning never stops - we just learned that Prof. Bengo becomes "Officer of the Order of Canada", Prof. Ng is joining the board of Drive.ai; Frank Chen from a16z is telling us VC will not care about AI startups in few years. Of course, we all learned that "Not Hotdog" from Silicon Valley is an actual deep learning apps. All of these events are covered in this issue.

We also have two more new segments for you - The first is Fact-Checking. Routinely, we look at news posted on AIDL and decide if they are faked. This time we fact-check the claim "Facebook AI Created Its Own Unique Language", which is widely circulated from popular tech outlets last week.

Another interesting feature is "Member's Submission". This time we have Neel Shah, an active member of AIDL, tells us more about research trend in India.

As always if you like our newsletter, feel free to subscribe and forward it to your colleagues/friends!

Artificial Intelligence and Deep Learning Weekly

News




Factchecking

Fact-Checking : "Facebook AI Created Its Own Unique Language"

This time we check the claim "Facebook AI Created Its Own Unique Language". Is it true? Did Facebook really create an AIese?

When you hear such extraordinary claims from popular outlets, you should be alerted and feel suspicious. The next thing you should do is to look at the sources. In this case, the paper referred to is "Deal or No Deal? End-to-End Learning for Negotiation Dialogue" by FAIR.

So what did Facebook researchers actually do? First of all, their goal was to create bots which can negotiate. What they do first is to train a seq2seq model based on an English database. Of course, such bots would speak English, rather than any made-up language such as Esperanto or Toki Pona.

What is this machine language everyone refers to then? As it turns out, the researchers find that

... that models trained to maximize the likelihood of human utterances can generate fluent language, but make comparatively poor negotiators, which are overly willing to compromise.

So they use different strategies to evolve the bot such that they are better in negotiation. Of course, this evolution would result in a slightly different language from English, but it is more appropriate to call it a speaking mode rather than a unique language. Calling it a unique language seems to imply a difference like between English and French. Yet the difference we see here is more like English we use in chatting versus in a business setting.

AIDL Weekly rate this claim as False.

(20170728: The original link of the paper hosted at Amazon s3 was gone, so we replace it with the more updated version, retrieved from arxiv, as of today.)

Artificial Intelligence and Deep Learning Weekly

Blog Posts

Open Source


Video

Member's Submission

 

AIDL Weekly Issue #18 - Andrej Karpathy, Prof. Bengio's MS Advisory and The Goldberg Debate


Issue 18  

Editorial

Thoughts from Your Humble Curators

We took a short break last week, but so many happenings in our little world of AI/DL! Dr. K being hired by Tesla is certainly huge to us. So is Prof. Bengio advising Microsoft. We also follow closely the Yoav Goldberg's debate, which happens around 10 days ago. Why the interest? Because Goldberg's Debate would make you ask deep questions about our field(s), such as is NLP really solved by deep learning? Should we use arxiv as the publication platform? Does deep learning really live up to its hype on every field? We present a fairly extensive list of viewpoints in our blurb.

On technical news, we are loving "One Model to Learn Them all" (branded as "MultiModel"), which is perhaps the first model which can learn from varieties of modalities including speech, image and text. If you are into computer vision, the late Tensorflow object detection API would also keep you entertained.

As always, if you find our newsletter useful, make sure you subscribe and forward to your colleagues/friends!

Artificial Intelligence and Deep Learning Weekly

News




Blog Posts


Open Source


Jobs

Video

 

AIDL Weekly Issue #17 - Andrew Ng's First Interview After Baidu, and More

Editorial

The Organic Growth of AI - Thoughts From Your Humble Curators

The Last two months have been very eventful - GTC, F8 and Google I/O. We have AlphaGo vs the best human player. This week is a bit slower. We see Apple come up with the Core ML - this is Apple playing catch up to Google but could very well be very impactful long-term given Apple's culture of tight integration and iOS market size and position.

Some multi-threaded happenings this past week:

  • On celebrity in the field: We saw the first interview of Prof. Andrew Ng after he left Baidu, giving yet another interesting and inspiring interview with Forbes
  • On infrastructure: There is Kaggle hitting 1 million developers, and Coursera is getting its series D. Both are training grounds of budding machine learning researchers/engineers.
  • On new techniques: Facebook is able to train a Resnet-50 in one hour (Wow!) DeepMind also comes up with a technique in relationship modeling which improves around absolute ~30% accuracy. The common thread of both research is that the techniques themselves are surprisingly simple.
  • On our AIDL Facebook group: We have seen record sign ups in recent weeks. Just last week, we added ~1800 members, and it's continuing apace. We should be hitting 30K members in the not too distant future.

As always, if you like our newsletter, subscribe and forward it to your colleagues/friends!


An announcement here - one of us (Arthur) is having a vacation, so no press for AIDL Weekly next Friday, we will resume Issue #18 at Jun 23.

Artificial Intelligence and Deep Learning Weekly

News







Blog Posts





Jobs

Member's Question

Udacity DL class vs Hinton's ?

AIDL Member Fru Nck asked (rephrase): Can you tell me more about the Hinton's Coursera class vs the Google's Udacity class on deep learning?

Here are couple of great answers from other members:

By Karan Desai: "Former focuses more on theory while latter gets you through Tensorflow implementations better, keeping the concepts a little superficial. In my opinion both courses were made to serve different purposes so there isn't a direct comparison. Meanwhile you can refer to Arthur's blog."

By Aras Dar: "I took them both. For the beginners, I highly recommend Udacity and after Udacity, you understand Hinton's course much better. Hinton's course is more advanced and Udacity course is built for beginners. Hope this helps!"

Afterthought from me: I didn't quite take the Udacity class. But by reputation, it is more a practical class with many examples. If you only take Ng's, Hinton class is going to confuse you. In fact it confuses many PhDs I know. So go for the Udacity class and few others first before you dive into Hinton's.

Artificial Intelligence and Deep Learning Weekly

 

AIDL Weekly #16 - Apple Neural Engine, Neuromorphics and AlphaGo's Retirement

Editorial

Editorial

Last week we learned that AlphaGo retired after her glorious career: beating the best human Go player and a team of 5 top players in the world. We include 4 pieces in this issue just to wrap up the event.

None of these should be surprising to AIDL readers though. As we have predicted in Issue 15:

Go joins the pantheon of games like Chess, where computers have proven to be better than humans. As with Chess, research funding will move from Go to some other more complex A.I.-vs-human research projects.

It still leave us a question: what is the meaning of researching AlphaGo? We believe Dr. Andrej Karpathy gives a very good answer - he believes that AlphaGo's research only solve a very narrow sense of AI. Unless we are talking about yet another computer game, it's rather hard for AlphaGo's research to transfer. Take a look of the Blog section to see Dr. K's article.


We also heard about all about new chips last week, Apple Neural Engine and ARM new processor. So that's why we also include the Bloomberg piece on Apple Neural Engine. On the same vein, you may heard neuromorphics which purportedly used spiking neural networks, are they real? And are they getting better than the vanilla deep learning chips such as TPU. We include two pieces to analyze the matter.


As always if you like our newsletter, feel free to subscribe and forward it your colleagues!

Artificial Intelligence and Deep Learning Weekly

News






Blog Posts





Jobs

About Us

 

AIDL Weekly #15 - AlphaGo vs The World

Editorial

AlphaGo vs The World - Thoughts From Your Humble Curators

Back 10 years ago, no one would believe computer Go program can ever beat humans. Many experts estimate it would take 25-50 years to make Go to compete in 9-dan, not become the world champion.

This is exactly what happened yesterday - AlphaGo defeated Ke Jie, the strongest human Go player according to Go Ratings. Go joins the pantheon of games like Chess, where computers have proven to be better than humans. As with Chess, research funding will move from Go to some other more complex A.I.-vs-human research projects.

The big question is - what's the next big game for A.I. to challenge humans? Our guess is the next target is Starcraft/Warcraft. No doubt it would require another sets of technical breakthrough to defeat such complex game with so many game-states.

But before that, deep learning made some real history today.


Other than coverage of AlphaGo (4 items), we also cover SoftBank and statistics of NIPS. As always, if you like our letter, feel free to subscribe/forward it to your colleagues!

Artificial Intelligence and Deep Learning Weekly

News






Blog Posts


Jobs

Video

Videos of AlphaGo vs Ke Jie First Two Matches

Match 1

Match 2

Artificial Intelligence and Deep Learning Weekly


Member's Question

Question by Nishanth Gandhidoss‎: Is the following three is what AI is all about to learn for a Data Scientist?
Computer vision
Natural language processing
Reinforcement learning
A: (By Arthur) On the terms - "reinforcement learning" (RL) is usually used as one sub-branch of machine learning, usually goes parallel with "supervised learning" (SL) and "unsupervised learning" (UL). Briefly, RL usually means that you don't have correct output in your training (unlike SL), what you have is just a reward and the reward could be delayed. That makes RL very different from SL, and usually it has its own class of techniques."computer vision" (CV) and "natural language processing" (NLP), on the hand, is more applications for machine learnings. You can use techniques from SL, UL and RL in either one of these fields. So that's why the terms CV, NLP and RL seldom compare with each others.

On data scientist's learning of all these subjects - it depends on the type of jobs. More conventionally (say 5 years ago), being data scientist usually means processing data in table forms (R dataframe, SQL etc). But nowadays due to deep learning, data scientist can also be asked to work on high-dimensional data such as computer vision and NLP (through word vector). So I am not surprised that some job description include CV and NLP. If you have some knowledge about them, you do have an edge.

Artificial Intelligence and Deep Learning Weekly

 

AIDL Weekly Issue 14 - Google I/O 2017, AMD Vega and Review of Five Basic Deep Learning Class

Editorial

The Two Battlefronts of Nvidia

This week, we cover Google I/O: TPU v2 is a monster, equivalent to 32 P100s. Then there is new free TPU research clusters, TensorFlow lite which is a package for easy TF development on embedded device, automatic search of network architecture. All the amazing features!

Google could have taken quite a bit of attention away from Nvidia GTC by announcing TPU v2 last week, as The Next Platform's Nicole Hemsoth nicely put it:

[......] we have to harken back to Google’s motto from so many years ago… “Don’t be evil.” Because let’s be honest, going public with this beast during the Volta unveil would have been…yes, evil.

In other news, AMD also announced a competing product, the Radeon RX Vega. However, AMD has been having a hard time against Nvidia due to software issues (more details in this issue) even when its specs are slightly better and the card is cheaper. This is the power of the software moat. Hardware commoditizes fast but software makes things sticky.


Other than Google I/O and AMD new GPU card, we also include several nice resources and links this week including:

  • Arthur's Review on Five Basic Deep Learning Classes
  • Adit Despande's github on using Tensorflow.

As always, if you like our letter, feel free to subscribe and forward it to your colleagues!!
We also just reach 20000 members in our AIDL forum, so come join us!

Artificial Intelligence and Deep Learning Weekly


A Correction on Issue #13 Editorial

About the Editorial of Issue 13: Peter Morgan is kind enough to correct us - both Nervanna and TPU are based on ASIC, rather than FPGA. We have corrected the web version since.

Artificial Intelligence and Deep Learning Weekly

News




Blog Posts


Open Source


Jobs

Video

Member's Question

Thought/Anecdote from a participant of GTC 2017

Ajay Juneja share this on AIDL:
"Thoughts from the admin of the Self-Driving Car group this week (I attended the Nvidia Conference):

  1. Bi-LSTMs (Bi directional LSTMs) are everywhere, and working quite well. If you aren't using them yet, you really should. For those of us from the mechanical engineering world, think of them a bit like making closed-loop feedback control systems.
  2. The convergence of AI, VR, AR, Simulation, and Autonomous Driving. It's happening. Need to generate good data for your neural nets, quickly? Build a realistic simulator using Unreal Engine or Unity, and work with gaming developers to do so. Want to make your VR and AR worlds more engaging? Add characters with personality and a witty voice assistant with emotion to them, while using cameras and audio to determine the emotional state of the players. Want to prototype a new car or building or surgery room? Build it in VR, create a simulator out of it. We need to cross pollinate these communities and have everyone working together 🙂
  3. Toyota signed with Nvidia. That's 8 of 14 car companies... and they have signed the 2 largest ones (VW and Toyota). I hear rumblings from the AI community saying "If you want to build a self driving car TODAY, your choices are Nvidia and... nothing. What can I actually buy from Intel and Mobileye? Where are the engineers to support it? Qualcomm may have something for the 845 but they are drunk on mobile profits and again, no one knows their tools."

500K Nvidia Developers vs. next to nothing for Intel and Qualcomm's solutions.

I believe Nvidia has its moat now."

Artificial Intelligence and Deep Learning Weekly

About Us

 

AIDL Weekly #13 - Special Issue on GTC 2017

Editorial

The Moat of Nvidia - Thoughts From Your Humble Curators

There are many tech conferences each year. But none impressed us as much as GTC 2017. We curated 4 pieces about the conference, but in this Editorial, we'd to explain the incredible moat of Nvidia. And, we think this moat is getting stronger.

First, by "moat", we mean competitive advantage. So what's Nvidia's moat? Some of you might quickly point out its hardware platforms such as its GTX, Quadro and Teslas (or Pascal or Volta) series of GPU cards, and software platform, CUDA. Beyond the obvious IP and chip design moat, there is also powerful software lock-in. Indeed, as developers, we compile code with CUDA daily. CUDA is an easy to learn extension of C and is quick to produce results. The surrounding rich software support makes it easy to get up and running, and has high switching costs, once enough efforts has been spent on top of it.

But increasingly, Nvidia is branching out into new areas of computing, creating new moats. It just tripled its data center business in a yoy basis. It has to do with the fact that they own both the hardware/software platform. And deep learning is not going anywhere soon.

Now, this moat is further strengthening in GTC 2017. Why? First, it announced that it is going to train 100k developers just this year, creating more potential customers steeped in their wares. This is a smart move - behaviors are hard to change. Secondly, they announced a new cloud platform initiative (curated under "Nvidia GPU Cloud"), which makes it easier for newcomers to start building on Nvidia's platform. Now, it remains to be seen what the competitive dynamics would be with other large cloud platforms like Google, Amazon, and Microsoft which are also Nvidia's customers. Nvidia might just see its own platform more as an educational platform and not necessarily a major revenue contributor like AWS long-term.

Currently, there are two potential competitors of Nvidia, one is AMD, but AMD is still struggling to come up with a new GPU to compete. Then there is a ASIC-platform, but most of them are still under development (Intel's Nervanna) or proprietary (Google's TPU). So virtually Nvidia is monopolizing the deep learning computing platform.

In this issue, we further analyze on Nvidia's training plan, the new V100, new partners on Drive PX and its Cloud move. We also cover Medical Imagenet and other news.

As always, if you like our letters, please subscribe and forward it to your colleagues!

Edit at 20170514: Peter Morgan is kind enough to correct us - both Nervanna and TPU are based on ASIC, rather than FPGA. We have corrected the web version since.

Artificial Intelligence and Deep Learning Weekly

News





Blog Posts


Open Source



 

AIDL Weekly Issue #12 - Lyrebird, Recursion Pharmaceuticals, and Campas

Editorial

Thoughts From Your Humble Curators

We start to see how machine intelligence can be applied in controversial manner, two related pieces this week:

  • Lyrebird - which astounded us by not only mimicking multiple politicians, but they claim only one minute of training data is enough.
  • Campas - which provides sentence judgement based on software.

We also discuss Recursion Pharmaceuticals and what makes deep learning particularly useful in the company.

As always, if you like our newsletter, remember to subscribe and forward to your colleagues!

Artificial Intelligence and Deep Learning Weekly

News




Blog Posts



Open Source

Video

Member's Question

Difference Between ML Engineer and Data Scientist?

Q: (From Gautam Karmaker) Guys, what is the difference between ML engineer and a data scientist? How they work together? How their work activity differ? Can you walk through with an use case example?"

A: (From Arthur, redacted)

"Generally, it is hard to decide what a title means unless you know about the nature of the job, usually it is described in the job description. But you can asked what are these terms usually imply. So here is my take:

ML vs data: Usually there is the part of testing/integrating an algorithm and the part of analyzing the data. It's hard to say how much the proportion on both sides for each job. But high dimensional data is more refrained form simple exploratory analysis. So usually people would use the term "ML" more, which mostly means running/tuning an algorithm. But if you are looking at table-based data, then it's like to be "data" type of job. IMO, that means at least 40% of your job would be manually looking at trends yourself.

Engineer vs scientist: In larger organization, there is usually a difference between the one who come up with the mathematical model (scientist) vs the one who control the production platform (engineer). e.g. If you are solving a prediction problem, usually scientist is the one who train, say the regression models, but the engineer is the guy who turn your model to create the production system. So you can think of them as the "R" and the "D" in the organization.

Both scientist and engineer are career tracks, and they are equally important. So you would find a lot of companies would have "junior", "senior", "principal", "director", "VP" prefixed the both track of the titles.

You will sometimes see terms such as programmer or architect replacing "engineer"/"scientist". Programmer implies their job is more coding-related, i.e. the one who actual write code. Architect is rare, they usually oversee big picture issues among programmers, or act as a balance between R&D organizations."

Artificial Intelligence and Deep Learning Weekly

About Us

This newsletter is published by Waikit Lau and Arthur Chan. We also run Facebook's most active A.I. group with 19,000+ members and host a weekly "office hour" on YouTube.

Share on Twitter | Share on Linkedin

Artificial Intelligence and Deep Learning Weekly

©2017-2019 Artificial Intelligence and Deep Learning Weekly
| Sponsorship

 

AIDL Weekly Issue 11 - Groq: A Company No One is Talking About

Editorial

Thoughts From Your Humble Curators

Perhaps the biggest news last week is about Groq, a company started by Google's ex-employees who work on Tensor Processing Unit (TPU). We talk about the company and its current principals.

Of course, ICLR 2017 also held last week. We have two links this issue focused on the conference.

Other than Groq and ICLR 2017, we also cover:

  1. Notes on Stanford cs228n, a Bayesian network class,
  2. a note from Athelas' Dhruv Parthasarathy on image segmentation,
  3. another criticism of Neuralink.

As always, if you like AIDL Weekly, don't forget to subscribe and forward to your colleagues!

Artificial Intelligence and Deep Learning Weekly

News


Blog Posts







Video

 

AIDL Weekly Issue 10 - F8, Brain-Computer Interface, Apple's SDC Permit

Editorial

Thoughts From Your Humble Curators

The next big platform everyone will be fighting over is your mind. Check out Elon Musk's Neuralink and Facebook's brain-typing and skin-hearing.

Last week also featured F8, which happened on April 18th and 19th. F8 gave us another week filled with some far-out news: Augmented reality? Caffe2.ai? Brain computer interface? Check, check and check. We have 4 items in this issue which cover all these cool stuffs.

We also had a very interesting live-streamed office hour with Sumit Gupta, VP of HPC, AI and Machine Learning at IBM. We went in-depth into what Sumit thinks are bottlenecks in Deep Learning today and other topics. Check out the video below.

Other than F8 and the IBM interview, we also cover:

  • Development of SDC, by Apple and Baidu,
  • Neuralink and its criticism.

As always, if you like our newsletter, subscribe and forward it to your colleagues!

Artificial Intelligence and Deep Learning Weekly

News





Blog Posts


Open Source

Jobs

Video