Sunday, March 13, 2016

Alphago probably isn't learning from Lee Sedol

There has been quite a bit of discussion about whether Alphago can learn from the games it plays against Lee Sedol. I think not. At least, not directly. 
The heart of the program is the “policy network” a convolutional neural network (CNN) that was designed for image processing. CNNs return a probability that a given image belongs to each of a predefined set of classifications, like “cat”, “horse”, etc. CNNs work astonishingly well, but have the weakness that they can only be used with a fix size image to estimate a fixed set of classifications.
The policy network views go positions as 19×19 images and returns probabilities that human players would make one of 361 possible moves. This probability drives with the Monte Carlo tree search for good moves that has been used for some time in go computers.The policy network is trained on 30 million positions (or moves) initially. 
CNN (aka “deep learning”) behavior is pretty well understood. The number of samples required for learning depends on the complexity of the model. A model of this complexity probably requires tes of thousands of example positions before it changes much. 
The number of samples required to train any machine learning program depends on the complexity of the strategy, not on the number of possible positions. For example, Gomoku ("five in a row", also called goban) on a 19×19 board would take many fewer examples to train than go would, even though the number of possible positions is also very large.
Another point: Any machine learning algorithm will eventually hit a training limit, after which it won’t be able to improve itself by more training. After that, a new algorithm based on a new model of game play would be required to improve the play. It is interesting that the Alphago team seems to be actively seeking ideas in this area. Maybe that is because they are starting to  hit a limit, but maybe it's just because they are looking into the future.
So Alphago probably can’t improve its play measurably by playing any single player five times, no matter how strong. That would be “overfitting”. The team will be learning from the comments of the pro players and modifying the program to improve it instead.
Interesting tidbit: Alphago said the chances of a human playing move 37 in game 2 was 1 in 10,000. So the policy network doesn’t decide everything.

15 comments:

Base Information said...

Thanks for given information. this is very useful for me.
Interesting Read about:- Business Intelligence

Martin Dineal said...

Great article – Impressive: We think you might be interested to know more about our company.Techno Data Group helps you get your hands on world’s best class customized B2B Contact List, which can make your sales and marketing campaigns cost effective. Achieve Your Marketing Goals and Grow Your Business.
IBM DBMS Users Email & Mailing List

Priya Sethuraman said...

Truely a very good article on how to handle the future technology. After reading your post,thanks for taking the time to discuss this, I feel happy about and I love learning more about this topic. keep sharing your information regularly for my future reference. This content creates a new hope and inspiration with in me. Thanks for sharing article like this. The way you have stated everything above is quite awesome. Keep blogging like this. Thanks.
Hadoop Training in Chennai
Hadoop Training
Best Hadoop Training in Chennai
Best Hadoop Training Institute in Chennai

jayashree said...

i saw your blog it is really good and very much interesting too, thus i like your information what you have posted so please update latest information too.

Best Laser Clinic In Chennai | Best Root Canal Treatment In Chennai | Best Implant clinic In Chennai

Swethapriya said...


keep sharing your information regularly for my future reference. This content creates a new hope and inspiration with in me

Best Dental Clinic in Velachery

jayashree said...

This blog is very well good, so please say about those things in a very well good manner and i am looking forward more things from your blog.

Best Laser Clinic In Chennai

Deepika said...

I read your articles very excellent and the i agree our all points because all is very good information provided this through in the post.

Best Dental Clinic In Vellore

Mahalya sree said...

Everything is fine, am happy about your blog. Thanks admin for sharing the unique content, you have done a great job I appreciate your effort and I hope you will get more positive comments from the web users.
Architectural Firms in Chennai
Architects in Chennai

Philips Huges said...

Wonderful blog.. Thanks for sharing informative Post. Its very useful to me.

Installment loans
Payday loans
Title loans

subash karthi said...
This comment has been removed by the author.
Karthi Keyan said...

very very amazing explaintion....many things gather about yourself...yes realy i enjoy it
Digital Marketing company in Chennai

Philips Huges said...


Its a wonderful post and very helpful, thanks for all this information. You are including better information regarding this topic in an effective way.Thank you so much

Personal Installment Loans
Payday Cash Advance loan
Title Car loan
Cash Advance Loan

Unknown said...

Great article, its helps us alot.
Best Business Intelligence Companies in Mumbai
Top Business Intelligence Services in Thane

anupavi said...


This is extremely helpful info!! Very good work. Everything is very interesting to learn and easy to understood. Thank you for giving information.

SEO Company in India

Aashi siva said...


Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us

Vmware Training in Chennai

Web Designing Training in Chennai

AWS Training in Chennai

Linux Training in Chennai

Microsoft Azure Training in Chennai

Post a Comment

Please leave a comment