Recommender System Conference, Vancouver - 2018

You will find link to presentation and code for Tutorial
"Concept to Code: Learning Distributed Representation of Heterogeneous sources for Recommendation"
here. Please stay tuned.


This blog will help self learners on their journey to Machine Learning and Deep Learning journey.
I plan to come up with week by week plan to have mix of solid machine learning theory foundation and hands on exercises right from day one.

Your suggestions and inputs are most welcome. Please contact me at omsonie at gmail.com

ML: Supervised and Un-supervised learning

1. Andrew Ng's Coursera Course

  • Sup Un-sup Learning (40min)
  • Linear Regression one var (1 hr 15min)
  • Linear regression one var Matrix ()
  • Linear regression multiple features (1hr)
  • Octave (1hr 20min)
  • Logistic regression (1hr 10min)
  • Regularisation (40min)
  • Neural_Network (1hr)
  • NN Back-propagation (1 hr 15min)
  • Advice apply ML Bias, Var (1hr)
  • ML System Design (1hr)
  • Support Vector Machine (1hr 37min)
  • Un-sup: Clustering (40min)
  • Un-sup: PCA (1 hr 10min)
  • Anomaly Detection (1hr 30min)
  • Recommender System (1hr)
  • Large Scale Machine Learning (1hr)
  • Example Photo OCR (50min)

  • Ref: Machine Learning Exercises in Python - http://www.kdnuggets.com/2017/07/machine-learning-exercises-python-introductory-tutorial-series.html

2. Andrew Ng's Stanford CS229 Course

  • Supervised Learning, Discriminative Algorithms (30 pages)
  • Generative Algorithms (14 pages)
  • Support Vector Machines (25 pages)
  • Learning Theory (11 pages)
  • Regularization and Model Selection (8 pages)
  • Regularization and Model Selection ( 3 pages)
  • Unsupervised Learning, k-means clustering (3 pages)
  • Mixture of Gaussians ( 4 pages)
  • The EM Algorithm (8 pages)
  • Factor Analysis ( 9 pages)
  • PCA Principal Components Analysis (6 pages)
  • ICA Independent Components Analysis (6 pages)
  • Reinforcement Learning and Control (15 pages)
  • Boosting algorithms and weak learning (11 pages)

  • Ref: http://cs229.stanford.edu/

3. Yaser S. Abu-Mostafa - CAL TECH Course

  • The Learning Problem - (slides 20)
  • Is Learning Feasible? - (slides 20)
  • The Linear Model I - (slides 25)
  • Error and Noise - (slides 23)
  • Training versus Testing - (slides 21)
  • Theory of Generalization - (slides 19)
  • The VC Dimension - (slides 25)
  • Bias-Variance Tradeoff - (slides 25)
  • The Linear Model II - (slides 23)
  • Neural Networks - (slides 25)
  • Overfitting - (slides 22)
  • Regularization - (slides 22)
  • Validation - (slides 23)
  • Support Vector Machines - (slides 21)
  • Kernel Methods - (slides 21)
  • Radial Basis Functions - (slides 23)
  • Three Learning Principles - (slides 23)
  • Epilogue- (slides 18)

  • Ref: https://work.caltech.edu/telecourse

4. Probability - Tom Mitchell - CMU

  • Naïve Bayes (51 pages)
  • NNaïve Bayes Logistic Regression (17 pages)
  • NNaïve Bayes (35 pages)
  • NGaussian Naïve Bayes (31 pages)
  • NNaïve BayesLogReg ( 17 pages)
  • NGenerative/Discriminative classifier (36 pages)

  • Ref: http://www.cs.cmu.edu/~ninamf/courses/601sp15/lectures.shtml



Neural Network and Deep Learning

1. Neural Networks by Geoffrey Hinton - Toronto Coursera


2. Andrej Karpathy Deep Learning cs231n


3. Deep Learning - Nando de Freitas


4. Deep Learning by Google - ud730 (Udacity)


5. Deep Learning by Andrew Ng (NEW) - DeepLearning.AI


6. Deep Learning Tutorial



Computer Vision - CNN



NLP

  • CS 224N / Ling 284 by Christopher Manning is a great course to get started.
  • CS224d: Deep Learning for Natural Language Processing,
  • Stanford class by David Socher (founder of MetaMind)
  • For more details see How do I learn Natural Language Processing?
  • On word embeddings: http://ruder.io/word-embeddings-1/
  • word2vec Parameter Learning Explained - Xin Rong
  • GloVe: Global Vectors for Word Representation - Jeffrey et. al.
  • Visualizing Data using t-SNE - Laurens et. al.
  • How to Use t-SNE Effectively - https://distill.pub/2016/misread-tsne/
  • Distributed Representations of Sentences and Documents - Quoc et. al.

  • Ref: http://cs224d.stanford.edu/


RNN - LSTM

  • The Unreasonable Effectiveness of Recurrent Neural Networks - Andrej Karpathy
  • Understanding LSTM networks - Christopher - http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  • Deep Learning, NLP, and Representations - http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/
  • Attention and Augmented Recurrent Neural Networks - https://distill.pub/2016/augmented-rnns/
  • Unfolding RNNs: Concepts and Architectures - http://suriyadeepan.github.io/2017-01-07-unfolding-rnn/
  • Doc2Vec Tutorial - https://rare-technologies.com/doc2vec-tutorial/
  • Distributed representations of sentences and documents – Le & Mikolov
  • Doc2vec model Example - https://amsterdam.luminis.eu/2016/11/15/machine-learning-example/
  • Building Skip-Thought Vectors for Document Understanding - https://www.intelnervana.com/building-skip-thought-vectors-document-understanding/
  • Tweet2Vec: Character-Based Distributed Representations for Social Media - Dhingra et al. - http://www.cs.cmu.edu/~wcohen/postscript/acl-2016-bd.pdf
  • Recent work in combining attention mechanism in LSTM Recurrent Neural networks with external writable memory has meant some interesting work in building systems that can understand, store and retrieve information in a question & answering style. This research area got its start in Dr. Yann Lecun’s Facebook AI lab at NYU.
  • The original paper is on arxiv: Memory Networks. There’re many research variants, datasets, benchmarks, etc that have stemmed from this work, for example, Metamind’s Dynamic Memory Networks for Natural Language Processing
  • Skip-Thought Vectors - https://arxiv.org/pdf/1506.06726.pdf
  • Character-Aware Neural Language Models - https://people.csail.mit.edu/dsontag/papers/kim_etal_AAAI16_slides.pdf
  • Ref: http://colah.github.io/
  • Recurrent Neural Networks for Collaborative Filtering - Erik Bernhardsson


RBM & DBM

  • Deep Boltzmann Machines - Ruslan et. al.
  • A BetterWay to Pretrain Deep Boltzmann Machines - Ruslan et. al.
  • An Efficient Learning Procedure for Deep Boltzmann Machines - Ruslan et. al.
  • A Beginner’s Tutorial for Restricted Boltzmann Machines - Deeplearning4j.org
  • RBM for Collaborative Filtering - Ruslan et. al.
  • RBM and recommender systems - Xavier Chapuis
  • A Practical Guide to Training Restricted Boltzmann Machines - Geo rey Hinton


Reinforcement Learning

  • Introduction to Deep Learning and Self Driving Cars [MIT 6.S094]
  • 1. Introduction to Deep Learning and Self-Driving Cars (1h30m)
  • 2. Deep Reinforcement Learning for Motion Planning (1h30m)
  • 3. Convolutional Neural Networks for End-to-End Learning of the Driving Task (1h20m)
  • 4. Recurrent Neural Networks for Steering Through Time (1h15m)
  • 5. Deep Learning for Human-Centered Semi-Autonomous Vehicles (35min)
  • Ref: http://selfdrivingcars.mit.edu/
  • David Silver’s (Google Deepmind) Video Lectures on RL
  • Book: Reinforcement Learning: An Introduction - Rich Stutton
  • Andrew Ng: CS229 - Reinforcement Learning and Control (15 pages)


Books

  • Machine Learning: A Probabilistic Perspective - Kevin P Murphy
  • Patttern Recognition and Machine Learning - Christopher M. Bishop
  • Machine Learning - Tom Mitchell
  • The Elements of Statistical Learning - Hastie, Tibshirani, Friedman

  • Deep Learning - http://www.deeplearningbook.org
  • Learning Deep Architectures for AI (2009) provides a good but academic introduction paper to the eld. http://goo.gl/MkUt6B
  • Deep Learning in Neural Networks: An Overview (2014), another excellent but academic introduction paper to the eld. http://arxiv.org/abs/1404.7828
  • Deep Learning Tutorial - LISA lab, University of Montreal

  • Mining of Massive Datasets (Cup) Paperback use pre formatted date that complies with legal requirement from media matrix – 20 Jun 2014 by Anand Rajaraman (Author), Jeffrey David Ullman (Author)

  • Reinforcement Learning : An Introduction" - Richard S. Sutton

Books with Codes

  • Python Machine Learning - Sebastian Raschka
  • Building Machine Learning Systems with Python - Luis Pedro Coelho and Willi Richert
  • Programming Collective Intelligence - Toby Segaran
  • Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems

Foundation & Math

  • Book: Bayesian Reasoning and ML David Barber
  • Video: Linear Algebra for machine learning” and was created by Patrick van der
  • Course: Coding the Matrix: Linear Algebra through Computer Science Applications - Philip Klein
  • Book: Linear algebra and its applications - Gilbert Strang


Python

  • Python tutorial: http://docs.python.org/tutorial/
  • Beginners: http://www.greenteapress.com/thinkpython/
  • Intermediate: http://www.diveintopython.net/
  • Google Python style guide , 2013
  • Video: Google Python Class Day 1 Part 2
  • Video: Google Python Class Day 1 Part 3
  • Video: Google Python Class Day 2 Part 2
  • Video: Google Python Class Day 2 Part 3
  • Video: Google Python Class Day 2 Part 4
  • http://code.google.com/edu/languages/google-python-class
  • Book: introduction_to_computation_and_programming_using_python - John Guttag
  • A Complete Tutorial to Learn Data Science with Python from Scratch - Analytics Vidhya

  • Python Numpy http://cs231n.github.io/python-numpy-tutorial/
  • pandas 0.18.1 documentation Cookbook
  • https://scipy.org/
  • http://scikit-learn.org
  • Video: SciKit - Pycon (3 hrs)
  • Scikit Learn Machine Learning Tutorial
  • Advanced SciKit Learn Tutorial
  • Deep Learning in Python - Analytics Vidhya

Frameworks & Libraries

  • Tensorflow: https://www.tensorflow.org/
  • Tensorflow for Deep Learning Research - http://web.stanford.edu/class/cs20si/
  • Theano: http://deeplearning.net/software/theano/tutorial/
  • Keras: https://keras.io/
  • Torch: http://torch.ch/
  • Caffe: http://caffe.berkeleyvision.org/
  • Lasange: https://lasagne.readthedocs.io/en/latest/


Papers

Advertising

  • Simple and scalable response prediction for display advertising Olivier Chapelle, Criteo et. al.
  • An Empirical Evaluation of Thompson Sampling, Olivier Chapelle, Yahoo et. al.
  • Improving Ad Relevance in Sponsored Search, Yahoo
  • Click Modeling in Search Advertising- Challenges & Solutions, Yahoo
  • Multi-armed Bandit, Cameron Davidson
  • Field-aware Factorization Machines for CTR Prediction
  • Ad Click Prediction- a View from the Trenches, Google
  • A Logistic Regression Approach to Ad Click Prediction
  • Online Advertising and Large Scale model fitting
  • Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks
  • Training Large-scale Ad Ranking Models in Spark
  • Delivering Guaranteed Display Ads under Reach and Frequency Requirements
  • Real-Time Bidding based Display Advertising: Mechanisms and Algorithms

Recommendation System

  • Data Mining Methods for Recommender Systems, Xavier Amatriain et. al.
  • Netflix The Recommender Problem Revisited
  • Deep Neural Networks for YouTube Recommendations, Google
  • Deep Learning for Recommender System, Telefonica Research
  • Repeat buyer prediction for eCommerce Training
  • Collaborative Deep Learning for Recommendation
  • Wide & Deep Learning for Recommender Systems, Google
  • A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems, Microsoft
  • A Survey and Critique of Deep Learning on Recommender Systems
  • Amazon Food Review Classification using Deep Learning and Recommender System
  • Applying Deep Learning to Collaborative Filtering, Hulu
  • Collaborative Deep Learning for Recommender Systems, Hong Kong
  • Collaborative Filtering and Deep Learning Based Recommendation System For Cold Start Items, UK
  • Comparative Deep Learning of Hybrid Representations for Image Recommendations
  • Deep content-based music recommendation
  • Factorization Meets the Neighborhood- a Multifaceted Collaborative Filtering Model
  • Improving Scalability of Personalized Recommendation Systems for Enterprise Knowledge Workers
  • Relational Stacked Denoising Autoencoder for Tag Recommendation
  • Restricted Boltzmann Machines for Collaborative Filtering
  • Session-based Recommendations with Recurrent Neural Networks
  • The application of Deep Learning in Collaborative Filtering
  • The WellDressed Recommendation Engine
  • Toward Fashion-Brand Recommendation Systems Using Deep-Learning: Preliminary Analysis
  • Enhanced Deep Convolutional Neural Network for Move Recommendation in Go
  • Explainable Restricted Boltzmann Machines for Collaborative Filtering
  • Recommender Systems for Large-scale E-Commerce

Others

  • Reliable Effective Terascale Linear Learning System
  • ImageNet Classification with Deep Convolutional Neural Networks
  • Visualizing and Understanding Deep Neural Networks
  • A Practical guide to training Restricted Boltzman Machine
  • Deep learning using genetic algorithms
  • Genetic Algorithms in Search, Optimization, and Machine Learning
  • A Scalable Tree Boosting System, Tianqi Chen (author)


Kaggle Winners' Interviews

  • How to Rank 10% in Your First Kaggle Competition
  • Beating Kaggle the easy way
  • How to win Machine Learning competitions
  • Want to Win Competitions Pay Attention to Your Ensembles
  • 4 Idiots' Approach for Click-through Rate Prediction
  • Secret Sauce Behind 9 Kaggle Winning Ideas
  • Winning the KDD Cup Orange Challenge with Ensemble Selection
  • Ensemble of Collaborative Filtering and Feature Engineered Models for Click Through Rate Prediction
  • 3-idots Approach for Display Advertising
  • A Beat the benchmark with Vowpal Wabbit - Display Advertising Challenge | Kaggle
  • Feature Engineering and Classifier Ensemble for KDD Cup 2010
  • BellKor solution to the NetFlix Grand Prize
  • The BigChaos Solution to the Netix Grand Prize
  • Netflix algorithm Prize Tribute Recommendation Algorithm in Python
  • Large-scale Parallel Collaborative Filtering for the Netflix Prize
  • Netflix Prize and SVD
  • Netflix Tech Blog_ Netflix Recommendations_ Beyond the 5 stars
  • What tools do Kaggle winners use


Copyright © 2015-2018 DeepThinking.AI