1. Ronan Collobert Jason Weston Leon Bottou Michael Karlen KorayKavukcouglu PavelKuksa NATURAL LANGUAGE PROCESSING (ALMOST) FROM SCRATCH

2. INTRODUCTION • Common Approaches in NLP • Using Task-specific features • Knowledge injection about structure of data • Expertise from Linguists • Approach used in the paper • No task-specific feature engineering • Minimal prior Knowledge

3. Common Benchmark Datasets • Part-Of-Speech (POS) Tagging • Syntactic Parsing • Chunking • Shallow Parsing • Named Entity Recognition • Person, Location etc. • Semantic Role Labelling

4. The Network • Words to Feature Vectors • Look up Table • Random initialization vs Unsupervised Pre-training • Extending to any Discrete Features • Extracting Higher level Features from Word Feature Vectors • Window Approach • Sentence Approach

5. Window Approach Network

6. Sentence Approach Network

7. Training Schemes • Word-Level Log Likelihood • Only words are taken independently for optimizing the weights • Sentence-Level Log Likelihood • Optimization function takes into account all the tags as well as transitions between tags • Stochastic Gradient • Standard Optimization Algorithm

8. Results