PyTorch Python Tutorial | Deep Learning Using PyTorch | Image Classifier Using PyTorch | Edureka

2. Agenda What is Deep Learning? What is PyTorch? Creating a Neural Network PyTorch v/s TensorFlow Use-Case of PyTorch Summary

3. What is Deep Learning? We will begin by looking at what led to formation of Deep Learning. Copyright © 2017, edureka and/or its affiliates. All rights reserved.

4. AI, Machine Learning and Deep Learning

5. What is Deep Learning? A collection of statistical machine learning techniques used to learn feature hierarchies often based on artificial neural networks Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer

6. Programming languages for Neural Networks Multiple languages allow support for working with Neural Networks! Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

7. Maximised support! LisP AI & Deep Learning Training www.edureka.co

8. Python for Deep Learning Let’s focus on the libraries Python has to offer! Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

9. Python Libraries 4 Python Deep Learning libraries we’ve found to be the most useful and popular! AI & Deep Learning Training www.edureka.co

10. Python Libraries 4 Python Deep Learning libraries we’ve found to be the most useful and popular! AI & Deep Learning Training www.edureka.co

11. What is PyTorch? The new kid in the block! Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

12. PyTorch – An Introduction PyTorch is a Python based scientific computing package Actively used Python Support Dynamic Computation Graphs Easy to Use API Fast and Feels Native Support for CUDA AI & Deep Learning Training www.edureka.co

13. PyTorch - Origin PyTorch is a cousin of lua-based Torch framework. • PyTorch is not a simple set of wrappers to support popular language. • AI & Deep Learning Training www.edureka.co

14. PyTorch Installation Let’s up open up PyCharm and see how to install PyTorch! Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

15. Major PyTorch Concepts An insight into what PyTorch has to offer to the developers! Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

16. PyTorch - Concepts Imperative N-Dimensional Array Running on GPU Tensor Terminologies Node in Computational Graph - To store Data and Gradient Variable Module Neural Network Layer - Stores State or Learnable Weights AI & Deep Learning Training www.edureka.co

17. The NumPy Bridge! Breakthrough performance! AI & Deep Learning Training www.edureka.co

18. The NumPy Bridge! Converting Torch Tensor to NumPy array and vice versa. • Torch Tensor and NumPy array will share underlying memory locations. • NumPy PyTorch Torch Tensor NumPy array AI & Deep Learning Training www.edureka.co

19. Autograd Module Records Operations Training Dataset Replay Backwards Compute Gradients Automatic Differentiation Negative of the slope Calculate the minima of the trace function Saves time by calculating differentiation of parameters at forward pass AI & Deep Learning Training www.edureka.co

20. Autograd Module Central Class of Package torch.Tensor Attribute Track all Operations .requires_grad as True After computationCall .backward() Stored in .grad Attribute Gradients Computed Stop Tracking History Call .detach() torch.no_grad() AI & Deep Learning Training www.edureka.co

21. Creating a Neural Network An insight into how a Neural Network can be created and used Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

22. Creating a Neural Network Autograd Start torch.nn package Neural Networks Construct using Define models and differentiate them Methods Forward(input) Returns output nn.Module Layers Simple Feed-Forward Network AI & Deep Learning Training www.edureka.co

23. Training Algorithm Back Propagation Update Parameters Inputs Calculate Loss PyTorch Variables train_batch Derivatives calculated automatically labels_batch output_batch AI & Deep Learning Training www.edureka.co

24. Creating a Neural Network Training procedure for a Neural Network is as follows: Define the Neural Network Iterate over Dataset Process the Input Compute the Loss Propagate Gradients Back Update the weights weight = weight - learning_rate * gradient AI & Deep Learning Training www.edureka.co

25. Creating a Neural Network Torch.Tensor Multi-dimensional array with support for autograd operations like backward() Neural Network module for encapsulating parameters to move them to GPU nn.Module nn.Parameter Tensor which is registered as parameter when assigned as attribute to Module Autograd.Function Implements forward and backward definitions of an autograd operation AI & Deep Learning Training www.edureka.co

26. Creating a Neural Network At this point, we covered: Defining a Neural Network • Processing Inputs and Calling Backwards • Computing the Loss • Updating the Weights of the Network • AI & Deep Learning Training www.edureka.co

27. PyTorch v/s TensorFlow An insight comparing both of the frameworks Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

28. Comparison – An overview Static Computational Graphs Dynamic Computational Graphs Cannot make use of Standard Python Flow Control Can make use of Standard Python Flow Control Support for Python Debuggers Cannot use native Python Debuggers Cannot inspect Variables and Gradients Dynamically Dynamic Inspection of Variables and Gradients Production Research AI & Deep Learning Training www.edureka.co

29. PyTorch – Use Case Let’s look at the working and generation of an image classifier. Copyright © 2018, edureka and/or its affiliates. All rights reserved. AI & Deep Learning Training www.edureka.co

30. Problem Statement Generating an Image Classifier which predicts data based on an Image-Set by constructing a Neural Network which is used by companies like Google to create Image-to-Text Applications such as Translation etc. Using this we optimise accuracy of data obtained! AI & Deep Learning Training www.edureka.co

31. Dataset What about data? Standard Python Packages can be used to load data into numpy array. Then can be converted into a torch.*Tensor. Image Audio Text • Scipy • SpaCy • Pillow • Librosa • Cython • OpenCV torchvision package helps to avoid writing boilerplate code AI & Deep Learning Training www.edureka.co

32. CIFAR10 Let’s use the CIFAR10 dataset. • The images in CIFAR-10 are of size 3x32x32, i.e. 3-channel colour images of 32x32 pixels in size. • AI & Deep Learning Training www.edureka.co

33. Flow Diagram Start Load the Dataset Read the Dataset Normalize test Dataset using torchvision Pre-processing of dataset Train the Network Define Convolution Neural Network (CNN) Define Loss Function Repeat the process to Decrease the Loss Test the Network Based on Trained Data Make Prediction on the Test Data End AI & Deep Learning Training www.edureka.co

34. Training an Image Classifier We will do the following steps in order: Load and Normalize CIFAR10 using torchvision Define Convolution Neural Network Define Loss Function Train the Network on Training Data Test the Network on Test Data Update the weights weight = weight - learning_rate * gradient AI & Deep Learning Training www.edureka.co

35. Step 1: Loading and Normalizing CIFAR10 Using torchvision, it’s extremely easy to load CIFAR10! Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Load Data Update the weights Normalize Data Convert into Tables Inputs AI & Deep Learning Training www.edureka.co

36. Step 2: Define a Convolution Neural Network 3-Channel Images Load and Normalize CIFAR10 Define CNN Red Green Blue Define Loss Function Train the Network Test the Network Update the weights AI & Deep Learning Training www.edureka.co

37. Step 3: Define a Loss Function and Optimizer Load and Normalize CIFAR10 Define CNN Define Loss Function Train the Network Test the Network Update the weights Classification Cross-Entropy Loss AI & Deep Learning Training www.edureka.co

38. Step 4: Train the Network Load and Normalize CIFAR10 Training Define CNN Loop over Data Iterator Define Loss Function Feed the Inputs Train the Network Optimize Test the Network Update the weights AI & Deep Learning Training www.edureka.co

39. Step 5: Test the Network on Test Data Check for Changes in Model Load and Normalize CIFAR10 Predict Output Class Label Define CNN Define Loss Function Incorrect Train the Network Check against ground-truth Test the Network Update the weights Correct Add sample to Correct Predictions AI & Deep Learning Training www.edureka.co

40. Results The results seem pretty good. Network did learn something! • Let us look at how the network performs on the whole dataset. • AI & Deep Learning Training www.edureka.co

41. Session In A Minute What is Deep Learning? Programming Languages Deep Learning Libraries PyTorch PyTorch v/s TensorFlow Use-Case Implementation AI & Deep Learning Training www.edureka.co

42. Copyright © 2018, edureka and/or its affiliates. All rights reserved.