1 / 35

Use 3D Convolutional Neural Network to Inspect Solder Ball Defects

Use 3D Convolutional Neural Network to Inspect Solder Ball Defects. Presenter: Bing- Jhang Lin 林炳彰 E-mail: lknight8631@gmail.com. Outline. Introduction Methods Experiments. Introduction. Head-in-Pillow (HIP) is a solder ball defect. HIP pattern.

rmoreland
Télécharger la présentation

Use 3D Convolutional Neural Network to Inspect Solder Ball Defects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Use 3D Convolutional Neural Network toInspect Solder Ball Defects Presenter: Bing-Jhang Lin 林炳彰 E-mail: lknight8631@gmail.com

  2. Outline • Introduction • Methods • Experiments

  3. Introduction • Head-in-Pillow (HIP) is a solder ball defect. HIP pattern https://alphaassembly.com/Utility/Blog/2012/October/Head-in-Pillow-An-Easily-Missed-and-Costly-Soldering-Defect

  4. Introduction • The HIP defect can be caused by the following reasons: • Surface of solder ball oxidation • Poor wetting of the solder • Distortion of the Printed Circuit Board (PCB) by the heat of the soldering process

  5. Introduction • The current diagnosis of the HIP defects usually uses 2D X-Ray inspection machine, but it is hard to find out the position of the defect. http://stevezeva.homestead.com/Head-in-pillow_X-Ray.JPG http://ih.constantcontact.com/fs163/1108958702593/img/143.jpg?a=1113547242057

  6. Introduction • Some reliable methods are destructive tests, so they are not recommended for use.

  7. Introduction • Although it is hard to find out the HIP location from the 2D X-Ray image, we can reconstruct the 3D PCB model by different angles of the 2D X-Ray images.

  8. Introduction • The 3D PCB model not only provides us with more detailed information but also represents the location of HIP defects more clearly. • In recent years, Deep Learning has been widely used in many computer vision tasks, especially the Convolutional Neural Network (CNN) has an outstanding performance in object recognition.

  9. Introduction • In this project, we propose a 3D CNN to inspect the HIP defects. • Our network combines the advantages of many state-of-art CNNs like:VGG, GoogleNet Inception, ResNet, and DenseNet. • We also design two types of the convolutional blocks and use the dense connectivity method to connect these blocks. VGG: Visual Geometry Group

  10. Introduction • Because the HIP problems do not happen often, so we use the data augmentation method to expand our 3D solder ball data. • In our experiments, our network can achieve excellent results on testing data; the size of our network is more refined; and the training and execution time is faster than other CNNs.

  11. Methods 3D Solder Balls Volumetric Data 3D Convolutional Neural Network

  12. 3D Solder Balls Volumetric Data • The 3D PCB volumetric data are reconstructed by different angles of 2D X-Rays images. • In order to obtain clearer 3D PCB volumetric data during the reconstruction, we control the camera to take a 2D X-Rays image every 2.81 degrees.

  13. 3D Solder Balls Volumetric Data

  14. 3D Solder Balls Volumetric Data • The 3D PCB volumetric data size is 1472 × 1176 × 46 pixels. • Since the volumetric pattern of solder ball is an elliptical sphere, the middle slice of the volumetric contains the biggest solder ball cross section.

  15. 3D Solder Balls Volumetric Data • The maximum bounding box size of the solder ball from the middle slice of the PCB volumetric data is 46 × 46 pixels, so the size of each 3D solder ball is 46 × 46 × 46 pixels.

  16. 3D Solder Balls Volumetric Data • We totally received 277 3D solder balls.

  17. 3D Solder Balls Volumetric Data • For an image object recognition task, the amount of data is not sufficient, so we also use the Data Augmentation method to obtain more data.

  18. 3D Solder Balls Volumetric Data • We rotate the 3D solder ball data six times to obtain more data.

  19. Methods • Middle slice pattern in different planes. z y x xz Plane yz Plane xy Plane Diagonal (Right Rear to Left Front) Diagonal (Left Rear to Right Front) 3D Solder Ball

  20. 3D Convolutional Neural Network Our network: Input Convolution 3×3×3 Stride: 2, ReLU: True Dense Connection Block Num of Connection: 2 Convolution 1×1×1 Stride: 2, ReLU: True Global Average Pooling (GAP) ×4 Softmax Dropout

  21. 3D Convolutional Neural Network • Our network contains four dense connection blocks. • The block can fully utilize the features of each layer and avoid some important features lost in deep layers. • Because the data may be in different sizes, we add the Global Average Pooling (GAP) layer before the last layer to let our network adapt to different sizes of input.

  22. 3D Convolutional Neural Network Dense Connection Blocks: Convolution Block (Inception) Concatenate ReLU Convolution Block (Inception) Concatenate ReLU Input

  23. 3D Convolutional Neural Network • We use the dense connectivity method to connect our convolutional blocks. • The output of previous layer will be connected with the current output of the convolutional block and do the ReLU activation function. • The features of the input will be fully used in the deep network and avoid some important information loss in the deep network. ReLU: Rectified Linear Unit

  24. 3D Convolutional Neural Network • Inception Block: Convolution 3×3×3 Stride: 1 Max Pooling 3×3×3 Stride: 1 Convolution 1×1×1 Stride: 1 Concatenate Input Tensor Average Pooling 3×3×3 Stride: 1 Convolution 1×1×1 Stride: 1

  25. 3D Convolutional Neural Network • Inception with Residual Block: Convolution 3×3×3 Stride: 1 Max Pooling 3×3×3 Stride: 1 Convolution 1×1×1 Stride: 1 Concatenate Input Tensor Avg Pooling 3×3×3 Stride: 1 Convolution 1×1×1 Stride: 1 Convolution 1×1×1 Stride: 1 ReLU Convolution 3×3×3 Stride: 1 ReLU Convolution 1×1×1 Stride: 1

  26. 3D Convolutional Neural Network • The convolutional blocks are composed of many different convolutional paths. • The different convolutional paths allow the following layer to learn various features, so the network can choose better features to adapt to the data. • The residual method is a good way to preventgradient vanishing problem, so we can more easily increase the depth of the network.

  27. Experiments Solder Ball Data Experiments

  28. Solder Ball Data • We currently have 277 balls of 3D solder, of which 38 are labeled data, and the rest are unlabeled data. • The volumetric size of each solder ball is 46 × 46 × 46 × 1 pixels. (The last channel represents the grayscale channel.) • Half of labeled data (19) are defective solder balls, and the other are normal solder balls.

  29. Solder Ball Data • We split the labeled data into two parts: training and testing. • The training part has 32 solder balls, and the testing part has 6 solder balls. • We rotate to expand the labeled data 6 times, so the training part has 192 (=32 × 6) solder balls, and the testing part has 36 (=6 × 6) solder balls.

  30. Solder Ball Data • Because the defective solder balls do not often appear on the PCB, so that the 239 unlabeled data are temporarily considered as normal solder balls in evaluating stage.

  31. Experiments • The information of each model.

  32. Experiments • The Result on testing data.

  33. Experiments • The Result on unlabeled data.

  34. Experiments • When the patterns of these solder balls are skewed near PCB border, we find that the models may misclassify the normal solder balls into defect. • The skew situation often happens on the PCB border, and the pattern of skew solder ball is blurry which may cause misclassification.

  35. Experiments

More Related