1 / 14

Project Implementation for ITCS4122

Project Implementation for ITCS4122. Jianping Fan Department of Computer Science University of North Carolina at Charlotte http://www.cs.uncc.edu/~jfan. You can select one of them. Semantic image classification & Visual Analytics Image Clustering & Visual Analytics

eericson
Télécharger la présentation

Project Implementation for ITCS4122

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project Implementation for ITCS4122 Jianping Fan Department of Computer Science University of North Carolina at Charlotte http://www.cs.uncc.edu/~jfan

  2. You can select one of them • Semantic image classification & Visual Analytics • Image Clustering & Visual Analytics • Deep Image Segmentation & Visual Analytics • Interactive Data Analytics

  3. 1. Build a system for supporting semantic image classification & visual analytics ---you are encouraged to use deep learning for feature extraction System Components: a. Image analysis and feature extraction (1) Image segmentation using JSEG You can download jseg image segmentation source code from: http://vision.ece.ucsb.edu/segmentation/jseg/ (2) Feature extraction: extract color histogram, color moments, Tamular textures (moments et al.) from each homogeneous regions (3) Region classifier training You can download software from: http://www.cs.cornell.edu/people/tj/svm_light/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/

  4. (4) Region merging After the labels for the homogeneous regions are obtained, the neighboring regions with the same label will be merged as one big region for the relevant salient object (5) Interesting point detection You can download software from: http://vision.ucla.edu/~vedaldi/code/sift/sift.html Detecting the object via SIFT features (6) Object-Based feature extraction Extract color histogram, color moments, textures, and SIFT from salient objects

  5. (7) Concept classifier training positive samples negative samples positive support vectors margin negative support vectors You can download software from: http://www.cs.cornell.edu/people/tj/svm_light/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/

  6. (8) Visualization for features and classification results (9) User-system interaction for result evaluation and integrating human advises for further data analytics. Deep Learning approach for this project Outputs: Features & List of Object Classes & Concepts Visualization of Category-driven Data Distributions Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … Input Images Parameter Fine-Tune

  7. You need to build classifiers for: • Salient Objects • Tree, grass, red flower, yellow flower, sky water, sand, road, car, building • Image Concepts • Garden, beach

  8. ---you are encouraged to use deep learning for feature extraction 2. Automatic Image Clustering & Visual Analytics • You can download 1,000,000 images from • You can extract one of the following visual features: color histogram or SIFT features, you can also try CNN features, you can get source code from • http://clickdamage.com/sourcecode/index.php • You can apply AP clustering technique to get an optimal partition of these 1,000,000 images according to their visual similarity, you can get AP clustering source code from http://www.image-net.org/ http://bonsai.hgc.jp/~mdehoon/software/cluster/software.htm

  9. Image Query Result Clustering • You can go to Google Images and type in keywords such as ``apple”, ``bank”, ``cloud” et al • Download the returned images (more than 1000 images), total images will be 100,000. • Extract color histogram or SIFT by using • http://clickdamage.com/sourcecode/index.php • Apply AP clustering to get an optimal partition of the returned images http://bonsai.hgc.jp/~mdehoon/software/cluster/software.htm

  10. Visualization for features and classification results User-system interaction for result evaluation and integrating human advises for further data analytics. ---you are encouraged to use deep learning for feature extraction Outputs: Features & Image Clusters Visualization of Data Distributions Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … Input Images Parameter Fine-Tune

  11. You can use DeepLab v3, U-nets, Mask RCNN,… to do semantic image segmentation • Integrating visualization to evaluate the segmentation results • Leveraging human advises to improve deep image segmentation 3. Deep Image Segmentation & Analytics

  12. Project Requirements • Project implementation: you need to spend time to understand the project and the relevant source code and compile these source codes and make them work in your computer and try to make some revisions. • Penalties: TA will test your system by using his images, so it is easy for us to check whether you really do the project or not. If we find that you did not spend enough time to work on your project, you will receive big penalties on your project grade.

  13. 4. Traditional Tools for Interactive Data Visualization Input Data Feature Extraction & Data Clustering Data & Knowledge Visualization User Assessment & Feedbacks Constraint-Driven Data Clustering

  14. Group or not • Each group may have no more than 5 students • We encourage individual project & bonus is expected for individual project • Bonus will be given to good implementation such as improving on algorithms or including nice interface & visualization

More Related