270 likes | 283 Vues
Lecture 1 4 : Convolutional Neural Networks on Surfaces via Seamless Toric Covers. Jiacheng Cheng Feb, 21, 2018. 1. Convolutional neural networks on surfaces via seamless toric covers.
E N D
Lecture 14: Convolutional Neural Networks on Surfaces via Seamless Toric Covers Jiacheng Cheng Feb, 21, 2018
1 Convolutional neural networkson surfacesviaseamlesstoriccovers HaggaiMaron,MeiravGalun,NoamAigerman,MiriTrope,NadavDym,ErsinYumer, Vladimir G. Kim, YaronLipman
Problemstatement Easy Hard
DeepLearning Geometric Deep Learning
Translations • Two dimensional,commutative Isometries ofR^2 • Convolution • Linear • Translationinvariant • Pooling • Non-linear(max) • Sub-translation invariant
Translations onsurfaces? • Translationonsurface≝locallyEuclideantranslation • Flowalongnon-vanishingvectorfields
Flat torus! • Translations “modulo1” • Full translation invariance on the flattorus !
Only thetorus! Index of vectorfield Euler characteristic • Poincaré-Hopf:Foracompactorientablesurface • Index–ameasureofthecomplexitynearavanishingpoint • Non-vanishingvectorfieldimpliesgenus1-torus
15 CNNonflattorus Cyclic padding
16 Recap • CNNiswell-definedoverflat-torus • RoadblocksforCNNonsphere-typesurfaces • Topological:NolocallyEuclideantranslationsonspheres • Geometrical:Theflattorusisflatandoursurfaceisnot
17 Solution: Map the surface to a flattorus
19 MappingtheTorustotheflatTorus ! Aigerman and Lipman,2015
22 Pull-back Translations: pull-back Euclideantranslations Two dimensional,commutative Conformalmaps ! Pull-backconvolution Linear Theorem: Translationinvariance Pull-backpooling Non-linear(max) Sub-translationinvariant
24 Newlayers projection cyclicpadding
Datageneration Inputimage Labels
26 Testphase • Aggregation from differenttriplets • “Magnifyingglass” • Scale factor asweights • + + + =
27 Human bodysegmentation Train: 370models FAUST, MIT, SCAPE,ADOBE Test: 18models SHREC07
Easyfunctions Raw • Normals • Average geodesicdistance • Wave kernelsignature Complex
Human bodysegmentation Train: 370models FAUST, MIT, SCAPE,ADOBE Test: 18models SHREC07
Biological landmarksdetection • Train: 73teeth from BOYER • Onlycurvatureandscalefactor Test: 8 teeth fromBOYER
32 Biologicallandmarks
35 Conclusion • CNN of sphere-typesurfaces • Wedefinedameaningfulconvolutiononsurfaces • Learns from rawfeatures • ReusingCNNsoftwareforimages • Limitationsandfuturework • Scope:Onlyspheretypesurfaces • Nocanonicalchoicefortriplets(andconvolutions) • Learn aggregationoperator