Ioan Buciu

1. Imaging and computer science applications to neurosciences Ioan Buciu Department of Electronics and Telecommunications Faculty of Electrical Engineering and Information Technology, University of Oradea, 410087, Romania e-mail: ibuciu@uoradea.ro

2. Contents • Principles of nuclear magnetic resonance (NMR). Relaxation processes. Spin echo • Nuclear magnetism. Magnetization • Nuclear magnetic resonance. "FID" signal • Relaxation processes and constants • Spin echo • Principles of MR imaging; magnetic field gradients • Gradients of magnetic field • Selective excitation of a section • Reading gradient • Phase coding gradient • Contrast in MRI images

3. Contents • The architecture of an MRI system • The magnet • Emission circuit • Receiver circuit • Reconstruction circuit • Central computer

4. Contents • Basics of image processing and analysis applied in medical imaging • Image enhancement and filtering • Image segmentation • Mathematical morphology applied to medical imaging • Biologically plausible neural models for image representation in the Human Visual System • Redundancy, coding and compression principles in the neural system • Dense, sparse and local data representation • Simple and complex neural cells model • Gabor approach

5. Contents • Biologically plausible neural models for image representation in the Human Visual System • PCA approach • ICA approach • NMF and variants • Compressed sensing • Automatic medical classification and recognition techniques • Introduction for supervised and unsupervised classification approaches • Artificial Neural Networks (ANNs) • Support Vector Machines (SVMs)

6. Human Visual System and the brain • The Brain: • Born with 1000 trillions of neural cells (neurons) ! • BUT we left with ~ 14 trillions neurons … • Neural cells are not isolated – connections are formed linking cells to perform neural processes (learning processes – how to move, memorizing objects, recognizing objects, sounds, familiar faces, activities, etc., cognitive processes). • Some neurons are responsible for visual processes – image compression, image encoding, image representation, image understanding.

7. HVS • Human Visual System – information flow

8. HVS • LGN –Lateral Geniculate Nucleus – 6 levels - luminance information perception (specialized ganglion cells). • Visual cortex – V1 … AIT. • V1 (primary visual cortex) – 6 levels  conducts pulses toward higher neural layers when stimulated by: • Oriented edges; • Various spatial frequencies; • Various temporal information; • Some particular spatial locations;

9. HVS – Visual pathway • Retina, LGN and V1 (primary visual cortex)

10. HVS – Visual pathway • Monocular visual field: 160 degree wide. • Binocular visual field: 200 degree wide. • Processing is not uniform ! 25 % of cortex is devoted to the central 5 degrees of the field of view. • Processes in the retina can be modeled via difference of Gaussians. • A Gaussian: • Difference of Gaussians (ON center OFF surround):

11. HVS – Visual pathway • The eye does not keep the absolute luminance value. • It only keeps relative light values. • Compression occurs in the periphery not in the fovea. • Receptive field: the region in space (visual scene) in which the presence of a stimulus alters (triggers) the neuron to respond (fire). • The receptive field is defined by type, size, and shape. • Receptive field is modeled via convolution kernel R: • where - original image, - neuron response.

12. HVS – The Eye • Cons and rods – convert light into signals further transmitted to the brain (visual cortex). • 2 eyes  2 retinas • no. of cones in each retina: 6 millions • no. of rods in each retina: 125 millions • Optic nerves: 1.5 millions (fibers for each eye). • Rods and cones must be interconnected to nerve fibers on a many-to-one basis. (Consequences: the images - visual scenes - are not represented internally as we see them !!! – image representation + image understanding following image encoding + information compression). Compression rate: 100:1

13. HVS – The Eye • Cons - generate achromatic (graylevel) responses (luminance decomposition). • Cons – sensitive only in low light levels (good for night vision). • Rods – photoreceptors responsible for color vision. • Color - perception to different wavelengths of visible light

14. HVS – The Eye • Different ranges gives rise to different color response. • 3 types – S, M, L – • sensitive only in • high light levels ! (can we see colors in low illuminated scenes ?) • Dogs have only 2 types, • bees have 4, and • mantis shrimps have 10 !

15. HVS – The Eye • Cones – S – blue response (445 nm – peak) • Cones – M – green or yellow response (535 nm – peak) • Cones – L – red response (575 nm – peak)

16. HVS – The Eye and Beyond • Opponent Processing: - Retina performs “matrix operations” to represent color (decomposition) in the opponent color system (Y, Y – B, R – G).

17. HVS – The Eye and Beyond • The output of the three cone color is transformed into an achromatic channel (such as luminance - Y) and two chromatic channels (opponent channels): – U, V – chroma components • High-level HVS (Human Vision System) is much more sensitive to the variations in the achromatic channel than in the chromatic channels. • Same principles are exploited in standard JPEG compression.

18. HVS – Eye and Color R G B

19. HVS – Eye and Color • RGB  YUV • Y = 0,3 R + 0,6 G +0,1 B; • U = B – Y; • V = R – Y Y U (Cb) V (Cr)

20. Human Perception - Limitations • Human perception – the organization, identification and interpretation of sensory information necessary to represent (internally) and understand the environment. • Human perception – thorough the 5 traditional (or more ?) human senses: • 1. Sight (human vision) – Human Visual System - ability of the eyes to detect images of visible light. • 2. Hearing (human audition) – Auditory System – the sense of sound perception - vibration. • 3. Taste. • 4. Smell. • 5. Touch.

21. Human Perception - Limitations • Can we sense everything, anything ? NO ! • OUR SENSES ARE LIMITED. • Human Auditory System – sounds (acoustic stimuli)  frequency – the number of vibrations that are produced per second – Hertz (Hz) – in nature mixture of sounds with diff freq components. • 1 Hz – 1 vibration. • Frequency range of human hearings: 20 Hz – 20.000 Hz. • Bounds degrade with age ! (middle-aged adult: up to 14 KHz) • Low frequency : human’s heartbeat. • High frequency: a scream.

22. Human Auditory System • Sound out of range  infrasound and ultrasound. • Bats: 100.000 Hz. • 1 Hz / second = 1 vibration / second.

23. Human Auditory System and its limitations • fs – sampling frequency – is its value important ? Definitely YES ! • f = 7 Hz; • red - fs = 10 Hz; • blue - fs = 8000 Hz; • This is temporal • representation

24. Human Auditory System and its limitations • Frequency representation: Analyze mixture of sounds – temporal analysis vs. frequency analysis (1D - Fourier Transform) Temporal representation Frequency representation

25. Human Auditory System and its limitations • Fourier Transform Temporal representation Frequency representation

26. Human Auditory System and its limitations • What’s human auditory system (human sound perception) got to do with it (frequency decomposition) ? • Answer: basic compression principle ! – Psychoacoustic Models – Perceptual Codecs - Spectral Masking – mp3 !

27. Human Visual System and its limitations • Human Visual System (HVS) – images – formed as visual stimuli • Like sounds, images can be also decomposed into frequency components. • Image decomposition in frequency  same as for sounds: 2D - Fourier Transform. • Like sounds, visual spectrum (frequency components) is bounded. • HVS is less sensitive to low and high frequencies – Consequence: We can remove high frequencies from an image without degrading (perceptually speaking) the image quality (degradation is not visually noticeable).

28. Human Visual System and its limitations • Image decomposition in frequency  same as for sounds: 2D - Fourier Transform. • Low frequency components correspond to smooth regions in the image, while high frequency components are associated to details in the image (edges, corners). Low frequency image High frequency image Low - High frequency image

29. Human Visual System and its limitations 2D – Fourier Transform First column: low and high frequency image, respectively Second column: corresponding 2D – FT (ONLY magnitude)

30. Human Visual System and its limitations

31. Human Visual System and its limitations a) b) c) DC component AC components high frequency AC components low frequency a) Image b) FT – Magnitude c) FT – Shifted magnitude d) FT - phase d)

32. Human Visual System and its limitations

33. Human Visual System and its limitations • Image reconstruction from magnitude and phase of FT: a) b) c) a) Original Image b) Reconstruction of image with magnitude only c) Reconstruction of image with phase only

34. Human Visual System and its limitations • Full image reconstruction magnitude + phase of FT: a) b)

35. Human Visual System and its limitations Original image 50 components 200 components 1000 components

36. Human Visual System and its limitations Original BMP – 813 KB JPEG – 100 % – 284 KB JPEG – 75 % – 72 KB JPEG – 10 – 15 KB

37. Human Visual System and its limitations Crop of original BMP Crop of JPG – 100 % Crop of JPG – 75 % Crop of JPG – 10 %

38. Human Visual System and its limitations Original BMP FT spectrum JPG – 100 % FT spectrum JPG – 10 % FT spectrum JPG – 75 % FT spectrum

39. Human Visual System and its limitations • Limitations of Visual Perception • 1. Space Perception – 3D (length, width or depth and height) + 4th ? which is time perception  4D (3 spatial dimensions + 1 temporal dimension – direction) 3D space 4D – space !

40. Human Visual System and its limitations • Limitations of Visual Perception • Flatland – 2D imaginary world – 1884 Edwin Abbott: A flatworld concept

41. Human Visual System and its limitations • Limitations of Visual Perception • Tessaract - tesseract, also called an 8-cell or regular octachoron or cubic prism, is the four-dimensional analog of the cube; the tesseract is to the cube as the cube is to the square (Wikipedia). A cube A 3D projection of an 8-cell performing a simple rotation about a plane which bisects the figure from front-left to back-right and top to bottom. 3D projection of an 8-cell performing a double rotation about two orthogonal planes.

42. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • A) Unreal (impossible) objects - Oscar Reutersward’s Triangle: • The brain perceives the object locally (as local components) followed by combining them as a whole: 3D mental reconstruction A paradox !!!!

43. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • B) Ambiguity (Perception ambiguity)

44. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • C) Distortion Illusions - neural cells are directional sensitive !

45. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • D) Camouflage – Stimuli have similar properties.

46. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • E) Simultaneous contrast.

47. Human Visual System and its limitations • Limitations of Visual Perception • 2. Human Visual Perception can be foolish: • F) Motion Illusion.

48. Human Visual System and its limitations • Limitations of Visual Perception • 3. HVS is only sensitive to visible light: • The wavelengths of electromagnetic radiation between roughly 370 nm and 730 nm account for light visible to the HVS.

49. Human Visual System and its limitations • Limitations of Visual Perception • 4. Bandwidth of vision defined by the – Contrast Sensitivity Function - spatial frequency components are visible up to 60 cpd.

50. Human Visual System and its limitations • Limitations of Visual Perception • 4. Perceptual masking – contrast masking: