1. Eye-RIS

2. Vision System • sense – process - control • autonomous mode • Program stora

3. Outline • hardware • system overview • AnaFocus, SIS Q-Eye chip • DiCop • ALTERA NIOS II RISC uP, MLP • software • development environment

4. System overview • Early-processing • Extract information from the image flow • Output: reduced set of data • Pixel-wise simultaneous processing • Not a loop! • Post-processing • Inputs are abstract entities • Output complex decisions

5. System overview • Two main parts: • Smart Image Sensor (SIS) – Q-EYE chip • RISC processor – Altera Nios

6. Q-Eye chip • 176 x 144 (QCIF) resolution • Each cell: • Multi-mode optical sensors • Pixel memories • Linear and non-linear analogue processors • Binary processors • Interconnected with 8 neighbours

7. Q-Eye chip • „This means that each pixel can both sense the corresponding spatial sample of the image and process this data in close interaction and cooperation with other pixels.” • „Smart Image Sensor Q-Eye is also called Focal-Plane Processor (FPP) because it processes images at the same physical layer where they are sensed.”

8. Conventional Vision System Be aware of the data size of F!

9. Eye-RIS Vision System In this case: f << F

10. Architecture of the Q-Eye chip

11. The cell array • 176x144 interconnected sensing-processing cells. Each of these cells contains mixed-signal circuitry capable of: • Sensing images; including adaptive linear and high-dynamic range acquisition • Storing up to 7 grey scale images and up to 4 binary images • Performing grey-scale image additions, subtraction and multiplication by a constant • Performing spatio-temporal image filtering (low-pass, high pass, band pass and stop band) with programmable bandwidth. • Performing grey scale image convolutions • Performing grey scale to binary, and binary to grey scale conversions • Performing generic combinational and sequential binary operations • Performing Hit-and-Miss binary operations • Real-time loading and downloading of images • Ultra fast calculation and input/output of certain image features such as centroids, mean values, pixel coordinates…

12. Block Diagram of a Q-Eye cell

13. Altera Nios II • General-purpose RISC processor core. • Harvard architecture. • Full 32-bit instruction set, data path, and address space. • 32 general-purpose registers. • 32 external interrupt sources. • Single-instruction 32 x 32 multiply and divide producing a 32-bit result. • Single-instruction barrel shifter. • Access to a variety of on-chip peripherals, and interfaces to off-chip memories and peripherals through Avalon Bus. • Hardware-assisted JTAG debug module enabling processor start, stop, step and trace. • Instruction and Data Cache Memory. • JTAG Debug Module. • Exception and interrupt controller.

14. Summary of Q-Eye chip • Chip: Q-Eye • Technology: 0,18 um • Power Supply: Dual (1.8 V, 3.3 V) • # of Processor Elements array: 176×144 • Accuracy: ~8bit • Density of Processor Elements: 422 PE × mm2 • Die Size: 60 mm2 • Package size: 64×67×63 mm • Power Consumption of system: 700mW • Maximum frame rate: > 10 000 • I/O type: Binary or grayscale type images

15. Software • Eye-RIS ADK • Nios II: ANSI C/C++ • Q-EYE: CFPP

16. Programming

17. Programming Q-Eye chip • The CFPP Image Processing Library (IPL) • Spatio-temporal filters • Arithmetic operation between images • Thresholding • Morphological and logic binary operations • The Extended Image Processing Library (EIPL) • Blob management • Classifying functions • Linear and non-linear digital processing of grey-level images • Geometrical transformations

18. Programming Altera Nios II • C,C++, Assembly • The Eye-RIS Basic Library (EBL): • Control the execution of the CFPP code • I/O management • Sending and/or receiving images to/from the PC either to be displayed or saved to disk • Printing error or information messages to a console • Timer management

19. Programming Eye-RIS (sync)

20. Programming Eye-RIS (async)

21. CFPP global memories // Declaration extern fpp_int value; extern fpp_bool flag; extern fpp_time texp; // Access void foo() { value = 123; int foo = value; if(flag) { FPPTime_write(texp, 500); int readTExp = FPPTime_read(texp); } }

22. DEMO