Images In Java

1. Images In Java Paul Cockshott ALMA TADEMA RIVALES INCONSCIENTES 1893

2. Introduction • The AWT and how to put a simple image onto the screen • Layout managers • The Jimage Class,

3. Summary of this section • At the end of this lecture you should have an idea of how to display a JPEG image on the screen, and how to load it into the Jimage class to carry out further image processing.

4. Agenda • AWT Images • Image Producers and Consumers • Jimage class • Pixel Representations • JPEG files

5. Overview • AWT abstract windows toolkit, supported by JavaSoft • Operating system independent layer for windowing in Java • Fiendishly obscure • Designed around requirements of images being streamed off the web

6. Connections • Simple image display program to show how to display a JPEG file • Pipeline model of image production • Jimages act as image consumers • Jimages allow arithmetic on image • Jimages provide output to AWT images and JPEG

7. How to display a picture 1 import java.awt.*; import java.awt.image.*; import java.util.*; class JPEGshow extends Frame { ... static public void main(String[] args) { if (args.length == 1) new JPEGshow(args[0]); else System.err.println("usage: java JPEGshow <image file>"); } } This is a standard Java Program class with a public static voidmain method

8. Constructor for JPEGshow See slide on these JPEGshow(String filename) { super("JPEG show Example"); add( new ImageCanvas(getToolkit().getImage(filename) ), BorderLayout.CENTER); setSize(700, 540); show(); } Read in a JPEG file Size of a frame

9. The toolkit • Each frame has associated with it a toolkit object the provides an interface to OS specific operations. • CreateImage • CreateMenu • CreateLabel • CreateMenuBar …. etc

10. Roll your own ImageCanvas class ImageCanvas extends Component { Image image; ImageCanvas(Image image) {this.image = image;} public void paint(Graphics g) { g.drawImage(image, 0, 0, this);} } Constructor just stores the image Paint is called whenever a component must be shown, the Graphics object does the actual drawing, it has to be passed in because it is what knows about physically drawing on the screen

11. Image Class • Pipeline flow model of image processing • Images are just tokens linking producers and consumers ImageConsumer ImageProducer Image

12. ImageProducer Methods • addConsumer(ImageConsumer ic) This method is used to register an ImageConsumer with the ImageProducer for access to the image data during a later reconstruction of the Image. • removeConsumer(ImageConsumer ic) This method removes the given ImageConsumer object from the list of consumers currently registered to receive image data. • startProduction(ImageConsumer ic) This method starts an immediate reconstruction of the image data

13. ImageConsumer methods • void setDimensions(int width, int height)  The dimensions of the source image are reported using the setDimensions method call. • Void setPixels(int x, int y, int w, int h, ColorModel model, byte[] pixels, int off, int scansize) The pixels of the image are delivered using one or more calls to the setPixels method.

14. Image Class continued ImageConsumer ImageProducer Image Image.getSource Images contain a pointer to their producer which holds the actual data for the image. This can be recovered using the getSource method. This allows a consumer to get at the pixel data of an image by adding itself to the producer and starting production

15. Summary • AWT is operating system independent • Streaming image model • Images as tokens • Producer - consumer pipeline • See chapters 6 of textbook

16. Buffered Image Class • Standard AWT images are just tokens for data streams. • A BufferedImage actually contains the data. BufferedImage Raster Colour model

17. JPEGCodec class • This class has factory methods to create JPEG encoders and decoders: • createJPEGDecoder(InputStream s) • createJPEGEncoder(OutputStream d)

18. Read a BufferedImage FileInputStream in = new FileInputStream(“myfile.jpg”); JPEGImageDecoder dec= JPEGCodec.createJPEGDecoder(in); BufferedImage im = decoder.decodeAsBufferedImage();

19. getRGB • You can access the pixels of a buffered image using int getRGB(int x, int y) The default colour representation is: green alpha red blue Bit 31 Bit 0

20. Writing pixels • This can be done with the setRGB method. • This takes x, and y co-ordinates and a pixel encoded as a 32 bit integer • im . setRGB(2, 5, 255); • Would set pixel 2,5 to 255 = bright blue.

21. Creating sub images • You can create a sub area within a buffered image using the • public BufferedImage getSubimage( • int x, • int y, • int w, • int h); Method of BufferedImage

22. Jimage implements ImageConsumer • Library of image processing classes developed in the department • Available for student practicals • Algebraic rather than stream oriented • Interfaces to MMX hardware under windows

23. Algebraic orientation By this we mean the it is structured around algebraic expressions whose values are images Thus if A and B are images and is some operator then A  B is also an image

24. Arithmetic I+J Universal plus(Universal) I-JUniversal minus(Universal) I×JUniversal times(Universal) I÷JUniversal divide(Universal) ½I½Universal abs() Filtering Jimageconvolve(double[] k)convolve with symmetrical separable kernel. public abstract Jimageconvolve(double[][] kernel)with non separable kernel Jimage operators

25. Scaling • JimagegetScaledInstance(int nwidth, int nheight)This scales with bicubic interpolation. • JimagegetScaledInstance(int nwidth, int nheight, int ndepth)This method allows the depth as well as area of an image to be altered if it is reduced the planes are aggregated if increased they are interpolated.

26. More operations Data access int rgbpixel(int x,int y) Converts the plane information into a pixel in the direct color model of java. public abstract int upixel(int x, int y, int plane) - returns unsigned integer pixel public abstract float fpixel(int x, int y, int plane) Returns the pixel in the range -1 to +1.

27. Data Access • public abstract void setPixel(int x, • int y, • int plane, • double pix) • Pixel information in range -1 to +1 • public void setSubImage(int x, • int y, • int z, • Jimage im) • Update an area of an image with another one. The other one must not run off the edge of the one being written to. The source of the copying is the 0th plane of the source jimage.

28. Jimage input output • public void putJPEGImage( • java.lang.String fileName, • int quality) • throws java.io.IOException • Outputs the image to a jpeg file • public boolean getImage(java.lang.String fileName) • Initialise the Jimage from the specified file. The file must be jpeg or gif.

29. Jimage to AWT Image conversion • public java.awt.Image getAWTImage() • public java.awt.image.ImageProducer getProducer()

30. Jimage implementations

31. An example program Create Jimage with byte pixels • class Jimageshow extends Frame { • Jimageshow(String filename) { • super("Jimage show Example"); • Jimage raw=new ByteImage(100,200,3); • if (raw.getImage(filename)){ • Jimage cooked = (Jimage)raw.times(0.3); • add(new ImageCanvas(cooked.getAWTImage()), BorderLayout.CENTER); • setSize(700, 540); • show(); • } • } Multiply by 0.3 Convert to AWT for display

32. Pixel Representations When dealing with displays it is conventional to assume that pixels are bytes holding numbers in the range 0 to 255. 0 Is assumed to be black 1 Is assumed to be white or maximum brightness of any given colour. For multicolour displays with 3 colour components, the convention is to have 3 fields of range 0..255 to hold the colour information.

33. Pixel Representations AWT For multicolour displays with 3 colour components, the convention is to have 3 fields of range 0..255 to hold the colour information. The AWT does this with the class Color. • public Color(int rgb) • Creates a color with the specified RGB value, where the red component is in bits 16-23 of the argument, the green component is in bits 8-15 of the argument, and the blue component is in bits 0-7. The value zero indicates no contribution from the primary color component. • A Jimage returns this format with int rgbpixel().

34. Pixel Representations: Bytes The byte data type in Java does not take on the values 0..255. Instead it takes on the values -128 to 127. There are no unsigned bytes in Java. This creates a problem for the representation of pixels in Jimages. The solution adopted is to adopt the following representation • -128 = black • 0 = mid grey • 127 = white

35. Pixel Representations: Floats If byte pixels are signed then so must other representations be. The solution adopted is to adopt the following representation for floats • -1 = black • 0 = mid grey • 1 = white

36. Conversionsbetween representations unsigned bytes shorts float min value 0 -128 -2048 -1 maxval 255 127 2047 1 medianval m 127.5 -0.5 -0.5 0 range r 255 255 4095 2 As shown in table a pixel prin representation r is converted to a pixel ps in representation s by the operation: ps = ms+(rs(pr-mr))¸rr

37. Signed Pixels : advantages Signed pixels seem at first to be counter-intuitive but they have numerous advantages. • A value of 0 or mid grey can be viewed as the ‘most likely’ value that a pixel takes on in the absence of other information. • If you do arithmetic on images, in particular subtract one image from another, then negative values of pixels naturally arise. • Signed pixels allow straightforward implementation of contrast adjustments. For instance multiplying an image by 0.5 halves the contrast in the image.

38. Signed Pixels : contrast adjustment Signed pixels allow straightforward implementation of contrast adjustments. For instance multiplying an image by 0.5 halves the contrast in the image. ´ 0.5 ´ 0.5 1 0.5 0.25 -0.25 -0.5 -1 Finalcontrast range Initial contrast range

39. Image Multiplication ´

40. Image Addition +

41. Image subtraction -

42. What Is Convolution • Convolution takes a kernel of coefficients and multiplies each pixel in a neighbourhood by the corresponding coefficient, and then sums the result • x y p[I+x, j+y]*k[x,y] • Will give the convolved pixel at position i, j

43. 1 D convolution • A 1 D convolution takes a one dimensional array as a kernel and applies it first in the X and then in Y dimension. • This can often be performed faster than a 2d convolution

44. Image Convolution: smoothing marble = double[] k= {0.1,0.1,0.2,0.2,0.2,0.1,0.1}; Note sum of coefficients =1 marble.convolve(k)=

45. Image Convolution: sharpening marble = double[] k= {-0.3,1.6,-0.3} Note sum of coefficients =1 number terms is odd marble.convolve(k)=

46. Convolution in Java2D • Java 2D provides a standard library for convolution of buffered images • This uses the class Kernel and ConvolveOp

47. Kernels in JAVA 2D float[] blur={ 0.0f, 0.1f, 0.0f, 0.1f, 0.6f, 0.1f, 0.0f, 0.1f, 0.1f}; Kernel k= new Kernel(3,3, blur); im = new ConvolveOp(K).filter(im,null); This will blur the image im by applying the 3 by 3 kernel blur to it.

48. Importance of speed • Image may contain a million pixels, • Arithmetic may be required on each one • Important to optimise operations or they are very time consuming • May need to use assembler kernels • May need to use special purpose instructions

49. Multimedia Extensions MMX • Intel and other CPU manufacturers have been adding to the instruction sets of their computers new extensions that handle multi-media data. • The aim is to allow operations to proceed on multiple pixels each clock cycle

50. MMX 2 • Standard Intel register set 8 floating point registers 8 General Registers fp0 fp1 fp2 fp3 fp4 fp5 fp6 fp7 eax ebx ecx edx esp ebp esi edi 32 bit 64 bit