Image Processing Techniques using the Java TM Advanced Imaging TM (JAI) API Framework

1. Image Processing Techniques using the JavaTM Advanced ImagingTM (JAI) API Framework Bill Champlin UCCS / CS525 Spring ‘08

2. Agenda • Project Description • Project Purpose • Overview of JAI API Capabilities • Installing JAI • Programming Techniques • Programming Model and Application Development Steps • Example Image Processing Code Fragments • Conclusions / Recommendations • References

3. Project Description • The purpose of this project is to: • Discover the image processing (IP) capabilities of Sun Microsystems JAI API • Learn the programming techniques necessary to write JAI based applications • Report on how some other projects have leveraged JAI (see project paper) • Astronomy • Medical Imaging • Database • Transportation • Planetary Exploration (viewing surface images of Mars, Venus, moons of Jupiter, etc)

4. JAI API Capabilities • JAI is an API package that greatly simplifies developing applications requiring the use of IP algorithms • Provides over 80 pre-canned algorithms to save developing from scratch • Provides a high level API for using these algorithms • Extends existing Java 2D imaging capabilities • Consequently integrates well with Java 2D (AWT) and Swing APIs • Using organizations can integrate custom algorithms • Highly portable – runs on any hardware and O/S Java does • Algorithms supported include: • Vector math routines applied to the same pixels from one or more images i.e. add, subtract, or, xor, log, extrema (min/max) and creating a histogram • Image transformation routines i.e. convolving, edge detection, and filtering (low / high pass filters) • Geometric transforms including rotation, scaling, translation and warping • Frequency operator encoding / decoding (codecs) i.e. DCT/IDCT and DFT/IDFT

5. JAI API Capabilities – con’t • Additional Supported Capabilities • Overlaying one image on top of another • Creating the composite of two images using Transparency • Tiling of images, so only those visible portions need to be processed • Creating client/server imaging applications • Easy creation of custom color lookup tables (CLUTs) and running an image through a CLUT • Ability to apply different color systems to an image and convert between them i.e. CIE/CMYK/RGB

6. Installation • Download and install the Java Development Toolkit (JDK) or Java Run-Time Environment (JRE) if only running applications • Download and install JAI API from Sun’s Java website (see references) • Setup run path to find “java” and classpath to find 3 JAI jar files, plus maximum heap memory size option (-Xmx), but only if needed • Optionally download JAI demonstration tutor program (see references) • Example setting paths and running: Prompt>path=.\jre1.6.0_03\bin Prompt>java -Xmx32m –classpath .;./classes;jai_core.jar;jai_codec.jar;mlibwrapper_jai.jar; Tutor • Run from IDE or put “java” command as above into .bat file or run directly from command line

7. Programming Techniques • Programming model: • Creates a “pipeline” of IP objects • “Deferred execution” causes pipeline to execute only when final result is requested Programming Steps: Fetch images For each Algo. to execute: -Build parameter list -Invoke Static JAI “create” Operation -Do something with results i.e. pipe to next operation, display images, or save them

8. Programming Techniques – cont’dExample IP Code Fragments • Loading an image file from disk (correct codec automatically used) PlanarImage myImage = JAI.create(“fileload”, “C:/images/galaxyPicture.jpeg”); • Or alternatively, to load an image from the internet: PlanarImage myImage = JAI.create("url", new URL("http://viva.uccs.edu/~wchampli/cs525/images/wchampli.png")); • Rotating an image (where angle is a float initially set to 0 degrees): public void actionPerformed(ActionEvent event) { // method called if button pressed angle += 45; // increase rotation angle by 45 degrees ParameterBlock pb = new ParameterBlock(); // create parm list pb.addSource(myImage); // add image to parameter list pb.add(width / 2.0F); // x origin parameter is one-half image width pb.add(height / 2.0F); // y origin parameter is one-half image height pb.add((float) Math.toRadians((double) angle)); // angle parameter pb.add(new InterpolationNearest());//how to fill in angles parameters image = JAI.create("Rotate", pb, null); // execute the operation and get an image canvas.set(image);}//display result image (note:could pipe into another operation)

9. Conclusions/Recommendations • Conclusions: • JAI makes it easy to develop IP applications and it saves substantial programming effort over re-coding algorithms • JAI is more portable typical C libraries targeted for specific platforms • To date, JAI has been a well kept secret as information is scattered amongst various sources on the internet and it is not yet in widespread use • Wikipedia entry just appeared in mid April • JAI Tutorial not previously build-able due to missing files (until the last few weeks) • Sun’s JAI project team now appears to be actively working demos, fixing bugs, etc. so JAI use should now become more widespread • Recommendations: • Performance Improvement Suggestions: • Change underlying data storage to use Java I/O buffer classes to save copying between buffer storage types • Modify IP algorithm code to create multiple threads for processing large images to leverage multi-cores • Create an IP algo chaining editor tool to build algo pipelines and execute them (considering as a possible master’s project)

10. References Project Website: • Project Wiki: http://cs525javaimaging.pbwiki.com/ Resources: • Sun Microsystems. Programming in Java Advanced Imaging, Release 1.0.1, Palo Alto. CA, November 1999. http://java.sun.com/products/java-media/jai/docs/ • R. Santos. Java Advanced Imaging API: A Tutorial. RITA Vol. XI Number 1, 2004, (http://www.inf.ufrgs.br/~revista/docs/rita11/rita_v11_n1_p93a124.pdf) • JAI API doc page: http://java.sun.com/products/java-media/jai/forDevelopers/jai-apidocs/ • JAI API download page: https://jai.dev.java.net/binary-builds.html#Stable_builds_1.1.4 • JAI Tutorial: http://java.sun.com/developer/releases/jai/#jaidemo • Developer Forum: http://forum.java.sun.com/forum.jspa?forumID=540&start=0 • Sun’s Java JAI project: https://jai.dev.java.net/ Sample Using Projects: • JAI Success Stories: http://java.sun.com/products/java-media/jai/success/ • Univ. Washington Intelligent Transportation System: http://java.sun.com/products/java-media/jai/success/uw-its.html • LMCO Web based Electronic Light Table (WebELT): http://java.sun.com/products/java-media/jai/success/lmco.html • Note: JAI Logo Icon taken from 6 above - JAI Tutorial “images” directory; warping code on slide 14 taken from 6 above - JAI Tutorial JAIWarpDemo.java source code

11. Backup Slides

12. Rotation Example See code on slide 8

13. Gamma Scaling Example Gamma Scaling ramps up dimmer pixel values: int value = slider.getValue(); double gamma = value / 100.0; double x; // gamma is 0.0-1.0 for (int i = 0; i < 256; i++){ x = i / 256.0; // values are 0.0-1.0 x = 255.0 * Math.pow(x, gamma); tableData[0][i] = (byte)x; tableData[1][i] = (byte)x; tableData[2][i] = (byte)x; } ParameterBlock pb = new ParameterBlock(); pb.addSource(image); pb.add(new LookupTableJAI(tableData)); target = JAI.create("lookup", pb, null); display.set(target);

14. Warping Example Warping translates image pts thru a polynomial With coefficients set by control pt locations: float[] srcCoords = new float[200]; float[] dstCoords = new float[200]; warp = WarpPolynomial.createWarp(srcCoords, 0, dstCoords, 0,2*numPoints,1.0F/width, 1.0F/height, (float)width, (float)height,degree); float[][] tcoeffs = warp.getCoeffs(); int length = tcoeffs[0].length; coeffs = new float[2 * length]; for (int i = 0; i < length; i++) { coeffs[i] = tcoeffs[0][i]; coeffs[i+length] = tcoeffs[1][i];} ParameterBlock pb = new ParameterBlock(); pb.addSource(srcImage); pb.add(warp); pb.add(new InterpolationNearest()); dstImage = JAI.create("warp", pb);