HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML
 

As images come in different sizes and formats, image loading is often a complex and time consuming process. Image loading usually consists of fetching the image file, reading the header, decoding the compressed pixels' values, and then delivering the pixels. Java currently supports two types of images, JPEG and GIF. The loading mechanisms for these formats are already built-in. If other formats are involved, users will have to write their own loading programs.

Image loading may mean different things to different applications. Some applications may display images directly without explicitly using image properties or pixel values. In this case, image loading involves fetching the image as a single object. Image animation is an example of such an application. In some applications, pixels may have to be manipulated. Image loading for such applications will include fetching the image properties and pixel values explicitly, involving several stages. Image Analysis belongs to this category of applications. This article describes some loading issues, problems, and difficulties commonly encountered and a solution to some of these problems.

Asynchronous Loading
The Java environment, which is multithreaded, allows asynchronous loading of images. Whether it is from a network or local disk, it takes a finite amount of time to load an image; this duration is unpredictable. As an image is being loaded, the program which uses that image will require the loading status in order to proceed with its task. There can be two approaches to accomplishing this...polling and notification. In polling, the program that needs the image checks the loading status at regular intervals. In a multithreaded environment, this approach is inefficient because the program that waits for the images not only wastes CPU cycles but also hinders the progress of other tasks.

In the notification approach, the image loading program notifies the image user program as the image is being loaded. In this approach, CPU cycles are not wasted and the progress of other tasks is unhindered during loading. However, a proper synchronization mechanism is needed between the image loading and image using programs to trigger appropriate actions at different stages of loading.

The AWT imaging model facilitates image loading through notification. The image producer delivers the pixels to the image consumer asynchronously. The image observer monitors the delivery of pixels. The ImageObserver interface has just one method called imageUpdate(). Whether it is image loading or image drawing, whenever an asynchronous image operation is involved, the imageUpdate() method is invoked at regular intervals to notify the status of that image operation. Although the AWT component class has the built-in imageUpdate() method, it is often necessary to override the imageUpdate() method to provide a better control over the actions that are performed upon image loading.

A typical imageUpdate() method is shown in Listing 1. The arguments of the imageUpdate() method include the status flag, width, and height. The status flag is a bitmask with each bit in the flag representing an operation. If the imageUpdate () method returns false, the updates for that operation will not be notified further.

Image Loading Stages
As Java is used in a wide variety of applications, the loading requirements vary. For example, in image animation there is no need to explicitly extract pixels. In contrast, in image analysis it is a must.

Depending upon the applications, image loading can involve a number of steps. These steps can be grouped into the following three stages.

1. Fetching the Image Object
The getImage() methods in the Applet class or AWT Toolkit class will perform image fetching. Just the invocation of getImage() method does not force an image to load. Images are loaded only when they are needed. Thus, the getImage() method is just a request to load an image. Actual delivery of pixels takes place only when an image is required.

2. Fetching Properties
An image can be considered as a combination of raw pixel values and information about the image. This information may consist of geometrical parameters, viewing parameters, scanning parameters, ownership, image format, etc. The properties of an image can be obtained by explicit requests; i.e., by using the get methods in the Image class. There are three methods available for property fetching. The width and height of an image can be fetched by getWidth(ImageObserver observer) and getHeight(ImageObserver observer) methods respectively. The other image properties can be fetched by the getProperty(String name, ImageObserver observer). This method can be used to fetch any named property that is specific to an image format. All the properties may not be needed to view an image.

Getting the width, height, and other properties are asynchronous operations, which means that the properties requested may not be immediately redeemable if the image is not loaded. In such a case, these methods return a -1. As in any other asynchronous operation in AWT, an ImageObserver object monitors the status of the image property fetch. The ImageObserver receives updates about an Image operation through the imageUpdate() method. When the WIDTH bit is set, the width of the image is available; similarly, image height is available when the HEIGHT bit is set.

3. Extracting Pixel Values
When an image object is loaded, pixels are hidden within that object. In order to grab the pixel values, specific requests need to be made. The PixelGrabber class which implements the ImageConsumer interface is used for extracting pixels. A PixelGrabber object should first be created as:

PixelGrabber pg = new PixelGrabber(img, 0,0,width,height, pixMap, 0, width);

When the PixelGrabber object is created, resources are allocated to enable the pixel grabber operation. The actual fetching of pixels is done by the grabPixels() method, shown in Listing 2.

A request to extract pixels values will also trigger the delivery of pixels.

Image Loading Problems
In this section, some of the problems and difficulties encountered due to Java's image loading are described.

1. Image Painting Problem
As mentioned before, images are loaded only when they are needed. This could cause problems if the paint() method is executed before the image is fully loaded. If double buffering is not implemented, you can see the image slowly unfolding on the screen as it loads. Even with double buffering, if the actual paint() or update() is executed before it is drawn on the off screen graphics, the screen will be blank.

There are many solutions to the image paint problem. One of them is to force the image to load in advance. The AWT component and the Toolkit classes provide the prepareImage() methods to force image loading. When an image is loaded in advance, some resources are tied up for a longer time. When a large number of images is involved, loading in advance is not desirable. The other solution is to use the imageUpdate() method to paint the image only after the image is completely loaded.

2. Animation Problem
Late loading can also result in slow animation as the images are being loaded. It may also lead to undesirable flashing. A solution to this problem is to load all the images in advance and then start the animation. Although prepareImage() methods can be used to accomplish this, it may involve some additional programming. However, the MediaTracker class which uses prepareImage() and imageUpdate() has methods to accomplish advance image loading and tracking. One problem with advance loading is that the viewing area in the applet/application will remain blank until all of the images are loaded. This may not be desirable if the loading times are larger. A progressive painting solution can be resorted to in such cases.

3. Loading Images for Non-Display Applications
Although MediaTracker is easy to use, it can not be used universally. The MediaTracker constructor requires an AWT component to be an argument. Moreover, that component has to be visible on the screen. This is not a problem in multimedia types of applications, as images are always displayed in some subclasses of the AWT component class. However, in Image Analysis types of applications in which computations are often performed before display, a dummy component may have to be made visible in order to use the MediaTracker class. There can also be applications in which graphical user interfaces are separated from the image loading classes. In such cases, MediaTracker will force the use of a GUI component in the image loading classes.

4. Difficulty with imageUpdate() Method
The imageUpdate() method has to trigger appropriate actions depending on the status. So, the imageUpdate() method becomes the center of control whenever image related asynchronous operations are involved. This may involve some programming difficulties in large and complex applications such as:

  • A number of actions may have to be performed in different objects when certain status information is available. Because of the asynchronous nature, these objects may be in different states. Managing these will be difficult because the state of the objects has to be known. This would require maintaining state variables in each object and triggering actions based on their state.
  • If several images are to be loaded together, it is often necessary to do different things with different images. For example, you may need to display a few images in an image set, but perform analysis on all the images in the set. This would require the imageUpdate() method to identify the image object and take appropriate actions depending on the object.
  • Typically, in an applet or a small application, a single object handles image loading and drawing. The ImageObserver interface is implemented in that object itself. In other words, the same object will have the imageUpdate() method. However, in a large application, loading may be performed by one object and drawing may be performed in another. The status notification may go to different objects and these objects will have to be aware of what actions are to be performed. A solution would be to create a separate ImageObserver object where the loading and drawing status related to an image is reported.
5. Scattered Image Loading APIs
As we have seen in the Image Loading Stages section, a number of APIs are required to load images. These are scattered over different classes in the java.awt and java.awt.image packages. This would mean a longer learning curve to understand image loading. What is desirable, therefore, is a single class that encapsulates all the image loading functionality. The next section describes such a class.

A Multi Purpose Image Loading Class
An image loading class called ScreenImage was developed to solve some of the problems discussed in the previous section. Unlike MediaTracker, it does not require a visible AWT component. The ScreenImage class implements the ImageObserver interface and its functionality centered around the imageUpdate() method. Clients can register with the ScreenImage class for callback and the imageUpdate() method will notify the registered objects at different stages of image loading. Because of this callback, the ScreenImage eases the programming difficulty involved in using the imageUpdate() method by allowing the clients themselves to perform appropriate actions.

The ScreenImage class has a number of APIs that help to load images, fetch properties, and extract pixels. It also provides the prepareImage() methods to enable the applications to load images in advance. In addition, it saves image properties such as width and height, and pixels' values if extracted. The client applications can monitor the progress in loading by checking the loadStatus variable. Thus, a number of related image loading functions is available under one class. The complete code listing for ScreenImage class is shown in Listing 3. A ScreenImage object needs to be constructed for every image that is going to be loaded. The client application is neither required to be of the type component nor is it required to implement the ImageObserver interface. However, it has to implement the Callback interface, which contains just one method called performCallback(int satus), shown in Listing 4.

The registered clients receive a callback notification whenever the loading status changes. Unlike the imageUpdate() method, this method is invoked when a certain loading status is attained. Currently, it is invoked on the following occasions:

  • Height and width available
  • Image is ready for drawing
  • Pixel values are available
  • Error condition
A Sample Application/applet.

A sample application/applet was developed to illustrate the use of ScreenImage class. It is used to load images for three different types of applications. Figure 1 shows the Frame that runs the application/applet.

Figure 1
Figure 1

It can display images in four tiled viewports, run animation, and extract pixels. Images can be selected using the ImageSelector Panel. Animation can be performed in any of the four windows. As the animation loop is running, it can be dynamically switched to other ports by selecting a desired viewport. The pixel computation can be performed without even displaying images. Some of the methods used in this application/applet are explained next.

Displaying Images
The method shown in Listing 5 can be used in any application to load an image. This method can be a part of the image viewing class. The imageTable variable is a HashTable which saves the ScreenImage and its id. This is a convenient way of keeping track of whether a ScreenImage object has been created for a given image. The fileList variable contains the list of image file names. In case of applets, the setUrlInfo() method can be used.

Listing 6 is the program snippet to display an image. The actual image display takes place in a class called ImageCanvas.

All the methods in Listings 7, 8, and 9 belong to the ImageCanvas class, which performs image painting. In the ImageCanvas class, scnImage is an instance variable. The method in Listing 7 sets that variable. This method first removes itself from the callback list of the previous ScreenImage and then registers with the new ScreenImage object.

The imagePaint() methods shown in Listing 8 are used for painting images. The actual painting is done by the update() method. The drawOffScren() method, which is not shown here, implements the double buffering.

The method shown in Listing 9 is called by the ScreenImage method to notify the image loading status. The action is performed only when an image is ready (or when there is an error, but not shown in the code).

Extracting Pixels
A method for extracting pixels is shown in Listing 10. Pixel values can be extracted without even displaying the image. The code is similar to the loadAnImage() method, except for the fetchPixels() method. In this method, a delay is added between load() and fetchPixels() method. Without this delay, loadPixels() will hang in some platforms.

The method in Listing 11 is called by the ScreenImage object. It checks for the PIXELS_AVAILABLE status and invokes a method to generate statistics.

Summary & Conclusions
The asynchronous nature of Java's image loading may pose problems, including image painting and animation problems, and difficulty in using the imageUpdate() method. The ScreenImage class described here not only solves some of the problems but can also be used in a variety of applications. It encapsulates a number of image loading-related methods that are scattered over different AWT classes. In addition, it keeps track of image loading and saves image pixels. Just like MediaTracker, it simplifies the programming involved in image loading synchronization. If an application has a large number of images involved for simple viewing or animation, using the ScreenImage class may not be as desirable as it may result in unnecessary code overhead.

References

  1. Cohen et al, Professional Java Programming, Wrox Press, September 1996.
  2. Graham, Jim, Image loading related articles posted to comp.lang.java. usenet group.
  3. java.awt and java.awt.image API Documentation.
About the Author
Lawrence Rodrigues is a senior consultant with Compuware Corp., Milwaukee. He has been developing Java applets and applications, is a contributor to the book "Professional Java: Fundamentals," by Wrox Press, and is also a judge at JARS. Besides Java, his current interests include Image Visualization and Analysis, Computational Geometry and Image Data Compression.

	

Listing 1: The imageUpdate() method.

public boolean imageUpdate(Image img, int infoflags, int x, int y, int width, int height){
           if((infoflags & ImageObserver.ERROR) != 0){
                // Error  handling code.
               return false;
           }
        if((infoflags & (ImageObserver.WIDTH | ImageObserver.HEIGHT))!= 0){
             //Invoke methods that needs width and Height
          }
        if((infoflags & (ImageObserver.FRAMEBITS | ImageObserver.ALLBITS))!= 0)  {
            //Invoke methods that are waiting for image to be completely loaded.
       }
        return true;
  }

Listing 2: The grabPixels() method

        try{
            if(pg.grabPixels() != true){
              // Pixels not grabbed.
             }
          } catch (InterruptedException e){
             // Handle Exception 
        }

Listing 3. ScreenImage Class

/*
 *  @# ScreenImage.java 1.0 96/9/20
 *  Copyright (c) 1996 Lawrence Rodrigues
 */
import java.io.*;
import java.awt.*;
import java.util.*;
import java.net.URL;
import java.awt.image.*;

 public class  ScreenImage extends Object implements ImageObserver{
       static  public final int EMPTY = 0;
       static  public final int ABORTED = 1;
       static  public final int ERRORED = 2;
       static  public final int LOAD_REQUESTED = 4;
       static  public final int HT_WID_KNOWN = 8;
       static  public final int IMAGE_READY = 16;
       static  public final int PIXELS_AVAILABLE = 32;
       public Image  image;
       public int width = -1, height = -1;
       public int origPixels[];
       public ColorModel cm;
       public URL url= null;
       public String fileName = null;
       public String dir;
       public String imageName;
       public int imageId;
       public int loadStatus = EMPTY;
       private Vector clientObjs = new Vector();
       private Callback  ClientObj;
  public ScreenImage(){
         loadStatus |= EMPTY;
  }
  public void setWidHt(int wd,  int ht){
         width = wd; height = ht;
         loadStatus |= HT_WID_KNOWN;
  }
  public void setParams(int rwIm[], int wd, int ht, ColorModel colCm){
         width = wd; height = ht;
         cm = colCm;
         origPixels = rwIm;
         loadStatus |= (LOAD_REQUESTED | HT_WID_KNOWN| PIXELS_AVAILABLE);
   }
   public void setFileInfo(String path, String name, int id){
          if(path == null || name == null) return;
          dir = path;
          imageName = name;
          imageId = id;
          fileName = new String(dir+File.separatorChar+imageName);
          url = null;
   }
  public  void setUrlInfo(URL ul, String name, int id){
          if(ul == null || name == null) return;
          url = ul;
          imageName = name;
          imageId = id;
          fileName = null;
   }
  public  boolean requestImageLoad(){
          if((url == null) && (dir == null)) return false;
          if(url != null) image = Toolkit.getDefaultToolkit().getImage(url);
          if(fileName != null) image = Toolkit.getDefaultToolkit().getImage(fileName);
          if(image == null) return false;
          return true;
  }
  public  boolean fetchWidHt(){
          if(image == null) return false;
          int wid = image.getWidth(this);
          int ht = image.getHeight(this);
          if((wid == -1) || (ht == -1)) return false;
          setWidHt(wid, ht);
          return true;
  }
  public  void load(){
          if(loadStatus == EMPTY){
             if(!requestImageLoad()) return ;
          }
          fetchWidHt();
  }
  public boolean fetchPixels(){
         if((width  <0) || (height <0)){
             load();
             return false;
         }
         int pixMap[] = new int[width*height];
         PixelGrabber pg = new PixelGrabber(image,0,0,width,height, pixMap, 0, width);
         try {
	       pg.grabPixels();
          } catch (InterruptedException e){return false;}
         if((pg.status()  & ImageObserver.ABORT)!=0){
             notifyClientObj(ERRORED);
             return false;
         }
         setParams(pixMap, width, height, cm);
         notifyClientObj(PIXELS_AVAILABLE);
         return true;
     }
  /** 
    * Prepares the image to be constructed with its original width & height
    */ 
  public boolean prepareImage(){
         return prepareImage(width, height);
  }

  /** 
    * Prepares the image to be constructed with a given width and height.
    */ 
  public boolean prepareImage(int wid, int ht ){
         if(image == null) return false;
         if(Toolkit.getDefaultToolkit().prepareImage(image, wid,ht, this)){
            loadStatus |= IMAGE_READY;
         }
         return true; 
  }
    /**
      * Implements ImageObserver interface. Called at regular intervals when
      * image related operations are taking place. This methods notifies the
      * clients when a certain status becomes avaialale.
      */
  public boolean imageUpdate(Image img, int infoflags, int x, int y, int width, int height){
         if((infoflags & ImageObserver.ERROR) != 0){
             notifyClientObj(ERRORED);
             return false;
         }
         if((infoflags & (ImageObserver.WIDTH | ImageObserver.HEIGHT))!= 0){
             setWidHt(width, height);
             notifyClientObj(HT_WID_KNOWN);
         }
        if((infoflags & (ImageObserver.FRAMEBITS | ImageObserver.ALLBITS))!= 0)           
		{
            loadStatus |= IMAGE_READY;
            notifyClientObj(IMAGE_READY);
            return false;
          }
        return true;
   }
   /**
     * A callback  object can register itself here. 
	 It gets added to the callback list.
     */
   public void registerForCallback(Callback cbObj){
          clientObjs.addElement(cbObj);
   }
   /**
     * Callback object can remove itself from the callback list.
     */
   public void unregisterCallback(Callback cbObj){
          clientObjs.removeElement(cbObj);
   }
   /**
     * Notifies all the registered client objects.
     */
   public void  notifyClientObj(int status){
          for (Enumeration e = clientObjs.elements() ; e.hasMoreElements() ;) {
               ((Callback)(e.nextElement())).performCallback(this, status);
          }
   }
}

Listing 4:  Callback  Interface

    public interface Callback{
           public void performCallback(ScreenImage img, int action);
    }

Listing 5: The loadAnImage() method

private boolean loadAnImage(int num){
           ScreenImage im = new ScreenImage();
           if(im == null) return false;
           imageTable.put(Integer.toString(num), im);
           im.setFileInfo(path, fileList[num], num);
           im.load();
           return true;
   }

Listing 6: The displayImage() method

ImageCanvas CurScreen = new ImageCanvas(width, height);

public void displayImage(int num){
               ScreenImage img = (ScreenImage)imageTable.get(Integer.toString(num));
            if(img == null){
               if(!loadAnImage(num)){
                    return;
                 }
           curScreen.setScreenImage(img);
           curScreen.imagePaint();
}

Listing 7: The setScreenImage() method

public void setScreenImage(ScreenImage im){
             if(scnImage != null){
                scnImage.unregisterCallback(this);
                }
             scnImage = im;
             scnImage.registerForCallback(this);
             pixImage = im.image;
        }

Listing 8: The imagePaint() methods

   public  boolean imagePaint(Image img){
            if(img == null) return false;
            pixImage = img;
            drawOffScreen(img);
            return true;
    }
   public  boolean imagePaint(){
            return imagePaint(pixImage);
   }

Listing 9: The ScreenImage() method

 public void performCallback(ScreenImage img, int action){
             if(action == ScreenImage.IMAGE_READY){
                imagePaint(pixImage);
             }
  }

Listing 10: The loadPixels() method

   public void loadPixels(int id){
           ScreenImage im;
           im = (ScreenImage)imageTable.get(Integer.toString(id));
           if(im== null){
              //Not loaded
              im = new ScreenImage();
              imageTable.put(Integer.toString(id), im);
              im.registerForCallback(this); 
              im.setFileInfo(path, fileList[id], id);
              im.load();
               try{
                   mainThread.sleep(100);
              }
              im.fetchPixels();
          }
          else {
              //Already loaded.
               im.registerForCallback(this); 
               im.fetchPixels();
           }
     }

Listing 11: The ScreenImage() method

public void performCallback(ScreenImage img, int status){
           if(status == ScreenImage.PIXELS_AVAILABLE){
              generateAndDisplayStats();
         }

 

All Rights Reserved
Copyright ©  2004 SYS-CON Media, Inc.
  E-mail: [email protected]

Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.