HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML
 

The multimedia objects in Sun's Java Development Kits are so primitive that they're worthless for serious development. Fortunately, Sun has overhauled Java's multimedia capabilities with the release of the Java Media Framework. In this article I'll explain why the JMF architecture is a significant improvement and show you how to use these objects in your applets or applications.

Struggling with AudioClip
If you've ever used the applet's AudioClip class, you've probably grieved over its limited functionality. The most irritating restrictions are its inability to determine the length of an audio file or how much of a file has already played. To be fair, AudioClip was designed for no-frills playback of AU digital audio content. Consequently, it can't play most digital audio files, or any digital video content or Musical Instrument Digital Interface (MIDI) files (see Listing 1).

JMF to the Rescue
Unlike AudioClip, JMF is a strategic API that's part of Sun's Media and Communications APIs. It supports most popular digital audio and video file formats along with MIDI files. Furthermore, you can use JMF 2.x to record (or capture) both digital audio and video files.

There are two types of JMF releases: pure and performance packs. The pure version can run on any Java platform (i.e., Linux and other forms of UNIX), while the performance packs contain code to maximize performance for either Win32 or Solaris platforms.

Understanding JMF Requires Time
Besides supporting a greater number of media formats, JMF also offers a richer selection of multimedia objects. Foremost among these improvements is the ability to manipulate time. In fact, virtually everything in JMF revolves around the clock interface and its ability to monitor time.

JMF clocks measure two types of time: TimeBases and Media Time. The former represents the constant flow of time from a known starting point (Greenwich Mean Time is a TimeBase). By contrast, Media Time describes the amount of time that has been consumed in a media stream. Unlike a TimeBase, Media Time can be stopped, can flow backward (rewind) or can advance at irregular intervals (fast forward).

Clocks use state to determine when they should be active. By default, clocks begin in stopped state and you start them by invoking their syncstart() method. Once started, a clock attempts to synchronize (or correlate) its TimeBase with its Media Time. Clocks remain started until you issue a stop(), the media stream ends or an exception occurs (see Figure 1).

Figure 1
Figure 1:

Getting Everything Under Control
Although time is an important aspect of multimedia, a robust multimedia platform provides additional features such as resource management, error handling and support for threads. JMF supports these features in the controller interface, an extension of the clock interface.

Controllers enhance the clock interface by dividing the stopped state into five stages: unrealized, realizing, realized, prefetching and prefetched (see Figure 2).

Figure 2
Figure 2:

Controllers leave the unrealized state and enter the realizing state when they attempt to access the resources necessary to manipulate multimedia content. For instance, an audio controller will attempt to reserve resources on a sound card during realization. Once these resources are obtained, the controller becomes realized.

If you're using a nonrealtime operating system such as Windows 98, NT or Solaris, your programs may run irregularly. By contrast, multimedia devices consume large quantities of data at specific intervals.

If your application can't obtain enough time slices to fulfill the demands of these devices, you'll hear the grating sound of audio breakups or see video frames dropped.

The most common technique to prevent such problems is to pool buffers before they're needed. If your application can't satisfy the device's buffer demands, you can withdraw data from the previously filled buffer pool and stream it to the device, thereby preventing audio-visual hiccups.

The process of filling a controllers buffer pool is called prefetching. A controller leaves realized state and enters prefetching state when you request that it prefetch buffers (the numbers of buffers required will vary by controller). After these buffers have been obtained, the controller reaches the prefetch state. Once prefetched, a controller may be started.

The controllers subdivision of stopped state gives you increased granularity of control over multimedia resources. For instance, if a controller is able to realize, you know that all required hardware resources are functioning and available for use.

Another advantage of the multistate approach is the ability to detect errors. For instance, if the controller implodes while it's realizing, you know that it failed trying to acquire or initialize a multimedia resource. By contrast, if it failed to prefetch, the problem involved filling the prefetch buffers. Similarly, if it failed to start, it's probably due to an invalid media time or clock operation.

The final benefit of multiple states is the ability to maximize threads. For example, state transitions that occur rapidly(i.e.,stopped to realizing and realized to prefetching)are done synchronously, while the transitions that can take extended periods of time (i.e.,realizing to realized and prefetching to prefetched)are performed asynchronously.

Realizing must be threaded since some hardware devices have long initialization times. Likewise, prefetching is threaded because input/output operations are often lengthy.

Never Assume
Controllers provide four primary methods to manipulate states: realize(), prefetch(), start() and stop(). The first method causes the controller to transition from unrealized to realizing and then to realized state. Similarly, prefetch() switches it from realized to prefetching and then to prefetched. The last two methods, start() and stop(), cause the controller to move into started and stopped state, respectively (see Table 1).

Table 1

Because state transitions may occur asynchronously, you should never assume that the transition has completed when the controller's prefetch() or realize() methods return to your application. Rather, you should listen for the appropriate state transition event to be sent from the controller. Failure to listen for these events could result in an exception if you call an inappropriate method for a given state.

To receive controller events, you should register as a listener of the controller. Since controller events utilize the JDK 1.1 event model, if you're familiar with JavaBeans or AWT programming, you already know how to handle them.

// Add ourselves as a listener to the controller
player.addControllerListener(this);

Events permit you to monitor the state of a controller. Although most of the code in your listener method will be handling state transition events (see Listing 2), you should also listen for errors, media status events or error conditions. For instance, a well-written JMF application listens for the EndOfMediaEvent so that it's aware when playback stops:

// the EndOfMediaEvent indicates we've run out of data
else if (event instanceof EndOfMediaEvent)
{
// rewind (i.e. set media time to 0
player.setMediaTime(new Time(0));
}

The Good Stuff
So far, we've concentrated on primitive objects and interfaces. Now we'll delve into the three components that let you play content: DataSource, Player and Manager.

DataSources are objects that retrieve data, place it in buffers and stream these buffers to client applications. Their main responsibility is to convert dedicated audio and video file formats into industry standard formats such as Pulse Code Modulation (PCM). This conversion process ensures that client applications can work with virtually any file format.

There are two types of DataSources: pull and push. PullDataSources supply buffers when your application requests them. Examples include those that read .AVI, .WAV and MIDI files. By contrast, PushDataSources stream data to your application when the data is available. TV and radio broadcasts are examples of PushDataSources since the content is continually being pushed toward you. They are more complex to use since you must handle situations in which a buffer arrives and you're busy doing something else.

MediaHandlers retrieve buffers from a DataSource, massage the data and transport the resultant buffers to a multimedia device. The most common type of MediaHandler is the Player, a MediaHandler that also implements the controller interface.

When you request that a Player start(), it obtains data from its associated DataSource, updates its state information and sends the data to its destination device.

Managing the Missing Piece
If you simply want to play an audio file, the multitude of methods surfaced by DataSource and Player objects is intimidating. Furthermore, it can be a daunting task to find a MediaHandler to process a DataSource's output. Therefore, JMF provides the Manager object to shield you from these implementation details. The Manager not only finds the appropriate DataSource for your content, but it also constructs a Player, connects the Player to the DataSource and returns a Player reference to your application.

The only prerequisite for using the Manager is that you must pass it a MediaLocator (see Listing 3). MediaLocators inform the Manager of the data's location and the protocol that should be used to retrieve the data.

Once you've created a MediaLocator, you can call the Manager's createPlayer() method to construct a Player.

// create a Player with a Media Locator
player = Manager.createPlayer(mrl);

The first thing you should do upon successful creation of a Player is to add yourself as a listener. If you delay this call, you may miss vital warning events emanating from the Player.

Another technique you should practice is enclosing the initial interaction with the Player in try/catch blocks.

try
{
// create a Player based on the medialocator
player = Manager.createPlayer(mrl);
// be sure to immediately listen for events...
player.addControllerListener(this);
}
catch (NoPlayerException e)
{
System.out.println("Can't find a player for " + mrl);
}

This is crucial because JMF Players can throw a multitude of exceptions during object creation.

New JMF programmers remember to create the Player inside a try/catch block, but frequently use methods that depend on player creation outside the block. For instance, should the Player creation in Listing 4 fail, the failure will be caught. However, the code then attempts to add itself as a listener to a nonexistent Player, which will result in an unhandled exception.

You Choose the Level of Complexity
The beauty of JMF is that casual programmers can use the Manager to create a Player, then issue the start() method to immediately begin playback of multimedia content. The following code illustrates the quickest technique to commence playback. It isn't necessary to understand all of the state gyrations a controller goes through, but you should listen for error events and catch exceptions.

public void start()
{
// when the Web page is loaded, immediately start playback
if (player != null)
{
player.start();
}
}

JMF also satisfies the cravings of advanced programmers for low-level access to multimedia objects. For instance, an advanced application may test for the existence of multimedia resources. If resources are available, it will then prefetch buffers to improve responsiveness. Finally, it will automatically restart playback when the media stream finishes. All of these actions are possible if you listen and react to controller events.

To implement such a system, the applet in Listing 5 calls the Player's realize() method when the applet's init() method is invoked. If it receives a RealizeCompleteEvent, it knows that the audio hardware is functional. Consequently, it issues prefetch() to fill up the Player's buffers.

When the PrefetchCompleteEvent is sent by the Player, the applet displays a set of buttons to start and stop playback. Since prefetch() has filled the Player's buffers, this applet will start playback faster than one written by a casual programmer who calls start() while the Player is in the unrealized state.

Finally, when the Player runs out of media to process, it will fire the EndOfMediaEvent to the applet. The applet in turn rewinds the file to the beginning and restarts playback by calling start(). This will cause the media to repeat forever until the user hits the stop or pause buttons on the applet.

Is There More?
As we've discovered, JMF provides the multimedia building blocks for use in your applets and applications. The clock interface permits you to control the direction and speed of playback. The controller extends the clock to provide resource management, error handling and event tracking. DataSources and Players retrieve and process, respectively, multimedia data. Finally, the Manager object simplifies the usage of DataSources and Players.

In the next part of this series we'll explore how you can energize your JMF applications with emerging Internet multimedia standards.

Author Bio
Linden deCarmo is the author of Prentice Hall's Core Java Media Framework. He is a senior software engineer at NetSpeak Corporation, where he develops advanced telephony software for IP networks.
He can be reached at: [email protected]

	

Listing 1: 

import java.applet.AudioClip; 

public class LessPrimitive extends java.applet.Applet 
{ 
   AudioClip audioClip1; 
   AudioClip audioClip2; 

   // Applet role 
   public void init() 
   { 
  
      // create two audio objects 

      audioClip1 =  getAudioClip( get- 
      CodeBase(), "clock1.au" ); 
      audioClip2 =  getAudioClip( get- 
      CodeBase(), "clock2.au" ); 

      // play two audio files in loop mode 
  
      audioClip1.loop(); 
      audioClip2.loop(); 

      try 
      { 
        // we still have NO clue how 
        // long these files will take if 
        // we want to play them one time 
        // however, we put them in loop 
        // mode, so they will play forever 
        Thread.sleep( (long) 2000 ); 
      } 
      catch (InterruptedException e) 
      { 
        System.out.println("Someone 
        woke us up unexpectedly..."); 

      } 

        // stop the first clip after 2 
        // seconds 

        audioClip1.stop(); 

        try 
      { 
        // we still have NO clue how 
        // long these files will take if 
        // we want to play them one time 
        // however, we put them in loop 
        // mode, so they will play forever 
        Thread.sleep( (long) 7000 ); 
      } 
      catch (InterruptedException e) 
      { 
        System.out.println("Someone 
        woke us up unexpectedly..."); 

      } 
  
      // stop the 2nd after another 7 
      // seconds 

      audioClip2.stop(); 

   } 

} 

Listing 2: 

public synchronized void controllerUpdate(ControllerEvent event) 
{ 
 // this event is received when the 
 // Controller enters Realized state 
 if (event instanceof RealizeCom- 
 pleteEvent) 
 { 

        // event handling code goes 
        // here..... 
  } 
 // this event is received when the 
 // Controller finishes prefetching.... 
    else if (event instanceof Prefetch- 
    CompleteEvent) 
    { 
        // event handling code goes 
        // here..... 
  
    } 

} 

Listing 3: 

MediaLocator mrl = null; 
URL url = null; 

// The applet tag should contain the 
// path to the source media file, rela- 
// tive to the html page. 

if ((mediaFile = getParameter("FILE")) == null) 
{ 
    System.out.println("Invalid 
    media....."); 
} 
try 
{ 
    // create a MediaLocator based on 
    // the filename 
    url = new URL(getDocumentBase(), 
    mediaFile); 
    mediaFile = url.toExternalForm(); 
} 
catch (MalformedURLException mue) 
{ 
    System.out.println("Bad URL......"); 
} 
try 
{ 
  // Create a media locator from URL 
  if ((mrl = new MediaLocator(media- 
  File)) == null) 
  { 
   System.out.println("Can't build URL 
   for " + mediaFile); 
  } 

catch (MalformedURLException mue) 
{ 
    System.out.println("Invalid media 
    file URL!"); 
    System.exit(0); 
} 

Listing 4: 

try 
{ 
    // create a Player based on the 
    // medialocator 
    player = Manager.createPlayer(mrl); 
} 
catch (NoPlayerException e) 
{ 
    // exceptions will be caught 
    // here... 
    System.out.println("Can't find a 
    player for " + mrl); 

} 

// if an exception occurred in Manag- 
// er.createPlayer(mrl) 
// then the player reference will be 
// NULL and the line below will gener- 
// ate another exception 
player.addControllerListener(this); 

Listing 5: 

 // this applet will load the audio 
 // file specified by the HTML page and 
 // play it forever until stop or pause 
 // is pressed 

import java.applet.Applet; 
import java.awt.*; 
import java.awt.event.*; 
import java.util.*; 
import java.net.URL; 
import java.io.IOException; 
import java.util.Properties; 
import javax.media.*; 
import java.lang.String; 
import java.net.MalformedURLException; 

public class AlohaJMF extends Applet implements ControllerListener 
{ 
  
    // this is the object that will 
    // play the media files.. 
    Player player = null; 

    // displays progress during download 
    Component progressBar = null; 
  
  
    // video window 
    Component visualComponent = null; 

 
    // GUI controls for position, 
    // start, stop etc. 
    Component controlComponent = null; 
  
  
    int controlPanelHeight = 0; 
    Panel panel = null; 

    // attributes of the video window.... 
    int videoHeight = 0; 
    int videoWidth = 0; 

    // init reads the HTML params and 
    // creates a Player based on those 
    // parameters...... 
    public void init() 
    { 
     setLayout(null); 
  
     // create the panel where we'll 
     // show our Player 
     panel = new Panel(); 
     panel.setLayout( null ); 
     add(panel); 
     panel.setBounds(0, 0, 320, 240); 

     // HTML page will tell us what to 
     // play... 
     String mediaFile = null; 
  
     MediaLocator mrl = null; 
     URL url = null; 

     // Get the file to play from HTML... 
  
     if ((mediaFile = 
     getParameter("FILE")) == null) 
         System.out.println("HTML page 
         contains bogus file."); 

     try 
     { 
         // create a MediaLocator based 
         // on the filename 
         url = new URL(getDocument- 
         Base(), mediaFile); 
         mediaFile = url.toExternal- 
         Form(); 
     } 
     catch (MalformedURLException mue) 
     { 
         System.out.println("Invalid URL"); 
     } 
  
     try 
     { 
         // Create a media locator from URL 
         if ((mrl = new MediaLocator- 
         (mediaFile)) == null) 
         { 
           System.out.println("URL er- 
           ror for:" + mediaFile); 
     } 
else 
     { 
       try 
       { 
         // setup Player for media 
             player = Manager.cre- 
            atePlayer(mrl); 
            // listen for events imme- 
            // diately.... 
            player.addController- 
            Listener(this); 
             } 
             catch (NoPlayerException e) 
             { 
                   System.out.println- 
                   ("Player creation 
                   failed for " + mrl); 
                System.exit(0); 
             } 
            } 
     } 
     catch (MalformedURLException mue) 
     { 
         System.out.println("Invalid 
         media file URL!"); 
         System.exit(0); 
     } 
     catch (IOException ioe) 
     { 
         System.out.println("IO excep- 
         tion creating player for " + mrl); 
         System.exit(0); 
     } 
  

    } 

    // this method is called on applet 
    // creation and also when the page 
    // is reloaded 

    public void start() 
    { 
        // automatically begin playback 
        // when the page is loaded.... 
     if (player != null) 
     { 
         player.start(); 
     } 
    } 

    // make sure to stop AND close all 
    // resources when stop is called.. 
    // otherwise, we'll have resource 
    // leaks and frustrated users... 
    public void stop() 
    { 
        if (player != null) 
        { 
            player.stop(); 
            player.deallocate(); 
        } 
    } 

    public void destroy() 
    { 
     player.close(); 
    } 
  

    // controllerUpdate is required to 
    // listen for JMF events 
  
    public synchronized void con- 
    trollerUpdate(ControllerEvent event) 
    { 
     // Has our Player dead? 
     // If so, nothing to do..event is 
     // bogus. 
     if (player == null) 
         return; 
  
     // When the player is Realized, 
     // get the visual and control com- 
     // ponents and add them to the 
     // Applet 
     if (event instanceof RealizeCom- 
     pleteEvent) 
     { 
         if (progressBar != null) 
         { 
          panel.remove(progressBar); 
          progressBar = null; 
         } 

         int width = 320; 
         int height = 0; 
         if (controlComponent == null) 
               if (( controlComponent = 
              player.getControl- 
                    PanelComponent()) 
                    != null) 
          { 
  
           controlPanelHeight = 
                 controlComponent.get- 
                   PreferredSize().height; 
           panel.add(controlCom- 
                 ponent); 
           height += control- 
                 PanelHeight; 
          } 
  
         if (visualComponent == null) 
          if (( visualComponent 
                   = player.getVisual- 
                   Component())!= null) 
          { 
              panel.add(visual- 
                    Component); 
              Dimension video 
                    Size = visual 
                    Component.getPre- 
                    ferredSize(); 
              videoWidth = 
                    videoSize.width; 
              videoHeight = 
                    videoSize.height; 
              width = video- 
                    Width; 
              height += video 
                    Height; 
              visualComponent.setBounds(0, 0, videoWidth, videoHeight); 
          } 

         panel.setBounds(0, 0, width, 
         height); 
         if (controlComponent != null) 
         { 
          controlComponent.set- 
                Bounds(0, videoHeight, 
                width, controlPanel 
                Height); 
                 controlCompo- 
                nent.invalidate(); 
         } 
  
      } 
  
      // the EndOfMediaEvent == no 
        // more data 
     else if (event instanceof EndOfMe- 
     diaEvent) 
     { 
         // jump to beginning of content 
         player.setMediaTime(new 
         Time(0)); 
         // immediately restart playback... 
         player.start(); 
     } 
     else if (event instanceof Con- 
     trollerClosedEvent) 
        { 
            // player is done, kill 
            // GUI elements 
         panel.removeAll(); 
     } 
    } 

} 
  



 

All Rights Reserved
Copyright ©  2004 SYS-CON Media, Inc.
  E-mail: [email protected]

Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.