HomeDigital EditionSys-Con RadioSearch Java Cd
Advanced Java AWT Book Reviews/Excerpts Client Server Corba Editorials Embedded Java Enterprise Java IDE's Industry Watch Integration Interviews Java Applet Java & Databases Java & Web Services Java Fundamentals Java Native Interface Java Servlets Java Beans J2ME Libraries .NET Object Orientation Observations/IMHO Product Reviews Scalability & Performance Security Server Side Source Code Straight Talking Swing Threads Using Java with others Wireless XML
 

From the very first steps this column took, a journey of discovery was promised which, hopefully, has not disappointed. Using Visual Cafe as our development platform, we have ventured into areas of Java that others have feared to tread. From building complete applets with database connectivity to developing classes for sending and receiving e-mail, this column has so far tackled a wide variety of different problems. On the face of it, many of the topics appear difficult or far too complicated to have in an ordinary applet, application or servlet. But I think you'll agree that once the initial wall has been scaled, the rest of the problem becomes somewhat academic.

My previous two columns looked at interfacing to two of the more popular TCP services; POP and SMTP. This month, we will round off the TCP services with an implementation for a generic class to interface to the Usenet newsgroup network.

Usenet
The Usenet, or newsgroups, is a place where users exchange information in topics of conversation. These range from computing right through to knitting. In order to view or posts news items, a Usenet viewer program is required. This is simply a program that interfaces to a Usenet server and retrieves and post news articles. One of the more popular Usenet clients for the PC platform is Free Agent. Free Agent gives an interface to the newsgroups, downloading and uploading appropriate articles. It is this functionality we will build into a class that will allow any Java program to have access to the Usenet.

Connecting to a Usenet server is a simple matter of opening up a socket connection to the server which usually sits and listens for client connections on port 119. The procedure for performing this can be seen in the complete listing of the class. Like the POP and SMTP servers, communication with the Usenet server is through command strings passed as carriage-returned lines.

The available command set to communicate with the server is surprisingly small, with only a handful of commands required to get the most out of a server. But before we go into the commands, let's take a quick look at how articles are arranged on the server.

Articles
Each article that exists on the Usenet system is identified with a global message id. This is normally created using a mixture of time/date/posters domain, to ensure complete uniqueness across all the newsgroup servers. When a new article is posted to a server, the server then distributes the article throughout the network. This procedure propagates the article throughout the whole network.

Associated with each article is a header. This header, shown in Listing 1, describes the article, which includes the e-mail of the poster, the subject of the article, the newsgroup the article has been posted in and the date it was posted (to name some of the more useful fields).

Retrieving the article from the server is a simple matter of issuing the commands

HEAD <message-id>
BODY <message-id>

where <message-id> is the ID of the article, as stated in the Message-ID field of the article header. If the article exists, it will be sent back to the client. The header is sent first, then the body. With this, we can write a simple method to pull a particular article back from the server. Listing 2 illustrates this method.

Listing 2 shows the method, getArticle(...) which downloads the article from the server. First, it issues the HEAD command which asks the server to download the header of the given article. As each line is received, it is parsed so we may extract the e-mail address of the poster, the date it was posted and the subject of the article. If for some reason, the article is no longer available, then the server will return an error message line which contains a status code at the start of the line. If this code is either 423 or 430, we can assume the article is no longer available. If this is the case, the body is not downloaded and the method returns a null.

The body operates on somewhat the same principle, copying the lines of the article into a Vector for later use. Note that in this version we are actively filtering out all binary attachments as this can make the article body very large indeed. You may have noticed that the method returns a class of type cArticle. This is a simple wrapper class that represents a single Usenet article, with the body held in a Vector class.

Posting Articles
Posting an article to the Usenet server is much easier than downloading one. Using the command POST, the server will expect the head and then the body of the article, terminated with a single period (.) on a line of its own. The head is separated from the body by a single blank line.

Listing 3 shows the implementation of the postArticle(...) method. This method takes in the newsgroup the article is to be posted to, the e-mail of the poster, the subject and finally the actual body of the article formatted in a Vector, where each element is a single line.

The most important part of posting to the server is correctly formatting the header. If any of the syntax is wrong, then the article will be rejected. One of the features of the Usenet system is the ability to cross-post. This is where the same article is posted to multiple newsgroups at the same time. This is particularly handy as it saves on both bandwidth and time. To do this, simply list the newsgroups as a comma separated list. The server will then do the rest without any further intervention from the client.

Article Listings
So far we have seen how to post an article to the newsgroups, and how to request a particular article based on its message identification. However, this begs the question, how does one determine the message ids of the articles, so the client optionally may download them?

Fortunately, this is a fairly easy operation to perform. The server can be asked to return all the message ids for a particular newsgroup, since a certain date. For example, to request all the messages since the 9th of December 1997, midnight, for the newsgroup comp.lang.java.programmer, the following command would be issued

NEWNEWS comp.lang.java.programmer 971209 000000

The server will then return a list of message ids, each in their own line, with the list terminated using a period (.). This message id can then be used in the method described earlier to retrieve that article. Listing 4 demonstrates the communication with the server to determine the list of message ids.

The method returns a Vector of message ids back to the caller, or null if an error occurred. Again, as with the posting of articles, the formatting of the date is most important; otherwise, the server will recheck the command.

Summary
This article looked at the NNTP (News Network Transport Protocol) protocol, which is used to communicate with the Usenet servers. By building a simple class, the majority of the functionality required to send and receive articles is a simple method call away. Although only a subset of the available commands was implemented, other commands exist that allow further interrogation of the server. For example, a command exists that determines the available newsgroups.

Communication with the more popular TCP servers is performed using simple ASCII commands, which allow for very straightforward clients to be implemented. This is one of the real powers the Internet has to offer.

In the next column we will go further down the road of Java enlightenment and tackle another 'looks-far-too-complicated' topic area.

About the Author
Alan Williamson is on the Board of Directors at N-ARY Limited, a UK-based Java software company specializing in Java/JDBC/Servlets. He has recently completed his second book, focusing purely on Java Servlets. His first book looked at using Java/JDBC/Servlets to provide a very efficient database solution. He can be reached at (http://www.n-ary.com) and he welcomes all suggestions and comments.

	

Listing 1: Header.
 
Path: news.wisper.net!peer.news.zetnet.net!dispose.news.demon.net! 
From: [email protected] (AR Williamson) 
Newsgroups: comp.lang.java.programmer,comp.lang.java.help,comp.lang.java.gui 
Subject: Java Servlets 
Date: 28 Nov 1997 13:23:02 -0800 
Organization: N-ARY Limited 
Message-ID: <[email protected]> 
NNTP-Posting-Host: n-ary.com 
X-Trace: 880752184 11160 pvdl  206.184.139.132 
Lines: 31
 
Listing 2: getArticle() method.
 
public cArticle getArticle( String ID ) throws IOException 
{ 
  cArticle NG = new cArticle(); 
  String LineIn; 
  boolean bBad = false; 

  //- Get the Header 
  sendMessage( "HEAD " + ID ); 
  while ( (LineIn=In.readLine()) != null ) 
  { 
    if ( LineIn.charAt(0) == '.' ) 
      break; 
    else if ( LineIn.indexOf("423") == 0 || LineIn. 
     indexOf("430") == 0 ) 
      return null; 
    else if ( LineIn.indexOf("Subject:") != -1 ) 
      NG.Subject = LineIn.substring( LineIn.indexOf("Subject:")+9,LineIn.length() ); 
    else if ( LineIn.indexOf("Date:") != -1 ) { 
      try{ 
        NG.EntryDate = new Date( LineIn.substring( LineIn.indexOf("Date:")+6, 
                       LineIn.length() ) ).getTime(); 
      } 
      catch( Exception E ){ 
        NG.EntryDate = new Date().getTime(); 
      } 
    } 
    else if ( LineIn.indexOf("From:") != -1 ){ 
NG.Author = LineIn.substring( LineIn.indexOf("From:")+6,LineIn.length() ); 
      NG.Author = NG.Author.replace( '"', ' ' ); 
    } 
    else if ( LineIn.indexOf("Lines:") != -1 ){ 
      try{ 
int Lines = Integer.parseInt( LineIn.substring(LineIn.indexOf("Lines:")+7,LineIn.length()) ); 
        if ( Lines > 120 ) 
          bBad = true; 
      }catch(Exception E){} 
    } 
  } 

  //- Get the Body 
  sendMessage( "BODY " + ID ); 
  NG.vBody = new Vector(30,5); 
  LineIn = In.readLine(); 
  if ( LineIn == null || LineIn.indexOf("222") == -1 )  return null; 

  while ( (LineIn=In.readLine()) != null ) 
  { 
    if ( LineIn.length() != 0 && LineIn.charAt(0) == '.' && LineIn.length() == 1 ) 
        break; 
    else if ( LineIn.length() != 0 && LineIn. 
       indexOf("Content-type: ") != -1 ){ 
      if ( LineIn.indexOf("text/plain") != -1 ) 
          NG.vBody.addElement( LineIn + "\n" ); 
      else 
          bBad = true; 
    } 
    else if ( LineIn.length() != 0 && LineIn. 
       indexOf("begin 600") != -1 ) 
      bBad = true; 
    else if ( bBad == false ) 
      NG.vBody.addElement( LineIn + "\n"); 
  } 

  if ( bBad ) return null; 
  NG.Newsgroup = Location + Type + cArticle.USENET; 
  return NG; 
} 

Listing 3: postArticle() method.
 
public void postArticle( String _NG, String _from, String _Subject, Vector _Body ) 
	throws IOException 
{ 
  String LineIn; 

  //- Get the Header 
  sendMessage( "POST" ); 
  LineIn = In.readLine(); 
  if ( LineIn == null || LineIn.indexOf("340") == -1 ) return; 

  //- Post the Header fields 
  sendMessage( "Path: n-ary" ); 
  sendMessage( "From: " + _from ); 
  sendMessage( "Newsgroups: " + _NG ); 
  sendMessage( "Subject: " + _Subject ); 
  sendMessage( "Message-ID: <" + System.currentTimeMillis() + "@n-ary.com>" ); 
  sendMessage( "Date: " + new SimpleDateFormat
  ( "dd MMM yy HH:mm:ss").format(new Date()) + " GMT"); 
  sendMessage( "Organization: N-ARY Limited" ); 
  sendMessage( "Reply-To:" + _from + "\r\n" ); 

  //- Post the Body 
  Enumeration E = _Body.elements(); 
  while ( E.hasMoreElements() ) 
    sendMessage( (String)E.nextElement() ); 

  sendMessage( "\r\n.\r\n" ); 
  In.readLine(); 
} 

Listing 4.
 
public Vector getArticleList(String _Name, long _date) throws IOException 
{ 
  String D = new SimpleDateFormat( "yyMMdd HHmmss").format(new Date(_date)); 

  Vector V = new Vector( 50, 10 ); 
  sendMessage( "NEWNEWS " + _Name + " " + D ); 
  String LineIn; 
  while ( (LineIn=In.readLine()) != null ){ 
    if ( LineIn.charAt(0) == '.' ) 
      return V; 
    else if ( LineIn.indexOf( "230" ) == 0 ) 
      continue; 
    else 
      V.addElement( LineIn ); 
  } 
  return null; 
} 

Listing 5: Complete listing.
 
import java.applet.*; 
import java.io.*; 
import java.util.*; 
import java.net.*; 
import java.text.*; 

public class newscomms 
{ 
  DataInputStream In; 
  DataOutputStream Out; 
  Socket  OutPort; 

  public newscomms(){} 

  public boolean openPort(String _Host){ 
      try{ 
          OutPort = new Socket( _Host, 119 ); 
          In      = new DataInputStream( OutPort.getInputStream() ); 
          Out     = new DataOutputStream( OutPort.getOutputStream() ); 
          In.readLine(); 
          return true; 
      }catch( IOException E ){ 
          System.out.println( "newscomms.openPort(.): " + E );     
      } 
      return false; 
  } 

  public void closePort(){ 
    try{ 
        sendMessage( "QUIT" ); 
        OutPort.close(); 
    }catch( IOException E){} 
  } 

  public void sendMessage( String _M ) throws IOException{ 
    Out.writeBytes( _M + "\r\n" ); 
  } 

  public Vector getArticleList(String _Name, long _date) throws IOException{ 
    String D = new SimpleDateFormat( "yyMMdd HHmmss").format(new Date(_date)); 

    Vector V = new Vector( 50, 10 ); 
    sendMessage( "NEWNEWS " + _Name + " " + D ); 
    String LineIn; 
    while ( (LineIn=In.readLine()) != null ) { 
      if ( LineIn.charAt(0) == '.' ) 
        return V; 
      else if ( LineIn.indexOf( "230" ) == 0 ) 
        continue; 
      else 
        V.addElement( LineIn ); 
    } 
    return null; 
  } 

  public cArticle getArticle( String ID ) throws IOException { 
    cArticle NG = new cArticle(); 
    String LineIn; 
    boolean bBad = false; 

    //- Default the Newsgroup 
    int Location    = cArticle.UK; 
    int Type        = cArticle.PERMANENT; 

    //- Get the Header 
    sendMessage( "HEAD " + ID ); 
    while ( (LineIn=In.readLine()) != null ) 
    { 
        if ( LineIn.charAt(0) == '.' ) 
          break; 
        else if ( LineIn.indexOf("423") == 0 ||  
         LineIn.indexOf("430") == 0 ) 
          return null; 
        else if ( LineIn.indexOf("Subject:") != -1 ) 
          NG.Subject = LineIn.substring( LineIn.indexOf("Subject:")+9,LineIn.length() ); 
        else if ( LineIn.indexOf("Date:") != -1 ) { 
          try{ 
            NG.EntryDate=new  Date(LineIn.substring(LineIn.indexOf 
             ("Date:")+6,LineIn.length())).getTime(); 
          }catch( Exception E ){ 
            NG.EntryDate = new Date().getTime(); 
          } 
        } 
        else if ( LineIn.indexOf("From:") != -1 ){ 
          NG.Author = LineIn.substring( LineIn.indexOf("From:")+6,LineIn.length() ); 
          NG.Author = NG.Author.replace( '"', ' ' ); 
        } 
        else if ( LineIn.indexOf("Lines:") != -1 ){ 
          try{ 
            int Lines=Integer.parseInt(LineIn.substring(LineIn. 
               indexOf("Lines:")+7,LineIn.length())); 
            if ( Lines > 120 ) 
              bBad = true; 
          }catch(Exception E){} 
        } 
    } 

    //- Get the Body 
    sendMessage( "BODY " + ID ); 
    NG.vBody = new Vector(30,5); 
    LineIn = In.readLine(); 
    if ( LineIn == null || LineIn.indexOf("222") == -1 )  return null; 

    while ( (LineIn=In.readLine()) != null ){ 
      if ( LineIn.length() != 0 && LineIn.charAt(0) == '.' && LineIn.length() == 1 ) 
        break; 
     else if ( LineIn.length() != 0 &&  
          LineIn.indexOf("Content-type: ") != -1 ){ 
        if ( LineIn.indexOf("text/plain") != -1 ) 
          NG.vBody.addElement( LineIn + "\n" ); 
        else 
          bBad = true; 
      } 
      else if ( LineIn.length() != 0 && LineIn.indexOf("begin 600") != -1 ) 
        bBad = true; 
      else if ( bBad == false ) 
        NG.vBody.addElement( LineIn + "\n"); 
    } 

    if ( bBad ) return null; 
    NG.Newsgroup = Location + Type + cArticle.USENET; 
    return NG; 
  } 

  public void postArticle( String _NG, String _from, String _Subject, Vector _Body )  
     throws IOException 
	 { 
    String LineIn; 

    //- Get the Header 
    sendMessage( "POST" ); 
    LineIn = In.readLine(); 
    if ( LineIn == null || LineIn.indexOf("340") == -1 ) return; 

    //- Post the Header fields 
    sendMessage( "Path: n-ary" ); 
    sendMessage( "From: " + _form ); 
    sendMessage( "Newsgroups: " + _NG ); 
    sendMessage( "Subject: " + _Subject ); 
    sendMessage( "Message-ID: <" + System.currentTimeMillis() + "@n-ary.com>" ); 
    sendMessage( "Date: " + new SimpleDateFormat( "dd MMM yy HH:mm:ss").format(new Date()) + " GMT"); 
    sendMessage( "Organization: N-ARY Limited" ); 
    sendMessage( "Reply-To:" + _from + "\r\n" ); 

    //- Post the Body 
    Enumeration E = _Body.elements(); 
    while ( E.hasMoreElements() ) 
      sendMessage( (String)E.nextElement() ); 

    sendMessage( "\r\n.\r\n" ); 
    In.readLine(); 
  } 
}


 

All Rights Reserved
Copyright ©  2004 SYS-CON Media, Inc.
  E-mail: [email protected]

Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.