Enabling A File System As A Transactional Resource
Building An Adapter
Transactional support is fundamental to application development.
While most data sources are transactional in nature,
some data sources like the file system are not.
In typical J2EE deployments, applications often need to interface
with other applications through file-based messages. Such
deployments would benefit from an adapter that would buffer all
file operations and provide a transactional view to the file system.
This article, targeted at developers, demonstrates how J2EE
Connector Architecture and JTA specifications can be implemented
to build such an adapter, XAFileConnector. We'll also
suggest enhancements to extend the adapter functionality in
light of the new proposed draft connector specifications.
Most applications are bound to the local file system. They
use the file system to store and share data. Often the success
of interactions with the file system needs to be tied to the
successful completion of other actions involving other
Enterprise Information Systems (EIS). For instance, we often
encounter situations where a record is to be deleted from the
database only after it has been successfully written to a file.
While the same can be easily built into the application logic,
it would be infinitely more neater if we could develop an
extension on the file system that would help us include the
file operations such as "create", "read", "update", and "delete"
as part of a transactional unit of work that could be committed
or rolled back as a group. This extension is called a
Resource Adapter. The ubiquitous presence of the file system
and the role it plays in Enterprise Application Integration
strategy justify the need to build such a resource adapter.
For a J2EE-managed environment, i.e., an environment characterized
by the presence of a container for runtime support,
J2EE recommends Connector Architecture as a standard way of
integrating heterogeneous EISs with the application server. The
Connector Architecture specifies application- and system-level
contracts to be implemented by the EIS. The resource adapter
implements the EIS side of the contracts. The adapter uses a
native interface specific to the underlying EIS, runs in the containers
address space, and manages access to the EIS resources.
Adherence to the Connector contracts on one hand ensures that
the same adapter can be deployed on different vendors' containers,
while ensuring that application and tool developers have a
standard interface to program to for accessing different EIS systems.
Among the set of contracts that need to be implemented by
the resource adapter to seamlessly plug-in to any J2EE platform
is the XAResource transaction contract. This contract defines the
communication interfaces between the parties involved in a distributed
transaction. Implementation of this interface by the
adapter enables seamless propagation of transaction context and
allows the associated resource manager (RM) to participate in a
two-phase commit protocol along with other resources.
With this introduction we proceed to discuss what these
contracts are and what it takes to implement these contracts
to build an adapter.
Understanding and Implementing the Contracts
The application contract manifests itself as a Common
Client Interface (CCI). CCI is the client side of the adapter
and performs the following roles:
Specifies how application components connect to the EIS
through a resource adapter
Provides an API for coding EIS function calls and retrieving
results
Connecting to the EIS
ConnectionFactory is the application's gateway to the EIS. It's
instantiated at the time of deployment by the container and a reference
to it, or its serialized representation, is bound to the JNDI
namespace. Later the application components do a JNDI lookup
to get hold of the factory to create connection objects to the EIS.
The ConnectionFactory does not represent the actual connection
repository, nor is the connection the actual physical connection
to the EIS. Both are client-side proxies that nevertheless maintain
handles to the actual factory and connection objects, respectively.
The container plays the role of an intermediary between the real
objects and their proxies, and the mechanism allows the container
to inject value-added services by intercepting calls made on
the proxies before they're actually delegated to the real objects.
When a ConnectionFactory encounters a getConnection
request, it delegates it to the ConnectionManager instance.
public Connection getConnection()
throws ResourceException {
return (Connection)connMgr.allocateConnection
(mngdConnFactory, new ConnectionRequestInfoImpl());
}
In a managed environment the ConnectionManager
implementation is provided by the container. The
ConnectionManager maintains a pool of connections corresponding
to each factory. The ConnectionManager checks
whether it can service the request from the pool. If not, it
requests the real factory (ManagedConnectionFactory) to
create a new connection.
How does the container know about the availability status
of a connection?
The Connector Architecture prescribes an elaborate mechanism
of listeners to implement callbacks. Whenever a new connection
is created, the container registers a listener with it. The
connection raises event notifications to intimate the container
about the happenings on the connection. When a close is
called on the connection proxy it terminates its association
with the physical connection. The physical connection in turn
generates an event. All the listeners registered on the connection
can react to the event. The container on its part uses the
event to change the availability status of the connection from
in-use to available, i.e., if the connection is not participating in
a transaction. If the connection is participating in a transaction,
the container waits for the transaction to commit before it
can make the connection available (see Listing 1).
Invoking EIS Functions
An InteractionSpec implementation holds properties for
driving an interaction with an EIS instance. In our case each
property maps to a File Operation that may be performed.
public static final int CREATE = 10;
public static final int UPDATE = 20;
public static final int DELETE = 30;
public static final int MOVE = 40;
public static final int SKIP = 50;
public static final int READ = 60;
Since CCI is EIS independent it cannot use any of the EIS
data structures. To get over this, CCI introduces the concept
of a record. A record is a generic representation of data that's
exchanged with the EIS. More specific implementations to
represent hierarchical, tabular data collections can also be
implemented. XAFileConnector extends the MappedRecord,
which is a key value representation of record elements for
both input and output records (see Listing 2).
CCI defines an interaction interface that allows a client to
interact with EIS. An interaction represents a single communication
with the EIS, an EIS function call. An interaction
instance is obtained from a connection. The interaction
instance is required to maintain its association with the connection
instance. The interaction delegates its execution to
the connection, which in turn delegates its execution to the
associated ManagedConnection (see Listing 3).
Transaction Contract
A transaction as we know it is a set of operations performed
in such a way that the state of the system after the
transaction is either the cumulative effect of state changes due
to successful individual operations or is the initial state before
the commencement of the transaction in case one or more
operations happens to fail. Transactions can be classified into
two types based on the number of resources participating.
1. A local transaction performs operations on a single RM.
The local transactions can be further classified based on
the manner of demarcating transaction boundaries.
javax.resource.cci.LocalTransaction: These run under
the control of the RM. The transaction object can be
obtained from the connection object and used to
demarcate transaction boundaries somewhat analogous
to the way JDBC setAutoCommit(false) and commit APIs
are used to demarcate transaction boundaries.
javax.resource.spi.LocalTransaction: Work in this type
of transaction can accrue over multiple client components.
A client component may do some work on a
resource and pass control to another component, which
may again do some work while still being part of the
same transaction. The transaction context is passed
along with the control, and it contains the following
information:
- A reference to the transaction manager (TM), the
external entity that is used to demarcate transaction
boundaries
- A globally unique transaction identifier, XID,
generated by the TM
- A transaction timeout time
2. Global/distributed transactions - work in this type of
transaction can span multiple RMs. The transaction manager
demarcates the transaction boundaries. A TM coordinates
the activities on the participating RMs using the
XAResource interface. An RM knows only about the work it
does for its transaction branch. The XAResource interface
implements the two-phase commit protocol between the
RMs and the TM. The XAResource interface allows for a
one-phase optimization in case only one resource is participating.
With this brief introduction to transactions we'll start with
how we can incorporate transactional ability into our file system
adapter.
For a file system adapter to be able to roll back a transaction,
i.e., revert to state prior to the start of a transaction, an adapter
must either memorize the original state or it must be able to get
back to the original state by performing certain anti-operations
that reverse the effect of the operations. To ensure feasibility and
consistency at all times, the anti-operations have to be performed
in the order that is exactly the reverse order in which the
operations were performed. Similarly, committing a transaction
should flush all transactional logs to free all memory.
For each file operation on the connection that modifies
the state of the file system (nonread only call), XAFileConnector adds a WorkItem to the associated transaction's
work list. If there's no active transaction associated with the
connection, it autocommits the work and no WorkItem is
created. The WorkItem stores enough information about the
associated operation to be able to reverse its effect in case of
a rollback or to do a cleanup of the backup in case of a commit.
A work area is designated to store the information needed
for restoration in the event of a rollback (see Listing 4).
Note how MOVE and CREATE file operation do not have
any associated cleanup during operation commit while
UPDATE and DELETE operations delete the backups.
Depending on the kind of transaction running,
javax.resource.cci.LocalTransaction, javax.resource.spi.
LocalTransaction, or XAResource keeps track of the work
being done in the transaction impending completion.
CCI Local Transaction implementations are obtained
directly from the connection, without the container having
any role to play. Since the container is not directly involved, it
is not intimated of the state of the transaction, so CCI implementations
of local transactions differ from SPI implementations
in the sense that they need to raise event notifications to
apprise the container of transaction life-cycle events. The
events help the container in connection pool management.
Since there's only a marginal difference in functionality, the
same class is used to implement both forms of local transactions
with a flag identifying the type of transaction the current
instance represents.
Our implementation model exhibits the following relationships
shown in Figure 1.
Figure 1
Each ManagedConnection can provide an XAResource
implementation for distributed transaction management. The
beginTransaction call associates the application component's
thread of control with a global transaction. The TM generates a
globally unique XID and calls start on all open RMs that are
linked with the thread. The RMs are enlisted with the transaction.
To guarantee scalability of the Connector Architecture, it
recommends that a physical connection and hence the associated
XAResource can participate in multiple transactions, but
at any point only one of these transactions can be active. To
ensure this, the start call first checks that the associated connection
isn't running a local transaction nor is the XAResource
instance associated with an active transaction. Once assured,
it associates the passed XID with the XAResource instance and
activates the transaction. All work done henceforth on this
physical connection, until the transaction is suspended or
completed, accumulates against the active transaction.
The start call is accompanied by the following flags:
TMNOFLAGS: Indicates that it is a new transaction. A call
to getTransactionState with flag true creates new entries
for the new transaction and initializes the instance.
TMRESUME: Indicates a suspended transaction is being
resumed. A call to getTransactionState with flag false
means no new entries need to be made. The state of the
transaction is retrieved and the current instance is initialized
with that state. The state of the transaction is also
changed from SUSPENDED to NOT SUSPENDED.
TMJOIN: A two-phase optimization, the TM, when it
detects that a particular RM instance is already participating
in a transaction, instead of creating a new branch
decides to merge all work done on a single RM against a
single XID. By doing this it saves on running the two-phase
protocol on all branches separately. No new entries have to
be made and the state of the transaction remains throughout
NOT SUSPENDED (see Listing 5).
The application component calls commit to make the
effects of the transaction permanent. The TM first calls end
for each involved RM, from the AP's thread of control, to dissociate
the thread from the global transaction. The TM then
executes the two-phase commit protocol.
STAGE 1 is the prepare stage or the voting stage. TM calls
prepare for each RM that was associated with the global
transaction. RMs express their readiness to commit the
transaction. A single negative vote ensures rollback. A prepare
attempt on a suspended transaction would throw back a
protocol exception. Similarly it throws back an exception,
albeit a different one, when the transaction has been marked
earlier for rollback due to internal errors. Prepare stage also
offers a two-phase optimization. A transaction that has not
changed the state of the RM doesn't need to go through the
second phase if prepare returns XA_RDONLY (see Listing 6).
Stage 2 is the commit stage. If all RMs return success from
prepare, the TM records a decision to commit the transaction
and calls commit for each RM. The XA specification allows
for an optimization in this procedure. If the TM has dealt
with only one subordinate RM in the global transaction, it
can omit Phase 1 by directly calling commit with the
onePhase flag set to true (see Listing 7).
Deployment Process
Before we can start using the adapter, we need to deploy
it. All adapter interfaces, classes, native libraries, etc., along
with the Deployment Descriptor (DD), are packaged in a .rar
archive. In situations where multiple J2EE applications need
to share the XAFileConnector, it can be deployed directly into
an application server as a standalone unit. However, the
resource adapter module may also be bundled with a J2EE
application, if it is needed only by a single application.
The Deployment Descriptor provides a lot of information
to the container, including:
General information such as name and version of adapter,
vendor name, licensing requirements, etc.
Names of concrete classes for certain core interfaces; for
example, we have to provide the name of the class that
implements the ManagedConnectionFactory Interface.
Information about the kind of support built into the
adapter. This allows the container to perform certain optimizations.
XAFileConnector offers full support for
XATransaction. No authentication support is needed as all
native calls to connect to the resource, i.e., the local file
system, are made in the current user's context.
Configurable properties that are dependent on the deployment
environment. The property naming conventions follow
the JavaBean standard. XAFileConnector uses a property by
the name of workingFolder to store the name of the folder
where transaction logs of the running transactions are kept.
The deployer sets this property at the time of deployment by
invoking the corresponding setter method. The method
checks for the existence of the folder and sufficient access
permissions before returning (see Listings 8 and 9).
To be able to successfully deploy an adapter, a better
understanding of the deployment process is essential. We will
briefly walk through the responsibilities of the deployment
code and how it interacts with the adapter code.
The deployment code looks at the DD to find out which
class implements the ManagedConnectionFactory
Interface. It dynamically creates an instance of the
same and configures properties on the instance.XAFile The
Connector uses only one property workingFolder as discussed
earlier.
In a managed environment the deployment code instantiates
a ConnectionManager specific to the container and obtains a
ConnectionFactory instance from ManagedConnectionFactory,
as shown in the following implementation:
public Object createConnectionFactory(ConnectionManager cxManager)
throws ResourceException
{
return new ConnectionFactoryImpl(this, cxManager);
}
Note how the ConnectionManager instance is
passed around while creating the ConnectionFactory.
The mechanism helps associate the two factories and
the ConnectionManager. The ConnectionManager is
the ConnectionFactory's interface to the container.
The deployment code then binds the ConnectionFactory
instance in the JNDI namespace from where it's later
looked up by the application component.
Writing a Client
Clients needing to interact with the file system will have
to use the CCI interface. They cannot use the java.io package
to access the file system if they want the transactional support.
While the approach may look nonintuitive and tiresome,
which in fact it is, it is not as big a pain as you would
imagine it to be.
By this time we would have understood how to acquire a
connection to the file system.
Once connected we can obtain an interaction from the
connection.
Interaction interaction = conn.createInteraction();
Interaction maintains an association with the connection
from which it was created and executes on the same connection.
A connection object also serves as the source for
RecordFactory. RecordFactory hands out all types of
records. Records are the means of exchanging data, both
incoming and outgoing, with the files system.
RecordFactory rf = cf.getRecordFactory();
MappedRecord in = rf.createMappedRecord("IN");
MappedRecord out = rf.createMappedRecord("OUT");
Assuming that we wish to create a file, the only parameters
we would need to pass are the folder name in which to
create the file and the file name. Initialize the records with
the input parameters.
in.put("SOURCE_DIR", "E:\\bea\\weblogic700\\FILES1");
in.put("SOURCE_FILENAME", "create.txt");
Now the only information that needs to be passed is our
intention of actually creating a file. This is specified when
we instantiate an object of the class InteractionSpecImpl.
interaction.execute(new
InteractionSpecImpl(InteractionSpecImpl.CREATE), in, out);
Handle the exceptions and examine the OUT record for
returned values.
Close all open resources.
What Further?
The proposed draft connector specification makes provisions
for inbound communication through a Message Inflow
Contract and also includes a Work Management Contract.
The inclusion of the two would make it feasible for the
adapter to work as a poller.
The Work Management Contract would allow the adapter
to monitor folders for incoming files. The activity can be performed
by submitting work to the container. The contract
would save the adapter the task of creating its own threads
thus allowing the container to exercise better control over its
runtime environment.
The Message Inflow Contract would allow the adapter to
trigger action in the event of receiving a file. The situation
can be thought of as analogous to a JMS situation, with the
adapter instead of an MDB delivering messages by polling on
a folder instead of a queue.
Better concurrency control can also be built into the
adapter using the new improved I/O support found in newer
Java runtimes, 1.4 onwards.
References
JTA specifications: http://java.sun.com/products/jta
JCA specifications: http://java.sun.com/j2ee/'connector/download.html
Author Bios
Ashish Garg is a senior technical specialist with Infosys Technologies Ltd. He has
more than four years of experience in J2EE technologies.
ashish_garg@infosys.com
Sandip Mane is a senior technical specialist with Infosys Technologies Ltd. and specializes in WebLogic
msandip@infosys.com
"Enabling A File System As A Transactional Resource"
Vol. 8, Issue 11, p. 28
Listing 1
public void close() throws ResourceException
{
if (mConn != null)
{
mConn.sendEvent(ConnectionEvent.CONNECTION_CLOSED,
null, this);
if (localTransaction != null &&
localTransaction.inTransaction) {
localTransaction.rollback();
localTransaction = null;
}
mConn = null;
}
return;
}
Listing 2
public class FSInRecord extends HashMap
implements MappedRecord
{
static final String SOURCE_DIR
= new String("SOURCE_DIR");
static final String SOURCE_FILENAME
= new String("SOURCE_FILENAME");
static final String DESTINATION_DIR
= new String("DESTINATION_DIR");
static final String DESTINATION_FILENAME
= new String("DESTINATION_FILENAME");
static final String DATA
= new String("DATA");
static final String SIZE
= new String("SIZE");
public FSInRecord(String fileName,
String newName, String sourceDir,
String destinationDir, int size)
{
this.put(SOURCE_DIR, sourceDir);
this.put(SOURCE_FILENAME, fileName);
this.put(DESTINATION_DIR, destinationDir);
this.put(DESTINATION_FILENAME, newName);
this.put(SIZE, new Integer(size));
}
...
...
}
Listing 3
boolean execute(InteractionSpec ispec, Record input,
Record output) throws ResourceException
{
FSOutRecord out = (FSOutRecord)output;
FSInRecord in = (FSInRecord)input;
String sourceDir = (String)in.get(FSInRecord.SOURCE_DIR);
sourceDir = (sourceDir != null &&
sourceDir.endsWith(File.separator)) ? sourceDir :
sourceDir + File.separator;
String sourceFileName =
(String)in.get(FSInRecord.SOURCE_FILENAME);
String destDir =
(String)in.get(FSInRecord.DESTINATION_DIR);
destDir = (destDir != null && destDir.endsWith(File.separator))
? destDir : destDir + File.separator;
String destFileName = (String)in.get(FSInRecord.DESTINATION_
FILENAME);
String destFile;
if((sourceDir == null) || (sourceDir.equals("")))
throw new ResourceException("Invalid File Directory");
if((sourceFileName == null) ||
(sourceFileName.equals("")))
throw new ResourceException("Invalid File Name");
String sourceFile = new String(sourceDir +
sourceFileName);
if(sourceDir.equals(workingFolder))
throw new ResourceException("Invalid File Directory. It
can not be same as working folder directory.");
try
{
switch(((InteractionSpecImpl)ispec).getAction())
{
case InteractionSpecImpl.CREATE: {
File file = new File(sourceFile);
boolean success = file.createNewFile();
if(!success) throw new ResourceException("File could
not be created");
addWork(sourceDir, sourceFileName, null, null,
WorkItem.CREATE);
break;}
case InteractionSpecImpl.DELETE: {
destFileName = sourceFileName + new
java.util.Date().getTime();
destFile = new String(workingFolder + destFileName);
closeFile(sourceFile);
File destFileHandle = new File(destFile);
File sourceFileHandle = new File(sourceFile);
boolean status =
sourceFileHandle.renameTo(destFileHandle);
if(! status)
throw new ResourceException("File can not be deleted");
addWork(sourceDir, sourceFileName, workingFolder,
destFileName, WorkItem.DELETE);
break;}
case InteractionSpecImpl.MOVE: {
if((destDir == null) || (destDir.equals("")))
throw new ResourceException("Invalid Destination File
Directory");
destFile = destFileName != null ? destDir +
destFileName : destDir + sourceFileName;
closeFile(sourceFile);
File destFileHandle = new File(destFile);
File sourceFileHandle = new File(sourceFile);
boolean status =
sourceFileHandle.renameTo(destFileHandle);
if(! status)
throw new ResourceException("File can not be moved");
String renameTo = destFileName != null ? destFileName
: sourceFileName;
addWork(sourceDir, sourceFileName, destDir, renameTo,
WorkItem.MOVE);
break;}
...
...
}
}
catch(IOException ie)
{
throw new ResourceException(ie.getMessage());
}
return true;
}
Listing 4
public WorkItem(String sourceDir, String sourceFile,
String destinationDir, String destinationFile, int
workName, ManagedConnection mConn) {
this.workName = workName;
this.sourceDir = (sourceDir.endsWith(File.separator)) ?
sourceDir : sourceDir + File.separator;
...
...
}
public void executeCommit() throws ResourceException {
switch(workName) {
case CREATE: break;
case UPDATE:
case DELETE: //delete the backed up file
String fullNameWithPath = new String(destinationDir +
destinationFile);
File toBeDeleted = new File(fullNameWithPath);
closeFile(fullNameWithPath);
if (!toBeDeleted.delete()) throw new
ResourceException("Clean Failure: failed to delete ... " +
toBeDeleted.getPath());
break;
case MOVE: break;
}
}
Listing 5
public void start(Xid xid, int flags) throws XAException {
if (mConn.localTransactionCCI != null) {
throw new XAException(XAException.XAER_OUTSIDE);
}
activate(xid);
BitSet states = null;
switch (flags) {
case TMNOFLAGS:
states = getTransactionState(associatedXID, true);
break;
case TMRESUME:
states = getTransactionState(associatedXID, false);
states = updateState(states, SUSPENDED, false);
break;
case TMJOIN:
states = getTransactionState(associatedXID, false);
break;
}
XIDStates.put(associatedXID, states);
}
Listing 6
public int prepare(Xid xid) throws XAException {
BitSet states = getTransactionState(xid, false);
boolean suspended = currentState(states, SUSPENDED);
boolean markedForRollback = currentState(states,
MARKED_FOR_ROLLBACK);
boolean readOnly = currentState(states, READ_ONLY);
if (suspended) {
throw new XAException(XAException.XAER_PROTO);
} else if (markedForRollback) {
throw new XAException(XAException.XA_RBOTHER);
} else if (readOnly) {
return XA_RDONLY;
}
states = updateState(states, PREPARED, true);
XIDStates.put(xid, states);
return XA_OK;
}
Listing 7
public void commit(Xid xid, boolean onePhase) throws
XAException {
BitSet states = getTransactionState(xid, false);
boolean prepared = currentState(states, PREPARED);
if (!onePhase) {
if (!prepared) {
throw new XAException(XAException.XAER_PROTO);
}
}
ArrayList workList = (ArrayList)XIDWork.get(xid);
executeActionsList(workList, true);
removeTransationHistory(xid);
}
Listing 8
public void setWorkingFolder(String workingFolder)
{
if (workingFolder == null ||
workingFolder.equals("")) throw new
RuntimeException("Invalid value for Working folder in the
Deployment Descriptor.");
File workingDir = new File(workingFolder);
if (!workingDir.isDirectory()) throw new
RuntimeException("Working folder specified is not a valid
Directory.");
if (!workingDir.canWrite()) throw new
RuntimeException("Working folder specified does not have
write permissions.");
this.workingFolder = workingFolder;
}
Listing 9
public void setWorkingFolder(String workingFolder)
{
if (workingFolder == null ||
workingFolder.equals("")) throw new
RuntimeException("Invalid value for Working folder in the
Deployment Descriptor.");
File workingDir = new File(workingFolder);
if (!workingDir.isDirectory()) throw new
RuntimeException("Working folder specified is not a valid
Directory.");
if (!workingDir.canWrite()) throw new
RuntimeException("Working folder specified does not have
write permissions.");
this.workingFolder = workingFolder;
}
All Rights Reserved
Copyright © 2004 SYS-CON Media, Inc.
E-mail: info@sys-con.com
Java and Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. SYS-CON Publications, Inc. is independent of Sun Microsystems, Inc.