#ifndef MxNPort_h_seen
#define MxNPort_h_seen

// We use these AVIs
#include "DistArrayDescriptor.h"

namespace gov {
  namespace cca {

	/* $Id: mxn.interface,v 1.23 2002/01/08 19:22:32 kohl Exp $ */

	/**
 	 * MxN Parallel Data Redistribution Port
 	 *
 	 * CCA MxN Working Group
 	 */

	class MxNPort : public virtual gov::cca::Port {
	public:

	/**
	 * MxN Interface
	 *
	 * The purpose of this interface is to allow CCA components to
	 * identify and exchange data elements among parallel, decomposed
	 * data objects.  One parallel component with "M" instances might
	 * want to share or correlate data with another parallel component
	 * with "N" instances.  In some cases M == N, and the data mapping
	 * is straightforward, although M and N could have different data
	 * decompositions which would require remapping the data elements.
	 * In other cases, N == 1, and the parallel data is collected into
	 * a central location, such as for most visualization purposes.
	 * When M != N, then a full "redistribution" of the parallel data
	 * objects is required, to map each element of one data distribution
	 * to a corresponding element in another data distribution.
	 *
	 * There is no implicit support for any data "munging" here,
	 * such as spatial or temporal interpolation or units conversions.
	 * These functions are outside the scope of MxN data exchange
	 * and should be implemented in another interface, likely as a
	 * "filter" service, component or port.
	 *
	 * As the CCA definitions for Data Objects becomes more complete,
	 * this MxN interface should be expected to support a wide variety
	 * of data organizations and decompositions, including both
	 * structured and unstructured meshes.  But for the purposes
	 * of this initial interface description, the "Data Object" is
	 * left open and unspecified.  Ultimately, the MxN interface
	 * will require sufficient information from a given Data Object
	 * interface to be able to:
	 *
	 * 1. Identify the owner/location of any specific data element.
	 *
	 * 2. Extract and Assign the value(s) associated with a specific
	 *     data element.
	 * 
	 * As long as all necessary information is available to support
	 * these operations, then the MxN interface can function.  Such
	 * information includes, but may not be limited to:  pointers to
	 * actual data values, the data type(s), the dimensionality and
	 * cardinality of any arrays, the allocated data size, leading
	 * dimensions and boundary/ghost regions of any locally stored
	 * subarrays, the data layout (as in Row Major or Column Major),
	 * the element ids, coordinates, and vertex displacement of values
	 * for an unstructured mesh, the spatial grid type and mapping,
	 * some instantaneous temporal reference value, the units of the
	 * data values, the processor topology, the decomposition types
	 * along each axis for structured meshes, and the details of each
	 * parallel instance's portion of a data distribution (logical
	 * processor address, block or cycle size for rectangular
	 * decompositions, and any explicit decomposition bounds or
	 * patches).
	 *
	 * Note that some of the information above may be used purely for
	 * error checking and validation purposes in the MxN interface
	 * (e.g. spatial grid and units), but this information will be
	 * essential for *other* support interfaces (such as interpolation
	 * and units conversion) that must be present in any practical
	 * implementation.
	 *
	 * Typical use of this interface will proceed as follows (see
	 * interface specification below for method argument descriptions):
	 *
	 * Parallel components A and B "decide" to exchange parallel data.
	 * How this happens is a mystery left up to the applications
	 * programmer, but suffice to say that this could be done via
	 * explicit component connections at composition time, to choose
	 * which components / data objects should be coupled.  Or this
	 * could be arranged dynamically at run-time via some programmatic
	 * control, given some method extensions to this MxN interface.
	 *
	 * To allow any sharing and exchange of parallel data, each
	 * parallel data object must be registered with an instance of
	 * the MxN implementation - say, an "MxN-er" (which could be a
	 * port on a component or a system service, etc).  Data objects
	 * can be registered by any parallel component that has access
	 * to a given MxN-er.  Once all data objects have been registered,
	 * the MxN-er can support a variety of dynamic MxN interconnections
	 * among the parallel component instances.
	 *
	 * To register a data object with the MxN-er, a parallel component
	 * must describe the data and any distributed decomposition (i.e.
	 * who owns which pieces of it) and also make the actual data
	 * elements themselves available for reading and/or writing as
	 * part of subsequent MxN transfers.  All of this information
	 * about the data objects is currently encapsulated in the MxN
	 * interface using a generic "Data" interface placeholder, while
	 * the actual data interface is under development by the CCA Data
	 * Working Group.  The "registerData()" method takes this data
	 * object information, along with some basic synchronization
	 * and access mode settings, and produces an MxN data handle
	 * for use in constructing MxN connections and transfers:
	 *
	 *   Each A:    Data Adata;
	 *              registerData( Adata, True, ReadOnly, Adhandle );
	 *
	 *   Each B:    Data Bdata;
	 *              registerData( Bdata, False, WriteOnly, Bdhandle );
	 *
	 * Similar to the base "decision" to share parallel data, one
	 * other exchange must occur to actually pass some information
	 * between the parallel components, A and B (or elsewhere to a
	 * third party) about the actual data objects that are to be
	 * selected for a given MxN connection.  One of the parallel
	 * components, on either side of the MxN connection, (or else
	 * some third party) must be responsible for specifying the source
	 * and destination data objects for a connection, and when and
	 * how the data is to actually be transferred.  This can be
	 * accomplished via explicit method invocations over user-defined
	 * ports that pass data object handles or references, or could
	 * be handled by an additional MxN interface method, such as
	 * "getRegisteredData()", that could return a list of all data
	 * handles that have been registered to the given MxN instance.
	 *
	 * After the proper data handles have been exchanged/delivered,
	 * then a "communication schedule" needs to be generated for
	 * coordinating the communication between the parallel instances
	 * of A and B.  The communication schedule is determined based
	 * on the pair of data decompositions for the given data objects.
	 * The schedule can be re-used for any similar data objects or
	 * connections.  Only one side of an MxN connection, or some
	 * third party, need set up this communication schedule.  (Either
	 * a "push" or "pull" model is possible, as is an externally
	 * controlled coordination).
	 *
	 * To be efficient, and to be amenable to typical SPMD programming
	 * style, the "createCommSched()" call is made collectively by
	 * any or all parallel instances of the coordinating component
	 * cohort.  The same opaque communication schedule handle is
	 * returned to all collective callers.  This handle references
	 * the MxN mapping information for the given pair of decompositions:
	 *
	 *  All A:    createCommSched( Bdata.decomp, Adata.decomp,
	 *                csHandle );
	 *
	 * Next, given this handle to a communication schedule and the
	 * data handles for the MxN-registered source and destination
	 * data objects, a specific MxN "connection" can be made.  The
	 * "makeConnection()" method is used to construct a parallel
	 * communication channel for moving data values from the source
	 * data object to the destination data object.  The creation of
	 * this channel includes performing a synchronization among the
	 * parallel A and B component instances, as well as setting the
	 * "frequency" of MxN transfers (only for persistent MxN channels).
	 * Note that each A and B can have their own transfer frequency
	 * to account for any significant difference in their relative
	 * iteration timings, or for algorithmic reasons (in this example,
	 * A has a frequency of "4" and B has a frequency of "5", so A
	 * will send data every 4 iterations to B's 5).  Each side can
	 * also have its own synchronization override settings, to control
	 * the overhead from expensive synchronization operations that may
	 * not be deemed necessary for a given connection (in this example,
	 * A is prevented from synchronizing because B does not require it).
	 *
	 * Like the createCommSched() method, this invocation happens
	 * collectively, on one side of the MxN connection or via some
	 * third party controller.  A connection handle is returned,
	 * for use in actually executing transfers, or in later releasing
	 * the connection:
	 *
	 *  All A:    makeConnection( Bdhandle, Adhandle, csHandle,
	 *                NoSynch, Default, 4, 5, cHandle );
	 *
	 * Subsequent to the makeConnection() call, any synchronization
	 * specified in the invocation will now be active.  The parallel
	 * instances on either side of the MxN connection will exchange
	 * a simple flow control protocol (as part of the dataReady()
	 * method execution - see below) to maintain any required
	 * synchronization of their relative "iteration counts" or timing.
	 *
	 * When an actual transfer of data elements is desired from A to B,
	 * the "requestTransfer()" method must be invoked to request the
	 * transfer.  This method can be invoked by one of the connected
	 * parallel component cohorts or by the third party component
	 * (i.e. the component that created the communication schedule
	 * and made the connection :-).  The requestTransfer() method is
	 * non-blocking and returns immediately, before the transfer is
	 * completed.  A "transfer" handle is returned for monitoring the
	 * progress of the transfer.  (The waitTransfer() and testTransfer()
	 * methods can be used to wait for or check the completion of the
	 * transfer, respectively.  However, these calls are not strictly
	 * necessary for completion of the transfer.)
	 *
	 *  All A:    requestTransfer( cHandle, tHandle );
	 *            . . .
	 *            waitTransfer( tHandle );
	 *
	 * Once the transfer request has been posted, the data elements
	 * will be communicated from source to destination as each
	 * parallel instance become "ready".  The instances of A and B
	 * must each indicate that their portion of the data object is
	 * "ready" for the transfer, i.e. the parallel data object is
	 * in some "consistent" state for reading or writing.  The
	 * "dataReady()" method is used to indicate this state.  During
	 * an invocation of dataReady(), (or between calls to the
	 * non-blocking "dataReadyBegin()" and companion "dataReadyEnd()"
	 * method), the MxN-er will process all pending transfers that
	 * involve the given data object.  (A bool flag to the
	 * dataReady() and dataReadyBegin() methods indicates whether
	 * the iteration count or "time" should be incremented for the
	 * given data object - this allows multiple transfers per iteration
	 * as may be desired.)
	 *
	 *  All A:    dataReady( Adhandle, True, ReadWrite );
	 *
	 *  All B:    dataReadyBegin( Bdhandle, True, ReadWrite );
	 *            . . .
	 *            dataReadyEnd( Bdhandle );
	 *
	 * The transfer of elements need not take place "en masse" as
	 * a fully synchronized operation.  As each "pair" of mapped
	 * parallel instances (one from A, one from B - as specified in
	 * the communication schedule) becomes "ready", the data elements
	 * mapped between them can be transferred.
	 *
	 * Note that calls to dataReady() or dataReadyEnd() will *block*
	 * under MxN "flow control" until ALL pending transfers involving
	 * the given data object are completed.  For "one-shot" connections
	 * (frequency == 0) a single pending transfer will be posted for
	 * each invocation of requestTransfer().  For "periodic" transfers
	 * (frequency > 0), a series of pending transfers will be generated
	 * from a single invocation of requestTransfer(); as the iteration
	 * count or "time" associated with the given frequency elapses
	 * (as counted by invocations of dataReady() or dataReadyBegin()),
	 * a new transfer request is automatically posted.  Calling the
	 * requestTransfer() method again for a periodic connection will
	 * reset the iteration count offset for triggering transfers at
	 * the given frequency.
	 *
	 * More details of the methods in this interface are described
	 * below.
	 */

	/**
	 * AccessMode Enumerated Type
	 *
	 * Enumerated type for specifying the read/write access to data
	 * objects that have been registered with the MxN system.
	 *
	 * Can have the values:
	 *    ReadOnly, WriteOnly and ReadWrite
	 */
	enum AccessMode { ReadOnly, WriteOnly, ReadWrite };

	/**
	 * mxnDataHandle Type
	 *
	 * Someday we may need something more here, but for now it's an int.
	 */
	typedef int mxnDataHandle;

	/**
	 * int registerData( DistArrayDescriptor * & data, bool synch,
	 * 			enum AccessMode access, mxnDataHandle &dHandle )
	 *
	 * Registers a "Data Object" (whatever that is :-) with the MxN
	 * System (a.k.a. the "MxN-er" - a generic moniker to cover the
	 * component, port and service models :-).  This makes the data
	 * object available for "MxN parallel data redistribution," i.e.
	 * when registered its elements can be extracted / overwritten
	 * (see the "access" argument) with the values transferred from
	 * another parallel distributed data object.  The owner of the
	 * data object may not want anyone to have MxN access to this
	 * data object, so the owner has to specifically register the
	 * object to allow this activity.
	 *
	 * To perform any data transfers with this data object, the MxN-er
	 * will have to know the basic data organization and layout, and
	 * how the local data allocation fits into the context of the
	 * global data decomposition.  It is assumed that the "Data" object
	 * contains this decomposition information.
	 *
	 * @param data Input the "Data" object to be registered.
	 * @param synch Input indicates whether the (potentially
	 *            distributed) data elements in "data" are inherently
	 *            "synchronized", i.e. the collection of decomposed
	 *            units constitute an overall global array that has
	 *            some periodic consistency as the data is computed
	 *            in parallel and then reconciled (usually this input
	 *            will be set to "True", unless it's an embarrassingly
	 *            parallel decomposition...).
	 * @param access Input the access mode of the data object being
	 *            registered.  Can be ReadOnly, WriteOnly or ReadWrite
	 *            to dictate whether the registered data object can
	 *            be read, written or both by the MxN-er.
	 * @param dHandle Output a handle to this data object registration,
	 *            for subsequent use in unregistering the data via the
	 *            unregisterData() method.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int registerData( DistArrayDescriptor * & data, bool synch,
			enum AccessMode access, mxnDataHandle &dHandle ) = 0;

	/**
	 * int unregisterData( mxnDataHandle dHandle )
	 *
	 * Unregister a previously registered data object.  This takes
	 * the given data object "out of play" for MxN and breaks any
	 * existing MxN connections to the data object.  (It would be
	 * "courteous" of the user to explicitly break any MxN data
	 * connections using the releaseConn() method, but it is not
	 * strictly necessary.)
	 *
	 * @param dHandle Input the handle returned by registerData().
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int unregisterData( mxnDataHandle dHandle ) = 0;

	/**
	 * (Potential Method...?)
	 * int reregisterData( mxnDataHandle dHandle,
	 * 			DistArrayDescriptor *data, bool synch,
	 * 			enum AccessMode access )
	 *
	 * Re-register a previously registered data object.  This could
	 * invalidate existing MxN connections, depending on any changes
	 * to the data object's registration.  It is expected that any
	 * such changes will be handled by a call to modifyConn(), else
	 * the next requestTransfer() invocation will fail.
	 *
	 * @param dHandle Input the handle returned by registerData().
	 * @param data Input the "Data" object to be re-registered.
	 *            This re-registration allows changes in the data
	 *            object's size and any distributed decomposition.
	 * @param synch Input the new synchronization property of the
	 *            registered data object (see registerData()).
	 * @param access Input the new access mode of the registered data
	 *            object (see registerData()).
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int reregisterData( mxnDataHandle dHandle,
			DistArrayDescriptor *data, bool synch,
			enum AccessMode access ) = 0;

	/**
	 * mxnCommSchedHandle Type
	 *
	 * Someday we may need something more here, but for now it's an int.
	 */
	typedef int mxnCommSchedHandle;

	/**
	 * int createCommSched( DistArrayDescriptor *dst_decomp,
	 * 			DistArrayDescriptor *src_decomp,
	 * 			mxnCommSchedHandle &csHandle )
	 *
	 * Create a Communication Schedule for use in MxN Parallel Data
	 * Transfer.  The schedule is generated based on the mapping
	 * between the two given parallel data decompositions, and an
	 * opaque handle is returned which can subsequently be passed
	 * to the makeConnection() method to actually initiate an MxN
	 * communication "channel".  This method is called "collectively"
	 * by all parallel instances in a given cohort.
	 *
	 * @param dst_decomp Input the data decomposition information for
	 *            the destination data objects.
	 * @param src_decomp Input the data decomposition information for
	 *            the source data objects.
	 * @param csHandle Output a handle to identify the communication
	 *            schedule or mapping between the two decompositions.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int createCommSched( DistArrayDescriptor *dst_decomp,
			DistArrayDescriptor *src_decomp,
			mxnCommSchedHandle &csHandle ) = 0;

	/**
	 * int freeCommSched( mxnCommSchedHandle csHandle )
	 *
	 * Free the given Communication Schedule, as previously created
	 * by a call to createCommSched().  (Also a collective call.)
	 *
	 * @param csHandle Input the handle to the communication schedule
	 *            to be freed.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int freeCommSched( mxnCommSchedHandle csHandle ) = 0;

	/**
	 * mxnSynch Enumerated Type
	 *
	 * Enumerated type for specifying the synchronization required
	 * when making a MxN channel connection, as specified in the
	 * makeConnection() method.  When not "Default", can be
	 * used to force a specific connection synchronization type.
	 *
	 * Can have the values:
	 *    Default, Synch and NoSynch
	 */
	enum mxnSynch { Default, Synch, NoSynch };

	/**
	 * mxnConnHandle Type
	 *
	 * Someday we may need something more here, but for now it's an int.
	 */
	typedef int mxnConnHandle;

	/**
	 * Time Type
	 *
	 * This should really be its own object / AVI.
	 * But int is close enough for SC...  :-)
	 */
	typedef int Time;

	/**
	 * int makeConnection( mxnDataHandle dst, mxnDataHandle src,
	 *         mxnCommSchedHandle csHandle,
	 *         enum mxnSynch dst_synch, enum mxnSynch src_synch,
	 *         Time dst_freq, Time src_freq,
	 *         mxnConnHandle &cHandle )
	 *
	 * Create a connection or "communication channel" between two
	 * parallel data objects, as specified using two data handles
	 * returned by regsiterData() and a corresponding "communication
	 * schedule" returned by createCommSched().  This connection
	 * is to be used for an MxN Transfer of data elements from the
	 * "source" data object to the "destination" data object.
	 * This routine does *not* actually move any data, but rather
	 * sets up the desired MxN communication channel (and any
	 * synchronization) for use by the requestTransfer() method.
	 *
	 * @param dst Input handle for the destination data object.
	 * @param src Input handle for the source data object.
	 * @param csHandle Input the handle (as returned by the
	 *            createCommSched() method) to the communication
	 *            schedule or mapping between the two data
	 *            decompositions, to be used to perform this MxN
	 *            Transfer.
	 * @param dst_synch Input indicates whether a synchronization
	 *            is required among the instances of the parallel
	 *            cohort at the destination side of the connection.
	 *            The purpose of this synchronization is to make sure
	 *            that "consistent" collections of elements are
	 *            transferred.  If set to "Default", then the
	 *            synchronization type specified for the destination
	 *            data object (in the call to regsiterData()) will
	 *            be used for all synchronization requirements.
	 *            If *not* set to "Default", then dst_synch will
	 *            *override* the default destination data object
	 *            synchronization settings and either force / not force
	 *            synchronization at the destination throughout the
	 *            lifetime of the connection.  A synchronization type
	 *            of "Synch" implies that a loose synchronization will
	 *            be maintained among the given parallel instances
	 *            of the cohort.  If "NoSynch" then each of the
	 *            parallel instances can execute independently,
	 *            transferring their data elements as needed without
	 *            any synchronization within the cohort.
	 *            (Synchronization here is determined by common
	 *            iteration counts on all instances of a given
	 *            parallel cohort.)
	 * @param src_synch Input indicates whether a synchronization
	 *            is required among the instances of the parallel
	 *            cohort at the source side of the connection.
	 *            (Same as dst_synch, but at the other side of the
	 *            connection. :-)
	 * @param dst_freq Input the periodic "frequency" at which each
	 *            specific data transfer is to occur at the destination
	 *            data object.  If freq is 0, then this is a "one-shot"
	 *            transfer and no subsequent transfer requests will be
	 *            issued automatically after the initial request is
	 *            posted (see requestTransfer()).  If freq is > 0,
	 *            then this informs the MxN-er to automatically generate
	 *            new transfer requests at the given frequency after
	 *            the initial transfer is posted.  The frequency can
	 *            either be a scalar that increments some local
	 *            iteration count in each parallel instance, or can
	 *            represent some notion of simulated "time" associated
	 *            with each instance of the given data object.
	 * @param src_freq Input the periodic "frequency" at which each
	 *            specific data transfer is to occur at the source
	 *            data object.  Typically the same as "dst_freq",
	 *            but can be used to prevent a slowdown between two
	 *            parallel components with dramatically different
	 *            iteration timings.  The dst_freq and src_freq
	 *            parameters must either *both* be equal to 0, or else
	 *            *both* greater than zero - "one-shot" and "periodic"
	 *            connections cannot be "mixed".
	 * @param cHandle Output a handle for this MxN connection, as
	 *            needed for actual MxN transfer requests over this
	 *            communication channel (via requestTransfer()),
	 *            or to release the communication channel (via
	 *            releaseConn()) when no more transfers are needed.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int makeConnection( mxnDataHandle dst, mxnDataHandle src,
			mxnCommSchedHandle csHandle,
			enum mxnSynch dst_synch, enum mxnSynch src_synch,
			Time dst_freq, Time src_freq,
			mxnConnHandle &cHandle ) = 0;

	/**
	 * int releaseConn( mxnConnHandle cHandle )
	 *
	 * Release the current MxN data transfer connection: cease any
	 * loose synchronization that the MxN-er may be maintaining,
	 * release any flow control between the source and destination
	 * data objects, and clean up any internal state / storage for
	 * this particular parallel data transfer channel.
	 *
	 * @param cHandle Input the data connection handle returned by
	 *             makeConnection().
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int releaseConn( mxnConnHandle cHandle ) = 0;

	/**
	 * (Potential Method...?)
	 * int modifyConn( mxnConnHandle cHandle,
	 *         mxnCommSchedHandle csHandle,
	 *         enum mxnSynch dst_synch, enum mxnSynch src_synch,
	 *         Time dst_freq, Time src_freq )
	 *
	 * Modify a connection or "communication channel" between two
	 * parallel data objects, as created by a call to makeConnection().
	 * This routine updates the properties of the given connection
	 * without breaking any ongoing synchronization.
	 *
	 * @param cHandle Input the data connection handle returned by
	 *             makeConnection().
	 * @param csHandle Input the handle (as returned by the
	 *            createCommSched() method) to the new communication
	 *            schedule or mapping between the two data
	 *            decompositions, to be used to perform subsequent MxN
	 *            Transfers.
	 * @param dst_synch Input indicates the new synchronization property
	 *            for the destination side of the connection (see
	 *            makeConnection()).
	 * @param src_synch Input indicates the new synchronization property
	 *            for the source side of the connection (see
	 *            makeConnection()).
	 * @param dst_freq Input the new periodic "frequency" at which each
	 *            specific data transfer is to occur at the destination
	 *            data object (see makeConnection()).
	 * @param src_freq Input the new periodic "frequency" at which each
	 *            specific data transfer is to occur at the source
	 *            data object (see makeConnection()).
	 * @return execution status (0 if OK, !=0 if error)
	 */
	/*
	 * virtual int modifyConn( mxnConnHandle cHandle,
	 * 		mxnCommSchedHandle csHandle,
	 * 		enum mxnSynch dst_synch, enum mxnSynch src_synch,
	 * 		Time dst_freq, Time src_freq ) = 0;
	 */

	/**
	 * mxnTransferHandle Type
	 *
	 * Someday we may need something more here, but for now it's an int.
	 */
	typedef int mxnTransferHandle;

	/**
	 * int requestTransfer( mxnConnHandle cHandle,
	 *         mxnTransferHandle &tHandle )
	 *
	 * Request the execution of an actual MxN Transfer of data elements
	 * over the given communication channel, from the "source" data
	 * object to the "destination" data object.  Once the request for
	 * the transfer has been posted, the transfer will occur when
	 * each parallel instance, on both sides of the connection,
	 * has called "dataReady()" to indicate the readiness of the data
	 * object to have data elements read or written.
	 *
	 * This operation assumes that there is *no* discrepancy between
	 * the global coordinate system, or the global grid/mesh associated
	 * with "src" and "dst".  (This is *not* the place for spatial or
	 * temporal interpolation or units conversions - do that somewhere
	 * else, before or after the MxN parallel data exchange, using a
	 * "filter" component/port. :-)
	 *
	 * Invocation of the requestTransfer() method is a non-blocking
	 * call that returns immediately, before the transfer completes.
	 * The "tHandle" transfer handle output argument can be passed to
	 * waitTransfer() or testTransfer() to wait for or determine if
	 * the transfer has completed, respectively.
	 *
	 * @param cHandle Input the connection handle for the desired
	 *            communication channel.
	 * @param tHandle Output a handle for this data transfer, as needed
	 *            to check completion of the data transfer via the
	 *            waitTransfer() and testTransfer() methods.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int requestTransfer( mxnConnHandle cHandle,
			mxnTransferHandle &tHandle ) = 0;

	/**
	 * int waitTransfer( mxnTransferHandle tHandle )
	 *
	 * Wait until the current MxN data transfer is completed.
	 * This is a blocking method call that doesn't return until
	 * the transfer completes or an error is encountered.
	 *
	 * @param tHandle Input the data transfer handle returned by
	 *            requestTransfer().
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int waitTransfer( mxnTransferHandle tHandle ) = 0;

	/**
	 * int testTransfer( mxnTransferHandle tHandle )
	 *
	 * Test if the current MxN data transfer is completed.
	 * This is a non-blocking method call that returns immediately
	 * with a status code that indicates whether the transfer has
	 * completed or not, or whether an error has occurred.
	 *
	 * @param tHandle Input the data transfer handle returned by
	 *            requestTransfer().
	 * @return execution status (1 if completed, 0 if not, else error)
	 */
	virtual int testTransfer( mxnTransferHandle tHandle ) = 0;

	/**
	 * int dataReady( mxnDataHandle dHandle, bool update_time,
	 *            AccessMode access )
	 *
	 * Set the MxN connection status to "Ready" for the data object
	 * referred to by this MxN data handle.  Any pending data transfers
	 * that have been requested can now be executed (assuming that the
	 * other side of the given connection is also in a "Ready" state).
	 * The dataReady() method is a blocking call that will not return
	 * until all pending data transfers have been processed.
	 *
	 * @param dHandle Input the data handle to the MxN-registered data
	 *            object that is ready for MxN data transfer.
	 * @param update_time Input indicates whether the internal iteration
	 *            count or "time" value for this data object should be
	 *            incremented as part of this transfer cycle.  Setting
	 *            this value to "False" allows multiple parallel data
	 *            transfers within a single iteration, as needed.
	 * @param access Input the access mode being activated for the
	 *            given data object.  Can be ReadOnly, WriteOnly or
	 *            ReadWrite to dictate whether the data object can
	 *            now be read, written or both by the MxN-er to
	 *            execute parallel data transfers.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int dataReady( mxnDataHandle dHandle,
			bool update_time, AccessMode access ) = 0;

	/**
	 * int dataReadyBegin( mxnDataHandle dHandle, bool update_time,
	 *            AccessMode access )
	 *
	 * Same as dataReady(), but in a Begin / End set of methods that
	 * allow other processing while the pending data transfers are
	 * in progress.
	 *
	 * Sets the MxN connection status to "Ready" for the data object
	 * referred to by this MxN data handle.  Any pending data transfers
	 * that have been requested can now be executed (assuming that the
	 * other side of the given connection is also in a "Ready" state).
	 * The dataReadyBegin() method is a non-blocking call that will
	 * return immediately, before any pending data transfers are
	 * fully processed.
	 *
	 * @param dHandle Input the data handle to the MxN-registered data
	 *            object that is ready for MxN data transfer.
	 * @param update_time Input indicates whether the internal iteration
	 *            count or "time" value for this data object should be
	 *            incremented as part of this transfer cycle.  Setting
	 *            this value to "False" allows multiple parallel data
	 *            transfers within a single iteration, as needed.
	 * @param access Input the access mode being activated for the
	 *            given data object.  Can be ReadOnly, WriteOnly or
	 *            ReadWrite to dictate whether the data object can
	 *            now be read, written or both by the MxN-er to
	 *            execute parallel data transfers.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int dataReadyBegin( mxnDataHandle dHandle,
			bool update_time, AccessMode access ) = 0;

	/**
	 * int dataReadyTest( mxnDataHandle dHandle )
	 *
	 * Companion to dataReadyBegin(), tests whether all pending data
	 * transfers have completed.
	 *
	 * This is a non-blocking call that returns True iff all pending
	 * data transfers have completed.  Otherwise, dataReadyTest()
	 * returns False.
	 *
	 * The dataReadyTest() method does *not* change the MxN connection
	 * status from "Ready" to "Not Ready" for the given data object.
	 * The dataReadyEnd() method must still be called to complete
	 * the iteration.
	 *
	 * @param dHandle Input the data handle to the MxN-registered data
	 *            object, as previously passed to dataReadyBegin().
	 * @return data transfer status (1 if completed, 0 if not, else
	 *            error)
	 */
	virtual int dataReadyTest( mxnDataHandle dHandle ) = 0;

	/**
	 * int dataReadyEnd( mxnDataHandle dHandle )
	 *
	 * Companion to dataReadyBegin(), waits for all pending data
	 * transfers to complete.
	 *
	 * This is a blocking call, such that all pending data transfers
	 * must complete before the dataReadyEnd() method returns.
	 * Before returning, dataReadyEnd() un-sets the MxN connection
	 * status from "Ready" to "Not Ready" for the given data object.
	 *
	 * @param dHandle Input the data handle to the MxN-registered data
	 *            object, as previously passed to dataReadyBegin().
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int dataReadyEnd( mxnDataHandle dHandle ) = 0;

	/**
	 * int getRegisteredData( mxnDataHandle * &dHandles, int &nHandles )
	 *
	 * Provide a list of all data objects registered with the MxN-er.
	 * Essential routine for any practical "discovery" process,
	 * like for the Viz Proxy, to connect up to another data object.
	 *
	 * @param dHandles Output an array of data handles to MxN-registered
	 *            data objects, as registered with registerData().
	 * @param nHandles Output the number of elements in dHandles.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int getRegisteredData( mxnDataHandle * &dHandles,
			int &nHandles ) = 0;

	/**
	 * int getDataName( mxnDataHandle dHandle, char * &name )
	 *
	 * Returns the data field name for the given data handle.
	 * It is assumed that this would be handy for programmatically
	 * determining who you want to connect to via MxN...
	 *
	 * @param dHandle Input a data handle to an MxN-registered
	 *            data object, as registered with registerData().
	 * @param name Output the name of the given data field.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int getDataName( mxnDataHandle dHandle, char * &name ) = 0;

	/**
	 * int getDataBounds( mxnDataHandle dHandle,
	 * 		int * &upper, int * &lower, int &rank )
	 *
	 * Returns the rank and Global Upper and Lower Bounds for the
	 * data field represented by the given data handle.
	 * It is assumed that this would be handy for creating a local
	 * array that would be amazingly enough the exact size needed
	 * to MxN couple with the given data field (like for VizProxy :-).
	 *
	 * @param dHandle Input a data handle to an MxN-registered
	 *            data object, as registered with registerData().
	 * @param upper Output an array of the Global Upper Bounds for data.
	 * @param lower Output an array of the Global Lower Bounds for data.
	 * @param rank Output the size of the upper and lower arrays.
	 * @return execution status (0 if OK, !=0 if error)
	 */
	virtual int getDataBounds( mxnDataHandle dHandle,
			int * &upper, int * &lower, int &rank ) = 0;

	}; // MxNPort

  } // namespace cca
} // namespace gov

#endif // MxNPort_h_seen