We design and implement the one-to-one process communication and collective communication methods for the topology class, the collective communication capabilities includes broadcasting, scattering, gathering, and reducing. The distributed mutex and barrier synchronization class are also implemented across all the processes in the topology.

A collective (one-to-many or many-to-one) communication operation is executed by having all processes in the topology call the communication routine with matching arguments. The visualization tool can illustrate the underlying message routing for each collective communication methods. 

If user specify the "machine file" when using "mturun", the processes in the logical topology are mapped to host machine listed in user's machine file in a round-robin fashion, starting from process 0 and host 1. Otherwise, all the processes in the topology are mapped to the local host where "mturun" is executed.

We have the following process topology defined so the user can use these topology directly instead of constructing his/her own process topology using the channel interface. Currently, we support the following Topology class:
 

Arbitrary Topology	Star Topology	Ring Topology	Torus Topology
Fully Connected Topology	LinearArray Topology	Grid Topology

The arbitrary "Topology" class is designed to support user's arbitrary process topology, user need to provide the topology matrix which represents  a strongly connected process graph, the mininum spanning tree algorithm is implenmented to support one-to-many and many-to-one process communication primitive. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
Topology(int **topmap, int numProcs, int myID)

DESCRIPTIONS: 
    (1) Constructor of  Topology class 
    (2) Build the process topology based on  user specified topology matrix.

PARAMETERS: 
       topmap       --->  2-D topology matrix which represents the process topology. 
      numProcs    --->  total number of processes in the topology.
      myID           --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        The user defined process topology is constructed and corresponding communication channel is set up. 

//--------------------------------------------------------------------------------------------
~Topology( )

DESCRIPTIONS: 
        Destructor of the Topology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.

//--------------------------------------------------------------------------------------------
TopologySend( int destId,  void *sendbuf,  size_t message_size )

DESCRIPTIONS: 
   (1) One-to-One communication, send a message to desination process "destId". 
   (2) Non-blocking send.

PARAMETERS: 
     destId,               destination process
     sendbuf:           initial address of the send buffer.
     message_size:   message size, number of bytes to be sent 

RETURN VALUE: 
        If no error occurs, TopologySend( ) returns the actual number of bytes of the message sent(NOTE, this return value may not be the same as  message_size). Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE:
     (1) Process may block indefinitely. 
     (2) Vector time will be updated. 

//--------------------------------------------------------------------------------------------
TopologyReceive( int sourceId,  void *recvbuf, size_t message_size )

DESCRIPTIONS: 
   (1) Receives a message from source process "sourceId". If "sourceId" is CM_ANY, it will receive the first   
         message that arrives. The user may use the function GetLastSenderId() to determine the sender. 
   (2) Blocking receive --> process is blocked until it receives a message from process "sourceId".

PARAMETERS: 
      sourceId,          source process
      recvbuf:            initial address of the receive buffer. 
      message_size :  maximum number of bytes to receive (upper bound). 

RETURN VALUE: 
        If no error occurs, TopologyReceive() returns the actual number of bytes of the received message (NOTE, this return value may not be the same as message_size). If the channel has already been shutdown, it returns 0. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     (1) Process may block indefinitely. 

//--------------------------------------------------------------------------------------------
TopologyBroadcast( int rootID, void *sendbuf, size_t message_size,  void *recvbuf, size_t recv_size,  bool visual

DESCRIPTIONS: 
   (1) One-to-many process communication, topology broadcasts a message from the process with id "rootID" to all processes of the topology, including itself. 
   (2) The argument rootID and  visual  must have identical values on all processes. On return, the contents of root's communication buffer has been copied to all processes.

PARAMETERS: 
      rootID:             id of the broadcast root process.
      sendbuf:            initial address of the send buffer. 
      message_size :  number of bytes to broadcast. 
      recvbuf:            initial address of the receive buffer. 
      recv_size :         maximum number of bytes to receive (upper bound). 
      visual :             boolean variable which control if underlying message passing is visulized in the visualization tool.

RETURN VALUE: 
        If no error occurs, TopologyBroadcast( ) returns the actual number of bytes broadcast out for the root process, and the actuall number of bytes received for all the other processes. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     NONE. 

//--------------------------------------------------------------------------------------------
TopologyGather(int rootID, void* sendbuf, size_t send_size, void* recvbuf, size_t recv_size,   bool visual )

DESCRIPTIONS: 
   (1) Many-to-One process communication, each process in the topology (rootID process included) sends the contents of its send buffer to the root process "rootID", the root process receives the messages and stores them in process ID order sequentially. 
   (2) The receive buffer is ignored for all non-root processes.
   (3) The argument rootID and  visual must have identical values on all processes. 

PARAMETERS: 
      rootID:             id of the broadcast root process.
      sendbuf:            initial address of the send buffer. 
      send_size, :       number of bytes to send 
      recvbuf:            initial address of the receive buffer. 
      recv_size :         maximum number of bytes to receive (upper bound). 
      visual :             boolean variable which control if underlying message passing is visulized in the visualization tool.

RETURN VALUE: 
        If no error occurs, TopologyGather( ) returns the actual number of bytes received  for the root process, and the actuall number of bytes send out for all the other processes. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     NONE. 

//--------------------------------------------------------------------------------------------
TopologyScatter( int rootID, void* sendbuf, size_t send_size, void* recvbuf, size_t recv_size,  bool visual )

DESCRIPTIONS: 
   (1) One-to-many process communication, inverse operation of  "TopologyGather", the rootID process sends out  a message contains in the "sendbuf", the message is split into N (number of processes in the topology) equal segments, the ith segement is send to the process in the topology with id "i".
   (2) The argument rootID and  visual must have identical values on all processes. All arguments are significant on process "rootID", while for other processes, only recvbuf, recv_size are significant.

PARAMETERS: 
      rootID:             id of the broadcast root process.
      sendbuf:            initial address of the send buffer. 
      send_size, :       number of bytes to send 
      recvbuf:            initial address of the receive buffer. 
      recv_size :         maximum number of bytes to receive (upper bound). 
      visual :             boolean variable which control if underlying message passing is visulized in the visualization tool.

RETURN VALUE: 
        If no error occurs, TopologyScatter( ) returns the actual number of bytes send out for the root process, and the actuall number of bytes received for all the other processes. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     NONE. 

//--------------------------------------------------------------------------------------------
TopologyReduce( int rootID, void* sendbuf, void* recvbuf, size_t message_size, void* ( *Reduce_func )(void *, void *),   bool visual )

DESCRIPTIONS: 
   (1) Many-to-One process communication, topology performs a global reduce operation (sum, max, min etc.) across all the processes in the topology, the reduction operation is defined by  a  user defined reduce funtion " void* ( *Reduce_func )(void *, void *) ".  It combines the elements provided in the  sendbuf of each process in the topology, using the user defined reduction function, and returns the combined value in the recvbuf  of the rootID process.
   (2) The argument rootID and  visual  must have identical values on all processes. On return, the reduced result is stored in the recvbuf  of  the  rootID process .
   (3). The reduction operation is assumed to be associative.

PARAMETERS: 
      rootID:             id of the broadcast root process.
      sendbuf:            initial address of the send buffer. 
      send_size, :       number of bytes to send. 
      recvbuf:            initial address of the receive buffer. 
      recv_size :         maximum number of bytes to receive (upper bound). 
      visual :             boolean variable which control if underlying message passing is visulized in the visualization tool.

RETURN VALUE: 
        If no error occurs, TopologyBroadcast( ) returns the actual number of bytes broadcast out for the root process, and the actuall number of bytes received for all the other processes. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     NONE. 

int GetLastSenderId()
       Descriptions:
                   The function should be used after a message is received
        Parameters:
                    None
        Return Values:
                     Upon successful, it return the ID of sender of the latest message received by the current process.
                   
        Note:
                     The user is responsible for checking the successfulness of the latest receive. If the latest receive  
                     failed, this function may return an undesirable value which means nothing. 

In the fully connected topology, each process is connected with other N - 1 processes, suppose there are N processes in the topology. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
FullyConnectedTopology( int nunProcs,, int myID)

DESCRIPTIONS: 
    (1) Constructor of  FullyConnectedTopology class 
    (2) Build the fully connected process topology.

PARAMETERS: 
      nunProcs      --->  total number of processes in the topology.
      myID            --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        Build a fully connected process topology. 

//--------------------------------------------------------------------------------------------
~FullyConnectedTopology( )

DESCRIPTIONS: 
        Destructor of the FullyConnectedTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.

In the star opology,  the process which is in the center is connected with other N - 1 processes, suppose there are N processes in the topology, other processes only connected with the center process. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
StarTopology(int nunProcs,  int myID,  int center_node_ID = 0)

DESCRIPTIONS: 
    (1) Constructor of  StarTopology class 
    (2) Build the star process topology.

PARAMETERS: 
      nunProcs      --->  total number of processes in the topology.
      myID            --->  the unique identifier of the current process.
      center_node_ID  --->  the unique identifier of the process which is in the center of the star topology. Defaultly, process 0 is in the center of the star topology

PRECONDITION:
        None
POSTCONDITION:
        Build a fully connected process topology. 

//--------------------------------------------------------------------------------------------
~FullyConnectedTopology( )

DESCRIPTIONS: 
        Destructor of the FullyConnectedTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.

In the linear array topology, N processes in the topology form a linear array, process i is conneced with process "i - 1" and process "i + 1", except process 0 and N - 1 (process 0 is only connected with process 1, whereas process N - 1 is only connected with process N - 2). This topology may be useful if the computation is of the pipeline patter. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
LinearArrayTopology( int num_process, int myID )

DESCRIPTIONS: 
    (1) Constructor of  LinearArrayTopology class 
    (2) Build the fully connected process topology.

PARAMETERS: 
      nunProcs      --->  total number of processes in the topology.
      myID            --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        Build a fully Linear Array  process topology. 

//--------------------------------------------------------------------------------------------
~FullyConnectedTopology( )

DESCRIPTIONS: 
        Destructor of the LinearArrayTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.

In the Ring topology, N processes in the topology form a circular ring topology, process i is conneced with process "i - 1" and process "i + 1", except process 0 and N - 1 ( process 0 is connected with process 1 and N - 1). This topology may be useful if the computation is of the pipeline patter. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
RingTopology( int num_process, int myID )

DESCRIPTIONS: 
    (1) Constructor of  RingTopology class 
    (2) Build the circular ring process topology.

PARAMETERS: 
      nunProcs      --->  total number of processes in the topology.
      myID            --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        Build a fully Linear Array  process topology. 

//--------------------------------------------------------------------------------------------
~RingTopology( )

DESCRIPTIONS: 
        Destructor of the RingTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.
 

In the Grid topology, all the processes in the topology forms a 2-D N x M grid, there are  N rows and M columns, totally there are N x M processes in the topology. Process 0 located at the upper left corner of the grid, whereas process N - 1 is located at the lower right corner of the grid. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
GridTopology( int Row, int Column, int myID )

DESCRIPTIONS: 
    (1) Constructor of  GridTopology class 
    (2) Build the 2-D Row x Column Grid  process topology.

PARAMETERS: 
      Row              --->  grid row dimension size.
      Column        --->  grid column dimension size. 
      myID            --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        Build a fully connected process topology. 

//--------------------------------------------------------------------------------------------
~GridTopology( )

DESCRIPTIONS: 
        Destructor of the GridTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.

In the Torus topology, all the processes in the topology forms a wrap-around 2-D N x M grid, there are  N rows and M columns, totally there are N x M processes in the topology. Process ID is mapped to nodes in the Torus in row major order, process 0 located at the upper left corner of the grid, whereas process N - 1 is located at the lower right corner of the grid. All the point-to-point and collective communication methods are the same with those defined for arbitrary topology class, please refer to the above for relevant information. The class interface is defined in system header file: Topology.h

//--------------------------------------------------------------------------------------------
TorusTopology( int Row, int Column, int myID )

DESCRIPTIONS: 
    (1) Constructor of  TorusTopology class 
    (2) Build the 2-D Row x Column Torus  process topology.

PARAMETERS: 
      Row              --->  grid row dimension size.
      Column        --->  grid column dimension size. 
      myID            --->  the unique identifier of the current process.

PRECONDITION:
        None
POSTCONDITION:
        Build a fully connected process topology. 

//--------------------------------------------------------------------------------------------
~TorusTopology( )

DESCRIPTIONS: 
        Destructor of the TorusTopology class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys all the underlying channels and releases all the system resources.