Currently, we provide an abstraction of communication channels with a control process and vector-time class to support the channels. These interfaces are subject to change as our implementation evolves. If you are interested in how we integrate the programming environment and the run-time visualization, you may visit our ThreadMentor web site, which presents our design for a user-level thread library and runtime visualization system.
 
 

To support remote process creation, we have designed and implemented a central control process, cmrun, which creates remote processes automatically. Users can specify the number of user processes and the hosts on which these processes will execute.

Both synchronous channels (blocking send and blocking receive) and asynchronous channels (nonblocking send and blocking/nonblocking receive) are implemented in the channel library to support different types of one-to-one process communication. The logical time (VectorClock) class is also designed so that users can design/implement various distributed algorithms on top of channels.

In the following sections, we present the interface details for the Channel, VectorClock and Topology classes. Users can download the library and a couple of testing programs from the "Download & Install" section below.

Note: Since the control process use "rsh" or "ssh" to spawn the user process on remote hosts, our software assumes that any of the machines listed in user's machinefile are accessible via "rsh" or "ssh" without requiring password entry, you may need to contact your system administrator for help on this.. 

How to use the Control Process: cmrun

• SYNTAX: 

 cmrun   -{rsh | ssh} [-np nprocs] [-mf machinefile] {-pf programfile | userprog [user_arg0,user_arg1, ... ]} {-help}
 
• COMMAND LINE OPTIONS:

 
• rsh | ssh      ----  remote host access protocol.
• nprocs --- number of user processes in one computation, default value is 1.
• machinefile --- file that contains the hostname on which the user processes will be running. If this file is not given, the application process will executes at the current host. This is a list of all available machines 
• userprog --- the user program to run; 
• user_arg0, user_arg1, ... --- the command line arguments for user's program. 
• programfile --- a file contains all the user programs to run in one computation. 
• EXAMPLES:

"cmrun" can be used in two ways, one for creating user processes that all execute the same program in one computation, and the other for creating user processes that execute different user programs which are specified in a configuration file: programfile.
 
1. cmrun [-np nprocs] [-mf machinefile] userprog [user_arg0, user_arg1, ... ] 

 
• (a) cmrun -np 4 foo 

-- 4 processes will be spawned on the local host, each process will execute program "foo".
 
• (b) cmrun -np 4 -mf machinefile foo 2 8 

-- 4 processes will be spawned, each process will execute the program named "foo" with command line arguments "2 8" Execution locations will be taken from "machinefile" and assigned in round-robin fashion to the four processes. 

A specific example " machinefile" is given here. In this case, two processes will run on host bear.cs1.mtu.edu, 1 on bova.csl.mtu.edu and 1 on brin.csl.mtu.edu. 

Note: the hosts listed in machinefile must be in the user's ".rhosts" in the user's home directory. Here is an example .rhosts file. 
 

2. cmrun [-mf machinefile] -pf programfile

  programfile --- a file containing all of the user programs to run in one computation. Here is an example program file. Assume there are n machines in machinefile and m processes in programfile. If n = m, then there is a one-to-one correspondence between the machines in machinefile and the processes listed in programfile. If n > m, then the m processes are assigned to the first n machines in machinefile. If n < m, then the processes are assigned to the machines in a round-robin fashion.

File Format:

1. Machine File:

A machinefile is a file that contains a list of the possible hosts on which you want your program to run. 

If user does NOT specify the machinefile when he/she fire up "cmrun", by default, all the user process will run on the local host where the control process "cmrun" is running. If user specify an set of machines in his/her own machine file, "cmrun" will then create user process on those machines.

The machine file is very simple and is just a list of machines user want his/her program to urn. A specific example " machinefile" is given here. Each host name is specified on one line of the machinefile.

Sample machine file for Linux machines in th CS lab of Michigan Tech.: 
bear.csl.mtu.edu 
bova.csl.mtu.edu 
brin.csl.mtu.edu 
anthony.csl.mtu.edu

The machinefile should be kept in the same directory as user's executable files and named something appropriate like machinefile. The name of your machinefile will be used as an argument to the "cmrun" option -mf.

Note: Make sure the hosts listed in your machinefile also listed in the your ".rhosts" file in your home directory. Here is an example .rhosts file. 
 

2. Program File:

A   programfile is a file that contains a list of the executable file the user want to run on various machines. 

If user does NOT specify the  programfile, then all the user process will execute the same program user specifies on the command line, in this case, we say it's like the "SPMD" computation model.

If user want to execute different programs on different machines, he/she can  specify a lists of executable programs in the  programfile,  the control process will then map each executable program to different machines listed in user's "machinefile". In this case, we say it's like the "MPMD" computation model.

The machine file is very simple and is just a list of executable programs user to urn. A specific example programfile file is given here. Each executable program name and command line arguments are specified on one line of the programfile.

Sample program file:
/home/grad/pichen/Test/ringtest1
/home/grad/pichen/Test/ringtest2
/home/grad/pichen/Test/ringtest3 
/home/grad/pichen/Test/ringtest4

The programfile should be kept in the same directory as user's executable files and named something appropriate like programfile The name of your programfile will be used as an argument to the "cmrun" option -pf.

Note: Make sure the absolute path of each executable file is specified in the program file.
 

3. .rhosts File: 

The .rhosts files list hosts and users that are trusted by the local host when a connection is made using the "rshd" service. The .rhosts file resides in a user's home directory and specifies the remote machines and remote user names that the user may use to remotely log in to the local machine. 

Each line of these files has the format:

hostname [username] 

hostname may be given as a host name (typically, a fully qualified host name in a DNS environment), an address, or a + character indicating that all hosts are to be trusted. username, if specified, may be given as either a user name on the remote host or a + character indicating all users on hostname. 

Here is an example .rhosts file. 

User Process ID:

1.  Control process assigns  process id  to each user process when control process spawn user process on remote hosts, user process id is in the range between 0 and nprocs - 1. nprocs is the number of processes user specified on the command line when control process is executed

 
2.  void getMyId( int *myid) :

          To determines the process ID of the calling process, user can call the function "getMyId( )". "myid" will be the process id of the user process assigned by the control process. The demo program shows how to do this.
 
3.  void getNumProcs(int *numprocesss):

      To determines the total number of user processes created by the control process, use can call the function "getNumProcs( )". "numprocesss" will be the number of user processes created by the control process. The demo program shows how to do this.

SynOneToOneChannel provides a synchronous inter-process communication channel (IPC), it's transparent of the physical location of user's processes and provide the user a communication mechanism to exchange information between a pair of processes. The class interface is defined in system header file: Channel.h

//--------------------------------------------------------------------------------------------
SynOneToOneChannel(int destID, int myID, char *channelname = 0)

DESCRIPTIONS: 
    (1) Constructor of SynOneToOneChannel class 
    (2) Build a two-way synchronous communication channel between process with the unique identifier "myID" and another process whose ID is "destID".

PARAMETERS: 
       destID             --->  the unique identifier of the destination process to which we want to build a channel. 
       myID               --->  the unique identifier of the current process which is calling this constructor.
       channelname ---> channel name for visual system display purposes, default parameter. if user does not specify the name, system would generate the default name for this channel.

PRECONDITION:
        None
POSTCONDITION:
        A two-way synchronous communication channel is constructed between the process with unique identifier "myID" and the process whose ID is "destID". 

//--------------------------------------------------------------------------------------------
~SynOneToOneChannel( )

DESCRIPTIONS: 
        Destructor of the SynOneToOneChannel class 
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys the channel and releases all the system resources.

//--------------------------------------------------------------------------------------------
 int Send( Channel_Message_t message, size_t message_size)

DESCRIPTIONS: 
   (1) Send a message through the current channel. 
   (2) Blocking send --> it will not return until receiver has received the message. 

PARAMETERS: 
     message:           pointer to the user's message
     message_size:   message size, number of bytes to be sent 
RETURN VALUE: 
        If no error occurs, Send() returns the actual number of bytes of the message sent(NOTE, this return value may not be the same as  message_size). Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE:
     (1) Process may block indefinitely. 
     (2) Vector time will be updated. 

//--------------------------------------------------------------------------------------------
int Receive(Channel_Message_t message, size_t message_size)

DESCRIPTIONS: 
   (1) Receives a message through the current channel. 
   (2) Blocking receive --> process is blocked until it receives a message or the channel is closed

PARAMETERS: 
      message         : pointer to the buffer where received  message is to be stored. 
      message_size : number of bytes intent to receive. 

RETURN VALUE: 
        If no error occurs, Receive() returns the actual number of bytes of the received message (NOTE, this return value may not be the same as message_size). If the channel has already been shutdown, it returns 0. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     (1) Process may block indefinitely. 
     (2) Vector time will be updated. 

AsynOneToOneChannel is an asynchronous inter-process communication channel (IPC), it's transparent of the physical location of user processes and provides the user a asynchronous communication mechanism for information exchange between a pair of processes. The class interface is defined in system header file: Channel.h

//--------------------------------------------------------------------------------------------
AsynOneToOneChannel(int destID, int myID, char *channelname = 0, float reliability = 1.0)

DESCRIPTIONS: 
     (1) Constructor of AsynOneToOneChannel class 
     (2) Build a two-way asynchronous communication channel between process with the unique identifier "myID" and the destination process whose ID is "destID"..

PARAMETERS: 
       destID             --->  the unique identifier of the destination process to which we want to build a channel. 
       myID               --->  the unique identifier of the current process which is calling this constructor.
       channelname ---> channel name for visual system display purposes, default parameter. if user does not specify the name, system would generate the default name for this channel.
       reliability         ---> channel reliability, default value (1.0) means the channel is fully reliable. If its value is 0.7, then it means 30% of user's message will get lost.

PRECONDITION:
        None
POSTCONDITION:
         A two-way asynchronous communication channel is constructed between the process with unique identifier "myID" and the process whose ID is "destID".

--------------------------------------------------------------------------------------------
AsynOneToOneChannel(int destID,  int myID,  float (* ReliabilityFunc)(void),  char *channelname = 0)

DESCRIPTIONS: 
     (1) Constructor of AsynOneToOneChannel class 
     (2) Build a two-way asynchronous communication channel between process with the unique identifier "myID" and the destination process whose ID is "destID".. 

PARAMETERS: 
        destID             --->  the unique identifier of the destination process to which we want to build a channel. 
       myID               --->  the unique identifier of the current process which is calling this constructor.
       channelname  ---> channel name for visual system display purposes, default parameter. if user does not specify the name, system would generate the default name for this channel.
       ReliabilityFunc---> pointer to user defined channel reliability function, which implements user's channel reliability policy.

PRECONDITION:
        None
POSTCONDITION:
        A two-way asynchronous communication channel is constructed between the process with unique identifier "myID" and the process whose ID is "destID".

//--------------------------------------------------------------------------------------------
~AsynOneToOneChannel ( ) 

DESCRIPTIONS: 
        Destructor of the AsynOneToOneChannel
PARAMETERS: 
        None
PRECONDITION:
        None
POSTCONDITION:
        Destroys the channel and release all the system resources.

//--------------------------------------------------------------------------------------------
int Send( Channel_Message_t message, size_t message_size)

DESCRIPTIONS: 
   (1) Send a message through the current channel. 
   (2) Nonblocking send. 

PARAMETERS: 
     message:           pointer to the user's message
     message_size:   message size, number of bytes to be sent 
RETURN VALUE: 
       If no error occurs, Send() returns the actual number of bytes of the message sent(NOTE, this return value may not be the same as  message_size). Otherwise, MTU_SOCKERROR (-1) is returned.

NOTE: 
     (1) Vector time will be updated. 

//--------------------------------------------------------------------------------------------
int Receive(Channel_Message_t message, size_t message_size)

DESCRIPTIONS: 
   (1) Receives a message through the current channel. 
   (2) Blocking receive, process will be blocked until it receives a message or the channel is closed

PARAMETERS: 
      message         : pointer to the buffer where to store the to-be-received  message. 
      message_size : number of bytes to receive. 

RETURN VALUE: 
        If no error occurs, Receive() returns the actual number of bytes of the received message (NOTE, this return value may not be the same as message_size). If the channel has already been shutdown, it returns 0. Otherwise, MTU_SOCKERROR (-1) is returned. 

NOTE: 
     (1) Vector time will be updated.

//--------------------------------------------------------------------------------------------
int NonBlockingRecv(Channel_Message_t message, size_t message_size )

DESCRIPTIONS: 
   (1) Receives a message through the current channel. 
   (2) Nonblocking receive --> if there is message in the channel, the process receives the message and return, if the channel is empty, it returns immediately with blocking. 

>PARAMETERS: 
      message         : pointer to the buffer where to store the to-be-received  message. 
      message_size : number of bytes to receive. 

RETURN VALUE: 
       1. If no error occurs and there is message pending for read, Receive() returns the actual number of bytes of the message receives
          NOTE:   this return value may not be the same as message_size). 
       2. If the channel has already been shutdown or there is no message pending, it returns 0. 
       3. Otherwise, MTU_SOCKERROR (-1) is returned.

NOTE: 
     (1) Vector time will be updated.

//--------------------------------------------------------------------------------------------
bool IsMessageAvailable()

DESCRIPTIONS: 
        (1) Nonblocking check whether the current channel is available for reading (receiving) or not, which actually poll the channel status without blocking the caller. 

RETURN VALUE: 
         true:   there is a pending message to receive in the channel
         false:  there is NO pending message to receive in the channel.

//--------------------------------------------------------------------------------------------
VectorClock( )                                  --->  constructor

 ~VectorClock( )                                ---> destructor

 VectorClock( const VectorClock& ) ---> copy constructor

 VectorClock& operator=(const VectorClock&) ---> copy assignment

//--------------------------------------------------------------------------------------------
int& operator[ ]( int& i )

DESCRIPTIONS: 
   (1) Overloaded subscript "[]" operator overloading.
   (2) User can access/change vector clock by using "Vectorclock[ i ]". 

PARAMETERS: 
      i  :  process id. 

RETURN VALUE: 
       Local vector clock value.

NOTE: 
     NONE.

EXAMPLES:
      1. To increment the local vector clock:
          Declare a VectorClock object somewhere in your source code like following:
                    VectorClock vectorTime;    // vector time 
          Increment local vector time for process i, assume process id is i:
                     vectorTime[i]++;

//--------------------------------------------------------------------------------------------
void Print( void  )

DESCRIPTIONS: 
   dump vector clock to stdout in the format <i, j, ..., l>

PARAMETERS: 
      none

RETURN VALUE: 
       none
 
 
 

Distributed Monitor

Below we provide several sample demo programs for you to get familar with ConcurrentMentor. Here we give the templete makefiles for Byzantine Generals Problem as templete.  You can make corresponding changes to write you own makefile for other demo programs and your own programs.

1. Byzantine Generals' Problem

• Source files: byzagree.cpp, List.cpp, List.h

2. Distributed Dining Philosophers' Problem

• Source files: diningphils.cpp

• Source files: snapshot.cpp

• Source files: routing.cpp, routemap

• Source files: sort1.cpp

6. Ring Topology Test Program

• Source files: ringtest.cpp

• Source file: Torus.cpp

• Source files: tcpring.cpp, udpring.cpp

 

Installing Channel/Topology Library for Linux:

• Since the control process use "rsh" or "ssh" to spawn the user process on remote hosts, our software assumes that any of the machines listed in user's machinefile are accessible via "rsh" without requiring password entry.

• Download the following file to your computer:

Redhat Linux version 7.1 without visual support (CMLinux_Novisual.tar.gz, 100K)
Redhat Linux Version 7.1 with visual support(CMLinux_Visual.tar.gz, 1.2M)
 
• Unzip the downloaded file into a .tar file by typing the following at the command prompt.
• gzip -d CMLinux_Novisual.tar.gz           //for the version without visual support 
• gzip -d CMLinux_Visual.tar.gz                //for the version with visual support

 
• Untar the resulting .tar file by typing the following at the prompt: (Depending on which version you downloaded).
• tar -xvf ConcurrentMentor.tar , this will create a directory "ConcurrentMentor".

 
• Change your PATH environment variables to include the full path to the "ConcurrentMentor/bin" and "ConcurrentMentor/include" that were unarchived by the previous step.

 
• To run one of our demo testing programs, go to ConcurrentMentor/test/ and pick one of the testing program you are interested in, and change to the corresponding directory. Type make to compile and then read the comment at the header of the .cpp file to see how to run the program.

 
• If you run into any problems, please go to our Help Section below.

Installing Channel/Topology Library for Solaris:

• Since the control process use "rsh" or "ssh" to spawn the user process on remote hosts, our software assumes that any of the machines listed in user's machinefile are accessible via "rsh" without requiring password entry.

 
• Download the following file to your computer:

Sun Solaris 2.8 (CMSolaris.tar.gz (105K))

• Unzip the downloaded file into a .tar file by typing the following at the command prompt: (depending on what version you downloaded).
• gzip -d CMSolaris.tar.gz

 
• Untar the resulting .tar file by typing the following at the prompt..
• tar -xvf ConcurrentMentor.tar , this will create a directory "ConcurrentMentor".

 
• Change your PATH environment variables to include the full path to the "ConcurrentMentor/bin" and "ConcurrentMentor/include" folder that were unarchived by the previous step.

 
• To run one of our demo testing programs, go to ConcurrentMentor/test and pick one of the testing program you are interested in, compile it and run it. The README file will show you how to compile and run the test program.

 
• If you run into any problems, please go to our Help Section below.

Installing Channel Library for Windows

• Since the control process use "ssh" to spawn the user processes on remote hosts, our software assumes that any of the machines listed in user's machinefile are accessible via "ssh" without requiring password entry. 

 
• Download the following file to your computers

Windows2000/NT (CMWindows_Novisual.tar.gz, 620K)

Unzip and Untar the download file by using any one of the shareware programs
 * WinZip
 * WinRar 
After you have one of these programs simply double click on the downloaded file and it will be uncompressed for you.

 
• Make sure that you have installed a copy of ConcurrentMentor on every machine that you want to run ConcurrentMentor on. And also the directory structure of ConcurrentMentor should be the same on all these machines ( it is better that you install ConcurrentMentor directly in your home directory, i.e. C:\ConcurrentMentor.
• Set your PATH environment variables to include the full path to the directory of "ConcurrentMentor\bin". This way, you can run cmrun anyway on your machines. Remember this setting should be done on any machine you intend to run ConcurrentMentor on.
• To make a program, you can use the visual c++ environment. Create a Win32 Console project in "ConcurrentMentor\test\ directory. You must set the link to wsock32.lib and also channel.lib(topology.lib) which is in the directory of "ConcurrentMentor\lib". Also you must set \I C:\ConcurrentMentor\include to let the compiler to find your include files there.
• After make, install a copy of the executable to each machine. Remember the same directory structure for ConcurrentMentor on every machine.
• If you run into any problems, please go to our Help Section below.

 This section will be a collection of all questions asked about ConcurrentMentor and their solutions. 

1.Compiling your own programs 

Linux

• Make sure you are in the directory with the program you wish to compile. 
• Make sure you includes system header file: Channel.h into your source file.
• Type the following at the command line to create your executable:

{g++} -g -O2 -c {fileexec}  {filename.cpp}  -I{path}/Linux/include -D_LINUX_  {path}/Linux/lib/novisual/libchannel.a -lnsl
where: 
• {g++} is your compiler ( g++ or c++). 
• {path} is the path to the "Linux" directory. 
• {fileexec} is the name of the executable file you wish to create. 
• {filename.cpp} is the name of the C++ source file. 
• Make sure that your PATH environment variable points to the Linux/bin where "cmrun" is located 
• Run your executable. 

Solaris

• Make sure you are in the directory with the program you wish to compile 
• Type the following at the command line to create your executable:

{g++} -g -O2 -c {fileexec} {filename.cpp}  -I{path}/Solaris/include -D_SOLARIS_  {path}/Solaris/lib/novisual/libchannel.a -lsocket -lnsl
where: 
• {g++} is your compiler (g++ or c++). 
• {path} is the path to the "Solaris" directory. 
• {fileexec} is the name of the executable file you wish to create. 
• {filename.cpp} is the name of the C++ source file. 
• Make sure that your PATH environment variable points to the Solaris/bin where "cmrun" is located 
• Run your executable. 

• To be constructed!