Mini-Programming Project

Due on Friday, December 11, 2009 @ 11pm

100 points

Build a User-Level Multithread System

In this programming project, you are to put all you have learned in this class together and complete the design and implementation of a user-level multithreaded system. You do not start from scratch, however. In fact, you are provided with (1) a working system in object format, (2) a portion of the working system in source format for you to complete, and (3) several working examples in source format. Your job consists of (1) reading the incomplete system and understanding its mechanism, (2) implementing the missing parts, and (3) testing your version. Note that you should not use ThreadMentor in this project. You are to create your own!

The language of choice is ANSI C.

  • You must ssh to the Linux system on dummy1.csl.mtu.edu to do this mini-project, because this package DOES NOT work on any other lab machines!
  • Your implementation is system dependent and all system dependent components are in mtuThreadCore.c which does not require any modification.
  • You may use Windows or Sun Solaris to do and test your work; but, our grader will use the indicated machine for grading.

System Description 1: User Callable Functions

This part consists of the following functions, each of which has a direct counterpart in ThreadMentor threads:

All functions are in file new_mtuThreadTop.c. Note that implementing functions THREAD_JOIN(), THREAD_SUSPEND() and THREAD_RESUME() is your job. As a result, only dummy templates are provided in file new_mtuThreadTop.c.

System Description 2: Synchronization Primitives

This part consists of the following synchronization primitives for mutex locks, semaphores and channels. You have learned all of them in this course.

Mutex Locks

There are four functions for mutex locks:

Full implementations of these four functions are in file mtuThreadMUTEX.c. Read the code carefully because you will need it for implementing semaphores.

Counting Semaphores

There are four functions for counting semaphores:

Only dummy templates of these four functions are in file new_mtuThreadSEM.c, because it is your job to implement semaphores. Note that type SEM_t is defined as a pointer to a structure SEMAPHORE. A possible data structure is provided to you in file mtuThread.h. Feel free to modify it for your implementation.

Channels

There are four functions for channels. This is a simplified and scaled-down version of ThreadMentor.

Only four dummy templates of these four functions are provided to you in file new_mtuThreadCHAN.c, because it is your job to implement channels. Note that type CHANNEL_t is defined as a pointer to a structure CHANNEL. A possible data structure is provided to you in file mtuThread.h. Feel free to modify it for your implementation.

More importantly, channel should use non-bloking send and blocking receive. You may arrange the messages in the FIFO order.

How to Use this System

To use this system, you need to know the seven functions listed at the beginning of this page. All seven functions start with THREAD_, since they are for controlling the execution of threads.

The most important function is THREAD_CREATE() for creating threads. Function THREAD_CREATE() has the following prototype:

THREAD_t  THREAD_CREATE(
               void (*Entry)(),    /* function to be run as a thread*/
               int Size,           /* stack size for this thread    */
               int status,         /* running or suspended?         */
               int Argc,           /* # of arguments to be passed   */
               char **Argv);       /* argument list                 */
The first argument is a pointer to a function that will run as a thread. In fact, when you call THREAD_CREATE(), the first argument is simply the function name. Note that this function has a return type of void, which means it does not return anything. When this function runs as a thread, the size of the stack space assigned to it is specified by the second argument. You may use THREAD_SIZE, which is defined in mtuThread.h, for this project. The third argument specifies if the created thread should be suspended. If it should be suspended, use THREAD_SUSPENDED; otherwise, use THREAD_NORMAL. The last two arguments, Argc and Argv, are exactly the same as they do in a main program (i.e., retrieving arguments from a command line). More precisely, you should collect the arguments to be passed to a thread into an argument list and passed as Argv. The number of arguments, which is an integer, is passed as Argc. In this project, since we only pass a single integer to a thread, we shall ignore Argv and only use Argc for passing an integer. Therefore, Argc does not have the same meaning as mentioned above.

The following is an example in file pingpong.c. It has two threads, Ping() and Pong().

#include  <stdio.h>
#include  <stdlib.h>

#include  "mtuThread.h"

void  Ping(int);                        /* thread prototypes        */
void  Pong(int);

SEM_t StopPing, StopPong;
int   Count;

void  Ping(int  ID)
{
     while (1) {                        /* loop forever             */
          SEMAPHORE_WAIT(StopPing);     /* stop until released      */
               printf("Ping-");         /* print Ping-              */
          SEMAPHORE_SIGNAL(StopPong);   /* release Pong             */
     }
     THREAD_EXIT();
}

void  Pong(int  ID)
{
     do {                               /* loop for 'Count' times   */
          SEMAPHORE_WAIT(StopPong);     /* stop until released      */
               printf("Pong\n");        /* append Pong to Ping-     */
          SEMAPHORE_SIGNAL(StopPing);   /* release Ping             */
          Count--;
     } while (Count > 0);               /* done if Count = 0        */
     THREAD_EXIT();
}

int  main(int argc, char *argv[])
{
     THREAD_t  THREAD_Ping, THREAD_Pong;

     Count = 10;
     if (argc != 2) {
          printf("Use %s #-of-iterations\n", argv[0]);
          printf("No. of iterations is set to %d\n", Count);
     }
     else 
          Count = abs(atoi(argv[1]));

     StopPing = SEMAPHORE_INIT(1);      /* Ping goes first          */
     StopPong = SEMAPHORE_INIT(0);      /* Pong stops until informed*/

     THREAD_Ping = THREAD_CREATE(Ping, THREAD_SIZE, THREAD_NORMAL, 
                                 1, (char **) 0);
     THREAD_Pong = THREAD_CREATE(Pong, THREAD_SIZE, THREAD_NORMAL, 
                                 2, (char **) 0);

     THREAD_JOIN(THREAD_Pong);          /* can only join Pong()     */

     return 0;
}
This example first creates two semaphores StopPing and StopPong. Then, two threads are created running functions Ping() and Pong(). Note that (1) we use default stack size THREAD_SIZE; (2) both threads will run once they are created since THREAD_NORMAL is used; (3) the fourth argument is 1 (resp., 2) for the thread running Ping() (i.e., Pong()); (4) the fifth argument is (char **) 0 since no argument list is passed to the created threads; and (5) function THREAD_CREATE() returns the information structure of the created thread into a variable of type THREAD_t, which is used for calling THREAD_JOIN(). The remaining of this program is clear since we have encountered it a number of times in class. Note that threads Pong() terminates itself after Count iterations; however, Ping() loops forever. This may cause a problem. After Pong() is done, Ping() might not be able to run further because the thread that can signal it (i.e., thread Pong()) has terminated. This system will detect that no thread is in the ready queue and yet the system is still running! Hence, it might tell you that a deadlock is likely to occur, although you know this is not the case. The scheduler written in this way is to show you that a scheduler indeed can warn you for possible deadlocks.

Please refer to philosophers (philo.c) , philosophers with four seats (philo-4.c), and philosophers with a righty (philo-w.c) for further examples that use mutex locks and semaphores.

The next example shows the use of channels to solve the producer and consumer problem. See buffer.c for the details.

#include  <stdio.h>
#include  <stdlib.h>
#include  "mtuThread.h"

void  Producer(int);
void  Consumer(int);

CHANNEL_t Chan;

void  Producer(int  ID)
{
     int  i, data;

     for (i = 0; ; i++) {
          data = ID*1000 + i;
          CHAN_SEND(Chan, &data, sizeof(int));
          printf("Producer %d has sent data %d\n", ID, data);
          THREAD_YIELD();
     }

     THREAD_EXIT();
}

void  Consumer(int ID)
{
     int *data, size;

     while (1) {
          CHAN_RECV(Chan, (void**) &data, &size);
          printf("     Consumer %d received %d\n", ID, *data);
          THREAD_YIELD();
     }

     THREAD_EXIT();
}

int  main(int argc, char *argv[])
{
     THREAD_t   Producer1, Producer2, Consumer1, Consumer2;
     int        i;
     char       ch;

     printf("Use Ctrl-C to kill this program\n");
     printf("Hit any key to continue... ");
     ch = getchar();
     printf("\n");

     Chan = CHAN_INIT("Channel", ASYNCHRONOUS, MANYTOMANY);

     Producer1 = THREAD_CREATE(Producer, THREAD_SIZE, THREAD_NORMAL, 
                      1, (char **) 0);
     Producer2 = THREAD_CREATE(Producer, THREAD_SIZE, THREAD_NORMAL, 
                      2, (char **) 0);
     Consumer1 = THREAD_CREATE(Consumer, THREAD_SIZE, THREAD_NORMAL, 
                      1, (char **) 0);
     Consumer2 = THREAD_CREATE(Consumer, THREAD_SIZE, THREAD_NORMAL, 
                      2, (char **) 0);

     THREAD_JOIN(Producer1);

     return 0;
}
This program creates two producers and two consumers. The producers keep sending integer messages to channel Chan, while consumers keep receiving integer mail messages from channel Chan. Note the use of THREAD_YIELD().

Program psort.c shows a very interesting way of using channels. This program uses multiple threads to sort non-negative integers. Refer to the program file for the details. In this program, 10 threads (the sorters), S1, S2, ..., S10 are created. Each of these threads, say Si, has a channel inChan shared with its predecessor Si-1, and a channel outChan shared with its successor Si+1. There is a generator G and a terminator T. The generator has a channel shared with S1 and the terminator T has a channel shared with S10.

The generator G sends ten random non-negative integers into the channel shared with S1. After that, G sends an end message END, indicating that there is no more data so that the job ends.

For each sorter, say Si, it first receives a number from its predecessor via its inChan and memorizes it. Then, Si receives a new number via inChan. If this new number is an END, there is no more incoming numbers and the number memorized by Si is printed. Then, Si prints its number, passes END to its neighbor, and exits. If the new number is not an END, we have two cases to consider:

  1. If the new number is larger than or equal to the number that Si memorized, the number is sent to Si+1 via Si's outChan. After this, Si waits for a new incoming message via its inChan.
  2. If the new number is less than the number that Si memorized, Si memorizes the new number and sends the old one to Si+1 via its outChan. After this, Si waits for a new incoming message via its inChan.

The job for terminator T is simple. It could only receive the END message. (Why?) Once T receives the END message, it exits the whole program. Convince yourself that the output from sorters is in increasing order.

Your Job

After reading mtuThread.h, mtuThread*.c and all examples, you will know the merit of this multithreading package. You should also find several places marked with "Your Job" in various files. These are the places where you have to fill in with your implementation. You can modify the definition file and the implementation file. However, you have to make sure that all sample programs will run correctly without any modifications. Also, make sure that you do not use busy waiting. If you have one busy waiting in the system, you will receive no credit no matter how good the other parts are.

The following lists the places where your work is required.

As you can see from the filenames, all programs are written in C. Consequently, you should use ANSI C rather than C++ for this mini-project. Without observing this requirement, it is likely we may not be able to compile and test your programs successfully, and you may not receive proper credit for your work.

Testing Your Implementation

Your system must run all eight sample programs, alter1.c, alter2.c, pingpong.c, philo.c, philo-4.c, philo-w.c, buffer.c and psort.c, without any modifications. In addition to these eight sample programs, design and submit the following for testing.

How Will We Test Your Implementation

We will use the following to test each of the six sample programs, your smokers-sem.c, exchange.c and ring-leader.c, and some other programs we will be writing for testing purposes. Hence, make sure you will have more comprehensive tests with a few more test cases.
gcc  xyz.c mtuThread.c -o xyz
xyz
where xyz is a program name (e.g., exchange). Note that since we will use our original version of alter1.c, alter2.c, pingpong.c, philo.c, philo-4.c, philo-w.c, buffer.c and psort.c to test your implementation, any changes you made to these programs will have no effect. Therefore, you should not make any change to these sample programs. We will also run your smokers-sem.c, exchange.c and ring-leader.c under our correct implementation to see if you have submitted correct solutions.

Two correct implementations of this system are provided in binary files mtuThread-Linux.o and mtuThread-Solaris.o, you may copy any of these to mtuThread.o in order to check your implementation with the following:

gcc  xyz.c mtuThread.o -o xyz
xyz

Note that the -D compiler command line switch can be used to select a version to compile your program on Fedora Linux or on Solaris. The default (i.e., no -D compiler switch used) is Fedora Linux. Read the source files to figure it out how to compile on Solaris.

By the way, we expect you to use the gcc compiler.

Click here to see the grading sheet.

How Will We Grade Your Implementation

We shall evaluate your implementation based on the following criteria:

In addition to running test programs above, we will also use other programs. The correctness of your implementation will be evaluated with all test programs.

Downloadable Material

Click here to download file proj.tar.gz. Then, move this file to your working directory and use the following commands to uncompress and untar:
gzip -d proj.tar.gz
tar xvf proj.tar
You will see the following files in your directory:
  1. mtuThread-*.o: our correct implementation (in object format) for your reference and testing purposes
  2. mtuThread*.h: the header files for our correct implementation
  3. mtuThread*.c: the implementation files of our correct implementation. However, three files are missing: mtuThreadTop.c, mtuThreadSEM.c and mtuThreadCHAN.c. They are supposed to be replaced by your version of new_mtuThreadTop.c, new_mtuThreadSEM.c and new_mtuThreadCHAN.c, which are included in this package (i.e., proj.tar.gz).
  4. Sample programs: alter1.c, alter2.c, pingpong.c, philo.c, philo-4.c, philo-w.c, buffer.c and psort.c.
After receiving these files, you should copy one of the correct implementation file to mtuThread.o, and immediately rename new_mtuThreadTop.c, new_mtuThreadSEM.c and new_mtuThreadCHAN.c to mtuThreadTop.c, mtuThreadSEM.c and mtuThreadCHAN.c, respectively. Then, you should use these files for your work. Do not submit new_mtuThreadTop.c, new_mtuThreadSEM.c and new_mtuThreadCHAN.c, since we will not read and use them.

Submission Guidelines

For detailed programming and submission guidelines, refer to previous programming assignments. Before start working on this project, rename new_mtuThreadTop.c, new_mtuThreadSEM.c and new_mtuThreadCHAN.c to mtuThreadTop.c, mtuThreadSEM.c and mtuThreadCHAN.c, respectively. Do not change the file names in the #include directives. Submit the following files only: Note that your readme and journal files must have file names README and JOURNAL rather than readme and journal.

You should enclose any place where you made a change and/or addition with #ifdef-#endif as follows:

#ifdef CHANGED
     put your changes/modifications/additions here
#endif
In this way, any modification/addition to the original files can be found easily. When you submit your program, don't forget to define CHANGED at the very beginning of mtuThread.h. However, if you only modify new_mtuThread*.c, this mechanism is not required.