Project2: A Simple Distributed Computing Platform (Due at 11:59:59pm on 04/12/2012 (EST))
Description:
You are asked to develop a replicator (client) that distributes a large job over a number of computers (a server group) on a single switched LAN (our BU04 lab). In this assignment, a large (simulation) job can be divided into a number of small jobs, each of which can be assigned to one machine from the server group for execution. The execution results of the small jobs can be merged once all of them successfully terminate.
System Architecture:
client server1 server2 server3 ...
| | | |
| | | |
| LAN | | |
|------------------|-------------|--------------|-----
The client and servers are running Network File System (NFS) so that user files are visible at $HOME directory. You may want to set up the following environment:
• $HOME/replicate.hosts: a list of (server) hostnames which participate in the simulation.
• $HOME/replicate_out: the directory that stores the small job execution result.
The simulation program "hyper_link" (binary) is provided. In this assignment, you don''t need to know or care what "hyper_link" does, and actually it is a computing intensive (CPU demanding) simulator. The command line arguments of "hyper_link" are job# 100000 999 1 2 2 100, where the job number determines the number of small jobs in your simulation. To allow the client to run a large job, the job# should be given in a tuple: start, end, and step. For example, the command "hyper_link 1 100 1 100000 999 1 2 2 100" yields 100 small jobs with the job# starting from 1 to 100. Each small job produces a screen output (see example below) at the end (if finished successfully). Your code needs to redirect the output to a file and save it in $HOME/replicate_out. For example,
./hyper_link 1 100000 999 1 2 2 100
will produce a screen output looks like:
1:100000:999:2:0.5:1.124571e+00:7.744712e-04:6.245814e-01:6.957320e-04:0.000000e+00:0.000000e+00.
Requirements:
1. The communications between the replicator and servers are achieved through remote procedure calls.
2. A user interface is required for the replicator to control the server. A command line interface will be acceptable. A (working) graphic user interface (GUI) will impress the instructor and earn up to 20 bonus credits. Your interface should at least support the following operations.
o show the current CPU load of a certain server (if the server is active).
o show the current server status (active or inactive).
o stop a certain server.
o restart a certain server.
o show the CPU load (with the mark of hi-threshold) in the real time (for GUI interface only).
3. The replicator has to make sure all small jobs are successfully finished.
o If a server crashes (or not responsive), the running job (not finished yet) will be killed and rescheduled for execution.
o If a server CPU load exceed the preset threshold (the higher threshold), the replicator stops the server (and therefore kills the job).
o The replicate should keep polling the CPU load of the stopped server. Once the load becomes lower than the lower threshold (a preset value), the server should be reactivated to run the jobs.
o The replicator can also stop any server (through user interface) if needed. Once happened, the unfinished job will be killed.
o If a job terminates abnormally (e.g., being killed), the replicator has to reschedule the job execution later.
Submission:
Create a folder with the name of your Linux account, and copy all source code (better be divided into client and server two subdirectories) files/directories into the created folder. Make sure you have sshed to grail.cba.csuohio.edu, and then type: .
$ turnin -c cis620w -p proj2 your_folder