Computational Science & Engineering Research Institute
Solving complex science and engineering problems
using high performance computers
In a fluidized diluted granular flow, grains
behave like gas molecules.
Applying the Bolztmann kinetic-collisional model to a
S. Dartevelle, Ph.D. student, Geological & Mining Engg. & Sci.
NASA Goddard Space Flight Center Beowulf Cluster.
64 2-processor PC's connected by a 100Mb/sec switch.
P. Merkey, Research Ass't Prof., Computer Science Dept.
Evolution of a non-evaporating fuel spray.
Colors represent droplet radii in microns.
F. Tanner, Assoc. Prof., Mathematical Sciences Dept.
NSF Major Research Instrumentation Sun Enterprise.
12 Sun Ultrasparcs and a small disk farm.
S. Seidel, Assoc. Prof., Computer Science Dept.
The CS&E Research Institute was created in fall, 2002.
The discipline of Computation Science & Engineering is the application
of computational technologies
(computer hardware, software, networks, etc.) to current problems in
science and engineering. Typically, these problem are in areas of
national interest, including areas of economic interest and security,
and areas of state-wide economic interest. Examples of specific problems
being studied by the CS&E faculty at MTU are described below.
The CS&E Research Institute was created to:
- provide an academic environment in which researchers (faculty and
students) can collaborate on problems and technologies of common interest,
- bring researchers together to develop large scale, long range,
interdisciplinary, research programs,
- foster the development of the CS&E Ph.D. program, and
- provide access to medium- and large-scale computational facilities that
researchers could not otherwise afford to acquire.
The creation of the CS&ERI was approved by:
- Curt Tompkins, President
- Kent Wray, Provost and Sr. Vice President
- David Reed, Vice President for Research
- Bruce Rafert, Dean of the Graduate School and Distance Learning
- Maximilian Seel, Dean of the College and Sciences and Arts
- Robert Warrington, Dean of the College of Engineering
- Linda Ott, Chair of the Department of Computer Science
- Alphonse Baartmans, Chair of the Department of Mathematical Sciences
- Ravindra Pandey, Chair of the Department of Physics
- Theodore Bornhorst, Chair of the Department of Geological and Mining
Engineering and Sciences
- Michael Mullins, Chair of the Department of Chemical Engineering
- William Predebon, Chair of Mechanical Engineering - Engineering Mechanics
- Timothy Schulz, Chair of Electrical and Computer Engineering
- Steven Seidel, Associate Professor of Computer Science
- Phillip Merkey, Research Assistant Professor of Computer Science
A complete description of the CS&E Research Institute can be found in the
Phillip Merkey is the director of the CS&E Research Institute.
What Is CS&ERI Research All About?
How is CS&E research different than traditional science
and engineering research?
- CS&E problems cannot be solved in a laboratory.
CS&E problems cannot be solved
by direct ("wet lab") experimentation or by field observation.
CS&E problems are too large to "fit" into a laboratory.
The only way to solve them is to simulate them inside a computer.
- CS&E problems cannot be solved on your desktop computer.
It would take years or centuries to solve CS&E problems on a typical PC.
Much much more powerful computers are required.
To solve the largest and most interesting problems, more complex
computer systems are an absolute necessity.
- CS&E research is interdisciplinary.
CS&E research is always a combination of sophisticated computational
methods applied to the specific science and engineering problems at hand.
CS&E researchers (students and faculty) have been trained in their
application areas (the science or engineering problems), they have
been trained in the latest computational techniques (i.e., computer
program design and implementation), and they understand how to use the
most powerful computer systems to solve their problems.
- CS&E research is different than research in computer science.
Research in computer science is centered on problems
such as how to build better, faster, and cheaper computers,
how to design, implement, and maintain software. Some typical research
areas in computer science are graphics, computer architecture,
programming languages, operating systems, networks, databases, and
the theory behind all of these topics. Computer science research is
not concerned with physics problems, problems in civil or mechanical
engineering, problems in oil field exploration, plate techtonics,
or atmospheric chemistry. On the other hand, CS&E research is very
much interested in applying the latest computer science advances in
graphics, computer architecture, programming languages and networks
to problems in physics, civil and mechanical engineering, oil field
exploration, and so on. The discipline of Computational Science and
Engineering is the application of computer science to problems in the
sciences and engineering.
- Important CS&E problems are:
- Energy efficiency: combustion processes and emissions
- Simulated crash testing of automobiles
- Environmental studies: pollution in the ground, water, and air
- Molecular dynamics: simulating the behavior of atoms and molecules
- Computational chemistry: simulating chemical processes
- Bioinformatics: genetic sequencing and recombinant studies
- Weather forecasting, including hurricane and tornado tracking
- Aeronautics: airframe design and dynamics
- National defense and security interests: bio/chemical terrorism,
- Nuclear stockpile stewardship
What Makes CS&E Computers Different Than The One On Your Desktop?
In most cases, computers used in CS&E research are ``parallel" computers
made up of tens or hundreds of ordinary computers all connected by a
network that allows the computers to ``talk" to each other while they work
on solving a problem.
- Parallel computing: It's not the number of processors;
it's the network.
People working on a project team not only do work, but they cooperate
with each other to
coordinate their efforts. This requires that team members communicate
with one another. The same feature distinguishes parallel computers
from ordinary PC's.
In the photos below it is important to notice more than just
the number of processors (CPUs) that each parallel computer has.
In most cases the photo at the right shows the network technology used to
connect the computers.
In some cases the cost of the network is about the same as the cost of
the computers it connects. This is convincing evidence that a fast
network is as important as fast computers.
- Parallel programs: A program for a parallel computer must
divide the work between those computers.
All of the computers work on the problem at the same time.
Each computer does its share of work just as members of a project
team all contribute to the final result.
- Parallel computers speed up the time to solution and parallel
computers solve larger problems: These two advantages go hand-in-hand.
If more workers are assigned to a task, then we can expect the task to
be completed sooner. Moreover, each computer working on the task comes with
its own memory. The more computers there are, the more memory is
available, and since more memory is available, more data can be
used to represent the problem. This means larger problems can be solved.
Problems such as predicting global warming require computers with as
much memory as possible.
I'm still confused.
Don't most researchers at MTU use computers?
How is CS&E research different than what these researchers do?
We all use computers for word processing, web browsing and email. None of
that begins to tax our computers.
When was that last time that your own computer spent more than
20 seconds actually computing something? The only time that most of us
usually have to wait for our computers is when large amounts of data (e.g.,
pictures, movies, web pages) are being moved from one place to another
or while a large software package is being ``loaded".
Desktop computers today are so powerful that for most day-to-day tasks
(except 3-D game playing)
they spend most of their time doing nothing at all. Most of the computers
on our desktops spend most of their time waiting for us to do something!
In the laboratory, a researcher might
use a computer to control the inputs to an experimental device or
to automatically collect data during an experiment. Afterwards a
computer is used to analyze the data that have been collected
and to display the results using commercially available software.
Sometimes the researcher writes a programs to analyze the data in a way
particularly fitting to the science or engineering problem at hand.
From the computer's point of view, none of the work it is required to
do is at all taxing.
The computer typically spends very little time doing this
work; most of its time is spent waiting for the laboratory devices or
for the researcher. Even though a computer and its software is an
important laboratory tool, the particular computer is not critically
important to the work being done. Any reasonably modern PC can usually
do these jobs without breaking a sweat.
On the other hand, many CS&E researchers do not work in a traditional
laboratory. They do not do experiments or make field observations. All
of their research is done inside the computer. Their work is based on
mathematical models of physical phenomena. A much of
their research effort consists of turning those mathematical models into
computer programs. Their computer programs
simulate cloud formation, the flow of pollutants through the soil,
the pressures on an aircraft wing, or the evolution of black holes.
All of these "experiments" are artificial but, if carefully designed
models are used, they
mimic the real world so accurately and in so much detail that the results
reveal what would actually happen in the real world.
It is computational scientists who tell us that global warming is
(or is not) happening. It is computational scientists who design new
drugs without ever using a test tube.
Many important problems today cannot be solved by traditional
laboratory studies. The largest
and most powerful computers are the only and best tools for
scientists and engineers to use to solve these problems.
OK, now I understand that CS&E research is focussed on science and
engineering problems that can only be solved using computers ...
... but there are so many different units at MTU with "computer" in their
names. What are the differences between ... ?
- The CS&E Research Institute: The CS&ERI was created as a home
for researchers to collaborate on computational problems and techniques
of common interest, to facilitate the development of long range research
programs, to support the CS&E Ph.D. prgram, and to provide access to
medium- and large-scale computational facilities that would not
otherwise be available.
- The CS&E Ph.D. program: The CS&E Ph.D. program is the
primary educational component of the CS&E Research Institute. A more detailed
description of the CS&E Ph.D. program is given in the next section.
- The Department of Computer Science: The CS Department offers
B.S., M.S., and Ph.D. degrees in computer science. It is concerned with all
aspects of undergraduate and graduate computer science education.
It has established research
programs in a number of areas including parallel computing and advanced
architectures, compiler technology, graphics, distributed systems, and
several others. Some of the CS Department's educational and
research activities are directed toward computational science and
engineering, but there are many other areas of computing technology that
fall within the scope of the CS Department.
- The Computer Science Ph.D. program:
The CS Ph.D. program was established in the fall of 2001. It was
originally part of the CS&E Ph.D. program. Now that these two programs
have separated, the CS Ph.D. program concentrates on graduate
education in the CS research areas such as those mentioned above, and the
CS&E Ph.D. program concentrates on interdisciplinary graduate education
and research in the application of computational techniques to
current problems in the sciences and engineering.
- The Computer Engineering component of the Electrical and
Computer Engineering Department: The CE "department" is closely
allied with the CS Department. Much of their curricula overlap.
Some might say that
computer engineering is more often concerned with hardware and
computer science is more often concerned with software, but this
is a gross oversimplification and in many cases simply incorrect.
Here is an example of the close association between the disciplines of
computer engineering and computer science.
Special purpose computers embedded in "appliances"
(anything from microwave ovens, cell phones, automobiles, to rockets)
might fall closest to the domain of computer engineering. Designing
programming languages, compilers, and software for those computers is
likely to be the domain of computer science. Further discussion of these
distinctions would be out of place here. The main point is that both
computer engineering and computer science are distinct from computational
science and engineering because CS&E research primarily focusses on problems
in the physical world, not problems in the world of computers.
- The Information Technology department in EERC: IT is MTU's
phone company, internet provider, and cable TV company. It is a
business and service branch of MTU. IT provides internet connectivity
across campus and to the outside world, but IT does not provide
computational services to MTU researchers.
The Computational Science & Engineering Ph.D. program
One of the primary goals of the CS&E Research Institute is to foster
graduate research in computational science and engineering.
This is being accomplished through the CS&E Ph.D. program.
- The CS&E PhD program is nondepartmental. The CS&E PhD program
is one of two active nondepartmental PhD programs at MTU created
under the "Ph.D. in Engineering" umbrella Ph.D. program originally
established more than 10 years ago. (The other active program in this
category is Environmental Engineering.)
- What does "nondepartmental" mean? There is no one department
that "owns" the CS&E PhD program. Instead, it is directly responsible
to the Graduate School. No other PhD program
at MTU has this relationship; all other PhD programs at MTU are
responsible to a specific "home" academic department. The Department
of Computer Science is responsible for the day-to-day administration
of the CS&E PhD program but, in principle, any MTU department may
take on this responsibility.
- Any graduate-degree-granting department at MTU may grant a CS&E
PhD. This is a tremendously important and distinguishing feature of
the CS&E PhD program at MTU. No other PhD program at MTU has this feature.
One consequence of this fact is that any MTU department can be a "home"
to a CS&E PhD student. The home department provides the student's
advisor, financial support, basic computational facilities, and all
other support that is usually provided to students in the department's
own PhD program. However, the student is subject to the degree requirements
of the CS&E PhD program rather than the degree requirements of the PhD
program of the home department. This frees the student and his/her
advisor to construct a truly interdisciplinary degree program that
best fits a computationally-intensive research emphasis.
- Recent history of CS&E PhD program.
Two recent events have had a significant impact on the development of the
CS&E PhD program.
- The CS&E PhD program was originally created to
house two groups of graduate students: those doing interdisciplinary
research in computational science (as applied to problems in the sciences
and engineering) and those studying current research in the discipline
of computer science (as applied to traditional areas of computer science
such as hardware, software, networks, etc.). In the fall
of 2001 the Computer Science PhD program was approved by the Board of
Control. Since then the CS&E PhD program has been able to focus solely
on the mission of producing PhD graduates in the interdisciplinary area
of computational science.
- In the fall of 2001 the CS&E PhD program underwent an external
review, as mandated periodically by the Graduate School for all MTU
One of the primary recommendations of that review was that a CS&E Research
Institute should be created to help foster growth in the computational
sciences at MTU. This goal was accomplished in the fall of 2002 with
the creation of the CS&E Research Institute.
Summary: The CS&E PhD program has more flexibility and
interdisciplinary breadth than any other PhD program at MTU. It is the
only PhD program at MTU that reports directly to the Graduate School,
and it is the only PhD degree at MTU that can be awarded by more than
one department. A primary goal of the newly created CS&E Research
Institute is to foster the development of the CS&E PhD program.
Current CS&E Projects
NASA: Earth and Space Sciences Support for NASA High
Performance Computing and Communication Program,
P. Merkey (PI), S. Seidel (Co-PI), $758,000.
NASA cluster: A cluster built from 64 ordinary PC's.
Cluster interconnection network:
The 100Mb/sec Ethernet switch that connects the PC's in this cluster.
You are looking at the "business ends" of 64 dual-processor PC's. Note
that there is only one keyboard and monitor. All of these PC's are
connected to form a single, parallel computer. They are connected by
an ordinary Ethernet switch (right) that is just like the one that
connects your desktop PC to all other PC's in your local network.
This "Beowulf" cluster was
provided to MTU by NASA Goddard Space Flight Center to support
parallel computing projects in the CS&ERI.
Hewlett-Packard: UPC Technology Development Project,
S. Seidel (PI) and Co-PIs P. Merkey and C. Wallace, $179,000.
Over the past two year Drs. Steven Seidel, Phillip Merkey, and Charles
Wallace have received funding from Hewlett-Packard to pursue four projects
centered on the development of the Unified Parallel C programming language.
UPC is a new language for high performance computing on distributed shared
memory architectures. One of this year's projects continues the work begun
last year on the MuPC run time system for UPC. New this year are projects
to formally specify the UPC memory consistency model, examine
programmability and usability aspects of UPC, and produce a specification
for UPC collective communication operations.
Programming parallel computers is a more difficult task
than programming ordinary computers because a parallel program
has to control many computers at once. Many familiar programming
languages used for scientific computing, such as C and Fortran, have
been modified so that they can be used to program parallel computers.
One of the newest parallel programming languages is UPC, which stands
for Unified Parallel C. This language is currently being studied by
research groups at UC Berkeley, George Washington University, and MTU.
There is current interest in UPC because it appears to have a
well designed collection of commands that allows programmers to
more easily write parallel programs to solve complex problems.
One of the current CSERI projects is to develop a portable
version of the UPC programming language that can run on any Linux-based
computer system. This version is called MuPC (roughly standing for
"Michigan Tech UPC"). One of the reasons for developing MuPC is
to help "spread the word" about UPC, so MuPC is freely available to
the CS&E research community in the U.S.
The Cray T3E: These three cabinets contain 60 processors
and lots of wiring to connect them all.
Inside: This is the inside of one of the cabinets. All of
those cables are used to interconnect the processors and to keep
them in step with one another.
In association with this work, the CS&ERI has received a Cray T3E
supercomputer. This machine originally cost about $1.5M.
It is a few years old now, but current models of this supercomputer
maintain their place as the fastest machines in the world.
This system was provided to MTU to support UPC-related research.
The CSERI has made this system available to researchers at many U.S.
institutions, including the UPC research groups at Berkeley and at
George Washington University.
NSF: Sandu's CAREER and ITR funding:
Sandu, et al., $450,000.
The newest CS&E cluster: 20 dual processor 2GHz PC's.
Two interconnection networks: The processors are
connected with standard Ethernet (top) and by a very fast Myrinet
fiber network (bottom).
The computer: This computer is a brand new "Beowulf" cluster.
It is similar to the cluster used for the NASA project (two frames
above) but it is built using the latest technology. Each PC is
about 1.5" thick and houses two 2 gigahertz processors. There are
16 of these PC's (in groups of 8) above and below the two switches
that connect them. These switches are shown on the right. The
top switch is a standard ethernet switch like that used for the
NASA cluster. The bottom switch uses very fast fiber optic
technology (that's why the wires are thinner) to connect the computers.
This system was acquired with the NSF Major Research Instrumentation
grant described below.
NSF: Computational Facilities for MTU's CS&E
S. Seidel (PI), and Co-PIs C. Friedrich, J. Jaszczak,
A. Mayer, $260,000.
This is an equipment acquisition grant to support research and training in
computationally intensive projects in several disciplines. Participating
researchers come from the departments of physics, mechanical
engineering-engineering mechanics, metallurgy and materials engineering,
geological engineering and sciences, and computer science.
The list of current departments has since grown to include mathematical
The computational system acquired in this project is a primary tool
used in their work. Other research projects that use this facility
are in the areas of computational fluid dynamics, materials science,
groundwater flow, and interprocessor communication. This computational
facility is being used to expand the scale of problems that
these projects address.
Over the past 15 years, computing at MTU has evolved from a centrally
located, centrally administered "computer center" to a completely
distributed, self-administered web of desktop PC's. This trend put
computers on our desks, but by doing so it cut up the computational
power of these computers into very small pieces. As explained above,
these PC's are more than adequate for doing day-to-day tasks, but
certain researchers require much greater computational power. For
many years, medium- and large-scale computers were not available at
MTU because all computing had been "distributed" to the desktop.
This NSF grant was one of the first to reverse this trend and make
larger computers again available to researchers at MTU
This multi-year grant was obtained several years ago. The first piece
of equipment acquired was the Sun Enterprise 4500, described below.
This computer is freely available to scientists and engineers for
computational work that is too large for their desktop machines.
During the past several years faculty and students from the College of
Engineering and the College of Sciences and Arts have used this
computer for a wide range of work including:
- The study of volcanic processes
Student: Sebastiene Dartevelle,
Ph.D. student, Geological and Mining Engineering and Sciences
Advisor: Bill Rose
- The study of volcanic processes
Student: Song Guo,
CS&E Ph.D. student, Geological and Mining Engineering and Sciences
Advisors: Gregg Bluth and Bill Rose
- Groundwater remediation studies
Student: Mark Erickson,
M.S. student, Geological and Mining Engineering and Sciences
Advisor: Alex Mayer
- Project title needed
Student: Hanyi Li,
CS&E Ph.D. student, Geological and Mining Engineering and Sciences
Advisor: Judith Budd
- Project title needed
Student: Krista Stalsberg-Zarling,
CS&E Ph.D. student, Mathematical Sciences
Advisor: Franz Tanner
- Discrete atomic modelling
Investigator: Jong Lee,
Material Sciences and Engineering
- Other regular users of cse0.cse.mtu.edu?
Sun Enterprise 4500: 12 server-class Sun processors.
The computer: This is 12 processor Sun Enterprise 4500.
Unlike the two "Beowulf" clusters above, all connections between
processors are contained within the cabinet.
This system was acquired with an NSF Major Research Instrumentation
grant to support CS&E research at MTU.
Collaborator and Sponsor: Helsinki University of Technology (HUT), Finland
Principal Investigators: Franz X. Tanner (MTU) and Martti Larmi (HUT)
Ph.D. Student: Krista Stalsberg-Zarling
The simulation of spray combustion requires the modeling of turbulent,
reacting multiphase flows, where the gas phase is described with the
time-dependent, compressible, Reynolds/Favre-averaged conservation
equations for mass, species, momentum and energy, together with the
equations for the turbulence model. The liquid phase is governed by a
stochastic formulation of the discrete droplet model, where the liquid
and gas phases are treated separately. The liquid-gas interaction, as
well as the species creation and depletion due to chemical reactions,
are coupled with appropriate source terms in the gas conservation
One of the main problems in the modeling and simulation of two-phase
flows, using the discrete droplet model approach, is the intrinsic
dependence of the liquid-gas coupling on the spatial resolution of the
gas phase equations. The objective of this research project is the
development of a general method which reduces or eliminates such
This figure shows a numerically simulated volcanic
plume/cloud 115 minutes after eruption. The simulation was done
using ATHAM (Active Tracer High Resolution Atmospheric Model).
Current CS&E Ph.D. Students
Students in the CS&E Ph.D. program are members of their
"home" department. Their advisor, their funding, their office, etc.
all come from their home department. Yet these students are
all in the same degree program and must meet the same set of
CS&E degree requirements. This results in graduates who understand
their own science or engineering research area and who know how to
apply the most advanced computational techniques to solve problems
in that area. No other Ph.D. program at MTU is designed to meet
these goals and no other Ph.D. program at MTU
permits this level of interdisciplinary flexibility.
- Song Guo, Geological and Mining Engineering and Sciences
- Advisors: Gregg Bluth and Bill Rose
- Hanyi Li, Geological and Mining Engineering and Sciences
- Krista Stalsberg-Zarling, Mathematical Sciences
- Ping Huo, Computer Science
- Sakke Karstu, Computer Science
Current CS&E Faculty and Students
Links to projects that receive support from the
CS&E Research Institute
Other CS&ERI Activities
During the past year several visiting CS&E researchers have given talks at MTU:
- David Walker, Professor of High Performance Computing,
Department of Computer Science, Cardiff University, Wales
- Lori Freitag, Mathematics and Computer Science Division,
Argonne National Laboratory
- William Carlson, Center for Computing Sciences, Bowie, Maryland
- George Corliss, Dept. of Electrical and Computer Engineering,
Marquette University, Milwaukee, WI
- Brian Wibecan, UPC Development Team, Hewlett-Packard Company, Nashua, NH
What's On the Horizon for the CS&E Research Institute?
- Hewlett-Packard Company has agreed to provide the UPC research
group with a 16-node AlphaServer SC cluster in the near future.
- The first CS&E Ph.D. students are expected to graduate in the
next academic year.
- The new CILIT Computer Science Hall contains a CS&E lab and additional
environmentally controlled computer rooms for specialized CS&E equipment.