Make room for JavaSpaces, Part 2 - JavaWorld January 2000
Tutorial Details:
Make room for JavaSpaces, Part 2
Make room for JavaSpaces, Part 2
By: By Eric Freeman
Build a compute server with JavaSpaces
've just returned from the second Jini Community Summit, where Ken Arnold and I led a birds-of-a-feather session on JavaSpaces. This encouraging meeting highlighted a number of interesting JavaSpaces applications, including Bill Olivier's JavaSpaces-based email system at the University of Wales. Also featured was the Department of Defense's explorations of battlefield management software, in which battlefield resources, such as tanks, are represented in a space.
Make room for JavaSpaces: Read the whole series!
Part 1. Ease the development of distributed apps with JavaSpaces
Part 2. Build a compute server with JavaSpaces
Part 3. Coordinate your Jini apps with JavaSpaces
Part 4. Explore Jini transactions with JavaSpaces
Part 5. Make your compute server robust and scalable
In addition, there was significant interest in continuing to advance the JavaSpaces technology, and a number of areas were suggested for its improvement. One problem that the conference attendees wanted to confront was the painful process of setting up and running JavaSpaces (as well as Jini) for the first time -- probably the greatest barrier to using the technology. This problem is now being addressed in a new working group started by Ken Arnold, called Out of the Box (for more information, see the Resources section at the end of the article). Another area of community effort involves the development of helper or utility interfaces and classes that provide a valuable set of tools for new JavaSpaces developers. This article is based around one of those tools in particular: the compute server.
Basically, a compute server is an implementation of a powerful, all-purpose computing engine using a JavaSpace. Tasks are dropped into the space, picked up by processes, and then computed; the result is written back into the space and, at some point, retrieved. Beginning JavaSpaces programmers often ask how to implement such a system; in fact, the JavaSpaces Technology Kit ships with two sample compute servers, which perform computations for ray tracing and cryptography.
Compute servers are quite useful even for advanced programming. At the Jini Summit, it was decided that a common set of interfaces should be created to standardize JavaSpaces-based compute servers so that programmers could avoid reinventing the wheel every time they wanted to implement one. As a result, the community has created a working group whose goal is to build a specification and reference implementation of a compute-server architecture. One aim of this article is to kick off that work.
The compute server explained
A compute server provides a service that accepts tasks, computes them, and returns results. The server itself is responsible for computing the results and managing the resources that complete the job. Behind the scenes, the service might use multiple CPUs or special-purpose hardware to compute the tasks more quickly than a single-CPU machine could.
Historically, compute servers have been used to take advantage of the resources of large farms of processors in order to tackle computationally intensive problems -- ray tracing, weather modeling, cryptography, and so-called "grand challenge" computational problems. More recently, compute servers have begun to move into the mainstream, and have been put to work in a variety of environments, from the creation of financial models for Wall Street to the construction of large-scale order-entry and fulfillment centers that service the Web. All of these applications use the compute server to break a large computational problem into smaller problems, and then allow a distributed set of processors to solve these smaller problems in parallel.
Compute servers come in two forms: special-purpose and general-purpose. The SETI@home project is a good example of a special-purpose compute server. SETI@home makes use of the idle cycles of common desktop PCs to search for artificial signals of extraterrestrial origin within radio telescope data. The SETI compute server (which is formed by a group of PC users who have agreed to run the SETI@home screensaver on their PCs) is considered a special-purpose server because it only performs one type of computation: it searches for messages from outer space.
General-purpose compute servers allow you to compute any task -- you can add new tasks at will, and the server automatically incorporates their corresponding code. This is a powerful idea. For example, a corporation might create a compute server out of all the idle machines on its network. Computationally intensive problems can then be run across this compute server, making use of unused resources on the network as they become available. This lets the corporation more effectively use its existing installed base of hardware, thus reducing its overall hardware requirements. While you could do this with a special-purpose compute server, the administrative costs would be a nightmare; every time a new application needed to be run, it would have to be installed across the entire organization. With a general-purpose server, this happens automatically.
Many space-based compute servers have been built over the years. While I was part of the Linda group at Yale University, we built several such systems. The most sophisticated of these was a system called Piranha, built by classmate David Kaminsky. Piranha adaptively configured itself to the available resources on the network; as the name suggests, CPUs began feeding on the tasks that needed to be worked on as they became available. All our attempts at building compute servers were a bit unsatisfying, however, because the tasks had to be statically compiled into the server, thus limiting it to special purposes. JavaSpaces vastly improves this situation by moving code between processes -- you can move code into the compute server any time you need a new task computed. This is exceedingly easy with JavaSpaces; in fact, the properties of the space (inherited from Jini and RMI) allow it to transparently import new code into the compute server for you.
Compute servers can be built quickly with just a few lines of code. However, creating a reliable and robust compute server is a challenge that requires knowledge of most JavaSpaces and Jini APIs. In this article, I will explore many of the subtle aspects of JavaSpaces programming. My goal is to develop a basic compute server that can compute arbitrary tasks. Along the way, you will get a better feel for some of the JavaSpaces APIs that Susanne and I described in our last article.
Compute server design
Let's start with a big picture of the compute server and its relation to JavaSpaces. The typical space-based compute server looks like the figure below. A master process generates a number of tasks and writes them into a space. When we say task in this context, we mean an entry that both describes the specifics of a task and contains methods that perform the necessary computations. One or more worker processes monitor the space; these workers take tasks as they become available, compute them, and then write their results back into the space. Results are entries that contain data from the computation's output. For instance, in a ray-tracing compute server, each task would contain the ray-tracing code along with a few parameters that tell the compute process which region of the ray-traced image to work on. The result entry would contain the bytes that make up the ray-traced region.
A space-based compute server
(Illustration by James P. Dustin, Dustin Design)
There are some nice properties the contribute to compute servers' ubiquity in space-based systems. First of all, they scale naturally; in general, the more worker processes there are, the faster tasks will be computed. You can add or remove workers at runtime, and the compute server will continue as long as there is at least one worker to compute tasks. Second, compute servers balance loads well -- if one worker is running on a slower CPU, it will compute its tasks more slowly and thus complete fewer tasks overall, while faster machines will have higher task throughput. This model avoids situations in which slow machines becoming overwhelmed while fast machines starve for work; instead, the load is balanced because the workers perform tasks when they can.
In addition, the masters and workers in this model are uncoupled; the workers don't have to know anything about the master or the specifics of the tasks -- they just grab tasks and compute them, returning the results to the space. Likewise, a given master doesn't need to worry about who computes its tasks or how, but just throws tasks into a space and waits. The space itself makes this very easy, as it provides a shared and persistent object store where objects can be looked up associatively (for more details on these aspects of spaces, please refer to November's column ). In other words, the space allows a worker to simply request any task, and receive a task entry when one exists in the space. Similarly, the master can ask for the computational results of its (and only its) tasks -- without needing to know the specifics of where these results came from.
Finally, the space provides a feature that makes this all work seamlessly: the ability to ship around executable content in objects. This is feasible because Java (and Jini's underlying RMI transport) makes underlying class loading possible. How does this work? Basically, when a master (or any other process, for that matter) writes an object into a space, it is serialized and transferred to that space. The serialized form of the object includes a codebase , which identifies the place where the class that created the object resides. When a process (in this case, a worker) takes a serialized object from the space, it then downloads the object from the appropriate codeb
Read
Tutorial at: Click here to view the tutorial
Rate Tutorial: Make room for JavaSpaces, Part 2 - JavaWorld January 2000
View Tutorial: Make room for JavaSpaces, Part 2 - JavaWorld January 2000
Related
Tutorials:
 3D graphics programming in
Java, Part 3: OpenGL
3D graphics programming in
Java, Part 3: OpenGL
|
Java security evolution
and concepts, Part 5
Java security evolution
and concepts, Part 5
|
A birds-eye view of Web services
A birds-eye view of Web services
|
Will Big Blue
eclipse the Java
tools market?
Will Big Blue
eclipse the Java
tools market?
|
Create your own type 3 JDBC driver, Part 2
Create your own type 3 JDBC driver, Part 2
|
Yes, you can secure your Web services documents, Part 1
Yes, you can secure your Web services documents, Part 1
|
Check out three
collections libraries
Check out three
collections libraries
|
Effort on the
edge, Part 1
Effort on the
edge, Part 1
|
Get the inside
track on J2EE architect certification
Get the inside
track on J2EE architect certification
|
My kingdom for
a good timer!
My kingdom for
a good timer!
|
Once again, only
introduction
Once again, only
introduction
|
Finally, getting hands in !
Finally, getting hands in !
|
Impressive
!
Impressive
!
|
Fixing the Java Memory Model, Part 2
Writing concurrent code is hard to begin with; the language should not make it any harder. While the Java platform included support for threading from the outset, including a cross-platform memory model that was intended to provide \"Write Once, Run Anywh
|
Template-Based Code Generation with Apache Velocity, Part 2
Template-Based Code Generation with Apache Velocity, Part 2
As described in part one of this series, code generation typically uses a template engine to transform some kind of "model" into compilable code, given the formatting specified by a template.
|
Extensible Code Generation with Java
Extensible Code Generation with Java Part 2
In part 1 of this series, we looked at the idea of generated code, which is code written not by hand but by another application. The appeal of generated code is that it can eliminate drudgery (and mistakes) i
|
Java Development on Eclipse, Part 2
Java Development on Eclipse, Part 2
Editor's note: In part one of this two-part series of excerpts from Eclipse, author Steve Holzner provided examples of how Eclipse makes it easier to create Java code from scratch. Continuing in that vein, in this we
|
Tech Tip: Using the Varargs Language Feature
Have you ever needed to pass in many instances of the same object type to a method, but you don't know at compile time how many instances there will be? Find out how the new varargs language feature makes it easy to handle situations like this.
|
NetBeans IDE 4.1
Out-of-the-box support for J2EE 1.4 and Web Services. Check out what early access release 2 can do for you!
|
New Technical Articles: 64-bit Programming on Solaris 10 OS for x86 Platforms
Four technical articles describe the new Sun Studio 10 software's 64-bit programming features on the Solaris 10 OS for x86 and AMD64 platforms. Important issues regarding the AMD64 ABI (Application Binary Interface), debugging, migration to 64-bits, and p
|
|
|
|
