Which Java VM scales best? - JavaWorld - August
1998
Tutorial Details:
Which Java VM scales best?
Which Java VM scales best?
By: By John Neffenger
A JavaWorld Exclusive! Results of new VolanoMark 2.0 server benchmark show how 12 virtual machines stack up
find it ironic that the one company with a not-so-secret objective to "kill cross-platform Java" (according to an internal Microsoft document; see Resources below) also is the Java vendor that kept our 100% Pure Java company (Volano) in business for the past year and a half. Without the impressive performance of Microsoft's Java virtual machine, we would not have acquired the many VolanoChat customers who run our chat server on high-traffic Web sites. My article in the December issue of JavaWorld summarized our experience and reported the performance scores of four Java virtual machines using our Java server benchmark, VolanoMark 1.0 .
While writing that article in November, I suggested to a Microsoft product manager that Sun might eventually trade places with Microsoft as the Java performance leader. "It'll never happen," was his quick reply. Although it might still be hard to argue with his response, most of the Java VM vendors have significantly narrowed Microsoft's lead.
Mostly in response to the December article, I have received more than 450 e-mail messages from 19 Java vendors trying to solve the technical problems they encountered while running VolanoMark and various Java server applications that the benchmark represents. Their efforts are now starting to reduce the great disparity that exists between Java virtual machines commonly used on the Internet today.
What is VolanoMark?
VolanoMark is a 100% Pure Java server benchmark characterized by long-lasting network connections and high thread counts. In this context, long-lasting means that the connections last several minutes or longer, rather than just a few seconds. The VolanoMark benchmark creates client connections in groups of 20 and measures how long it takes for the clients to take turns broadcasting their messages to the group. At the end of the test, it reports a score as the average number of messages transferred by the server per second. Its results have accurately predicted the real-world Java performance and scalability limits of our VolanoChat product line for almost two years now.
VolanoMark 2.0, which adds scalability measurements, is an update to the tool and is the first benchmark to be submitted to the SPEC Open System Group for use in its server-side Java benchmark suite. There are other Java benchmark programs under consideration for the SPEC suite that have different characteristics, such as the high connection-turnover rates of a Java Web server or the transaction-processing requirements of a Java database.
When trying to determine whether a product meets your performance and scalability requirements, the best approach is to write your own benchmark and obtain your own results. The next best approach is to run a benchmark program yourself on your own system. VolanoMark 1.0, for example, is available as a free download from Volano's Web site. (See the Resources section below.) In any case, when you read benchmark reports you should make sure you understand the characteristics of the test and the configuration of the system. Without information about the hardware configuration, the operating-system settings, and the Java virtual machine heap and stack options, benchmark scores are meaningless. Fortunately, SPEC will provide a set of well-defined rules for running its server-side Java benchmark suite when it becomes available early next year.
The benchmark tests
The tests presented in this report look at the scalability of 12 Java virtual machines on eight operating systems, using a common Intel hardware platform when possible.
The connection scalability tests are divided into two groups of Java virtual machines (VMs):
Java VMs available now as final production releases
Java VMs scheduled for release later this year
The processor scalability tests compare Microsoft's virtual machine on Windows NT with Sun's virtual machine on Solaris Intel Edition and Solaris SPARC Edition. Note that these processor scalability tests ran on different hardware than all the other tests.
The final tests show a direct comparison of all the Java virtual machines, using the local loopback interface with the client and server application running on the same machine.
Except for the processor scalability results, all tests ran the VolanoMark server on a 200-megahertz (MHz) Intel Pentium Pro processor with a 256-kilobyte (KB) L2 cache and 256 megabytes (MB) of RAM. The client connected to the server machine via a 10-megabit-per-second (Mbps) Ethernet network. For the VolanoMark network test, the client side ran on a 200-MHz Intel Pentium Pro processor with a 256-KB L2 cache and 128 MB of RAM using Microsoft's SDK for Java version 3.0 Pre-Release 1 (jview version 5.00.2750). The Apple JVM tests ran on a Apple Power Mac G3.
We ran the VolanoMark server and client applications with an increased initial heap size of 8 MB, an increased maximum heap size of 64 MB, and a decreased native stack size of 32 KB (using java -ms8m -mx64m -ss32k ), except as follows:
The Microsoft SDK has no heap and stack command options available.
The Apple MRJ failed to run with the increased heap sizes.
The Novell JDK failed to run with a stack size less than its 48-KB default.
We did not decrease the 4-gigabyte (GB) default maximum heap size of the TowerJ-generated executable.
With the exceptions stated above, the commands were entered as follows, starting with two connections and working up to 900 connections:
Server
java -ms8m -mx64m -ss32k COM.volano.Main
Client
jview COM.volano.Mark -rooms 1 -users 2 -count 10000
jview COM.volano.Mark -rooms 5
...
jview COM.volano.Mark -rooms 45
See the COM.volano.Mark command synopsis for a complete description of all options.
To give you an idea of the heap usage, consider that the VolanoMark server uses 6,136 KB of heap memory when it runs under JavaSoft JDK 1.1.6 with 200 connections ( -rooms 10 ) and with the garbage collector disabled.
All connection scalability tests were performed without restarting the server between tests, except for the JavaSoft JDK 1.1.6 and Apple MRJ 2.0 JVMs, which required a restart in order to reach 900 connections. Restarting these JVMs may have caused a variation in their scores due to the effects of the garbage collector, but not enough to alter the general picture of the results.
In this article, the SunSoft JDK refers to what Sun calls the Production Release for Solaris. The JavaSoft JDK refers to Sun's JDK 1.1 Win32 Release. See the Resources section for links to the download locations of each JVM.
Current Java VMs: Free and available now
Connection scalability of Java virtual machines that are freely available now. Bigger numbers are better. All JVMs ran the same test on the same Intel hardware platform except for Apple MRJ 2.0, which ran on a Power Mac G3.
These seven Java virtual machines are the ones we recommend to our customers for running the VolanoChat product live on production Web sites. The chart shows the VolanoMark 2.0 scores for seven Java VMs as the number of connections are increased. The number of connections are shown across the bottom, with the throughput in messages per second shown on the left. The scores are shown at up to 900 concurrent connections. Failed tests in the table are indicated by " --- " (no score shown).
Microsoft SDK 2.02 still stands alone as the only fast and scalable Java virtual machine. Our customers with the highest Web site traffic currently have no other viable choice for a JVM. I had to restart the JavaSoft JDK 1.1.6 virtual machine five times between tests in order to reach 900 concurrent connections, and none of our large customers have succeeded in running our VolanoChat product with JavaSoft's JVM on Windows NT. IBM's recent updates to its operating system, TCP/IP stack, and JVM have given it a huge 50 percent performance improvement over the VolanoMark 2.0 tests I ran without the updates (not shown here), but I could not get more than 400 concurrent connections with IBM's JVM.
We had been waiting for more than a year for a stable Solaris Java VM and Sun finally delivered.
The SunSoft JDK 1.1.5 VM for Solaris may be a bit laggard in its performance, but its reliability is rock solid. We had been waiting for more than a year for a stable Solaris Java virtual machine with native threads and a just-in-time compiler, and Sun finally delivered on April 6, 1998. For me this was the biggest Java news of this year, but it went completely unnoticed (perhaps because its robust performance was not immediately apparent to the trade press that covered its release). After working with Java virtual machines on the server since early 1996, I had begun to lower my expectations of quality -- but the SunSoft JDK 1.1.5 sets a new standard for stability. No other JVM comes close. We ran our VolanoChat demonstration server on this JVM for more than 32 days non-stop -- that's roughly 40 messages per second, 24 hours per day for 32 days, without any down time and without even going over its initial 8-MB heap size! I think it would have run forever, but I had to bring it down in order to upgrade the VolanoChat server. (In fact, we haven't rebooted the Solaris 2.6 operating system since we first set it up on our Internet Web and chat server machine on November 5, 1997.)
Number of connections
Java Virtual Machine
2
100
200
300
400
500
600
700
800
900
Microsoft SDK 2.02
1751
2525
2090
1767
1556
1365
1242
1126
1030
946
JavaSoft JDK 1.1.6
1420
2082
1997
1797
1578
1381
1235
1099
1011
951
IBM JDK 1.1.6
1235
2009
1843
1853
1603
---
---
---
---
---
SunSoft JDK 1.1.5
902
1221
1134
1051
888
725
322
178
145
102
Apple MRJ 2.0
119
1090
950
781
877
767
---
---
---
---
Linux JDK 1.1.6
836
688
429
---
---
--
Read
Tutorial at: Click here to view the tutorial
Rate Tutorial: Which Java VM scales best? - JavaWorld - August
1998
View Tutorial: Which Java VM scales best? - JavaWorld - August
1998
Related
Tutorials:
 Enhance your Java application with Java Native Interface (JNI)
Enhance your Java application with Java Native Interface (JNI)
|
Explore the
Dynamic Proxy API
Explore the
Dynamic Proxy API
|
Cut down on
logging errors with Jylog
Cut down on
logging errors with Jylog
|
Pick up
performance with generational garbage
collection
Pick up
performance with generational garbage
collection
|
Rumble in the
jungle: J2EE versus .Net, Part
1
Rumble in the
jungle: J2EE versus .Net, Part
1
|
Should you go
with JMS?
Should you go
with JMS?
|
J2ME devices:
Real-world performance
J2ME devices:
Real-world performance
|
Effort on the
edge, Part 1
Effort on the
edge, Part 1
|
Profiling CPU
usage from within a Java application
Profiling CPU
usage from within a Java application
|
An Introduction to Cryptography and the Java Cryptography Extension
An Introduction to Cryptography and the Java Cryptography Extension
The Java Cryptography Extension (JCE) provides APIs for performing cryptographic operations in Java code.
the JCE lets us scramble and unscramble data, annotate code and data with infor
|
Use AOP to maintain legacy Java applications
Use AOP to maintain legacy Java applications
This artical shows you how to use aspect-oriented programming (AOP) to gain an unprecedented view into the inner workings of even the most opaque of legacy applications.
Please note that this article assume
|
Declarative Programming in Java
Declarative Programming in Java
What makes EJB components special is the declarative programming model through which we can specify the services such as security, persistence, transaction etc., that the container should provide. An EJB only implements
|
A Collection of JVM Options
This document is a compilation of all the JVM options for various versions of the JVM on primarily SPARC/Solaris Platform. The descriptions for each option are taken mostly verbatim from the reference documents.
|
Light-Weight Visual Components Library for different platform: SWT, J2SE, J2ME, .NET
Light-Weight Visual Components Library for different platform: SWT, J2SE, J2ME, .NET
|
Garbage Collection in the Java HotSpot Virtual Machine
Garbage Collection in the Java HotSpot Virtual Machine
Gain a better understanding of how garbage collection in the Java HotSpot VM works, and learn to take full advantage of it when designing, developing, and deploying your Java applications.
|
Commons-Math: The Jakarta Mathematics Library
Commons-Math: The Jakarta Mathematics Library
The Java programming language and the math extensions in Commons Lang provide implementations for only the most basic mathematical algorithms. Routine development tasks such as computing basic statistics or s
|
Lucene in Action
Lucene in Action
Lucene is a gem in the open-source world--a highly scalable, fast search engine. It delivers performance and is disarmingly easy to use. Lucene in Action is the authoritative guide to Lucene. It describes how to index your data, includin
|
Java Resources
There are all Java freebies. Some of these are old, and not under maintenance. Download and use them at your risk. In case of queries, mail subrahmanyam_avb@technologist.com or varalakshmi_a@techie.com.
|
Introduction to Servlets, JSP, and Servlet Engines
Servlets are the Java Technologies' answer to CGI programming. They are programs which run on the server side and generate dynamic content. Why would one prefer to use Servlets over traditional CGI programming?
|
Struts, JavaServer Faces, and Java Studio Creator:
The Evolution of Web Application Frameworks Sun Microsystems' Craig McClanahan, the creator of the Apache Struts Framework, co-specification lead for JavaServer Faces 1.0, and prime architect for Sun Java Studio Creator's new release, explains all three.
|
|
|
|
