Cover Story (sidebar) / May 1998

Benchmarking Java

Tom R. Halfhill and Al Gallant

To demonstrate the dramatic difference a just-in-time (JIT) compiler makes, the BYTE Lab ran the Java version of our BYTEmark program (jBYTEmark) on a Dell 90-MHz Pentium system with Windows 95 and the two most popular Web browsers. We also compiled the jBYTEmark source code with two releases of the same Java development tool (Symantec's Visual Café). The results show that interpreters are no match for a good JIT compiler.

Netscape Communicator 4.01 doesn't let you turn off the JIT compiler, but Microsoft Internet Explorer 4.0 does. With the JIT compiler off, Explorer's performance was close to the jBYTEmark reference baseline of 1.0. (We established the baseline in 1996 by running jBYTEmark on the same Dell Pentium-90 system with Symantec Café 1.0.) The results were almost identical after recompiling jBYTEmark with Visual Café 2.0, the latest release.

With the JIT compiler turned on, Explorer roared through the benchmarks. It ran the integer routines 10 times faster and the floating-point routines nearly six times faster. Again, recompiling the program with Visual Café 2.0 made virtually no difference.

Communicator lagged behind Explorer, but it ran the integer tests more than eight times faster and the floating-point tests more than five times faster than the baseline.

Note that jBYTEmark is a fairly straight port from the C version of BYTEmark. It's a CPU-intensive synthetic benchmark program that uses a mix of algorithms found in modern applications. However, it does not include any Java-specific tests to measure such critical performance factors as object creation/destruction, garbage collection, or thread synchronization. Actual performance could vary widely from these results.

The JIT Benefit

Bar chart of
                  Java ByteMarks.

Copyright © 1994-1998 BYTE

Return to Tom's BYTE index page