Testing manual optimisation

I have just made a few simple tests to see how much code movement of the sort performed by my optimisation stage actually helps Java programs’ run time, and the results are fairly disappointing.

First, I tried moving calls to Array.length, using this code:

public class ManualArray {
public static void main(String[] args) {
char[] array = new char[10000000];

long startTime = System.currentTimeMillis();
for (int i = 0; i < array.length * 100; ++i) {
}
long normalTime = System.currentTimeMillis() - startTime;

startTime = System.currentTimeMillis();
int length = array.length;
for (int i = 0; i < length * 100; ++i) {
}
long factoredTime = System.currentTimeMillis() - startTime;

startTime = System.currentTimeMillis();
for (int i = 0; i < array.length * 100; ++i) {
}
long normalTime2 = System.currentTimeMillis() - startTime;

System.out.println("Normal: " + normalTime + " ms");
System.out.println("Factored: " + factoredTime + " ms");
System.out.println("Normal: " + normalTime2 + " ms");
}
}

The output from running this (compiled with javac 1.6.0_03-p3, running with java 1.6.0_03-p3, both from Sun) looks something like this:

$ java ManualArray
Normal: 2172 ms
Factored: 730 ms
Normal: 726 ms

Running with java -server to use the Java HotSpot Server VM, the results are not promising either:

$ java -server ManualArray
Normal: 9 ms
Factored: 0 ms
Normal: 0 ms

I then tested moving a simple method call:

public class ManualMethod {
public static void main(String[] args) {
TestCollection list = new TestCollection(1000000000);

long startTime = System.currentTimeMillis();
for (int i = 0; i < list.size(); ++i) {
}
long normalTime = System.currentTimeMillis() - startTime;

startTime = System.currentTimeMillis();
int length = list.size();
for (int i = 0; i < length; ++i) {
}
long factoredTime = System.currentTimeMillis() - startTime;

startTime = System.currentTimeMillis();
for (int i = 0; i < list.size(); ++i) {
}
long normalTime2 = System.currentTimeMillis() - startTime;

System.out.println("Normal: " + normalTime + " ms");
System.out.println("Factored: " + factoredTime + " ms");
System.out.println("Normal: " + normalTime2 + " ms");
}
}

And the results this time:

$ java ManualMethod
Normal: 1451 ms
Factored: 725 ms
Normal: 724 ms
$ java -server ManualMethod
Normal: 4 ms
Factored: 0 ms
Normal: 0 ms

This is rather disappointing, as it suggests that my optimisations will not actually help much. I am inclined to suspect that the reason the code movement does not help is that the JIT compiler in the JVM already does some optimisation of this sort (perhaps by inlining short methods), though it is also possible that the method call overhead is really low.

If I make TestCollection.size() slower by adding some useless code to it, then the optimisation makes a lot more difference:

$ java ManualMethod
Normal: 11253 ms
Factored: 835 ms
Normal: 11134 ms

If we use the server VM, the results are very strange:

$ java -server ManualMethod
Normal: 7 ms
Factored: 0 ms
Normal: 5773 ms

I cannot see why the first normal run should be so much quicker than the second: the reverse was observed in all the other tests before this, which was presumably due to the JVM doing some JIT compilation once it decided that this would be worthwhile.

Advertisements

Tags: , , ,

3 Responses to “Testing manual optimisation”

  1. Dmitry Says:

    First, HotSpot Server optimizes your loops away as they do not compute any result. You should do something inside the loops, and make sure the result is not thrown away, otherwise the results are meaningless. Google for something like “common microbenchmarking mistakes”.

    Second, HotSpot Server is capable of detecting that array.length and list.size() are loop invariants. If you have some code inside the loops that _may_ potentially change ‘array’ or ‘list’, your manual optimization would come into play.

  2. qwandor Says:

    Thanks Dmitry, that confirms what I was suspecting. I assume that HotSpot is inlining short methods when it can work out statically what methods are actually called.

  3. More tests « Code movement in Java programs with pure functions Says:

    […] Code movement in Java programs with pure functions A journal of my research for a BSc (Hons) project at VUW « Testing manual optimisation […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: