Using Closures, Method Handles and Extension Methods in Java 8 (JSR-335)

Using Closures, Method Handles and Extension Methods in Java 8 (JSR-335)

Update: A bunch of code from hotspot was just integrated into the bsd-port and some of it breaks the build. Use my distribution at the end of this blog post if you want to test the features.

In December JSR-335 was approved by the JCP and is now starting. I’ve joined as member of the expert group, but like with most recent JSRs all the real discussion will take place on the public mailing list. A lot of work has already been done in OpenJDK to support the features of this JSR which will ultimately be included in Java 8 if it completes successfully. Here is the current state of the JSR. Though JDK 8 is when official support would ship, you can already build and execute the draft features of JSR-335 by using the regular OpenJDK 7 build along with some compile-time and run-time tools. Before I go into the guts of how to build them, here are some examples of what you will be able to do:

  • Create closures that are converted to Single Abstract Method (SAM) implementations when used in context
  • Create interfaces that specify default behavior for methods that are unimplemented by the implementing class
  • Reference methods in source and then convert those references to SAM implementations when used in context

Building OpenJDK 7 on Mac OS X 10.6 (based on Darwin10Build):

  • Install mecurial: 
    sudo port install mercurial
  • Install hg-forest: 
    sudo port install hg-forest
  • Install soylatte to /usr/local/soylatte16-i386–1.0.3
  • Clone bsd-port: 
    hg fclone http://hg.openjdk.java.net/bsd-port/bsd-port
  • export OPENJDK_ROOT=`pwd`
  • Grab build script: 
    cd bsd-port; curl -O “https://gist.github.com/raw/617451/d4862a41b07a1196d5149efc1c40406f8bb77dc1/update.sh"
  • mkdir ALT_COMPILER_PATH;cd ALT_COMPILER_PATH;
    ln -s /usr/bin .SOURCE;ln -s .SOURCE/g++-4.0 g++;
    ln -s .SOURCE/gcc-4.0 gcc;cd ..
  • unset JAVA_HOME
  • Edit hotspot/make/bsd/makefiles/defs.make and change ifeq ($(ARCH), amd64) to ifeq($(ARCH), x86_64)
  • Update sources and build (on a fast machine will take 15m): 
    sh update.sh

Now you should have a fully working OpenJDK 7 64-bit distribution:

macpro:bsd-port sam$ ./build/bsd-amd64/j2sdk-image/bin/java -version
openjdk version “1.7.0-internal”
OpenJDK Runtime Environment (build 1.7.0-internal-sam_2011_01_22_16_40-b00)
OpenJDK 64-Bit Server VM (build 20.0-b03, mixed mode)

Now we want to build the Lambda compiler and runtime support:

OpenJDK 7 doesn’t actually support everything that we need to run code generated for Lambda though. We need to also build a java agent that will modify the bytecodes in classes as they are loaded to run on the older VM for some features. Here is how you do that:

  • Grab the source: 
    cd $OPENJDK_ROOT; svn checkout http://jsr335-lambda.googlecode.com/svn/trunk/ jsr335-lambda
  • export JAVA_HOME=$OPENJDK_ROOT/bsd-port/build/bsd-amd64/j2sdk-image
  • Build the agent: 
    cd jsr335-lambda/agent; ant jar
  • export AGENTJAR=$OPENJDK_ROOT/jsr335-lambda/agent/lib/jsr335-agent.jar

Now we can actually build Lambda code and execute it, with a lot of command line options. For example, here is a simple program that we might want to execute:

public class Test {
  public static void main(String[] args) throws InterruptedException {
    Thread t = new Thread(#{ System.out.println("ran"); });
    t.start();
    t.join();
  }
}

Let’s first compile it:

  • export PATH=$JAVA_HOME/bin:$PATH
  • export LANGTOOLS_JAR=$OPENJDK_ROOT/langtools/dist/lib/classes.jar
  • javac -J-Xbootclasspath/p:$LANGTOOLS_JAR -source 8 -target 8 Test.java

Now we have Test.class file that will only work in a VM that supports some of the bytecode that we are using. In this case we don’t need the transformations so we can execute this code like this:

macpro:jsr335 sam$ java -cp . Test
ran

Here is some code that will not run out of the box, that shows you how you can now write extension methods:

public class Test2 {
  interface Trait {
    int inc(int i) default Test2.inc;
  }
  static int inc(Trait t, int i) { return i+1;}
  public static class Test3 implements Trait {}
  public static void main(String[] args) {
    Trait t = new Test3();
    System.out.println(t.inc(1));
  }
}

If you try and run this without the agent you will get this:

macpro:jsr335 sam$ java -cp . Test2
Exception in thread “main” java.lang.AbstractMethodError: Test2$Test3.inc(I)I
 at Test2.main(Test2.java:9)

Now we execute with the agent enabled:

macpro:jsr335 sam$ java -javaagent:$AGENTJAR -cp . Test2
2

The last feature that you might want to use, are method handles. Essentially you can convert a method handle into a closure by using it in a SAM context. Here is an example of how they work:

public class Test3 {
    public void test() {
	System.out.println("Test");
    }
    public static void main(String[] args) {
	Test3 t = new Test3();
	Runnable r = t#test;
	r.run();
    }
}

In order to run this we need to enable InvokeDynamic on our VM and include the classes from langtools for some runtime support:

macpro:jsr335 sam$ java -XX:+UnlockExperimentalVMOptions -XX:+EnableInvokeDynamic -Xbootclasspath/p:$LANGTOOLS_JAR -javaagent:$AGENTJAR -cp . Test3
Test

That command line has everything you should ever need to execute code generated using the Lamba enhancements so using anything else is probably unnecessary. Now let’s put all of this together into the final example from the Lambda proposal itself:

import java.util.*;
interface Sortable<T, U extends Comparable<? super U>>  extends List<T>  {
   void sortBy(Extractor<? super T, ? extends U>  e) default Impl.sortBy;
   static class Impl {
       public static<T, U extends Comparable<? super U>>
                 void sortBy(Sortable<T, U>  sortable, final Extractor<? super T, ? extends U>  e) {
           Collections.sort(sortable, #{T a, T b ->  e.extract(a).compareTo(e.extract(b))});
       }
   }
}
interface Extractor<T,U> {
    public U extract(T t);
}
class Person {
    public Person(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }
    private String lastName;
    public String getLastName() { return lastName; }
    private String firstName;
    public String getFirstName() { return firstName;}
    public String toString() {
        return firstName + " " + lastName;
    }
}
public class Test4 {
public static class SortableList<T, U extends Comparable<? super U>> extends ArrayList<T> implements Sortable<T, U> {}
public static void main(String[] args) {
        Sortable<Person, String> list = new SortableList<>();
        list.add(new Person("Sam", "Pullara"));
        list.add(new Person("Brian", "Goetz"));
        list.add(new Person("Bob", "Lee"));
        list.sortBy(Person#getLastName);
        System.out.println(list);
    }
}

Which results in:

macpro:jsr335 sam$ java Test4
[Brian Goetz, Bob Lee, Sam Pullara]

The punchline is that if you read through all of this you get a bonus: here is the distribution of above build products that you can download and run on your Mac (assuming you are running a 64-bit build of snow leopard):
openjdk7-lambda-macosx-x86_64-01222010.zip