Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

JλVλ 8 (A Comprehensive Look): Part 1.1 - Lambdas Under the Hood

DZone's Guide to

JλVλ 8 (A Comprehensive Look): Part 1.1 - Lambdas Under the Hood

We continue our detailed exploration of lambdas in Java 8, and how specific methods and functions work in this iteration of Java.

· Java Zone ·
Free Resource

Download Microservices for Java Developers: A hands-on introduction to frameworks and containers. Brought to you in partnership with Red Hat.

Note: This article dives into the details of Lambda Expressions and its "under the hood" implementation in Java. This is in continuation to JλVλ 8: A Comprehensive Look article, where Lambdas expressions are discussed. So its recommended to first go through the discussion on Lambda Expressions here.

Lambdas Under the Hood:

Let's have a look under the hood and see what a class with lambda expressions gets compiled to.

public class LambdaTest {
  public void publicMethod() {
    Integer localVariable1 = 10;

    Thread t = new Thread(() -> System.out.println(localVariable1));
    t.start();

    List<String> list = Arrays.asList("A", "B", "C");
    Collections.sort(list, (p1,p2)->p1.compareTo(p2));
    list.forEach(x->System.out.println(x));
  }
}

After compiling the LambdaTest class above (javac LambdaTest.java), the following bytecode is generated if we look through java:

public class com.java8exploration.LambdaTest {
  public com.java8exploration.LambdaTest();
    Code: ...

  public void publicMethod();
    Code:
       0: ...
      11: invokedynamic #4,  0              // InvokeDynamic #0:run:(Ljava/lang/Integer;)Ljava/lang/Runnable;
      16: invokespecial #5                  // Method java/lang/Thread."<init>":(Ljava/lang/Runnable;)V
      19: ...
      44: invokedynamic #11,  0             // InvokeDynamic #1:compare:()Ljava/util/Comparator;
      49: invokestatic  #12                 // Method java/util/Collections.sort:(Ljava/util/List;Ljava/util/Comparator;)V
      52: ...
      53: invokedynamic #13,  0             // InvokeDynamic #2:accept:()Ljava/util/function/Consumer;
      58: invokeinterface #14,  2           // InterfaceMethod java/util/List.forEach:(Ljava/util/function/Consumer;)V
      63: return

  private static void lambda$publicMethod$2(java.lang.String);
    Code: ...

  private static int lambda$publicMethod$1(java.lang.String, java.lang.String);
    Code: ...

  private static void lambda$publicMethod$0(java.lang.Integer);
    Code: ...
}

If we look closely at the compiled code we see that our 3 lambdas are compiled to 3 private static methods in the same class, also, all the local variables that they were accessing are captured as arguments to those methods. These methods succinctly capture the lambda body, it's arguments, arguments in lexical scope (if any, through variable capture) and return type. If you look at the publicMethod(), you will see that the invokedynamic instruction is used for the execution of lambda expressions which simply means that the Java team chose to defer the implementation of objects for these lambdas to runtime instead of directly generating the bytecode to create the implementation objects representing the lambdas. Since lambdas are the implementation of any functional interface hence there could be several strategies to provide an implementation for lambda expression like inner classes (similar to the anonymous class example discussed above), dynamic proxies, method handles and maybe others. Using invokedynamic allows the JVM to generate the bytecode (using any specific implementation strategy) at runtime. This enables JVM to change the implementation strategy in the future without having to change already compiled code. Also, this lazy linking of implementation objects allows for caching, i.e. the subsequent calls to the same statement can use a cached version. 

Closure (Variable Capture) in Lambdas:

Let us look more closely at closure behavior (or variable capture). Let's now try to access instance fields and see how the compiled code captures that:

public class LambdaTest {
  private Integer insVar1 = 20;

  public void publicMethod() {
    Thread t = new Thread(() -> System.out.println(insVar1));
    t.start();

    List<String> list = Arrays.asList("A", "B", "C");
    Collections.sort(list, (p1, p2) -> p1.compareTo(p2));
    list.forEach(x -> System.out.println(x));
  }
}

Compiled Code:

public class com.java8exploration.LambdaTest {
  private java.lang.Integer insVar1;

  public com.java8exploration.LambdaTest();
    Code:
       ...

  public void publicMethod();
    Code:
      ...
      11: invokedynamic #5,  0              // InvokeDynamic #0:run:(Lcom/java8exploration/LambdaTest;)Ljava/lang/Runnable;
      16: invokespecial #6                  // Method java/lang/Thread."<init>":(Ljava/lang/Runnable;)V
      ...
      48: invokedynamic #13,  0             // InvokeDynamic #1:compare:()Ljava/util/Comparator;
      53: invokestatic  #14                 // Method java/util/Collections.sort:(Ljava/util/List;Ljava/util/Comparator;)V
      ...
      57: invokedynamic #15,  0             // InvokeDynamic #2:accept:()Ljava/util/function/Consumer;
      62: invokeinterface #16,  2           // InterfaceMethod java/util/List.forEach:(Ljava/util/function/Consumer;)V
      67: return

  private static void lambda$publicMethod$2(java.lang.String);
    Code:
       ...

  private static int lambda$publicMethod$1(java.lang.String, java.lang.String);
    Code:
       ...

  private void lambda$publicMethod$0();
    Code:
       ...
}

If you compare the first lambda expression with others, you will see that the first lambda is accessing an instance field, also, if you look closely you will see the lambda-prefixed method that is compiled/generated for this lambda is non-staticprivate void lambda$publicMethod$0. So if a lambda is accessing fields just from the enclosing scope then the method compiled/generated for lambda is static and all the accessed variables are captured as method arguments, whereas, in cases where instance fields are accessed, the method compiled/generated for lambda is not static so that it can access instance members and other variables from the enclosing scope are captured as method arguments.

Invoke Opcodes

Let's dig a bit deeper to understand the invoke opcodes/instructions in bytecode to better understand invokedynamic and how it is used by the JVM to transform lambda expressions into units that can be executed in the object-oriented JVM.

Let's explore a few concepts before we can jump in:

Constant Pool: When a Java class is compiled, it is compiled to class files with bytecode instructions/opcodes and these bytecode instructions refer to variables, methods, etc. from a table, which is called the class' constant pool, as symbolic references.
Java:

...
String x = "ab";
....

Compiled Code:

6: ldc #3

Constant pool:
...
#3 = String #40
...
#40 = Utf8 ab

Static Typed Languages: A programming language is statically typed if type checking is performed at compile time. Type checking is a process of checking if appropriate types are used where expected.

Dynamic Typed Languages: A dynamically typed language is one which does not verify types at compile time. In these languages, the type information is not available at compile time and they might not even have any constructs for "typed variables."

Strongly Typed and Weakly Typed Languages: Another classification for languages other than being statically or dynamically typed is being strongly or weakly typed. Strongly typed languages are ones where there are constraints on the types of values that are used in an operation; weakly typed languages are ones where the language/system can handle the required conversions on the types implicitly to make them suitable for a particular operation. Some examples of these classifications are: Java (is statically typed and strongly typed); Python (is dynamic typed and strongly typed); PHP (is dynamic typed and weakly typed); JavaScript (is dynamic typed and weakly typed).

Call Site: In simple terms, call site is the site/location where a (method) call/invocation is made. As an example, see the code below:

public class Child {
  protected void sameMethod() {
    System.out.println("Child sameMethod called");
  }

  protected void ChildM1() {
    System.out.println("Child M1 method called");
    this.sameMethod();
  }

  public static void main(String[] args) {
    Child child = new Child();
    child.ChildM1();
  }
}

In the code above, line #8 would be the call site for sameMethod (in an oversimplified sense).

But when we talk about dynamically typed languages or dealing with dynamic typing constructs, we need to augment this definition of call site because in dynamically typed language it is not known until runtime what method/invocation will be made so it counter it we can assume call site to also be an object representing the information/logic that is required to dynamically locate/construct and invoke methods/operations at runtime. This is represented by java.lang.invoke.CallSite in Java. This call site in Java acts as a container for a method handle. This method handle (explained below) is called the target of a call site. When a target is set for a call site, it is called linking. The target for a call site can also be changed, this is called relinking. java.lang.invoke.CallSite is an abstract class, it has 3 concrete sub-classes which are used:

  • ConstantCallSite - Once a target (method) is linked to the call site, it can not be changed.

  • MutableCallSite - call site is mutable, it's target can be changed, relinking is allowed.

  • VolatileCallSite - updates to the call site target are visible to other threads.

Method Handle: In simple terms, method handles are like pointers to methods. This is represented by the MethodHandle class. Method handles are typed, i.e., a method handles are clearly defined with return type and parameter type information. Method handles can be used to execute the underlying method that it points to. The primary intent of method handle is to create an executable reference for a method. This construct can also be put to use for looking up methods and accessing them in place of using the reflection API, but note that this is not the intent of this construct, it can be just used for looking up and invoking methods and not for reflecting upon the method itself. There is also some sort of interoperability between Reflection API objects and method handles, i.e., we can convert java.lang.reflect.Method objects to method handles using the Lookup.unreflect method. The following is an example of the usage of MethodHandle and related APIs:

public class MethodHandleTryOut {
  private final int i = 10;

  private static void staticM1(String msg) {
    System.out.println("Static method called: " + msg);
  }

  private void instanceM1(String msg) {
    System.out
        .println("Instance method called: " + i + ", " + msg);
  }

  public static void main(String[] args) {
    try {
      MethodType desc = MethodType.methodType(void.class, String.class);
      MethodHandle mh = MethodHandles.lookup()
          .findStatic(MethodHandleTryOut.class, "staticM1", desc);

      MethodHandle mh2 = MethodHandles.lookup()
          .findVirtual(MethodHandleTryOut.class, "instanceM1", desc);
      mh.invoke("Hey");
      mh2.invoke(new MethodHandleTryOut(), "Hey");
    } catch (Throwable e) {
      e.printStackTrace();
    }
  }
}

Output:

Static method called: Hey
Instance method called: 10, Hey

Let's now jump into the invoke instructions/opcode:

In simple terms, invoke instructions represent method invocation in bytecode. All references to methods in the invocations are symbolic and point to a constant pool entry. Whenever an invoke instruction is interpreted, the symbolic reference is resolved and replaced with a direct reference for future invocations. Also, when resolving a symbolic reference, JVM verifies the validity of the invocations, i.e., types are appropriate, the method is accessible, etc. Once the symbolic reference is resolved and verified, the JVM invokes the call, for this, it requires the ObjectRef (a reference to an object in case of an instance method call) and the arguments to be pushed on to the stack before the invoke instruction. When the JVM invokes the method, it creates a stack frame for the method in the Java Stack, it pops the ObjectRef (for instance, method calls) and arguments from the calling method's stack frame and pushes it into the called method's stack frame (this is JVM implementation specific, it may be implemented differently).

Different Invoke Opcodes:

  • invokestatic: This is used for class methods (static methods). When a class method is invoked it is known at compile time and it is static/early/compile-time bound to the call site.

Source Code:

public class Child {
  public static void childStaticM1() {
    System.out.println("Child childStaticM1 method called");
  }

  public static void main(String[] args) {
    Child child = new Child();
    child.childStaticM1();
    Child.childStaticM1();
  }
}

Output: 

Child childStaticM1 method called

Child childStaticM1 method called

Compiled Code:

public class com.java8exploration.invoke.Child
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Child();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 3: 0

  public static void childStaticM1();
    descriptor: ()V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=0, args_size=0
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #3                  // String Child childStaticM1 method called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: return
      LineNumberTable:
        line 5: 0
        line 6: 8

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=2, args_size=1
         0: new           #5                  // class com/java8exploration/invoke/Child
         3: dup
         4: invokespecial #6                  // Method "<init>":()V
         7: astore_1
         8: aload_1
         9: pop
        10: invokestatic  #7                  // Method childStaticM1:()V
        13: invokestatic  #7                  // Method childStaticM1:()V
        16: return
      LineNumberTable:
        line 9: 0
        line 10: 8
        line 11: 13
        line 12: 16
}

You can see that invokestatic is used for invoking the childStaticM1 method from the main method.

  • invokevirtual: This is used for instance methods, the method to bind is not known until runtime and it selects the method to invoke based on the actual runtime type of the object and, hence, it is dynamic/late/runtime bound. This, in effect, enables runtime polymorphism.

Source Code:

public class Parent {
  protected void parentM1() {
    System.out.println("Parent M1 method called");
    this.sameMethod();
  }

  protected void sameMethod() {
    System.out.println("Parent sameMethod called");
  }
}

public class Child extends Parent {
  protected void sameMethod() {
    System.out.println("Child sameMethod called");
  }

  public static void main(String[] args) {
    Parent child = new Child();
    child.parentM1();
  }
}

Output:

Parent M1 method called

Child sameMethod called

Compiled Code:

public class com.java8exploration.invoke.Parent
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Parent();
    ...

  protected void parentM1();
    descriptor: ()V
    flags: ACC_PROTECTED
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #3                  // String Parent M1 method called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: aload_0
         9: invokevirtual #5                  // Method sameMethod:()V
        12: return
      LineNumberTable:
        line 5: 0
        line 6: 8
        line 7: 12

  protected void sameMethod();
    ...
}

You can see that in the parentM1 method, invokevirtual is used to invoke sameMethod, and if we look at the main method in the Child class, we can see that we are creating an instance of the Child class with a Parent reference type (the result would be the same if the instance is created as a Child reference type), but when sameMethod is invoked from the Parent class, it invokes the Child class' sameMethod and not it's own. This is because invokevirtual is used, which selects methods based on the runtime type of the object, which is Child.

  • invokeinterface: This is used for the invocation of instance methods as well, it is different than invokevirtual in the sense that it is used in cases where the reference type of the object is an interface, i.e., the method call is through the interface type.

Source Code:

public interface InterfaceI {
  void interfaceMethod();
}

public class Child implements InterfaceI {
  @Override
  public void interfaceMethod() {
    System.out.println("Child interfaceMethod called");
  }

  public static void main(String[] args) {
    InterfaceI child = new Child();
    child.interfaceMethod();
  }
}

Output: Child interfaceMethod called

Compiled Code:

public class com.java8exploration.invoke.Child implements com.java8exploration.invoke.InterfaceI
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Child();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 3: 0

  public void interfaceMethod();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #3                  // String Child interfaceMethod called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: return
      LineNumberTable:
        line 6: 0
        line 7: 8

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=2, args_size=1
         0: new           #5                  // class com/java8exploration/invoke/Child
         3: dup
         4: invokespecial #6                  // Method "<init>":()V
         7: astore_1
         8: aload_1
         9: invokeinterface #7,  1            // InterfaceMethod com/java8exploration/invoke/InterfaceI.interfaceMethod:()V
        14: return
      LineNumberTable:
        line 10: 0
        line 11: 8
        line 12: 14
}

You can see that invokeinterface is used for child.interfaceMethod() invocation in the main method.

  • invokespecial: This is used for instance methods as well, but it is meant for special cases where we need to select the method to invoke based on the reference type and not on the actual runtime type of the object. This is the primary difference between invokespecial and invokevritual. THe following are the cases where we need to do that, i.e. to use invokespecial instead of invokevirtual:

1. When invoking instance initialization methods, i.e constructors and/or instance initializers (compiled as <init> methods in bytecode), because, let's say, a constructor is invoked and now it has to invoke its parent class constructor. But if invokevirtual is used in this case, it will in turn again to invoke the child class constructor because it selects methods based on the runtime type of the object (which would be the Child class in the example below).

Source Code:

public class Parent {
  public Parent() {
    System.out.println("Parent constructor called");
  }
}

public class Child extends Parent {
  public Child() {
    // There is an implicit statement added here: super();
    System.out.println("Child constructor called");
  }

  public static void main(String[] args) {
    Child child = new Child();
  }
}

Output:

Parent constructor called
Child constructor called

Compiled Code:

public class com.java8exploration.invoke.Child extends com.java8exploration.invoke.Parent
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Child();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method com/java8exploration/invoke/Parent."<init>":()V
         4: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         7: ldc           #3                  // String Child constructor called
         9: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
        12: return
      LineNumberTable:
        line 4: 0
        line 6: 4
        line 7: 12

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=2, args_size=1
         0: new           #5                  // class com/java8exploration/invoke/Child
         3: dup
         4: invokespecial #6                  // Method "<init>":()V
         7: astore_1
         8: return
      LineNumberTable:
        line 10: 0
        line 11: 8
}

You can see invokespecial being used for invoking the constructors (<init>).

2. When private methods are calledinvokespecial is used since private methods cannot be overridden in subclasses and, hence, when a private method from a class is called it should call the method from that same class and not from subclass if the runtime object is of sub-class type. So, if in this case, invokevirtual is used, it would select from the runtime object class/type and that would be wrong, so  invokespecial is used in such cases (in fact, this applies to all private method invocations).

Source Code:

public class Parent {
  public Parent() {
    System.out.println("Parent constructor called");
  }

  protected void ParentM1() {
    System.out.println("Parent M1 method called");
    this.sameMethod();
  }

  private void sameMethod() {
    System.out.println("Parent sameMethod called");
  }
}

public class Child extends Parent {
  public Child() {
    // There is an implicit statement added here: super();
    System.out.println("Child constructor called");
  }

  protected void sameMethod() {
    System.out.println("Child sameMethod called");
  }

  public static void main(String[] args) {
    Child child = new Child();
    child.ParentM1();
  }
}

Output:

Parent constructor called

Child constructor called

Parent M1 method called

Parent sameMethod called

Compiled Code:

public class com.java8exploration.invoke.Parent
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Parent();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=2, locals=1, args_size=1
         0: aload_0
         1: invokespecial #1                  // Method java/lang/Object."<init>":()V
         4: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         7: ldc           #3                  // String Parent constructor called
         9: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
        12: return
      LineNumberTable:
        line 4: 0
        line 5: 4
        line 6: 12

  protected void ParentM1();
    descriptor: ()V
    flags: ACC_PROTECTED
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #5                  // String Parent M1 method called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: aload_0
         9: invokespecial #6                  // Method sameMethod:()V
        12: return
      LineNumberTable:
        line 9: 0
        line 10: 8
        line 11: 12

  private void sameMethod();
    descriptor: ()V
    flags: ACC_PRIVATE
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #7                  // String Parent sameMethod called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: return
      LineNumberTable:
        line 14: 0
        line 15: 8
}

You can see here that invokespecial is used when the private void sameMethod is called from the parentM1 method in the Parent class.

If we change the private void sameMethod access modifier to protected/public/default in the Parent class, then the same method invocation is compiled with the invokevirtual opcode.

protected void ParentM1() {
    System.out.println("Parent M1 method called");
    this.sameMethod();
  }

  protected void sameMethod() {
    System.out.println("Parent sameMethod called");
  }

// Compiled Code:
  protected void ParentM1();
    descriptor: ()V
    flags: ACC_PROTECTED
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #5                  // String Parent M1 method called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: aload_0
         9: invokevirtual #6                  // Method sameMethod:()V
        12: return
      LineNumberTable:
        line 9: 0
        line 10: 8
        line 11: 12

Output:

Parent constructor called

Child constructor called

Parent M1 method called

Child sameMethod called

3. When methods or fields are accessed through the super keywordinvokespecial is used so that the method selected for binding is a method from the reference type of the object (i.e. parent class) and not the runtime type of the object.

Source Code:

public class Parent {
  public Parent() {
    System.out.println("Parent constructor called");
  }

  protected void ParentM1() {
    System.out.println("Parent M1 method called");
    this.sameMethod();
  }

  protected void sameMethod() {
    System.out.println("Parent sameMethod called");
  }
}

public class Child extends Parent {
  public Child() {
    // There is an implicit statement added here: super();
    System.out.println("Child constructor called");
  }

  protected void sameMethod() {
    System.out.println("Child sameMethod called");
  }

  protected void ChildM1() {
    System.out.println("Child M1 method called");
    super.sameMethod();
  }

  public static void main(String[] args) {
    Child child = new Child();
    child.ParentM1();
    child.ChildM1();
  }
}

Output:

Parent constructor called

Child constructor called

Parent M1 method called

Child sameMethod called

Child M1 method called

Parent sameMethod called

Compiled Code:

public class com.java8exploration.invoke.Child extends com.java8exploration.invoke.Parent
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   ...
{
  public com.java8exploration.invoke.Child();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      ...

  protected void sameMethod();
    descriptor: ()V
    flags: ACC_PROTECTED
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #5                  // String Child sameMethod called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: return
      LineNumberTable:
        line 10: 0
        line 11: 8

  protected void ChildM1();
    descriptor: ()V
    flags: ACC_PROTECTED
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #6                  // String Child M1 method called
         5: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: aload_0
         9: invokespecial #7                  // Method com/java8exploration/invoke/Parent.sameMethod:()V
        12: return
      LineNumberTable:
        line 14: 0
        line 15: 8
        line 16: 12

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=2, args_size=1
         ...
        13: invokevirtual #11                 // Method ChildM1:()V
        16: return
      LineNumberTable:
        line 19: 0
        line 20: 8
        line 21: 12
        line 22: 16
}

You can see in the bytecode that, in the childM1 method, the invokespecial opcode is used to invoke super.sameMethod.

  • InvokeDynamic: This JVM opcode was added in Java 7 and it was meant to support dynamically typed languages and to allow language writers and compilers to generate the invokedynamic opcode to defer handling invocations to runtime, because, for a dynamically typed language, the type of an object is not known explicitly at compile time and it is at runtime only where it's type could be determined and so it is essential to perform invocations for methods/operations at runtime by determining the type at runtime and deciding at runtime what method/operation to invoke. As we have seen, the other invoke instructions directly dictate the method that the instruction should be linked to (static linking), but the invokedynamic instruction facilitates the linking of the method indirectly at runtime (dynamic linking). Instructions other than invokedynamic are meant specifically for statically typed constructs of the Java language, i.e. Java in general, since, as described earlier with static typing, the types of variables are clearly defined and it can be known at compile time as to what method/operation should be in invoked for particular objects/operands, and so all these invoke instructors can link the methods directly. Let us understand this through an example:

Source Code:

public static void main(String[] args) {
    int x = 10, y = 20;
    String sx = "10", sy = "20";
    System.out.println(x + y);
    System.out.println(sx + sy);
}

Output:

30

1020

Compiled Code:

public static void main(java.lang.String[]);
    Code:
      stack=3, locals=5, args_size=1
         0: bipush        10
         2: istore_1
         3: bipush        20
         5: istore_2
         6: ldc           #7                  // String 10
         8: astore_3
         9: ldc           #8                  // String 20
        11: astore        4
        13: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
        16: iload_1
        17: iload_2
        18: iadd
        19: invokevirtual #9                  // Method java/io/PrintStream.println:(I)V
        22: getstatic     #2                  // Field java/lang/System.out:Ljava/io/PrintStream;
        25: new           #10                 // class java/lang/StringBuilder
        28: dup
        29: invokespecial #11                 // Method java/lang/StringBuilder."<init>":()V
        32: aload_3
        33: invokevirtual #12                 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
        36: aload         4
        38: invokevirtual #12                 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
        41: invokevirtual #13                 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
        44: invokevirtual #4                  // Method java/io/PrintStream.println:(Ljava/lang/String;)V
        47: return

If we look at the example above, we have the add (+) operation that is applied first to integer operands and then to String operands. If we look at the compiled code, we can see that when addition is done with integer types, the "iadd" operation is performed, but when addition is done with String type, StringBuilder is used to concatenate the operands and return a concatenated string. Since the code is statically typed, it is possible to know at compile time what operation/method to perform, whereas the same is not true when the code is dynamically typed, i.e. the types are not known at compile time.

Also, note that this should not be confused with  invokevirtual's runtime binding, because invokevirtual knows which method to link at the call site, it's just that it decides which object on which to execute the method at runtime, where, in the invokedynamic case, the method to link is not known until runtime.

invokedynamic facilitates the indirect linking of a method at runtime by initially executing a proxy method, which, when executed at runtime, decides what actual method to link to. This proxy method is called the bootstrap method. It is the responsibility of this bootstrap method to select/create and link the actual method to which the instruction should be linked. The bootstrap method returns a call site with the target method (method handle) set in it. Each invokedynamic instruction in the code is called a dynamic call site and, originally, this call site was in an unlinked state, i.e., no target for the call site is set. But, once the bootstrap method is executed, the target for the call site is set and the actual method to invoke is linked. Once the linkage is done, it is used directly for all future invocations instead of executing the bootstrap method again (i.e., the bootstrap method is executed only once).

Image title

Lambdas to Executable Methods in the Object-Oriented JVM

Lambdas are transformed into an appropriate form to be executable in the JVM using the invokedynamic instruction. Though a point to note here is that invokedynamic is not used in the true dynamic typed language sense, but it is used to leverage the dynamic linking feature of invokedynamic to defer the strategy used to implement lambdas into executable code.

Whenever any lambda expression is encountered in the code, an invokedynamic instruction is generated in place of the lambda expression which refers to the bootstrap method which in turn links the call site (links the method to execute) at runtime and the lambda expression itself is compiled to a private method within the same class with a pseudo name. This method is the representation of the lambda expression to a plain executable Java method. It captures the lambda body and it's arguments, the arguments in the lexical scope (if any, through variable capture) and return type.

Let's see some code:

Source Code:

public class LambdaTest {
  public void publicMethod() {
    Function<String, String> func = (s) -> s.toLowerCase();
    System.out.println(func.getClass() + " implements " + func.getClass().getInterfaces()[0]);
    System.out.println(func.apply("ROBIN"));
  }

  public static void main(String[] args) {
    (new LambdaTest()).publicMethod();
  }
}

Output:

The class com.java8exploration.LambdaTest$$Lambda$1/471910020 implements the interface java.util.function.Function 
robin

Compiled Code:

public class com.java8exploration.LambdaTest
Constant pool:
  ...
   #2 = InvokeDynamic      #0:#38         // #0:apply:()Ljava/util/function/Function;
  #15 = Class              #54            // java/lang/String
  #16 = Class              #55            // com/java8exploration/LambdaTest
  #18 = Methodref          #16.#56        // com/java8exploration/LambdaTest.publicMethod:()V
  #22 = Utf8               ()V
  #25 = Utf8               publicMethod
  #34 = MethodHandle       #6:#59         // invokestatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
  #35 = MethodType         #60            //  (Ljava/lang/Object;)Ljava/lang/Object;
  #36 = MethodHandle       #6:#61         // invokestatic com/java8exploration/LambdaTest.lambda$publicMethod$0:(Ljava/lang/String;)Ljava/lang/String;
  #37 = MethodType         #29            //  (Ljava/lang/String;)Ljava/lang/String;
  #38 = NameAndType        #62:#63        // apply:()Ljava/util/function/Function;
  #54 = Utf8               java/lang/String
  #55 = Utf8               com/java8exploration/LambdaTest
  #56 = NameAndType        #25:#22        // publicMethod:()V
  #57 = NameAndType        #81:#76        // toLowerCase:()Ljava/lang/String;
  #62 = Utf8               apply
  #63 = Utf8               ()Ljava/util/function/Function;
  #76 = Utf8               ()Ljava/lang/String;
  #81 = Utf8               toLowerCase
  #87 = Class              #92            // java/lang/invoke/MethodHandles$Lookup
  #88 = Utf8               Lookup
  #92 = Utf8               java/lang/invoke/MethodHandles$Lookup
  ...
{
  public com.java8exploration.LambdaTest();
    ...

  public void publicMethod();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=4, locals=2, args_size=1
         0: invokedynamic #2,  0              // InvokeDynamic #0:apply:()Ljava/util/function/Function;
         ...
        63: return

  public static void main(java.lang.String[]);
    ...
    invokevirtual #18                 // Method publicMethod:()V
    ...

  private static java.lang.String lambda$publicMethod$0(java.lang.String);
    descriptor: (Ljava/lang/String;)Ljava/lang/String;
    flags: ACC_PRIVATE, ACC_STATIC, ACC_SYNTHETIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokevirtual #19                 // Method java/lang/String.toLowerCase:()Ljava/lang/String;
         4: areturn
      LineNumberTable:
        line 8: 0
}
SourceFile: "LambdaTest.java"
InnerClasses:
     public static final #88= #87 of #91; //Lookup=class java/lang/invoke/MethodHandles$Lookup of class java/lang/invoke/MethodHandles
BootstrapMethods:
  0: #34 invokestatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #35 (Ljava/lang/Object;)Ljava/lang/Object;
      #36 invokestatic com/java8exploration/LambdaTest.lambda$publicMethod$0:(Ljava/lang/String;)Ljava/lang/String;
      #37 (Ljava/lang/String;)Ljava/lang/String;

If you closely analyze the compiled code above, you can see that:

  • At line-36, an invokedynamic instruction is used where the lambda expression is present.

  • At line-45, a private method named lambda$publicMethod$0 is generated, which represents the out lambda expression.

  • The invokedynamic instruction refers to the #2 index in the constant pool, which in turn refers to the #0 entry which points to the bootstrap method which is listed at the very bottom of line-60.

  • If you check the output of this example, you can see that when the lambda expression is executed, an instance is returned and assigned to "func" and its type, com.java8exploration.LambdaTest$$Lambda$1/471910020, implements java.util.function.Function.

In case of Lambda transformation in Java 8, the bootstrap method is a library method named metafactory in java.lang.invoke.LambdaMetafactory and this is responsible for linking the method that was generated to represent the lambda expression. This metafactory method is passed with arguments like method handles to the generated lambda method, it's method type, the return type of lambda execution (i.e., functional interface type), etc., by the JVM. If we have a look at the source of metafactory and related methods, we can see that it creates a new inner class on the fly at runtime using the internal ObjectWeb ASM library containing our lambda method code in the sam method block (functional interface's method, for this example it will be the "apply" method). If we check in our example that created the inner class, it is "LambdaTest$$Lambda$1/471910020." This method returns a linked call site object (with a target set to the lambda method in the created inner class) which replaces the bootstrap method call and for all future executions of the lambda expression it is used and the bootstrap method is not called again.

public static CallSite metafactory(MethodHandles.Lookup caller,
 String invokedName,
 MethodType invokedType,
 MethodType samMethodType,
 MethodHandle implMethod,
 MethodType instantiatedMethodType)
      throws LambdaConversionException {
  AbstractValidatingLambdaMetafactory mf;
  mf = new InnerClassLambdaMetafactory(caller, invokedType,
                                       invokedName, samMethodType,
                                       implMethod, instantiatedMethodType,
                                       false, EMPTY_CLASS_ARRAY, EMPTY_MT_ARRAY);
  mf.validateMetafactoryArgs();
  return mf.buildCallSite();
}

CallSite buildCallSite() throws LambdaConversionException {
  final Class<?> innerClass = spinInnerClass();
  ...
  Object inst = ctrs[0].newInstance();
  return new ConstantCallSite(MethodHandles.constant(samBase, inst));
  ...
}

Now an obvious question would come to your mind: why not directly create an inner class at compile time to implement the lambda expression, which would be easier than the invokedynamic approach? In fact, this was the approach Scala used to follow, but now it has switched to approximately the same approach as Java. The answer to this is the two main advantages of using the invokedynamic approach that we discussed earlier, i.e., defer the implementation to runtime to be able to switch implementation strategy in the future and to allow caching. So let's assume that in the future if the JVM itself gets first-class function support that we may use a direct approach instead of the current one, so if invokedynamic is not used and the resultant bytecode is already compiled/generated, then those classes would not be able to take advantage of a better and performant bytecode generation strategy for implementing lambda, if it comes in the future. Other than that, in fact, using invokedynamic can be a bit faster than the compile time inner class way, because with the compile time inner class way, depending on the number of lambdas, there can be a lot of classes that would have to be loaded and with the invokedynamic way the inner class is created on the fly and on demand. Also, with the compile time inner class way, a new instance of the inner class has to be created for each invocation, but, in the case of the invokedynamic, way it is cached for later use.

Download Building Reactive Microservices in Java: Asynchronous and Event-Based Application Design. Brought to you in partnership with Red Hat

Topics:
lambda ,lambda expression ,java ,java 8 ,functional programming

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}