Closures for Java (v0.6a)

Neal Gafter and Peter von der Ahé

We describe a proposed extension of the Java Programming Language that supports deferred execution of snippets of code. These correspond to lambda expressions in languages such as Javascript, C#, and Groovy.

Lambda Expressions

We introduce a syntactic form for constructing an anonymus function value:

Expression:

ExpressionLambda

Primary:

StatementLambda

ExpressionLambda:

# ( FormalParameters_opt ) Expression

StatementLambda:

# ( FormalParameters_opt ) BlockStatement

We refer to these two expression forms, expression lambdas and statement lambdas as lambda expressions. Evaluating a lambda expression does not cause its controlled statement or expression to be evaluated immediately, but rather it builds a function object that represents the code. The function object can be stored and invoked later to cause the controlled statement or expression to be evaluated.

Within the controlled statement of a statement lambda

• A return statement yields a result from the statement lambda.
• It is a compile-time error if some break or continue statement transfers control to a target outside the statement lambda.

Within the controlled expression or statement

• The meanings of names are the same as the meanings of names in the context in which the lambda expression occurs, with the exception that the formal parameters introduce variable names that are in scope within the controlled expression or statement (and may shadow variable names from enclosing scopes).
• this and super have the same meaning as they would where the lambda expression appears.
• A mandatory warning is required if some local variable or parameter from an enclosing scope is used, unless one of the following conditions holds:
  1. The variable is declared final, or
  2. The variable is not the target of any assignment (i.e., it is only initialized in its declaration), or
  3. The variable is annotated @Shared, or
  4. some enclosing construct is annotated @SuppressWarnings("shared").

The type of a lambda expression is defined by the set of declared argument types, result expressions, and thrown types of the lambda's body. While the type of a lambda expression is not denotable in source, a lambda expression may be converted to some object type by a lambda conversion (see later). The target of such a conversion it typically a function type (see below) or other user-defined interface type. It is a compile-time error if a lambda expression appears in a context where it is not subject to a lambda conversion. The conversion occurs entirely at compile-time.

Example: the following lambda expression takes two integers and yields their sum:

#(int x, int y) x+y

Function Types

We introduce a syntactic form for a function type:

Type:

FunctionType

FunctionType:

# Type ( TypeList_opt ) FunctionThrows_opt

TypeList:

Type
Type , TypeList

FunctionThrows:

throws FunctionExceptionTypeList

FunctionExceptionTypeList:

ExceptionType
FunctionExceptionTypeList | ExceptionType

Informally, a function type describes the set of functions that accept a given list of argument types, result in a value of the given type, and may throw the indicated checked exception types.

Example: the following assigns to the local variable plus a function that computes the sum of its two int arguments:

#int(int,int) plus = #(int x, int y) x+y;

A function type is defined to have the same meaning as a parameterized interface type of a system-generated interface that has a single method named invoke. [This paragraph is a placeholder for a more precise description, including the name of the interface, required for interoperability among compilers. We expect the generated interfaces to be in the package java.lang.function, which will be closed to user-defined code, and the naming scheme to be an extension of the JVM method signature scheme.]

While some have proposed to add function types as a first-class extenstion to Java's type system, we don't believe that is necessary. Interfaces are expressive enough to be used to represent function types. Unfortunately, doing so directly in user code is clumsy, error-prone, verbose, and difficult to read and write. The function type notation provides a more readable, convenient, and less error-prone shorthand, and the interfaces to which it is translated provide a standard for interoperability among libraries and languages.

Example. The variable declaration

#int(String) parseInteger;

is translated into

interface java.lang.function.IO<A1,throws X> { // system-generated
    int invoke(A1 x1) throws X;
}
IO<? super String,? extends Nothing> parseInteger;

We would expect the compiler, runtime system, and debuggers to display these types using the function type syntax rather than the translation involving generics.

Because a function type is an interface type, it can be extended by a user-defined interface and it can be implemented by a user-defined class.

# T_r ( T₀ ... T_n ) throws E₀ | ... E_m

public T_r invoke(T₀ x₀, ... T_n x_n) throws E₀, ... E_m

Lambda Conversion

A lambda expression may be converted to any compatible interface type by the lambda conversion, which is a kind of widening conversion.

There is a lambda conversion from a lambda expression to every interface type that has a single method m such that the lambda expression is compatible with m. A lambda expression is compatible with a method m iff all of the following hold:

• Either
  • The lambda expression is an expression lambda, and either
    • there is an assignment conversion from the type of its result expression to the return type of m; or
    • the result expression is a subtype of java.lang.Void and the method m has return type void; or
  • The lambda expression is a statement lambda, and either
    • its body cannot complete normally and there is an assignment conversion from the type of the expression in each of its return statements to the return type of m; or
    • all of its return statements yield no return value, and the method m has return type void or java.lang.Void
• m has the same number of parameters as the lambda expression.
• For each corresponding position in the parameter lists, both parameter types are the same.
• Every checked exception type that can be thrown by the body of the lambda expression is a subtype of some exception type in the throws clause of m.

Example: We can create a function object that adds two to its argument like this:

interface IntFunction {
    int invoke(int i);
}
IntFunction plus2 = #(int x) x+2;

 

Alternatively, using function types:

#int(int) plus2 = #(int x) x+2;

We can use the existing Executor framework to run a function object in the background:

void sayHello(java.util.concurrent.Executor ex) {
    ex.execute(#(){ System.out.println("hello"); });
}

Method References

A method reference is an expression form that allows a reference to a method to be used as shorthand for a lambda expression that invokes that method

Primary:

MethodReference

MethodReference:

Primary # TypeArguments_opt Identifier ( TypeList_opt )

(#(Type x0, Type x1 ...)tmp.<TypeArguments>Identifier(x0, x1 ...))

(#(Type x0, Type x1 ...){tmp.<TypeArguments>Identifier(x0, x1 ...);})

Where tmp is a temporary value that holds the computed value of the primary expression. The former translation is used when the resolved method has a non-void return type, while the latter is used when the resolved method has a void return type.

If the primary resolves to a type, then this is translated to

(#(Type x0, Type x1 ...)Primary.<TypeArguments>Identifier(x0, x1 ...))

(#(Type x0, Type x1 ...){Primary.<TypeArguments>Identifier(x0, x1 ...);})

when the resolved method is static, or

(#(Primary x, Type x0, Type x1 ...)x.<TypeArguments>Identifier(x0, x1 ...))

(#(Primary x, Type x0, Type x1 ...){x.<TypeArguments>Identifier(x0, x1 ...);})

when the resolved method is an instance method.

Exception type parameters

To support exception transparency (that is, the design of APIs in which the compiler can infer that exceptions thrown by a lambda expression are thrown by the API), we add a new type of generic formal type argument to receive a set of thrown types. [This deserves a more formal treatment]

TypeParameter:

throws TypeVariable TypeBound_opt

ActualTypeArgument:

throws ExceptionList

ExceptionList:

ReferenceType
ReferenceType | ExceptionList

What follows is an example of how the proposal would be used to write an exception-transparent version of a method that locks and then unlocks a java.util.concurrent.Lock, before and after a user-provided block of code. Given

public static <T, throws E extends Exception>
T withLock(Lock lock, #T()throws E block) throws E {
    lock.lock();
    try {
        return block.invoke();
    } finally {
        lock.unlock();
    }
}

This can be used as follows:

withLock(lock, #(){
    System.out.println("hello");
});

This uses a newly introduced form of generic type parameter. The type parameter E is declared to be used in throws clauses. If the extends clause is omitted, a type parameter declared with the throws keyword is considered to extend Throwable (instead of Object, which is the default for other type parameters). This type parameter accepts multiple exception types. For example, if you invoke it with a block that can throw IOException or NumberFormatException the type parameter E would be given as IOException|NumberFormatException. In those rare cases that you want to use explicit type parameters with multiple thrown types, the keyword throws is required in the invocation, like this:

Locks.<throws IOException|NumberFormatException>withLock(lock, #(){
    System.out.println("hello");
});

You can think of this kind of type parameter accepting disjunction, "or" types. That is to allow it to match the exception signature of a function that throws any number of different checked exceptions. If the block throws no exceptions, the type argument would be the type Nothing (see below).

Type parameters declared this way can be used only in a throws clause or as a type argument to a generic method, interface, or class whose type argument was declared this way too.

Throws type parameters are indicated in class file format by the presence of the character '^' preceding the type parameter.

Definite assignment

A variable V is (un)assigned after a lambda expression iff V is (un)assigned before the lambda expression

A variable V used in a lambda expression's body but declared outside the lambda is

• definitely assigned before the lambda expression's body iff V is definitely assigned before the lambda expression; and
• not definitely unassigned before the lambda expression's body.

The return statement

A return statement returns control to the invoker of the nearest enclosing method, constructor, or statement lambda.

The type Nothing

We add the non-instantiable type java.lang.Nothing. Values of this type appear where statements or expressions cannot complete normally. This is necessary to enable writing APIs that provide transparency for functions that do not complete normally. Nothing is a subtype of every obect type. The null type is a subtype of Nothing. At runtime, a NullPointerException is thrown when a value of type Nothing would appear. This occurs when a programmer converts null to Nothing.

Example: The following illustrates a function being assigned to a variable of the correct type.

interface NullaryFunction<T, throws E> {
    T invoke() throws E;
}
NullaryFunction<Nothing, Nothing> thrower = #(){ throw new AssertionError(); };

Reachable statements

The initial statement of a statement lambda is reachable.

An expression statement in which the expression is of type Nothing cannot complete normally.

Acknowledgments

The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal:

C. Scott Ananian, Lars Bak, Cedric Beust, Joshua Bloch, Gilad Bracha, Martin Buchholz, Alex Buckley, Maurizio Cimadamore, Ricky Clarkson, Stephen Colebourne, Danny Coward, Luc Duponcheel, Erik Ernst, Rémi Forax, James Gosling, Christian Plesner Hansen, Cay Horstmann, Kohsuke Kawaguchi, Danny Lagrouw, Doug Lea, Peter Levart, "crazy" Bob Lee, Mark Mahieu, Niklas Matthies, Tony Morris, Martin Odersky, Terence Parr, Tim Peierls, Cris Perdue, John Rose, Ken Russell, Stefan Schulz, Guy Steele, Mads Torgersen, Zdenek Tronicek, Jan Vitek, and Dave Yost.

Minor changes since 0.6 first published

• Added Peter von der Ahé as a coauthor.
• In the lambda expressions section, separated two semantic bullets that only apply to statement lambdas.
• The VM signature character for throws type parameters has been changed from # to ^.
• Other minor formatting improvements and additions to the acknowledgments.
• A "throws" type parameter implicitly extends Throwable, not Exception.
• The lambda conversion allows an expression lambda yielding java.lang.Void to be compatible with a method returning void
• Within a lambda expression, super has the same meaning as it would in the enclosing scope. Technically, this is not a change to the meaning of the specification, because it "goes without saying".
• Reworded the definite assignment section to use JLS terminology.
• Clarified that the type of a lambda expression is not denotable.

Changes in this revision (0.6) compared to BGGA 0.5

• The control invocation syntax has been moved to a separate specification.
• The term closure literal has been replaced by lambda expression.
• We overhauled the syntax for lambda expressions, borrowing loosely from Clang. There are now two forms of lambda expression: an expression lambda has a controlled expression, while a statement lambda has a controlled statement.
• The term closure object has been replaced by function object.
• The term closure conversion has been replaced by lambda conversion.
• Added new semantics for the return statement: it can now be used to return from a statement lambda.
• Renamed java.lang.Unreachable to java.lang.Nothing ala Scala.
• Removed support for the type name null. The previous version used null as a placeholder for an exception type when none can be thrown. The type Nothing now serves that purpose.
• Overhauled restricted. The concepts of restricted and unrestricted function types and closures have been removed. Now, all lambda expressions are restricted. The equivalent of the previous specification's unrestricted closure may be passed to a method by using the control invocation syntax (0.6b).
• Added support for method references: We added support for treating a reference to a method as a function using a newly introduced token #. The syntax is borrowed from cross-references in javadoc and the FCM proposal.

Changes that need to go into the prototype

• Change the syntax of lambda expressions.
• Remove the restricted/unrestricted distinction; remove JDK API retrofits.
• Make #int(int) a non-generic interface java.lang.function.II0.
• Integrate the closures prototype with recent openjdk javac changes, particularly the diagnostic improvements.
• Change the pretty-printing of function types to use the new function type syntax.