Download

• ArbitraryAsyncReturns.zip [22mb]
• INSTALL: Unzip the file. Quit VS. Double-click to install in this order: (1) Roslyn.VisualStudio.Setup.vsix, (2) Roslyn.Compilers.Extension.vsix, (3) ExpressionEvaluatorPackage.vsix. I don't think the others are needed.
• TRY IT OUT: the zip file contains a sample project
• UNINSTALL: I've usually been able to go to within Visual Studio to Tools>Extensions, search for Roslyn, and uninstall in the order (1) Expression Evaluators, (2) Compilers, (3) Language Service. Once that didn't work, and I recovered my VS by deleting the entire folder %userprofile%\AppData\Roaming\Microsoft\VisualStudio and %userprofile%\AppData\Local\Microsoft\VisualStudio. Doing so will reset all your VS settings to default.

Introduction

It’s an ongoing user request for async methods to return arbitrary types. We’ve heard this particularly from people who want to return value Task-like types from their async methods for efficiency. In UWP development, it would be nice also to return IAsyncAction directly rather than having to return a Task and then write a wrapper.

The initial C# prototype of async did allow to return arbitrary types from async methods, but that was cut because we didn’t find a neat way to integrate it with generic type inference. Here’s a recap of the problem:

var xf = f(async () => {return 3;});
var xg = g(async () => {return 3;});
var xh = h(async () => {return 3;});
var xk = h(async () => {return 3;});

T f(Func<Task<T>> lambda) => lambda().Result;
T g(Func<MyTasklike<T>> lambda) => lambda().MyResultlike;
T h(Func<MyPerverse<T>> lambda) => lambda().MyPerverseResult;
T k(Func<MyWeird<T>> lambda) => lambda.MyWeirdResult;

// How to we infer what "T" should be based on the return statements in the lambda?
// In full generality it would depend on the details of how the tasklike things (Task<T>,
// MyTasklike<T>, MyPerverse<T>, MyWeird<T>) are built, and in particular how
// their builder's SetResult(...) method relates to the generic type parameter of the Task
// property of that builder...

// f: the builder for Task<T> lets you do “return 3;” and it infers that T should be int
// g: the builder for MyTasklike<T> lets you do “return 3;” and it infers that T should be int
// h: the builder for MyPerverse<T> lets you do “return 3;” but it infers that T should IEnumerable<int>
// k: the builder for MyWeird<T> lets you do “return 3;” but this has no bearing on what T should be: type inference failure

The feature was cut because we had no solution for the full generality of situations like MyPerverse<T> or MyWeird<T>: no way to make the leap from the type of the operand of the return statement, to the generic type arguments of the Task-like type.

Proposal: let’s just abandon the full generality cases h and k. They’re perverse and weird. Let’s say that async methods can return any type Foo (with similar semantics to Task) or can return any type Foo<T> (with similar semantics to Task<T>).

More concretely, let's say that the return type of an async method or lambda can be any task-like type. A task-like type is one that has either have zero generic type parameters (like Task) or one generic type parameter (like Task<T>), and which is declared in one of these two forms, with an attribute:

[TaskLike(typeof(FooBuilder))] struct Foo { … }
struct FooBuilder { … similar to AsyncTaskMethodBuilder … }

[TaskLike(typeof(FooBuilder<T>))] struct Foo<T> { … }
struct FooBuilder<T> { … similar to AsyncTaskMethodBuilder<T> … }

All the type inference and overload resolution that the compiler does today for Task<T>, it should also do the same for other tasklikes. In particular it should work for target-typing of async lambdas, and it should work for overload resolution.

For backwards-compatibility, if the return type is System.Threading.Tasks.Task or Task<T> and also lacks the [TaskLike] attribute, then the compiler implicitly picks System.Runtime.CompilerServices.AsyncTaskMethodBuilder or AsyncTaskMethodBuilder<T> for the builder.

Semantics

[edited to incorporate @ngafter's comments below...]

The VB spec currently explains the semantics of async methods+lambdas like this:

If the async method is a Function with return type Task or Task(Of T) for some T, then an object of that type implicitly created, associated with the current invocation. This is called an async object and represents the future work that will be done by executing the instance of the async method. When control resumes in the caller of this async method instance, the caller will receive this async object as the result of its invocation.

This would be changed to say that, by assumption, the async method's return type is a Tasklike, which has an associated builder type (specified by attribute). An object of that builder type is implicitly created by invoking the builder's static Create method, and the return value of the async method is the result of the .Task property on that builder instance. (or void).

The spec goes on to explain the semantics of exiting the method:

If control flow exits through an unhandled exception, then the async object’s status is set to TaskStatus.Faulted and its Exception.InnerException property is set to the exception (except: certain implementation-defined exceptions such as OperationCanceledException change it to TaskStatus.Canceled). Control flow resumes in the current caller. Otherwise, the async object’s status is set to TaskStatus.Completed. Control flow resumes in the current caller.

This would be amended to say that if control flow exits through an unhandled exception ex then .SetException(ex) is called on the builder instance associated with this method instance. Otherwise, the .SetResult() is called, with or without an argument depending on the type.

The spec currently gives semantics for how an async method is started:

The instance's control point is then set at the first statement of the async method body, and immediately starts to execute the method body from there.

This would be amended to say that the builder's Start method is invoked, passing an instance that satisfies the IAsyncStateMachine interface. The async method will start to execute when the builder first calls MoveNext on that instance, (which it may do inside Start for a hot task, or later for a cold task). If the builder calls MoveNext more than once (other than in response to AwaitOnCompleted) the behavior is undefined. If ever the builder's SetStateMachine method is invoked, then any future calls it makes to MoveNext must be through this new value.

Note: this allows for a builder to produce cold tasklikes.

The spec currently gives semantics for how the await operator is executed:

Either ICriticalNotifyCompletion.UnsafeOnCompleted is invoked on the awaiter (if the awaiter's type E implements ICriticalNotifyCompletion) or INotifyCompletion.OnCompleted (otherwise). In both cases it passes a resumption delegate associated with the current instance of the async method. The control point of the current async method instance is suspended, and control flow resumes in the current caller. If later the resumption delegate is invoked, the resumption delegate first restores System.Threading.Thread.CurrentThread.ExecutionContext to what it was at the time OnCompleted was called, then it resumes control flow at the control point of the async method instance.

This would be amended to say that either builder.AwaitOnCompleted or builder.AwaitUnsafeOnCompleted is invoked. The builder is expected to synthesizes a delegate such that, when the delegate is invoked, then it winds up calling MoveNext on the state machine. Often the delegate restores context.

Note: this allows for a builder to be notified in when a cold await operator starts, and when it finishes.

Note: the Task type is not sealed. I might chose to write MyTask<int> FooAsync() which returns a subtype of Task, but also performs additional logging or other work.

Semantics relating to overload resolution and type inference

Currently the rules for Inferred return type say that the inferred return type for an async lambda is always either Task or Task<T> for some T derived from the lambda body. This should be changed to say that the inferred return type of an async lambda can be whatever tasklike the target context expects.

Currently the overload resolution rules for Better function member say that if two overload candidates have identical parameter lists then tie-breakers such as "more specific" are used to chose between then, otherwise "better conversion from expression" is used. This should be changed to say that tie-breakers will be used when parameter lists are identical up to tasklikes.

Limitations

This scheme wouldn't be able to represent the WinRT types IAsyncOperationWithProgress or IAsyncActionWithProgress. It also wouldn't be able to represent the fact that WinRT async interfaces have a cancel method upon them. We might consider allowing the async method to access its own builder instance via some special keyword, e.g. _thisasync.cancel.ThrowIfCancellationRequested(), but that seems too hacky and I think it's not worth it.

Compilation notes and edge cases

Concrete tasklike. The following kind of thing is conceptually impossible, because the compiler needs to know the concrete type of the tasklike that's being constructed (in order to construct it).

class C<T> where T : MyTasklike {
    async T f() { } // error: the return type must be a concrete type
}

Incomplete builder: binding. The compiler should recognize the following as an async method that doesn't need a return statement, and should bind it accordingly. There is nothing wrong with the async modifier nor the absence of a return keyword. The fact that MyTasklike's builder doesn't fulfill the pattern is an error that comes later on: it doesn't prevent the compiler from binding method f.

class C { async MyTasklike f() { } }
[Tasklike(typeof(string))] class MyTasklike {}

Wrong generic. To be a tasklike, a type must (1) have a [Tasklike] attribute on it, (2) have arity 0 or 1. If it has the attribute but the wrong arity then it's not a Tasklike.

class C { async MyTasklike f() { } } // okay: return type has arity 0
class C { async MyTasklike<int> f() { return 1;} } // okay: return type has arity 1:int
class C { async MyTasklike<int> f() { return "s";} } // error: should return an int, not a string
class C { async MyTasklike<int,int> f() { } } // error

Incomplete builder: codegen. If the builder doesn't fulfill the pattern, well, that's an edge case. It should definitely give errors (like it does today e.g. if you have an async Task-returning method and target .NET4.0), but it doesn't matter to have high-quality errors (again it doesn't have high quality errors today). One unusual case of failed builder is where the builder has the wrong constraints on its generic type parameters. As far as I can tell, constraints aren't taken into account elsewhere in the compiler (the only other well known methods with generic parameters are below, and they're all in mscorlib, so no chance of them ever getting wrong)

System.Array.Empty
System_Runtime_InteropServices_WindowsRuntime_WindowsRuntimeMarshal__AddEventHandler_T
System_Runtime_InteropServices_WindowsRuntime_WindowsRuntimeMarshal__RemoveEventHandler_T
System_Activator__CreateInstance_T
System_Threading_Interlocked__CompareExchange_T
Microsoft_VisualBasic_CompilerServices_Conversions__ToGenericParameter_T_Object