Thursday, March 8, 2012

Synchronous tasks with Task<T>

I extracted this thought from an email by Microsoft’s Brad Wilson and circulated within my company. Why not share it with you?

Brad starts with an important piece of advice: don’t make a synchronous activity async!

Ok, but how do you construct a Task<T> that you’ll consume within the context of a bundle of tasks? Brad shows how. Hey … thanks Brad!

-----
The following is an anti-pattern with tasks on a server:
return Task.Factory.StartNew(
    () => model.Deserialize(stream, null, type));
This will run your code on a new thread, forcing a context switch, which is unnecessary because your code is fundamentally synchronous. If you’re going to run synchronously, you should just run synchronously, and return a TaskCompletionSource that’s populated with your result. For example:
object result = model.Deserialize(stream, null, type);
var tcs = new TaskCompletionSource<object>();
tcs.SetResult(result);
return tcs.Task;
If Deserialize might throw, then a version with try/catch would be a better implementation of the Task contract:
 var tcs = new TaskCompletionSource<object>();

 try
 {
     object result = model.Deserialize(stream, null, type);
     tcs.SetResult(result);
 }
 catch(Exception ex)
 {
     tcs.SetException(ex);
 }

 return tcs.Task;

What do you mean by "fundamentally synchronous"?


I can assure you that the Deserialize method in question is synchronous.
model.Deserialize(stream, null, type));
That expression blocks until it returns the deserialized object. You see the stream parameter and think "this should be an asynchronous method". Maybe it should be, smarty pants; come back when you have written a deserializer that can reliably produce an object graph without reading the entire stream first.

While we're waiting for your DeserializeAsync implementation, let's push on the proposition that we should not move the execution of Deserialize to another thread.

Clearly, if this method is reading a stream, it could take "a long time" to complete. If we are running on the client, we'll freeze the UI until the deserialization completes. That can't be good. And it isn't. If we're running on the client, you should consider moving the execution of this method to another thread.

In this case, we're running on the server. There is no user waiting for the method to return so we don't care about speed on any particular thread. I'm sure the client cares about a fast response but the response isn't coming until the work is done ... on one thread or another.

We do care about total server throughput. We gain nothing by moving execution to another thread; in fact, we lose because of the thread context switching cost. This is why Brad says that spawning a new task with "Task.Factory.StartNew" is "an anti-pattern ... on a server". It's cool on the client; not cool on the server.

I'm ready with DeserializeAsync; now what?

Should we invoke the async method within a delegate passed to "Task.Factory.StartNew"? No, we should not!

This surprised me too ... until someone walked me through it ... until someone asked me "what do you think will happen on the thread you spawn?" I realized that all I would do on that new thread is dream up some way to wait for DeserializeAsync to finish. Of course DeserializeAsync spawns its own thread so I've got an original thread waiting for my task thread which is waiting for the DeserializeAsync thread. That's a complete waste of time ... and a pointless, resource-wasting context switch.

What's the point of TaskCompletionSource?

We're in this situation because for some (good) reason we want to expose a method - synchronous or asynchronous - as a Task. We don't want or need to spawn a new thread to run that method. We just want to consume it as a Task. The TaskCompletionSource is the wrapper we need for this purpose. It lets us return a Task object with the Task API that we, like puppeteers, can manipulate while staying on the current thread.

For another, perhaps better, and certainly excellent explanation of this, I recommend the following Phil Pennington video on TaskCompletionSource. Happy coding!

7 comments:

wekempf said...

I'm not sure I understand half of what you said here. Not because I don't understand the topic, but because you say several things that are simply wrong which makes it hard to understand what the underlying point may be.

For example, Task.Factory.StartNew doesn't spawn a thread. That doesn't mean there's not a context switch and/or overhead involved, but it is a pretty important concept you get wrong here.

Deserializing something from a stream most certainly is something that can be asynchronous. If that stream is one coming across the wire there's all sorts of room for improving throughput by not treating this as a synchronous process. That's why we have async I/O in the first place. I fully agree that this isn't a place for Task.Factory.StartNew, but the naive implementation of using a TaskCompletionSource but remaining synchronous makes no sense to me. What's gained by returning a Task that is never asynchronous? It doesn't really help with composability. Worse it seems to document behavior that's not there. If I see a DeserializeAsync (or more accurately, a method that returns a Task regardless of the name) method I assume it IS an async method and would be perfectly safe to call from a UI thread, for example. The implementation you show here isn't async and isn't safe to call from a UI thread.

Not knowing the details in the e-mails, I'm not sure I agree with Brad. I don't think it matters if you're talking server side code or client side code here. In fact, it's often more important you get the async story correct on the server side where you need to scale. This means spawning a thread when appropriate, using a thread pool work item when appropriate, and using some other async mechanism, such as async I/O, when appropriate. None of those usages are anti-patterns... unless used incorrectly. As a general principle it's true that threads and thread pools are often overused/misused (on both the server and client), but that doesn't make them an anti-pattern. You want to make maximum use of the CPUs in the system, after all.

Brad said...

My commentary wasn't about whether reading from a stream can be async (obviously it can), but about the efficiency involved when: (a) you are forced to return a Task<T>; (b) you are on a server, which has no concept of "background thread"; and (c) you are doing something which is fundamentally synchronous (for example, something which is CPU-bound). In this case, calling Task.Factory.New() will spin off a new thread in the thread pool (default execution context), but that thread pool is the same one shared with the server accepting requests. All you've done is introduced a thread context switch without any better scalability benefit.

Obviously, when you're not on a server, there is a concept of background threads, so such a thing does make sense. The story becomes complicated when you don't know whether the code you're writing will be used on a client or on a server (in which case, without any foreknowledge, you probably have to default to the slightly more expensive thread context switch so as to prevent blocking the UI).

Ward Bell said...

@wekempf - I'm sorry for my failure to explain effectively and I appreciate the opportunity to clarify if I can.

I don't think I said anything that was "simply wrong".

You say "Task.Factory.StartNew doesn't spawn a thread." Really? According to Stephen Toub it does; see his excellent "Task-based Asynchronous Pattern" white paper. Brad says the same thing in his comment below: "calling Task.Factory.New() will spin off a new thread in the thread pool (default execution context)". MS documentation says "Calling StartNew is functionally equivalent to creating a Task using one of its constructors and then calling Start to schedule it for execution"; Task.Start schedules the task for execution to the current TaskScheduler which "handles the low-level work of queuing tasks onto threads." So take me to school; what do you think "Task.Factory.New()" does?

I obviously agree that "deserializing something from a stream most certainly is something that can be asynchronous" because I said that myself in this post. Perhaps you are complaining about the "Deserialize" method which is synchronous. I agree again ... and said so. But neither of us was in the position of the person who wrote that method. I have no idea why he didn't write it asynchronously; I'm trusting he had good reason.

Why wrap an synchronous method call in a Task? I agree that you should have a reason. Perhaps the method is one of several possible strategies that are all manifested as Tasks because any of them could be asynchronous. This is just one way it could participate in a composed Task chain. I've been in this spot many times.

You'll find numerous examples of Tasks that return synchronously in Stephen Toub's aforementioned paper. I also encourage you to look at the Phil Pennington video on TaskCompletionSource that I referenced in my post.

You write "the implementation you show here isn't async and isn't safe to call from a UI thread." I assume by "unsafe" you mean that it would be blocking; to my mind that's not a matter of safety so much as a matter of poor citizenship (unless you're in WinRT in which case it will just fail). But this is besides the point. I said ... he said ... we all said ... that this advice concerns execution on the server, not on the UI. How can I make that more clear?

Spawning a thread for an IO-intensive operation is as sensible on the server as it is on the client. But that isn't the point here either. The advice is "don't use Task.Factory.StartNew on the server simply to execute another method ... whether that method is synchronous or asynchronous".

Ward Bell said...

@Brad - I think we're saying the same thing. You just said it more succinctly.

Ward Bell said...

As long as we're on this subject, let me recommend another Stephen Toub article, "Async Performance: Understanding the Costs of Async and Await" which discusses the overhead of asynchrony and when to avoid it even when you manifest the method as asynchronous

wekempf said...

Brad's statement "calling Task.Factory.New() will spin off a new thread in the thread pool (default execution context)" requires a bit of parsing, but is basically accurate. A statement that "Task.Factory.StartNew spins off a thread" isn't, however. What Task.Factory.StartNew does is dependent on several factors, but when using the default execution context it is equivalent to ThreadPool.QueueUserWorkItem. This may cause a thread to be created (what I assume "spun up" means, as that's the typical meaning applied here), but more likely there will already be running threads in the pool that will just grab the work and do it at some point. There's a context switch here, and definite overhead involved, but nowhere near what's involved with creating a thread.

I'm still looking in from the outside, and I'm sure I'm not understanding something important here, but some of the things in Brad's reply still don't make sense to me.

"You are forced to return a Task." Why would you be forced to return a Task, especially for something not asynchronous? You're explanation around this doesn't make sense to me either. "Perhaps the method is one of several possible strategies that are all manifested as Tasks because any of them could be asynchronous." But this one isn't asynchronous. "This is just one way it could participate in a composed Task chain." But you don't have to use tasks to compose an operation within a Task chain, so I'm still at a loss for why you'd want to, let alone need to, return a Task here.

"you are on a server, which has no concept of 'background thread'" This really confuses me. Servers have as much concept of background threads as clients do. In fact it's been my experience that servers use threads more than clients. Are you strictly trying to differentiate based on the word "backround" here, thinking there's something magical about the UI thread? That's certainly not always the case. I've worked on several server programs for which there is a definite "main thread of execution" and several "background threads" that handle certain operations asynchronously.

"All you've done is introduced a thread context switch without any better scalability benefit." I get the point being made here, but I think this is said a little too matter of factly. There are CPU bound operations for which it does improve scalability to run them on thread pool threads. This is the very reason that parallel loops are showing up in all sorts of frameworks, including in the TPL. Yes, this does mean you're using time within the thread pool that otherwise could have gone to servicing I/O requests, but that doesn't automatically mean you're not improving the overall throughput and scalability of the server. As general advice I'm in agreement, but without talking specific scenarios I can't go beyond the "general" in that qualification.

Finally, Ward, I think you misunderstood me, not the other way around. I fully understand you're talking about server side code. I maybe wasn't fully clear when I made the statement about blocking the UI thread, as that's simply the scenario most people are cognizant of. My point was that returning a Task is supposed to be a clear statement that the call is asynchronous in nature. When the operation isn't asynchronous you can have a serious problem with such signatures. The blocking of the UI thread is the obvious example, but there are other examples as well, including scenarios in server programs. In any process that you need to maintain responsiveness a call to a method you expect to be asynchronous that instead performs a lengthy, possibly blocking, operation can cause serious problems.

Ward Bell said...

@wekempf - ok, I'll split that hair with you and concede that the thread to which work is delegated may already exist in the thread pool in which case it would not be created. The fundamental advice: think twice before moving work to a new server thread because of the context switch ... stands as I wrote it and was really the point of the post.

We are of the same mind on avoiding spurious use of Tasks. I leave for another day a proper treatment of when you might find yourself wrapping synchronous behavior in Tasks as I've reached a point of exhaustion on this thread.

But there is one final point of such great importance that I must call it out.

The distinction between a foreground thread and a background thread is critical in a UI. The work done on the foreground thread directly affects the user experience. If the foreground thread bogs down, the user perceives the application as slow, non-responsive, or crashed. You simply have to keep the foreground thread moving crisply which means you should move all "long running" work to a non-UI "background" thread.

There is no user on the server and therefore no comparable distinction between foreground and background ... emphasis on comparable.

As you say, there can be advantage to moving IO work onto other server threads so that the scheduler can better manage resources. Under these circumstances you may designate the request thread as "foreground" and the supporting worker threads as "background". But when you do this kind of multi-threading, it's because you want to maximize throughput and shorten the total request time if you can (as when offloading several IO activities to multiple threads running in parallel).

Such multiple threading aims at server resource maximization. The client-side multi-threading aims at human resource maximization.

One way to grasp the difference: you'd be willing to suffer worse total throughput on the client in order to maintain a crisp user experience by keeping the UI thread free of tasks that can be offloaded.

I'm sure that this makes sense to you and hope it is helpful to others.