Never In Doubt: April 2012

We are asked occasionally whether DevForce supports second level caching, that is, does DevForce have some means on the server of remembering previously queried entities between client requests.

This is rarely a real problem in a DevForce application because (a) DevForce applications are usually rich client applications and (b) the DevForce EntityManager cache typically delivers the performance benefits folks seek from a server side cache.

My response, although grounded in long experience, is not always persuasive. Second level caching should improve scalability in theory and theory often trumps reality.

In this post I discuss how to tell if you would benefit from server-side caching and how you might be able to use an Entity Framework second level cache to achieve it.

I haven't tried to install an Entity Framework second level cache. In this post I provide links to information about how to implement second level caching. The links come from reliable sources and it looks like this kind of caching should work. If you try it, please let me know how it goes. I haven’t tried it myself because I find that, for almost all of our customers, this is a solution looking for a problem. But if it makes sense for your application and you give it a try, I'm counting on you to get back to me with your results… and maybe contribute some guidance and code to help others.

Perceived Problem

You have a great many users who repeatedly query for the same entities. Those entities hardly ever change but for some reason they keep asking for them and for some reason you can’t cache them on the client.

You’ve measured and these queries account for a significant percentage of database hits. Moreover, they’re really bogging the database down. You’ve determined conclusively, after careful study of production traffic, that these repetitive database queries are choking your database. You’re pretty sure that server-side caching would provide significant relief.

Are you sure?

Honestly, I don’t think this happens often … which is why you should have the measurements that prove poor performance is traceable to this cause. Don’t guess that this is the problem. Don’t forecast that it is going to be a problem. You need proof.

The interest in second level caches arises most often among people who are evaluating DevForce and haven’t yet built an application with it. Such inquiries are typically speculative. Trust me, you can waste a lot of time investigating something that isn’t going to make any real difference in your application. It might make matters worse.

But suppose you’ve demonstrated that this is a real problem in your working application. You’ve established that the client app can’t cache these entities locally for some reason (perhaps it’s a web client) … which may be why you’re looking into caching on the server.

Maybe it really is time to consider EF “Second Level Caching”

You could try caching query results in a Query Interceptor. But that will require code you must write and if an EntityServer (aka, BOS) is involved you’ll have to make it thread safe. Consider EF “Second Level Caching” before rolling your own.

If there’s a 2^nd, there must be a 1st

Time for some definitions. The “first level cache” is the local cache of entities retrieved by some persistence manager. The DevForce EntityManager is a first level cache on the client. EF’s ObjectContext is a first level cache on the server.

When writing with EF Code First, you create a DbContext which is wrapper around an ObjectContext. You can think of your DbContext as a first level cache if you wish.

On the EntityServer (aka, the BOS) DevForce creates a new ObjectContext for each client request.

This EntityServer is in-process in a 2-tier deployment.

These first level caches do a great job of holding frequently requested entities. But they (and their entities) disappear when the EntityManager or ObjectContext disappear. The EntityManager on the client can live a long time, the life of the user session perhaps. The ObjectContext, on the other hand, evaporates after each client request.

If you had a “second level cache” it would sit outside of the EF ObjectContext. It would outlive the ObjectContext and would be shared by multiple instances of ObjectContexts. When your query reaches a new, empty ObjectContext, EF makes a request to the database. A second level cache could intercept that database request and satisfy it with previously retrieved results.

That sounds like the perfect resolution to your problem. If you had a second level cache, it could hold query results for the entities that clients are clamoring for … and the database pressure would be reduced.

Unfortunately EF doesn’t have an out-of-the-box second level cache.

From time to time you’ll hear someone argue that NHibernate is better than EF because NHibernate does have a second level cache. Often the person making this argument (a) is unaware of the limitations of a second level cache, (b) doesn’t realize that DevForce client-side caching usually eliminates the need for a second level cache and (c) has no evidence that the application would benefit from a second level cache. There’s the whiff of FUD in the air.

Let’s deodorize. As it happens, there is an open source second level cache for EF!

In brief, it’s a plug-in that intercepts EF requests to the EF “Store Provider” (the component that turns EF store queries into SQL queries on a database, be it SQL Server, Oracle, or something else).

The second level cache checks if it’s holding results for that query; if so, it returns them from its cache, short circuiting the call to the database; if not, the query passes through to the Store Provider. Then the second level cache intercepts and caches the returned results for next time. I’m simplifying of course; you’ll want to dig into the resources mentioned below for full details.

Notice that the component includes a tracing interceptor as well.

You don’t have to design your application for second level caching up front. You can add the second level cache component later … when you know you need it. It’s presence (or absence) is largely transparent to DevForce and EF. You’re just “wrapping” the Store Provider in this caching component; it looks like a normal Store Provider to EF.