DevForce “Blends”

We are seeing an increasing interest among developers in using Blend to design application views. I think this is great. Silverlight and WPF are great technologies for building productive business applications. But you can’t be as productive and effective as you should be if you only work in XAML. You need the tools.

And Blend 4 is an incredibly great tool. I say this even though I’m only competent to use about $1.30 of it.

All of this is preface to a forum question we received:

I’m attempting to mock data at *design* time. … Has anyone ever managed to get this to work? …Using DF to avoid creating dozens of mock repositories on a large project? Now that would be a coup!

My answer follows:

We are well on our way to making DevForce play well with Blend. Actually, we probably are THERE. But we haven't documented it well and I don't feel that I'm particularly close to anything worthy of calling a "Best Practice." Let's say I have some "ok" practices :-/

Here's visual evidence of an existence proof: yes, you truly can use a DevForce EntityManager and entities at design time:

This example comes from my slight re-design of the "Simple ComboBox in Silverlight" sample code.

[Aside: the existing sample is on our web site now. The revised one will be released with our forthcoming "DevForce Cookbook." Look for the Cookbook to arrive within a matter of weeks.]

The original point of the sample was to demonstrate how to wire up a ComboBox to DevForce entities AND bind that ComboBox to other DevForce entities.

The previous version looked bloody awful; this one just looks bad.

However, it's beautiful for the purpose of this forum post! You can plainly see employee, "Ima", in on the Blend art board. "Ima" does not exist in the Northwind tutorial database. She's a fake Employee, ginned up in a MainPageDesignViewModel that derives from the production MainPageViewModel. The view doesn't know; it's just binding to a ViewModel (VM).

All of the fakery happens in the design VM's constructor, shown here:

    public MainPageDesignViewModel() {

      EntityManager = new NorthwindManager(false);
      EntityManager.DefaultQueryStrategy = QueryStrategy.CacheOnly;

      var designEmp = new Employee {
        EmployeeID = 1, FirstName = "Ima", LastName = "Blast", ReportsToEmployeeID = 2,
      };
      var designManagerEmp = new Employee {
        EmployeeID = 2, FirstName = "Slim", LastName = "Ornun", ReportsToEmployeeID = null,
      
     EntityManager.AttachEntity(designEmp);
      EntityManager.AttachEntity(designManagerEmp

      PotentialManagers = new List<Employee> { designManagerEmp };
      CurrentEmployee = designEmp;
      Messages.Add("We're in DesignTime now !!!");
    }

The design version of the VM (a) creates two fake employess (Ima and her boss, Slim), (b) creates a disconnected, cache-only EntityManager (EM) - to ensure there is no attempt to talk to the database, and (c) adds the two employees to that EM. Because both fakes are in cache, DevForce can implement the navigation between Ima and Slim via the "Manager" property. I neglected to show this in the ComboBox but you can see it in the message which reads "Ima Blase whose manager is Slim Ornun".

It is only a few short hops from here to doing the cache-to-file trick that you want. In fact, I've done what you want on a different project and we'll provide a similar, smaller example in one of the Cookbook recipes .... although perhaps not in the first Cookbook edition. Trust me, it works great.

To be precise, it worked great for test runs of your application that you wanted to be fast and free of the need to talk to a database. It didn't work in Blend (although for different reasons than Jason cites).

Until DF 2010 v. 6.0.5, it simply wasn't viable to use DF entities in Cider (Visual Studio visual designer) or Blend because we required that the Desktop (aka Web aka "Full DotNet") model carry the same assembly name as the Silverlight model. No problem for the CLR but most tools, including design tools, just choked.

Using the same assembly name is no longer required. In fact, is heavily discouraged.

Now this ComboBox sample dates from 6.0.2 (I think) ... certainly before we got rid of the assembly name restriction. It did not Blend.

All I had to do was some assembly renaming plus clean and re-build. The view popped right up in Cider and Blend.

There was no data visualization though. Just a bunch of weird boxes. The XAML looked ok ... obviously it ran ok ... but it made no sense rendered in Cider or Blend.

A little dragging around was helpful but I still wanted to see sample data.

So I refactored the ViewModel to facilitate a derived, design-time version of that ViewModel and, presto, that's how I was able to get the layout you see in the image above.

[While I was at it, I replaced the button click wiring with Blend Behaviors ... and deleted the entire custom code-behind]

You can do the same things to your application, or wait for the Cookbook if you want to see the details.

As I mentioned, the cache-to-file variation is not far off. That will take some explanation, however, as there are a few places you can stumble on the way ... as you discovered. Please hang in there.

DevForce in the Cloud: Prism Explorer

On our second flight to Azure, we deployed our “Prism Explorer” sample application. The address is the same as before: http://dftest003.cloudapp.net/

The UI dates from Silverlight 2 (our first Silverlight release). We’ve ported it forward to DevForce 2010 and Silverlight 4 but haven’t touched the XAML. Pretty ugly but it gets the point across. Some day, some day …

Prism Explorer, aside from showing that DevForce and Prism play nice, demonstrates a variety of DevForce features such as:

• Arbitrarily complex queries, including anonymous projections
• Add, delete (enabled for Customer only in this case), and save
• Integrated validation (e.g., Customer requires a city)
• Lazy and eager load of related entities
• Online and offline (toggle “Connect” button)
• Save and restore entities to isolated storage file
• Out of browser

Here it is, running Out-of-Browser, after querying for “Customers (and their orders) starting with C”

Here’s another moment in the life of the UI – queried for all Employees and clicked Edit button on good ole Nancy Davolio:

I shot video of Kim and me deploying it. Raw footage is 40 minutes, much of that watching it build on my stressed-out notebook (Camtasia and VS competing for cycles). Hope to get it to 20 minutes after cutting out the dross.

Still, I was amazed to get this done in 50 minutes of clock time (paused video 10 minutes while publishing to Azure staging ). That includes commentary, fumbling, and miscues.

More on all of that in my next post.

DevForce in the Cloud: 1st Flight

Our first Azure DevForce “application” debuted today.

Sure it’s just Northwind customers in an un-styled, read-only Silverlight DataGrid. Kind of like putting a monkey in orbit. But it’s an important step for IdeaBlade and DevForce … and I’m feeling a wee bit proud.

Behind that simple (simplistic?) exterior are DevForce 2010, SQL Azure, Windows Azure, Silverlight 4, Entity Framework v.4, and .NET 4. That really is EF 4 running in the cloud, not on the client as it is in most Azure demonstrations. That really is the DevForce middle tier “BOS” in the cloud and a DevForce Silverlight client executing in your browser.

We’ll probably have taken this sample down by the time you read this post. But who knows; if you’re quick, you may still be able to catch it at http://dftest003.cloudapp.net/

Kudos to Kim Johnson, our ace senior architect / developer who made it happen. Thanks to Microsoft’s Developer Evangelist Bruno Terkaly who first showed us EF 4 in Azure when there were no other published examples.

I’ll be talking about DevForce and Azure much more as our “DevForce in the Cloud” initiative evolves. It’s clearly past the “pipe dream” phase and I anticipate acceleration in the weeks ahead.

Those of you who’ve been wondering if we’d get there (Hi Phil!) or whether to "play it safe" and build your own datacenter (really?) … wonder no more!

ObservableCollection / DataGrid Memory Leaks

Ran into a BEAR of a memory leak today. Or, rather, one of our customer’s did … while investigating our PrismExplorer sample that demonstrates DevForce 2010 and Prism v.2.2 playing well together.

This post is a re-post of my exchange on our IdeaBlade Forum. I figure it’s worth repeating here should you run into it and not think to look on our forum.

The lesson, I hasten to add, is for all Silverlight developers. DevForce entities are not the cause. If you bind a Silverlight DataGrid to objects that implement INotifyPropertyChanged, this post is for you.

The takeaway is this: Don't reassign the ObservableCollection<T> bound to an ItemsSource; repopulate it!

An now … the story. First, the customer reports a problem

When ever I bind a datagrid or listbox via a ViewModel to an ObservableCollection(results) in a Callback method from a simple query … I end up consuming memory which will never be released, … not after disposing of the EntityManger, View, Viewmodel, etc. This behavior can be reproduced in your Prism Explorer when querying Customers, Orders, and back a few times. You can see this in Performance Monitor (perfmon) under process->private bytes where the count keeps climbing up.

Here is my response:
-------------------------------------------------------

You are (mostly) correct and oh ... how awful!

As I demonstrate below, memory is recovered when you clear the EntityManager. It would be recovered if you disposed of the EntityManager itself. I'm not sure why you didn't see these effects.

However, you are absolutely right that something horrible is happening. I spent much of the day researching it.

The only good part of the story is that it has nothing to do with DevForce. The cause is something in Silverlight, probably in the Silverlight DataGrid.

I will explain but it will be a bit of a journey leading ... ultimately ... to a sad-but-necessary workaround.

If you want to skip the journey and go right to the destination, the lesson for your code is:

Don't reassign the ObservableCollection<T> bound to an ItemsSource; repopulate it!

And now ... my sad tale ...

The main "ModelExplorer" view binds a Silverlight DataGrid.ItemsSource to the QueryResults property of the supporting ModelExplorerViewModel (the VM).

"QueryResults" is defined as IList but is always an ObservableCollection<T> in practice (henceforth written as "OC<T>").

The problem starts with the repeated resetting of the QueryResults in the ModelExplorerViewModel after every query.

A peculiarity of PrismExplorer (PE) is that the main "ModelExplorer" view could be asked to display objects of any type in the DataGrid. That's its purpose as a demo screen. Your grids typically only show one kind of object. But PE must work with arbitrary query result types.

Because those types change, the OC<T> must change to match type.

The obvious thing to do was craft a new OC<T> after every query ... one designed for the query results type, assign that new OC to the QueryResults, and raise PropertyChanged on "QueryResults". The DataGrid dutifully hears the call and refreshes its display.

Unfortunately, it consumes an extra 300K every single time. "300K" is not a misprint.

Here's the code with the garbage collection call and logging (aside: we now inject ILoggerFacade into the VM now so that we can write to the Visual Studio Output window).

    public IList QueryResults {
      get { return _queryResults; }
      set {
        _queryResults = value;

        Log("=== Total Memory = " + GC.GetTotalMemory(true));

        RaisePropertyChanged("QueryResults", "QueryResultsCount");
        SelectFirstQueryResultsItem();
      }
    }

If you add this instrumentation, query for "All Customers", and click the "Query" button repeatedly, the Output window will show that your memory consumption is climbing ~300K each time. Here are actual measurement.

Prism Explorer: "=== Total Memory = 3406212";
Prism Explorer: "=== Total Memory = 3914916";
Prism Explorer: "=== Total Memory = 4336668";
Prism Explorer: "=== Total Memory = 4756884";
Prism Explorer: "=== Total Memory = 5177356";
Prism Explorer: "=== Total Memory = 5782900";
Prism Explorer: "=== Total Memory = 6202860";
Prism Explorer: "=== Total Memory = 6616220";
// Cleared the EntityManager
Prism Explorer: "=== Total Memory = 3655320";
Prism Explorer: "=== Total Memory = 4076292";
Prism Explorer: "=== Total Memory = 4498300";
Prism Explorer: "=== Total Memory = 4918260";
Prism Explorer: "=== Total Memory = 5358912";
// Cleared the EntityManager
Prism Explorer: "=== Total Memory = 3655384";

Note that DevForce isn't going anywhere. You are issuing the same query repeatedly and DevForce satisfies it from the EntityManager cache. However, the PE ViewModel is building a new OC<Customer> each time and reassigning QueryResults. The DataGrid is rebinding the ItemsSource to the new OC<Customer> each time ... and chewing up memory each time.

If we comment out "_queryResult = value", DevForce and PE will continue to do their things ... DevForce may query the database or satisfy from cache if it can. PE will always build a new OC<T>, assign QueryResults with that OC<T>, raise PropertyChanged, stimulate the DataGrid to rebind to the QueryResults. Exactly the same code paths.

The only programmatic difference is that QueryResults always returns the exact same (empty) object.

The memory consumption goes flat ... as it should.

Of course it climbs modestly when you issue a query that fetches new data; that is to be expected. Press "Clear" to clear the EntityManager cache and memory is recovered.

This is proof enough for me that the memory leak is in Silverlight ... perhaps in the Silverlight DataGrid. Our entities are involved in some way ... that's why clearing the EntityManager cache allows the GC to recover memory. But the DataGrid is clearly chewing up memory at a fast clip without our adding any new entities to memory.

What could possibly take 300k? The entity data involved are miniscule (it's Northwind! 7 Employees ~90 customers). All entities combined do not weigh 300K.

The memory consumption grows by 300K when there are no new entities. Evidently Silverlight or OC is in a deadly embrace with the entities that prevents garbage collection (there are events involved). But even if so ... the numbers are staggering. We aren't adding 300K of new data each time we click the button. You can't write enough event handlers to consume 300K. The cost of an OC isn't 300K.

In my research, I tripped over an old WPF blog post (can't find it now) that said something about WPF creating completely new data and control templates for each OC. Now THAT is a fast way to burn memory. I'm betting that Silverlight is creating and caching a complete visual representation of the DataGrid and the associated entities each time I reset the ItemsSource with a new OC<T>!

I tried a number of things to shake up the binding ... anything to make it let go of whatever it was holding:

• clear the _queryResults ( _queryResults.Clear() ) before re-assigning it with the new OC



• clear the _queryResults and set it null before re-assigning it with the new OC



• clear the _queryResults, set it null, raise PropertyChanged ... then assign with new OC

None of these attempts worked. Only pressing "Clear" worked.

What to do? The obvious answer is reuse the OC ... and that is the lesson for your application: Don't reassign the ItemsSource; repopulate it!

Of course OC<T> requires strongly typed contents. I tried OC<object> with the intention of simply clearing and refilling it with new results after each query. That executes without error. But it doesn't show anything on screen either ... pretty useless :-).

It actually worked if I use List<T> instead of OC<T>. But then I wouldn't have the benefits of ICollectionPropertyChanged and I'd have to manage adds, removals, clears by hand. Yuck.

The WorkAround

As a last resort, on each pass I check to see if the type of the fresh results matches the element type of the OC. If it does, I can re-populate the OC. If it does not match, I have to replace the ItemsSource with the new OC ... and pay the 300K tax.

If you always query for the same type, you'll be fine. You can change the query itself - there are numerous Customer and Employee queries in PE - without paying the tax. It's changing the result type that incurs the tax.

I tried to be clever and keep a dictionary of previously used OC types. If I was querying customers, I'd pull out the OC<Customer>; if I was querying employees, I'd pull out the OC<Employee>. That didn't help. The DataGrid doesn't remember OCs that it has seen before. Re-using old OCs didn't change its penchant for devouring another 300K

So all I can do is slow it down. If PE starts to blow, you can "Clear" the EntityManager and it will recover.

Here's the revised code (which will appear in the next PE release, minus the GC & Log calls)

    public IList QueryResults {
      get { return _queryResults; }
      set {

        ResetQueryResults(value);

        Log("=== Total Memory = " + GC.GetTotalMemory(true));

        RaisePropertyChanged("QueryResults", "QueryResultsCount");
        SelectFirstQueryResultsItem();
      }
    }


    private void ResetQueryResults(IList results) {

      if (null != _queryResults) _queryResults.Clear();
      if (null == results  0 == results.Count) return;

      var resultsType = TypeHelper.GetItemType(results);

      if (_queryResultsElementType == resultsType) {
        // Same results collection type; can refill it
        foreach (var item in results) {
          _queryResults.Add(item);
        }
      } else {
        _queryResults = results;
        _queryResultsElementType = resultsType;
      }
    }

    private Type _queryResultsElementType;

Here are memory reports:

// Query for All Customers

Prism Explorer: "=== Total Memory = 3411896"
Prism Explorer: "=== Total Memory = 3425356"
Prism Explorer: "=== Total Memory = 3416064"
Prism Explorer: "=== Total Memory = 3416076"
Prism Explorer: "=== Total Memory = 3416076"
Prism Explorer: "=== Total Memory = 3429460"
Prism Explorer: "=== Total Memory = 3416076"
Prism Explorer: "=== Total Memory = 3415020"
// All Employees
Prism Explorer: "=== Total Memory = 4203344"
Prism Explorer: "=== Total Memory = 4212964"
Prism Explorer: "=== Total Memory = 4213000"
Prism Explorer: "=== Total Memory = 4213000"
Prism Explorer: "=== Total Memory = 4213000"
// Clear Entity Cache ... followed by All Employees
Prism Explorer: "=== Total Memory = 3706416"
Prism Explorer: "=== Total Memory = 3588660"
Prism Explorer: "=== Total Memory = 3593996"
// First Employee
Prism Explorer: "=== Total Memory = 3590928"
Prism Explorer: "=== Total Memory = 3604372"
Prism Explorer: "=== Total Memory = 3589676"
// Employees named Smith (there are none)
Prism Explorer: "=== Total Memory = 3605584"

// All Employees (again)

Prism Explorer: "=== Total Memory = 3757472"
Prism Explorer: "=== Total Memory = 3762120"
Prism Explorer: "=== Total Memory = 3770336"
Prism Explorer: "=== Total Memory = 3762144"