MVVM Party: Data Service Required

Aren’t we all tired of the consultant weasel-words “it depends”? Every once in awhile it is possible to speak in unequivocal and unambiguous language. This is one of those moments.

A ViewModel should never reference the persistence machinery directly; it should always delegate the getting and saving of Model objects to a helper such as a DataService (aka Repository, aka ServiceAgent). 

Are there exceptions? I can’t think of a single one. I wouldn’t break this rule in an MVVM demo.

Now if I’m not using the MVVM pattern, fine. I just mandated a rule for ViewModel, not a rule for all UI architectures.

This really is one of the laws of ViewModel construction, taking a privileged place besides a very few other such laws as “The ViewModel may not have a reference to the View” and “The ViewModel may not refer to a type that requires a running view to construct.”

If you break these laws, you are not using MVVM. You’re doing something else. And I’m pretty sure you’re going to suffer for it.

You won’t suffer because the MVVM Police will get you. You’ll suffer because you added MVVM complexity to your application for nothing. You paid the price for a UI separation pattern … and destroyed that separation. You now have an application that is harder to read and maintain than if you’d put everything in code behind.

Who Put The Hot Sauce in Grandpa’s Oatmeal?

I’ll give you backstory and then I’ll repeat here what I said on the WCF RIA Services forum.

I frequently troll the WCF RIA Services forum to find out what RIA developers are thinking and <grin> to learn of the many ways in which IdeaBlade’s DevForce is superior.</grin>

I’m especially fond of Colin Blair’s contributions. Colin knows more about RIA Services than almost anyone; sometimes I think he knows more than the product team. But I especially like Colin’s posts because he always goes beyond the “how to”. He’s deeply interested in understanding what you are trying to do and in helping you move in a direction that gets you where you want to be. And I probably like his stuff because I agree with it. Except this time.

This time he gave bad advice.

A developer identified as “lein4d” posted the following question (amended for grammar and style, highlights added)

When using MVVM Pattern with WCF RIA Services, is it better to encapsulate the client side RIA code in service classes than to have it directly inside the ViewModel?

In many of the recent MVVM examples I've seen, the classes are separated into ViewModel, IServiceAgent, and ServiceAgent classes, in which only the ServiceAgent Class knows about RIA details.

I've been following this approach for a while, but I found it too cumbersome to create all the boilerplate code each time I create another ViewModel.[???] The application I'm creating doesn't have unit tests, and is not making use of design-time data.[!!!]

In this case, is it still necessary to abstract away the RIA service detail? I think it'll be more productive and write less duplicate code if I have DomainContext class directly inside ViewModel and load entities from there.

What is the rationale behind having IServiceAgent and ServiceAgent classes? As for sharing queries between ViewModels, isn't it sufficient to put shared queries in the server side?

This is a great question for which there is a definitive answer. Unfortunately, Colin gave him the wrong answer. He told lein4d that it is ok to call the DomainContext directly in the ViewModel. Kyle McClellan of the RIA Services team added “Feel free to use the DomainContext directly in your ViewModel.”

It is never ok to call the DomainContext (or the DevForce EntityManager) within a ViewModel. You should not be “feeling free” when you do that.

Although I am shy and generally unwilling to express myself forcefully, I found myself writing this spirited reply.

Why ViewModel Should Delegate Persistence To A Service

@Lein4d, why are you using ViewModel at all? Why not put everything in the code behind? It's a lot easier and there are fewer classes to worry about. The primary benefits of ViewModel are single responsibility, design/development independence, and testability. You don't seem to be interested in any of them … in which case the MVVM pattern isn't pulling its weight.

I don't have a lot of hard and fast rules. I do have a few. This is one of them.

"No persistence machinery in the ViewModel"

I'm going to try to persuade you of this even if you don't seem to care if your ViewModels are readable (sigh), don't test your ViewModels (sigh), and don't design with data (sigh).

I'm going to make my case solely on the Single Responsibility Principle and its implications for consistency and maintainability. If these characteristics don't matter to you ... stop reading ... and stop bothering with MVVM or any other pattern.

I ask all of my customers to write ViewModels that concentrate on providing state (data) and behavior (logic) that their views need. Complex tasks that are not strictly concerned with the view (e.g., how to query for entities or save them) should be delegated to supporting "services". A focused ViewModel is easier to understand and easier to test [oops ... I mentioned those points again]. It is also less vulnerable to changes that have nothing to do with the View.

The DomainContext itself is already a step along this road. Without it, in the extreme case, your ViewModel would be making its own connections to the server, issuing web requests on a background thread, waiting for the response, interpreting the response, mapping received data into entities and marshalling them back to the UI thread.  You wouldn't dream of writing that code in each of your Views or ViewModels. The DC abstracts that for you.

But it's not a sufficient abstraction for the ViewModel. There are still too many mechanics. Let's get concrete.

Suppose the ViewModel provides Customers to the View. That's really all the VM cares about ... some notion of a list of Customers. It wants to have a method like "GetCustomers". Why should it care about how that method is implemented?

Should that method return all customers or only active customers? Only customers that the current user is allowed to see? Can it be customers in cache or should it be customers fresh from the database? Should the query be for customers only or should the query include related entities as well? Should the customers be sorted? Is there an upper bound on the number of Customers to retrieve for this view? Should the query use an existing DomainContext or create a new one? What if the user isn't authenticated? Or not authorized? What if the DC can't connect to the server? What if the server returns an error?

These are all important considerations. None of them have anything to do with getting customers onto the screen ... which is the VM's job.

Will you worry about all of these considerations on day one? Probably not. But when they do become important to you, would you expect to edit all of your ViewModels to address them? Or would you expect to edit a Data Service to address them?

For me the answer turns on this: would changing the way I get the customers change the appearance of the view or how the user interacts with the view? If the answer is "no", then the code belongs in a Data Service, not the ViewModel.

Let's generalize that thought. You should be able to answer the following question for all code in your ViewModel:

Would changing this code alter the appearance of the view or how the user interacts with the view? If "yes", the code can stay. If "no", the code belongs elsewhere, perhaps in a helper class to which the VM delegates.

This is not a hypothetical issue or an academic discussion. You asked for advice rooted in experience. Presumably if you were certain, you wouldn't ask. So listen up to someone who has been working in this corner of .NET since 2003.

I've looked at a lot of customer code. That's part of my job description: work with customers who are building smart client / RIA applications. I’m kind of a code doctor. People call me when their code is sick.

Obviously I wasn't always looking at Silverlight code; usually it was Windows Forms where the corresponding collaborator is the "Presenter" in an MVP pattern. Every time I saw persistence code in the VM or Presenter, I saw trouble. Every one of those VMs or Presenters implemented their queries differently. Quite often the same conceptual query would appear in multiple guises. I'd see three or four variations on GetCustomers, all of them unintentionally different.

Why those differences?  Maybe the developers understood the business rules differently. Maybe the rules changed as the application evolved. Maybe some of the developers (or the same developer at different times) didn't understand how the persistence machinery works when they wrote the code. Maybe awareness of potential errors was acute on some days and not others.

Whatever the cause, it's a mess ... a mess spread across your code base. Get that stuff out of there. Put it in a Data Service where you can find it, watch it, and tune it.

P.S.: If you ever decide to test your ViewModels and/or test the many ways that your application accesses data, you'll be glad you made this move.

A final detail. Your question seemed to imply that you were creating a new "ServiceAgent" and interface for each ViewModel. You wrote "I found it too cumbersome to create all these boilerplate codes each time I create another ViewModel."

I would find that too cumbersome as well. I tend to write one "ServiceAgent" (DataService, Repository, whatever) per module. That's not a strict rule. But I do find that I use the same service across multiple VMs.  John Papa's tiny BookClub example has a single "BookService" that does the trick for both the BookViewModel and the CheckoutViewModel. You really don't need a lot of these services in your application.

Colin Replies

After I posted, Colin and I had a spirited exchange which you can read on the forum. I won’t repeat it all here. But I will summarize his objections and my counter arguments.

#1 “The primary purpose of the ViewModel is data binding”.

No it isn’t. The powerful data binding in the XAML platforms made MVVM feasible. Just because we can do it, doesn’t mean we should. I favor MVVM because it encourages three good things: Single Responsibility, Designer / Developer independence, and automated testing.

#2 [Paraphrasing] You don’t need a data service for a “Hello, World” application.

True enough. You don’t need MVVM either. More to the point, you probably shouldn’t be using MVVM in a “Hello, World” application.

#3 [Partial quotation? Misquotation? My interpretation] “I’m uncomfortable telling someone that they are a bad programmer because [they don’t do what I think they should do]”.

I didn’t call anyone a bad programmer. Lein4d asked if he should use a Data Service in his real world MVVM-base application. I didn’t say “maybe”. I said emphatically “yes”. We should be comfortable in our convictions and uncomfortable when we waffle.

I try to come clean when I’m unsure. Often the answer depends upon circumstances. Often I have opinions and experience but nothing I’d want to mandate and plenty of counter examples.

On this particular and rare occasion, I harbor no such doubts.

Conclusion

If you are going to adopt the MVVM pattern, realize that it adds complexity for a purpose: to improve your application’s intelligibility, maintainability, testability, and designability (ugh). Don’t pay the complexity cost and throw the benefits in the trash. Please delegate all persistence concerns to a helper service.

You’ll be glad you did.

“Sandbox” Editors with ClientUI Contacts

We’re developing a full-stack WPF/Silverlight development series in concert with Intersoft Solutions using DevForce and Intersoft’s “ClientUI” technology. You can learn about the series and keep up with the latest developments on our website.

Today I’m announcing the second installment of the series which replaces the “synchronous” editor with a “sandboxed” editor.

With the initial “synchronous” editor, users changes entered in the child window editor are visible immediately in the main view, the view that lists the contacts, as we see here:

That immediacy is a good experience in some screen layouts … say when the detail editor is embedded in the same screen as the list view. But many people feel that a child window editor implies a temporary workspace. Unsaved changes should be confined to the editor; changes should propagate to the main view only after the user saves them.

For example, the unfinished editing session in which we changed “Andre” to “Bob” should look like this:

In other words, the editing experience should take place “in a sandbox”, hence the name “sandboxed editor”. In the second sample in the DevForce/ClientUI series, we make some small changes to turn the “synchronous” editor into a “sandbox” editor.

In a post to follow very soon, I’ll describe the issues in detail and the code to get it done.I’ll be excerpting from the walkthrough document that accompanies the second code sample.

While that walkthrough is specific to the ClientUI sample, all of the principles and many of the techniques are applicable no matter how you build your DevForce application.

Go ahead … jump ahead of me and read that document. Or you can wait for me to re-cast it here in my blog in a few days. Either way, you can’t lose.

Happy holidays!

Update: Checkout the related post in which I answer an email about nested sandbox editors

Design with Database Data in DevForce

Recently I’ve migrated John Papa’s BookShelf (aka BookClub) example to DevForce. Download the DevForce version (BookShelfDF) from our web site. You can learn about John’s original version and get his code from the PDC 2010 web page.

The DevForce BookShelfDF includes a document describing the steps for migrating from RIA Services to DevForce and proceeds to describe the improvements that DevForce made possible. Among them is the ability to create design-time entities for display while developing your Views in visual design tools such as Blend and the Visual Studio designer (“Cider”). This post is a reprint of that section of the document.

Warning: this is a long-ish post.

ClientUI Sample: MVVM for WPF & Silverlight

We just published a new sample application in partnership with Intersoft Solutions. Here’s what excites me about it:

• Two solutions, in WPF and Silverlight, share the same source code and XAML
• Demonstrates MVVM and Repository patterns
• The views are Blendable and rely on design data delivered by ViewModels
• It looks great
• It’s concise and easy to read
• Includes a fifty page walkthrough that explains how it is put together and why
• A terrific mash-up of DevForce and some great Intersoft UI controls

Learn more about our evolving series of joint samples.

Download the code

See what Jimmy Petrus of Intersoft says about it

The Intersoft “ClientUI Sample” was the inspiration. It’s a lovely demo of a Forms-Over-Data application – a “Contact Editor” built with Intersoft controls. The UI presents two views: one lists all Contacts; the second is an overlay dialog for editing the currently selected contact and address. A “contact” is a person with an address.

The application appears and behaves the same in both WPF and Silverlight flavors. Intersoft’s APIs are identical for both WPF and Silverlight. They built both versions of the sample using the same code and xaml files, no changes required. They simply compiled separately against the WPF and Silverlight libraries. We thought that was pretty neat (and no mean feat).

Our two companies teamed-up to extend Intersoft’s “UI-only” sample with a DevForce entity model and our distributed persistence components so the editor could retrieve and save data from a database.

The DevForce APIs are the same for all .NET client platforms (ASP.NET, Windows Forms, WPF, Silverlight). As with Intersoft products, you write it once compile for the target libraries.

After integration with DevForce, the ClientUI sample became an end-to-end Contact editor that can save … delivered in WPF and Silverlight, built on a single code base.

Intersoft’s original ClientUI sample followed an MVVM design. That made our DevForce integration job easy. We hardly touched the views. We replaced the original, hand-coded model with a DevForce generated model that can exchange data with a database over the internet. We introduced a Repository and EntityManagerFactory, massaged the ViewModels, added a dash of configuration … and voila!

The conversion process is covered in detail - with digressions on patterns used and design choices - in a fifty page PDF that is part of the downloaded zip file.

We have our ideas for future development. We’ll surely demonstrate how to “sandbox” the editing dialog so that changes propagate to the main view only after they are saved. I’d like to add automated testing and maybe we’ll dare to use an IoC container. I’m sure you’ll tell us what you’d like to see.

Enjoy!

Prerequisites

The sample assumes you’ve installed IdeaBlade’s DevForce (6.0.7 or later, free version is fine) and Intersoft’s “ClientUI 2010” (v3.0.5000.11 or later, trial version is fine).

Silverlight Firestarter: See what’s coming

Want to know what Silverlight is all about and what’s coming in Silverlight vNext? Reserve a few hours on Thursday 12/2 for a free seminar from the folks that know: the Microsoft Silverlight team and friends.

You can catch it all online or be there in person and meet some wonderful people.

What part of “Free” do you not understand? Go register: http://jpapa.me/slfs10

P.S.: I’ll be there in person. Look me up.

Silverlight’s Bright Future

“Our strategy has shifted” may be the most unfortunate phrase ever uttered by a Microsoft exec. Bob Muglia, President of the Server and Tools Division, said it in his interview with Mary Jo Foley. Pundits and Silverlight detractors have been partying on it for more than a week.

It should just be good theater. Unfortunately, it has spooked the herd. More than a few of our customers and prospects are wondering if Silverlight is “dead”.

Some Silverlight defenders counter by reminding us that HTML 5 is not ready and runs on less than 1% of installed browsers.

But the merits and demerits of HTML 5 are beside the point. If you’re considering building in Silverlight, you have to know that Silverlight itself has a future. You don’t care about Silverlight 5. You want to know there will be a Silverlight 10. You want to believe that Microsoft sees Silverlight as a strategic, mainstream web development platform.

I believe Microsoft sees Silverlight as a strategic, mainstream web development platform … for many years to come.

Don’t take it on my authority; I don’t have any authority. Believe Bob Muglia when he sets the record straight. If I may paraphrase, Bob said Microsoft is investing heavily in HTML 5 because that is the standards-based technology with the potential for the broadest possible reach. Investment in HTML 5 does not mean abandoning Silverlight. It’s not a retreat from Silverlight. Far from it. Silverlight thrives … on the web, on the desktop, and on the phone.

Bob simply acknowledged that Microsoft will vigorously pursue multiple web technologies to meet diverse market needs.

Where is IdeaBlade going?

It’s IdeaBlade’s mission to help business application developers reach their users wherever they are, using whatever technology they want to use. Our DevForce product drives Console applications, “headless” services, Windows Forms, WPF, Silverlight, and HTML applications written with Web Forms and MVC.  Next month we’ll release our support for an OData API that enables any client … including any phone … to access your DevForce business model.  When there is a way to write applications in HTML 5, we’ll be there too.

We are always balancing our portfolio of client technologies. But to be clear, we believe Silverlight is now – and will remain for the foreseeable future – the best way to build and deliver business applications to the web. Bob Muglia said it too: “Silverlight provides the richest way to build Web-delivered client apps”, especially “enterprise application[s] … both inside and outside the browser.”

No HTML technology today can match Silverlight in capability or productivity for the web application developer. A comparable HTML 5 platform by all accounts is years away.

What’s all the Fuss?

The story at last year’s PDC was “three screens and a cloud” glued together by Silverlight. Barely a year later they’re ballyhooing HTML 5 as the glue. A third of the keynote goes to IE 9 and HTML 5; Silverlight is hardly mentioned. Then Bob says “strategic shift” and the stuff hits the fan.

What we have here is a failure to communicate. Microsoft evangelizes the shiny new balls but seems less adept at the sustained marketing necessary to keep existing brands burning bright – xBox excepted. The Silverlight team is going great guns on Silverlight vNext development back in Redmond but you would not know that at PDC and there is still no public target for a putative Silverlight 5 release. Other consumer-facing corporations invest in their brands, especially successful brands. Even soap is regularly “new and improved.” Microsoft can’t count on product development to keep the brand alive. They should be as relentless on the marketing front.

They tried mightily to recover the Silverlight momentum. Muglia apologized (yes – he said “apologize”) for his part in launching the controversy. Read his post carefully; he was adamant that Silverlight is a strategic technology with a long future. When you’re done reading Bob, read Scott Guthrie’s post which reiterates and elaborates on Bob’s themes; Scott is Vice President of the Developer Division and he ought to know.

Of course they also have to promote HTML 5 in a big way and Silverlight must pay a price for its previously exaggerated ambitions. The truth is that Silverlight will have less reach than we once hoped. It will still be cross-platform – running in many browsers and on both Mac and Windows. But the grand vision of Silverlight on every device on the planet cannot be realized. That was always a tall order and, over the last year, it became obviously unachievable. Apple blocked them successfully on iPhone and iPad … and will continue to do so as long as their market share permits.

Microsoft must have a future on these devices; they are too important to ignore. And Microsoft also must provide an alternative to Silverlight in other venues where Silverlight is shut out. We all know that some customers prohibit browser plug-ins … which precludes both Silverlight and Flash.

Silverlight was never the only game in town. You’ve heard about MVC, yes? MVC is an HTML technology and will likely be the backbone of HTML 5 development. Scott Guthrie is as avid a promoter and staunch ally of MVC as he has been of Silverlight. His blog posts for the last few years give each technology equal love.

MVC did not diminish Scott’s enthusiasm for Silverlight … and neither does the prospect of HTML 5.

Silverlight is not just the Phone

Reading some of the press, you might fear that Silverlight had been relegated to a phone development platform. They talked a ton about “Silverlight + Phone” at PDC. Silverlight is indeed the preferred platform for the Windows Phone 7; other than XNA it is the only development platform for WP7. I suspect Muglia was so phone-conscious that, in the interview moment, he just didn’t think of what else we use Silverlight for … hence his unfortunate “other sweet spots” comment.

Give him a break. WP7 is the hot new thing. According to Steve Ballmer, it’s Microsoft’s next billion dollar product. It’s a big deal and a key theme at PDC. Talking up Silverlight on the phone is called “staying on message”.

But Silverlight is also the premier client technology in many other spaces. For example, it’s the best way to build applications on top of SharePoint and that process got a lot easier with SharePoint 2010. We saw Silverlight front-ends on Azure. We are seeing Silverlight front-ends on Bing.

From our perspective, the most important fact: Silverlight remains the recommended client platform for distributed business applications, both inside and outside the browser. Bob Muglia said it. Scott Guthrie reiterated it.

The FUD Factor

Do you think it is curious that some in the technical press and the Silverlight detractors so eagerly cling to Muglia’s one inapt phrase? Why do they misrepresent and dismiss his clarification? Do these people really know something? Do they have secret inside Microsoft skinny? Do they mention unnamed sources? A “deep throat” of some kind?

No. All they do is read the same tea leaves everyone else is stirring: Muglia’s interview, the IE9 PDC publicity parade, the lack of a big Silverlight announcement … and throw in their own wishful thinking (egregious example here).

I have no inside scoop either. I’m reading what you’re reading.

I do have real-world experience developing with Silverlight (question: can you name a “Silverlight is Dead” article by anyone who has written a Silverlight application?)

I have my Microsoft contacts and I spoke to as many of them as I could reach. Every one of them … without exception and without equivocation … believes passionately that Microsoft is committed to Silverlight. They are dead certain that the Silverlight franchise is growing. They believe that HTML 5 is a big bet too … but that it does not dim Silverlight’s star. And no one expected a “strategic shift” announcement.

I worked at GE for twelve years. I know what a “strategic shift” looks like. We would never learn of a “strategic shift” by reading a Friday afternoon interview in a trade rag.  GE doesn’t work that way. GE would prepare us for a bombshell like that. The board would know. The senior executives would know. The press kit would be prepared. Company meetings would be scheduled at every level.

Microsoft is no GE … but it isn’t stupid either. There was no preparation. Muglia’s remarks were a bolt from the blue – as he said himself. There wasn’t supposed to be an announcement. All we have here is an interview by an executive trying to stay on message: Azure, WP7, IE9+HTML5.

If you’ve ever worked for a business or run a business, you know how this happens: a casual remark to a journalist gets wildly misunderstood and misrepresented. You don’t make a critical decision on this basis. You exercise prudent judgment and listen when company executives correct the record.

Well … what of HTML 5?

Now I must speak for myself, not IdeaBlade.

HTML is the way to get the broadest possible reach. HTML 4 is perceived to lack the pizzazz for the new breed of rich internet applications so everyone is pinning hopes on HTML 5. There are some problems with that:

1. Near zero reach for the next few years. A tiny fraction of browsers installed today support HTML 5. It takes a long time to replace the installed base of browsers (proof: even Microsoft can’t seem to kill off IE6). Plug-ins (like Silverlight and Flash) have been adopted much more readily than new browsers.

2. HTML 5 is not a standard yet. The WC3 doesn’t think it’s ready; see here and here. The browser wars of the 1990s are coming back and it’s going to be hell for the developers trying to write applications that actually work on all the browsers on all the devices. One wag described HTML 5 as “Write once, test everywhere!”

3. The developer story is missing. Applications don’t build themselves. You need libraries and tools to design, develop, debug, and test. None of this infrastructure exists for HTML 5. You need experience and best practices to build an application properly. No one has experience building HTML 5 applications. The essential foundation of human and physical capital is years away.

4. HTML 5 sets the lowest common denominator for user experience. The Silverlight team wrote convincingly about this in a recent post. The comparatively low HTML standard and limited capabilities may be good enough for some applications; will it be good enough for yours? Or will it prevent your RIA developers from delivering the rich experience and features that your application requires and that your market demands?

Ask yourself why your application is not on the web today? If HTML 4 were a viable option, why aren’t you there now? Is it because you can’t affordably deliver the rich responsive experience of your existing application? What is it about HTML 5 that will change that picture? What in HTML 5 will make your application better than it would be in HTML 4?

Silverlight remains more productive, more testable, more maintainable than any HTML-targeted approach. I can think of other excellent reasons to build an HTML application. But if the balance of considerations favored Silverlight last week, I can’t imagine why the distant prospect of HTML 5 should change the equation today … as long as you believe Silverlight is going to continue to grow and improve … as I clearly do.

DevForce and RIA Services Compared Part 1: Getting Started

Sorry folks. I've temporarily pulled my series of articles comparing WCF RIA Services with IdeaBlade's DevForce while I reconsider the approach. It will be back in another form.