Postcard from Alt.Net Seattle '09

I often intend to send postcards while traveling ... and as often find I've sent none. In panic and guilt, I write them from home.

Here it is, one week later, and I'm finally ready to say ... something.

First, it was a richly rewarding experience for me. I'm still bouncing on a turbulent sea of thoughts and ideas, still cringing at my errors, so it must have been great.

It is true, as some have said, that the sessions were less interactive than they could have been. Was it because we were mostly seated in class rooms that reduce all relationships to "teacher/student"? Were there too many people per session? Would more facilitation and examples of conversational styles have helped? Could we have insisted on fishbowls? There are clues all around for future conference conveners to consider next time.

Nonetheless, I want to salute Glenn Block and his team for creating and coordinating a wonderful event and making it so easy to attend and engage.

My second observation is that I was fortunate to be in the company of a startling number of charming, thoughtful, and passionate people. I hesitate to name any of them for fear of omitting someone ... someone who influenced me profoundly in one tidbit of conversation. As I write I am awash in memory of faces and names, all of whom were generous with their time and opinions.

I missed meeting many in the Alt.Net community who could not attend ... especially those who joined remotely on Twitter.

And now, a few notes on the sessions.

Ward Cunningham's Swimmer

The swimmer plays test scripts that are rendered visually in a dynamically generated activity diagram. You can see each actor's responsibility, situated in the actor's swim lane, played out vertically in time. Drill downs afford the close-up detail when you want it. But the diagram offers the high level view of the story that is otherwise difficult to grasp.

It was a hugely compelling demonstration of the confluence of testing and visualization. We all know the benefits of test automation. Here Ward introduces the human touch ... drawing upon our capacity for recognizing patterns (and trouble) so that the tests don't just pass ... they're not just green; they can come to life and reveal whether we're getting the user experience we truly want.

Another cool thing ... it looked like it would be easy to write and extend (a Ward Cunningham trademark). One person next to me whispered, "I think we could do that for Silverlight in a week."  I hope he does.

Is Persistence Ignorance (PI) Really Necessary?

I hosted and launched this deliberately provocative session. I've been speaking favorably of Entity Framework in a number of friendly venues and word has gotten back to me that the contrary view was unfairly represented. Worse, it has seemed to many that I've been taking swipes at NHibernate ... and grossly misrepresenting it.

Well I figured it was time to walk into the lion's den.

What did I learn? First, that Alt.Net folks - passionate as they are - are fully capable of civil discourse. Someone, detesting my perspective and lusting for battle, twittered "take off the gloves!" Frankly, I feel the punches were much more effective with the gloves on ... and I appreciate that I was permitted to leave the room with dignity intact.

On the merits, a couple of observations. Most importantly, I realize now that the way I've been talking recently veers much too closely toward questions such as "is NHibernate good or bad?" and "Is Entity Framework good or bad?". This can only harden positions in ways that are not helpful.

It didn't take long for me to discover that my knowledge of NHibernate as it is actually used is woefully inadequate. This severely undermined my central thesis which is that there are high costs to implementing a PI approach - costs not borne by reliance on a framework base class - and that such costs should be accounted for when evaluating the road you want to take. Had I known NHiberate (and its companion technologies!) better I would not have made certain errors (sorry about that) and would have been more effective in overcoming some of the counter arguments (which often took the form of "in NHibernate it just works").

I hasten to repeat: knowing NHibernate better would not have meant I was better equiped to tear it down. I was not out to attack NHibernate. No, knowing it better ... and having more hands on experience ... would have enabled me to be more clear and accurate about choices and trade-offs.

If I was pleased with anything I did ... and, believe me, I have rarely been so unsettled and uncomfortable ... I think it was in drawing a few distinctions.

I'm comfortable with my description of the essence of Persistence Ignorance itself. I didn't want to just regurgitate Jimmy Nilsson's inventory of PI "don'ts". Here's what I said:

• The machinery that makes instances of a type persistable is nowhere visible in the code for that type.
• You can construct a persistable type without conforming to the demands of any particular framework.

To my knowledge, at this time, there is no way to achieve this degree of PI; some approaches come much closer than others.

I'm also comfortable with my inventory of benefits from PI:

• Minimal friction in authoring and modifying the type
• Testability (no need to touch the database during unit test)
• Clarity (you can easily see what the type does)
• Opt-in to cross-cutting concerns (e.g., persistence, notification, property validation, security, localization, transaction are handled ... if at all ... by external means that you choose to apply)
• Design-first

I remain convinced that we can obtain these benefits with an approach that is less PI (e.g., leans on a framework base class) ... although it is harder to do so and most non-PI technologies don't try very hard.

The remainder of my arguments in that session fell apart. They did not stand up well under the criticism from those in the room and on twitter who know this space well. The fault is mine.

Let me state for the record that, all things being equal, I would prefer a PI (or, better, an Infrastructure Ignorant) approach. The challenge ... and it is one being addressed on all sides ... is that, today, all things are not equal.

Finally (for now), I'm coming to realize that what I'm really concerned about is how easy it is to develop your application within the constraints imposed by the application requirements (present and future), your knowledge of the tools, and the skills of the people on the project. Ian Cooper and Jeremy Miller have been helping me reframe the issues this way ... whether or not they know it or intend for me to evolve in this direction. Since requirements, knowledge, ease, and human capital are shifting all the time, there is no permanent answer to be found here.

Thanks to all who participated. I appreciate the opportunity to "learn, share, grow".

Messages or Entities?

This session seemed to bring revelation to many. I'm sure it's old hat to some ... but everywhere I ran into people who had "aha" moments in this one.

Udi Dahan was the source of much inspiration. As I strive to render him fairly, he may wish to chide me gently, as he did one enthusiast : "You may agree with me ... but I don't think I agree with you." Touche!

I missed the intro but I think the question on the table was "should one send messages or entities over the wire?" After ten minutes, Udi piped up and declared that the question itself was mistaken at which point he took the reins of the session (with everyone's consent and appreciation to be sure).

The question is mistaken because it does not account for the application context. Let's assume that a consequence of an HR application task is that an employee's database record will be updated. This fact alone is not sufficient to determine if the vehicle for such update should be a messaging or a persistence operation. You have to know what the business needs are both with respect to rules and the circumstances of the users.

Let's refine the case: the user updates the employee's marital status. Do we know how to decide now? No we do not. If this is an "innocent" operation with no known implications, a straight database update is perfectly acceptable. Indeed, absent any other information, it could be preferable ... because it is simpler and clearer than engaging a lot of messaging machinery.

Does the answer change if there are applicable validation rules? Not necessarily. If the rules can be applied base entirely on information immediately accessible at the point of change or the point of update ... no problem. For example, if the Employee type carries a rule that verifies that this particular user is authorized to change the marital status ... the application of that rule is easily accommodated within the familiar direct-persistence approach.

But if a change to marital status might trigger downstream processes ... especially if such processes belong to a separate and different business context ... then the persistence approach can come unglued. Changing the marital status ... or attempting to change the marital status ... may have consequences for the payroll system and the benefits system, both of which were designed and implemented separately from the employee information maintenance (EIM) system. These other systems could watch the database for changes to marital status (this is the usual expedient). But when you start to do this too many times, you begin to realize that everything would be clearer and more maintainable if the EIM had sent a "Changing Marital Status" message.

Lesson #1: The early place to introduce messaging is at the boundary between "contexts".

That's where the value is most obvious and where you are least likely to disrupt existing infrastructure commitments. Persistence operations may be just fine within a context even as that approach breaks down at the edges of the context. In DDD terms I would think one could see messaging as the opportune point to introduce an "anti corruption layer" between bounded contexts.

Lesson #2: Use messaging when you must express intent

I'm supposed to know this already. I thought I did. But I'm so used to using persistence frameworks exclusively that it is easy to think unambiguous intent is always manifest when I ship entity changes toward the database.

Do I need to know why I am changing the marital status? We know it will change, but what was the cause? If we don't care, no problem. But what if we do care? What if we need to distinguish one cause of status change from another ... even though the end result is the same?

We could extend the Employee record to capture the cause ... that's the usual approach. But this is the start of potentially cancerous growth; who hasn't seen a table that, after several years, has expanded to more than a hundred, sparsely populated columns. A message approach affords the opportunity to reason about the change and to record additional information about the change in "the right place."

Lesson #3: Messaging could make concurrency management easier

Consider what happens when user 'A' changes marital status and user 'B' changes the employee's home address "at the same time." Assume, for simplicity, that the affected data are in the same table row. Classic concurrency problem.

Concurrency violations are easily detected; that is not the issue. Concurrency resolution is the issue. Does 'A' win - and the address changes are lost? Does 'B' win - and the marital status change is lost? Do you blend them? Do you tell users 'A' and 'B' what you did?

There is no way of knowing a priori. In many (perhaps most applications), it is relatively harmless to pick an answer blindly and just go with it. Last one wins might be ok. Last one loses but has a chance to reapply his changes and then win? That could work. Resolution rules examine both records and chose or even blend the changes automatically? You can do that with at least one persistence framework I know.

But messaging starts to look like a more robust and more explicit approach.

Lesson #4: All messaging should be "fire and forget"

I'm trying to grok the implications of that one. It seems to suggest that I cannot use messaging to ask for information ... at least not in a Request/Response, firewall-guarded world.

Clearly that cannot not be and so my ignorance is shining ... again. Time to go learn something.

Lesson #5: Don't invent reasons to use messaging

We just talked about the forces in an application that militate in favor of messaging ... or at least in favor of such an approach at the boundaries of the application.

What if none of those forces are apparent in your business application? They might arise. Just as it might be the case that you will have to support thousands of users in a highly transactional, highly available environment.

But if you've looked seriously at the application requirements and none of these circumstances seem plausible ... even a year or more down the road ... then don't go there.

My beef with the messaging-hawks is usually right here.

Some of it is attitudinal. The hawks sniff derisively as they acknowledge that a pure persistence approach is "ok for simple CRUD apps" ... as if these applications were by nature simple and somehow unworthy of attention by a seasoned architect. In my experience, this grossly underestimates the complexity of applications which don't happen to face the challenges that messaging is most suited to meet. If I am building in earthquake country, I've got plenty to do ... without guarding against floods and hurricanes too. There is a genuine risk of over-engineering the solution ... and failing thereby to meet my client's needs.

Finally (I guess) is the illusion that adopting messaging (or DDD or anything else) is going to inoculate you against the hardships that will befall you when you learn ... months from now ... what you wish you knew today. This is fantasy. Evans talks about this a lot; neither you nor your client really knows what this application is supposed to do. By building it you discover what it should become. You strive to apply architectures that seem to best fit your current understanding of the application requirements ... and beyond that, in all humility, you cannot go.

What was cool about Alt.Net ...

... was meeting so many wonderful people and stumbling around in unfamiliar territory. Openly professed ignorance is hard on the ego ... and great for the soul.

Thanks again, Glen and all, for inviting me. I hope I will be invited again.