Monday, December 7, 2009

The Agony of EF


We’ve been working with EF v1 (.NET 3.5) for a little while now, using it sparingly when creating new code requiring CRUD functionality, and refactoring older code with similar characteristics. You may recall, from an earlier post, that our approach was to use EF to create a data-layer to replace the Datasets, Datatables and Datareaders we currently use with type-safe rich Entity Objects.  It was not and is not our intention to replace our existing business objects with EF.  To this end, we employed a repository pattern that serves up ‘detached’ EF objects which are then used to hydrate and persist our business objects.

All too easy

Initially, the results were very positive.  With some of the simpler cases we started with (mapping tables with few or no relationships) an immediate benefit was obvious.  We could replace the SELECT stored procedures with equivalent Linq-to-Entities statements, and remove entirely the INSERT, UPDATE and DELETE stored procedures in favor of EF persistence.  With simple use of the designer to import entities from the database into our model, and the implementation of some boilerplate repository code, we could eliminate at least four stored procedures per entity in our database.

A dream to some, a nightmare to others

Slowly, as we attempted to work with slightly more sophisticated cases, issues began to emerge; issues that are probably well documented across the web and perhaps most notably in the vote of no confidence.  The biggest issue we encountered was the general lack of tiering/layering support in EF.  Entity objects could be detached, but the consequences were painful. 

  • Relationships were not preserved.  Parent and child objects could not be traversed. Even worse, since the relationships were null entities rather than key columns, even the identifiers could not be accessed thus making it difficult to query and manually load related objects.  Lazy-loading disconnected relations isn’t possible and doesn’t even make sense.
  • Re-attaching detached objects was not straightforward.  Any non-trivial situation resulted in object graphs where some objects where attached while others detached, a mixture EF couldn’t handle.

These two alone were enough to make me doubt EF’s usefulness beyond trivial usage (or building a one-tier application), and we hadn’t even gotten to performance, or model merging issues.

Making the simple simpler; the complex more complicated

Nevertheless, I was determined to understand the limitations fully before discarding EF v1 (abandoning O/RM altogether, waiting for the promise of EF4, or switching O/RMs).  While our EF Repository pattern wasn’t mainstream usage, the issues we were facing with our data-layer are the same issues what would be faced by anyone using EF across layers and tiers.  The most likely parallel would be anyone attempting to use EF with WCF to create services.  In this camp, we were not alone, there are a considerable number posts lamenting the difficulties and a fair number of proposed solutions and attempted workarounds; some clever, some not so.

After investigating dozens of these solutions, implementing one other candidate with only partial success, I eventually came across Reattaching Entity Graphs with the Entity Framework.  This solution uses serialization to ‘detach’ the object graphs thereby preserving the relationships.   It also contains some very clever logic for reattaching the object graph while at the same time including a method for instructing the attachment process on how far to traverse down the various graph pathways when performing persistence.

After about a day or two I was able to integrate this solution into our code and solve our detachment and relationship issues.  Now I can look forward to facing the performance issues (this solution does require eager-loading the related entities using Include), the source control branch merging issues inevitable with one large edmx file, and the many others lying in wait for me.

A game of trade-offs

This experience reinforces one of the lingering doubts I have about O/RMs, and other tools, frameworks and patterns in general.  It often feels like a zero-sum gain, where we’re just moving the complexity around rather than reducing it.  Instead of struggling with writing stored procedures or dynamic sql and mapping the datasets and readers to object properties and back again (which I never found to be that onerous to begin with), we’re struggling with mapping objects to tables via Xml and persistence idiosyncrasies.  It doesn’t feel like an easier or better way necessarily, just a different way and different set of challenges. Maybe that’s just familiarity with the old method, and lack of familiarly with a new method, but maybe its not.

Thursday, December 3, 2009

Dealing with Design Debt Part 2½: The Smell of Fear

Its probably time for an update on the dealing with design debt series.  I promised part III would be the conclusion and it will be, but we’re not quite there yet, nevertheless there’s some story to tell now that will bridge the gap. 

Along with the business case made for the debt reduction project, there were also other business decisions that that had direct impact or were directly impacted by the project.  Most significant were the several projects on the schedule with direct overlap with key structures under construction as part of debt reduction.  To schedule these projects first would mean doing them twice, but not doing them first would mean delaying or missing revenue opportunities.

Measure twice, cut once.

From a purely technical perspective it made sense to complete the debt reduction project first, and tackle the dependent projects after.  This would allow us to design the solution for the new structure and implement once, rather than designing for the old structure then later retrofitting it for the new while also implementing twice.  However, the business case ran counter to the technical case.  These projects were estimated to have significant revenue impact, therefore delaying them until the completion of a long and risky debt reduction project was judged to represent a significant loss in revenue.

“It is a riddle, wrapped in a mystery, inside an enigma” - Churchill

Based on these projections, it was decided that the high revenue impact projects would be done prior to the debt reduction project, acknowledging the implied cost of implementing twice, and the ultimately less clean implementation.  Essentially, we had decided to take on debt during our debt reduction exercise.

Unfortunately, its not entirely clear whether that was the right decision or not.  The high revenue impact projects were completed but took longer than expected, pushed back the debt reduction project and made it a bit more complex.  Further, the actual revenue impact of those projects isn’t entirely clear nor is the impact of the additional debt and complexity particularly measurable. 

And there upon the rainbow is the answer to our neverending story

I guess the moral to this story is that, at least in this case, debt accumulation was an explicit rather than implicit trade-off decision.

Technorati Tags: ,