Saturday, November 10, 2012

Scratching the Surface

I just received my pre-ordered Microsoft Surface RT the other day, and aside from the fact that it was 2 weeks late and those at the Microsoft Store were clueless about exactly what happened or apparently how shipping or tracking even works, my first impression was profound disappointment.

Why him? Why not me?

I like Windows 8, I’ve been running it on my PC and my wife’s laptop for a few months now.  I have a windows phone 7 which I also like.  I find the unified Metro experience across devices and the integration between them appealing.  I was hoping that the Surface would basically be an iPad that ran Windows 8 instead of iOS.  I wanted the form factor of the iPad2 with the familiarity and convenience of windows (things like having separate accounts for my kids, Zune, printing, etc.). 

He’s good, you’re not.

However, the Surface is not an iPad.  Most strikingly due to its odd dimensions.  Its taller and narrower than an iPad2 by a lot. Its also heavier and thicker.  When I tried to use it like my iPad its odd shape makes it clunky.  Its not really meant to be held portrait (feels like a giant phone), and holding it landscape isn’t very natural either (like holding a mini wide-screen TV).  Not only that, but Surface has its own unique power adapter (I was hoping for micro usb to like the WP7, kindle fire, droid and virtually every other device).  Now I have yet another set of cables to wrangle.  My Windows 8 iPad dreams were dashed immediately. 

I’m better than him!

But then later I pulled it back out and kicked out the kickstand and hooked up the keyboard to see if there was anything salvageable about this thing before I sent it back.  Slowly it began to dawn on me that Surface isn’t meant to be an iPad its meant to be a laptop/netbook.  With the kickstand up, and the keyboard out I was able to type fairly effectively on my lap on the couch.   It even gives decent mouse control.  Once I started using it like an inverted laptop instead of an iPad it began to grow on me.

you’re worse! Much much worse.

I think I can actually do work on this thing, and that’s probably the point. That plays to Microsoft’s strengths, productivity apps and BYOD.  It remains to be seen whether this bridges the gap for people between work and leisure device.  I’m not convinced but at least intrigued.

Monday, November 5, 2012

(The long overdue) RIF Notes #20

“I have offended God and mankind because my work didn’t reach the quality it should have” – Leonardo da Vinci

Friday, August 24, 2012

RIF #19

“Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely” – Agile manifesto

“People can look to me as a teacher, but I consider myself a student of hip-hop”

Friday, July 27, 2012

Optical disks are a form of torture.

CD’s, DVD’s, Blu-Ray’s and Wii games are an aggravating lot.  Every time I go to put in a movie for the kids, whether the DVD player in the minivan or the Blu-Ray player in the house I find myself aggravated before I’ve even begun. 

Hari kari

Playing a movie involves an obsessive compulsive like ritual.  First I must daintily remove it from its delicate casing, without touching it, like a game of Operation.  Then I crane my neck from side to side while tilting it in the light to catch a glimpse of any smudges or blemishes, which there always are.  To be followed by gentle polishing and re-inspecting.  When I’m satisfied that I’ve done what I can I put it in the player and then wait patiently for what seems like minutes, to be greeted by a series of frustrating menu’s and images that I’m maddeningly not allowed to bypass no matter how many times I’ve already suffered through them, until I’m finally able to select play.  Then I wait for the inevitable complaints from the kids, “Dad it’s skipping again”, “It’s stuck”, which happens on virtually every movie new, old, or rented.  Of course, when it does skip and is stuck I really only have two options; continuously press fast-forward like an elevator button, hoping to jump past the glitch (which if it works at all more often than not it sends me careening too far into the future).  Or, more than likely, I’ll have to pop the movie out and go back to craning and polishing, sit through the menu’s again, and then fast-forward to where I was (while driving).

Wii games are no less frustrating, with the special ability to lock up or crash mid-level, and there own brand of boot up time and mandatory commercials to go through.

Remind me again why they’re better?

Makes me wonder what was so bad about cassettes and VHS.  They didn’t force me to re-watch anything or block menu functions.  They picked up where they left off, and I think I only had a tape get chewed a handful of times, certainly not every one every time.  They weren’t particularly delicate either. I didn’t have to treat them like priceless jewels.

Mal content

There was a glimmer of hope for a while with Netflix’s streaming service. They had the infrastructure, on nearly every device imaginable, but then the content dried up.  Now I can use Netflix, Amazon Prime, Vimeo, Hulu, HBOGO, iTunes, etc. to hodgepodge some content but only one some devices some of the time. 

I can only hope this craning and polishing is some kind of Miyagi-esque Karate training, the purpose of which I can’t quite see yet.

Monday, July 9, 2012

Blinded by my fear of being blind

“the act of observing will influence the phenomenon being observed”

Last year we implemented dynatrace as our APM tool to monitor and troubleshoot performance issues with our .NET applications.  Dynatrace is an incredibly powerful tool and immediately provided deep and rich analytics on the performance characteristics of our applications.  We had a bit of a bumpy road with the initial implementation, on-going difficulty tuning it, and occasional unpleasant side-effects.  Overall, however, I feel like a blind man who’s been given impaired vision. 

Out of the emptiness

Prior to dynatrace, performance optimization involved an extremely painstaking process of Trace.Write/trace.axd, real-time analysis was virtually impossible, and debugging major performance problems required dump files, WinDbg and luck.  But now, at a glance, I can see top consuming methods, database calls, and a whole host of other metrics that help me see how the application is behaving in the real-world with the ability to go back and view that behavior even after fact.  The learning curve on the tool is no joke, and it’s a balancing act to make sure we’re capturing enough information to make the right judgments but not so much that we’re overwhelmed or saturated.   Despite all this rich data its not always obvious what its telling me.

Garbage time

Recently we had one such elusive issue.  We’ve had a few incidents where for brief periods of time (30 minutes or so) our ASP.NET application performance significantly degraded (response times were 2-10 times their averages).  Dynatrace captured these events, and by drilling into the data it appeared that the slow response times were due to garbage collection suspension time.   For a long time that’s as far as we got with it.  We know performance degrades and suspect excessive garbage collection.  But what would be causing that, and why is it fairly rare and seemingly random?

Put me in coach

After hitting a wall, we brought in a dynatrace professional services ‘coach’ to help us interpret the data and what the tool was telling us.  They were equally unable to pinpoint the root cause, so they helped us configure additional Trending and Leak Analysis features to capture proprietary memory dump files for deeper analysis.  We ran this for a few days but didn’t experience an episode in that timeframe so we had nothing to capture.

Yay we broke it

On the fourth day we released a new version of our application and bingo, excessive garbage collection and poor response times.  We captured a few dump files, contacted our coach for a follow-up and waited for the episode to resolve.  However, this time it didn’t resolve.  The GC kept churning, and the response times continued to be bad, really bad.  When we realized this one wouldn’t resolve on its own, we began recycling app. pools, restarting webservers, even rebooting. No effect.  We toggled off a few of our recent dynatrace settings.  No effect.  Performance was still awful.  Meanwhile, we were scouring our recent change set for anything that could be responsible for GC, taking process dumps, contacting Microsoft, looking at CLR profilers, tweaking anything we could think of.  But the problem persisted. 

Ah ha!

Microsoft finally got back to us with a dump file analysis that pointed to NCache.  No stranger to blaming NCache, I brought down a node, and disabled NCache.  Still no effect, dynatrace was still showing excessive GC suspension time across the board. Basically every method in our application was affected.  We could see all the victims of GC suspension time but we just couldn’t pinpoint the culprit. 

Oh brother

After we ruled out NCache, I decided to bring the node back on-line and re-enable NCache.  When the node came back on-line, QA and other IT staff began to report improved response times, even though dynatrace was continuing to show the same pitiful response times and excessive GC.  Then I noticed that after the restart the dynatrace agent had malfunction and instrumentation on that node had been disabled.  The node without dynatrace was performing normally.

A bitter pill

It turns out that the Trending and Leak Analysis setting was not only causing the issue, but my attempt to shut it off initially hadn’t succeeded because the setting didn’t behave as expected and required an app. pool recycle to take affect.  Its not yet clear why that setting took days to manifest, why it had the effect it did, nor why dynatrace itself, the dumps we sent to Microsoft nor the dynatrace proprietary dumps identified it.

Throughout the entire incident, I was relying heavily on dynatrace. Every recycle, reboot, tweak and trick I was carefully measuring with dynatrace.  Even when it was suggested that we disable dynatrace at one point, I resisted because I’d have to go back to relying on subjective opinions about response times (not entirely true), and thought the likelihood low compared to the probability of something in our new code being the culprit.  I thought it had to be our code, that’s the only thing that had changed. I was wrong.

Now I can blame you for everything

The silver lining is that now that I know dynatrace is capable of causing excessive GC, its now a suspect in the original intermittent GC issues that we were ironically using dynatrace to troubleshoot in the first place.

Wednesday, June 27, 2012

RIF Notes #18

“Ignorance more frequently begets confidence than does knowledge” –Charles Darwin

I got a letter from the government the other day, I opened and read it, it said they were suckers

Saturday, May 19, 2012

Azure unbranded

It seems like Microsoft is dropping the name Azure from its cloud offerings? I guess I need some help from marketers understanding why Microsoft gravitates towards names for products that are so generic and bland as to be unsearchable terms.  I liked the name Azure, it made it clear when discussing and comparing technologies to know what you were referring to. And more importantly when trying to search for information “Windows Azure” and “SQL Azure” are pretty specific.  Try to search for “Windows” or even “.NET”.  Now we can add to that list “Cloud Services”, “CDN” and “Service Bus”, yeah I’m sure that’ll narrow it right down.

There must be some branding reason for it but it eludes me.  I have my own terms that might work better.

Windows Azure Compute The
Windows Azure Platform Thing
Windows Azure CDN It
Windows Azure Storage That
Windows Azure Traffic Manager This
Windows Azure Virtual Network Some
AppFabric Cache More
AppFabric Service Bus Stuff
AppFabric Access Control They
SQL Azure Them
SQL Azure Reporting services Which

Friday, May 18, 2012

RIF Notes #17

“Simplicity is prerequisite for reliability” - Edsger W. Dijkstra

Do you feel like you're falling?
You've taken this step
In front of you, is further from the truth
You fall apart in front of me again

RIF Notes #17

“Simplicity is prerequisite for reliability” - Edsger W. Dijkstra

Do you feel like you're falling?
You've taken this step
In front of you, is further from the truth
You fall apart in front of me again

Monday, April 30, 2012

You can’t have your cookie and cache it too

I recently discovered a poorly documented (in fact I couldn’t find any documentation about it) side effect of writing server-side cookies.  They cause outputcaching not to work, and they do so in a nonobvious way. 

If your outputcache directive looks like this:

<%@ OutputCache Duration="100" VaryByParam="none" location="Any" %> 

But somewhere in your page you have code that looks like:

Response.Cookies["user"]["value"] = "something";

The page will not be cached.  Sure, it’ll have the public cache-control/expires headers, but the value will always be 100 (duration), and it won’t be cached by IIS.  There won’t be any errors or warnings, it simply won’t cache properly.  In our case it was even less obvious because the cookie writing code had been written a long time ago in a base page, and was only triggered on certain pages with certain querystring parameters.  Even when I discovered pages that I had assumed to be cached were not, I saw the cookie but dismissed it because I mistakenly assumed the effect would be the other way around.  I assumed that a cookie on a cached page simply wouldn’t work right, the first request would write cookie and then every user thereafter would receive the cached page with the cached cookie value effectively giving everyone the same cookie.  You know what the say about assumptions.

After wasting a lot of time with the Failed Request Tracing, which proved impossible to decipher, I cycled back to my suspicious cookie logic.  Once I removed the cookie logic, caching behaved normally.  A quick audit uncovered a second place where some old rudimentary tracking logic was also writing cookies with a similar effect on caching. 

If you’re combining outputcaching with writing cookies on the server-side I’d suggest taking another look.


Technorati Tags: ,

RIF Notes #16

"One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision"Bertrand Russell

“Have you no backbone? have you no spine? Whatever happened to no one gets left behind?”

Wednesday, April 25, 2012

Jason speaks out…sort of.

I was recently interviewed on the Uhuru podcast.  Other than the fact that you could probably play a drinking game based on the number of times I say “sort of”, it turned out pretty well.


Tuesday, April 17, 2012

The 40 year old Twinkie

Last year I experienced a confluence of events that would be an exaggeration to say  changed my life, so I’ll opt to say they modified my life, not the least of which was turning forty.  Turning forty itself wasn’t that significant, more of a relief once the threshold had passed. I guess I had been subconsciously aware of its looming arrival.  Nevertheless, it got me thinking about my age and my health.  Until my thirties I had always been extremely thin, but that lost decade had seen me fluctuate between 160 and 190 lbs.  Each time I’d get heavy, disgust myself and then attempt to loose some weight fast, using Atkins, South Beach and other nonsense.  They worked temporarily but weren’t sustainable lifestyles. 

About the time I turned forty last year, a few other things happened.  I had been diagnosed which some severe reflux and some other minor health concerns that required that I alter my diet.  I went to Disney with my family and found myself once again disgusted with my 180 lb. self in family photos from the trip.  And I came across the Twinkie diet by chance.  The Twinkie diet crystallized for me what I always believed but for whatever reason was reluctant to accept.  And I’m sure I can list a lot of familiar reasons.  Ultimately, its all about calories, period.  No perfect mixture of proteins and vegetables of a certain color, not eating a bigger breakfast, or eating numerous tiny meals throughout the day. 

Inspired by its simplicity, I picked that last weight that I thought of myself as thin, 150 lbs.  and targeted the number of daily calories required to get there.  I used sites  like Livestrong and LoseIt.   Livestrong has the more complete food catalog but LoseIt integrated with fitbit, which I also picked up. None of those are critical, but mildly interesting.  I’m not especially anal about analyzing and tracking. Fitbit isn’t particularly accurate but it is effortless.  Therefore, I picked a low calorie number, 1500, to make sure that since I was going to be ball parking a lot, even if I was substantially off I’d still end up pretty low. I even threw in some P90X and Insanity workouts, but I had a hard time finding the time for those with any regularity, but they contributed.  After about 5 months I hit my 150 lb. goal, mostly on the wings of caloric restriction.

Unfortunately, 150 lbs. didn’t quite turn out to be a thin as I had expected and the Thanksgiving to Christmas time frame saw me gain back about six pounds.  But using the same Twinkie principles I was able drop 10 pounds again from 156 to 146, which is about as light as I’ve been since I was 25.  The beauty is that I still eat junk food, and go off the wagon pretty frequently, but as long as I’m roughly accounting for the calories the weight loss seems pretty sustainable.

Now if I could only suck back down to my 112 lb. high school wrestling weight, now that would be something.

Tuesday, April 3, 2012

Cuckoo for POCO Puffs: NHibernate edition

Migrating from EF model-first to code-first with FluentAPI wasn't terribly difficult and resulted in far simpler code.  Once completed, the resulting POCO's were pure enough to appear to be compatible with NHibernate.  The question I wanted to answer was how similar Fluent NHibernate is to EF’s code-first FluentAPI in terms of how it managed those POCOs.  What I was ultimately looking for was some kind of definitive answer that demonstrated that they were either similar enough as to make no difference or dissimilar in ways that made one clearly superior to the other.  The most thorough and recent comparison I had seen was Dino Esposito's in 2010, and that wasn't definitive nor was it necessarily tailored to our particulars.

Hey my man, what it look like?

What I found was that the model-first to code-first conversion was a much bigger switch than from code-first to NHibernate.  The resulting NHibernate implementation looked nearly identical to the EF code-first FluentAPI version. There were a few syntactic differences in the FluentAPI.  With NHibernate I had to use ".Not.Insert().Not.Update()" in cases where I had both a navigation property and a foreign key Id so that the insert/update statements didn't duplicate columns.  The Linq provider implementation was slightly different too.  Instead of DbSet properties of the context NHibernate uses a generic query method (session.Query<Myobject>()), and instead of Include() NHibernate uses Fetch() and ThenFetch().  I think anyone would agree that these are pretty minor differences.  From the perspective of the resulting code there was very little difference in complexity, organization or syntax.  For my humble purposes and based on this somewhat shallow analysis, it looks like they are similar enough as to make no difference.

The secret awaits eyes unclouded by ambition

Beyond the code I did discover some noteworthy differences between NHibernate and EF.  EF doesn’t have a built in mechanism to trace the generated SQL, which seems like such an obvious basic feature.  I overcame this by using extension methods before, but this is only a partial solution.  It will only trace SELECT statements, not the INSER, UPDATE and DELETE’s and it doesn’t include parameter values.  I took a look at EFProviderWrappers and wasn’t encouraged.  Its mainly sample code not an actively maintained project, it requires some odd configuration tricks, and doesn't support code-first’s DbContext.  While I could adapt this sample for my purpose, it really shouldn’t be necessary.  NHibernate will output the generated sql with a simple ‘ShowSql’ configuration setting.  Its not hard to hear the claimed lack of maturity and extensibility points in EF ringing true when the first, most basic one, is a hassle.

Those who are bound by desire see only that which can be held in their hands

On the flip side, I found that NHibernate dependencies to be fragile.  I downloaded the Fluent NHibernate 1.2 version which included Nhibernate 3.1.  Immediately upon running the application I began encountering a series of dependencies issues.  The two most troublesome of which were:

Could not load file or assembly 'NHibernate.ByteCode.Castle' or one of its dependencies. The system cannot find the file specified.
The type initializer for 'NHibernate.Cfg.Configuration' threw an exception. The invoked member is not supported in a dynamic assembly

I solved the former by adding it as a reference to every project that referenced the assembly that referenced NHibernate.  I don't find that to be a particularly satisfactory solution.  The latter was resolved by upgrading my other assemblies to use log4net 1.2.10, another odd nuisance.  While searching for answers to those two particular issues I discovered a frightening number of other compatibility woes plaguing others.  A well documented Microsoft supported and integrated Entity Framework requires far less fuss and googling.

You just get that sucker to the designated place at the designated time, and I will gladly designate his ass... for dismemberment!

So far, there’s not much to fight about.  Either I have to do a little extra work to get tracing, or a little extra work to make sure my versions and dependencies are straight.  There are, however, some more strategic considerations. 

  • NHibernate’s Level 2 cache is dreamy, the EFProviderWrapper is pitiful.  Until Microsoft or a third party writes a commercial Level 2 cache for EF, NHibernate has the advantage, with a caveat.  I’m not sure how well the Level 2 cache works or is to work with in practice, but more importantly the likelihood of our data access being all or mostly managed by an O/RM is a long way off, which somewhat devalues its usefulness in the short term. Still I can’t help but be enamored by the possibility.

  • Entity Framework is the preferred data access technology from Microsoft. That means documentation, support, integrated tooling, potentially wider adoption.  Its not hard to imagine EF reaching feature parity and even surpassing NHibernate in the near future, other than the fact it should have already done so and still has a ways to go.  The more troubling aspect of the “it comes from Microsoft” double edged sword is that to get new EF features we had to upgrade the entire platform from .NET 3.5 to .NET 4.0.  I’m hoping we don’t need .NET 4.5 to get EF 5.x features.  NHibernate is somewhat freer in this regard to add features independent of the full framework.

Well, I did you one better. I mastered "the art of fighting... without knowing how to fight".

Objectively, I might give the slight edge to NHibernate, all things being equal.  But a slight edge isn’t quite enough to go off into unchartered territory with NHibernate just yet.  I think we’ll continue down the path of converting our EF v1 style model-first code to Entity Framework code-first with FluentAPI (I’m still not sure how to refer to it), while taking comfort in the fact that as long as our POCO’s are pure its not a decision that is irrevocable.

Friday, March 30, 2012

Cuckoo for POCO Puffs: Prequel

What exactly am I hoping to accomplish by moving to EF code-first or NHibernate?

The short answer is that I'm looking to gain features that I'd otherwise have to implement while at the same time simplifying and reducing the amount the code we already have.  For the benefit of getting my own thoughts in order I want to articulate a slightly longer answer which will inventory the features and identify the potential areas of simplification and reduction.

And that is what grieves me the most

We have a substantial codebase which utilizes our own class-in-charge patterned persistence ‘framework’, the bulk of which still works by using the Enterprise Library and manually mapping ADO.NET structures (datarows, datareaders, commands) back and forth to the database. This approach first started in the .NET 1.1 era. The framework mainly consists of a few base classes that provide more of a template (for consistency) than anything else. This approach has lasted a long time and continues to work. Nevertheless, there’s a certain amount of tedium and ceremony with our existing approach. For every domain object there are potentially the four classic (CRUD) stored procedures that must also be created, plus any other variations to provide alternative means of retrieval or persistence, and of course all the mapping logic. This can mean an explosion of stored procedures to meet a variety of query alternatives, or one procedure with a multitude of conditional switches executing different query versions based on parameter combinations. Invariably, to keep the number of procedures manageable, the same stored procedure is used to serve multiple purposes at the expense of efficiency.

Ah. It must’ve been when I was younger

Our active record pattern-ish approach leads to additional difficulties because its not well suited to treating complex graphs as aggregates, we often find ourselves in the eager vs. lazy load conundrums, validation is ad-hoc, concurrency is seldom handled, and as I've lamented caching is non-trivial. One of the more obvious cases where our framework struggles is the oft encountered situation where one use-case implies that domain objects would be more efficiently eager-loaded as an entire graph, while those same domain objects are more efficiently lazy-loaded within another use-case in a different module within the system.  The same is true of caching.

Its possible to evolve our framework, use code generation, and steal bits and pieces from other frameworks, and I’ve done a little of that. But that’s an awful lot of wheel inventing and plumbing.

My children, I have watched you for a thousand years.

For a long time now there have been frameworks that provide a much richer set of services, both at the domain layer and data layer (and some debate about whether these frameworks could or should serve both). The frameworks that I'm most aware of are Entity Framework, NHibernate, CLSA.NET and SharpArchitecture. Therefore, these are the focus of my re-evaluation. The features of each that I'm interested in are:

Entity Framework 4.3

  • Sql Generation
  • Linq support
  • Change Tracking
  • Validation facilities
  • Concurrency support


  • Sql Generation
  • Linq support
  • Change Tracking
  • Validation facilities
  • Concurrency support
  • Level 2 cache
  • Logging/tracing


  • Implementation of DDD
  • Loose coupling


  • Validation
  • Security
  • Transaction manager
  • Business Rules engine
  • Rich databinding support
  • N-level undo
  • aggregate roots
  • Mobile objects/portability

Infidel Defilers. They shall all drown in lakes of blood

Our first foray into the frameworks was EF v1, and that did not deliver on the promise of code reduction or simplicity. At best it was a wash, making simple things simpler and difficult things more difficult, mostly just pushing complexity around. That's likely partially the fault of our approach and partly that of the the agony of EF v1.  

I wish to speak to you now

With EF 4.3 code-first and FluentAPI we've taken a fresh look, and with its apparent similarities to Fluent Nhibernate it made sense to look at both.  I still like the idea of deploying changes and not having to deploy both code and SQL scripts simultaneously. Of course, the downside is that we can't change them independently. Oftentimes we’ve found it advantageous to be able to optimize SQL in a stored procedure and change it without having to recompile and redeploy an application. Some might suggest that we could get the best of both worlds by combining O/RM's with stored procedures (because they support this), but I feel like this is the worst of both worlds. By the way, I'm sorry but O/RM's don't generate more efficient code than I could write.  Nevertheless, generated SQL provides the opportunity to query the same objects differently in different parts of the application and get more appropriate SQL. This is even more true with column level change tracking.  Having code that is smart enough to only update what was changed rather than sending all values back and forth, as well as knowing when no update is necessary, would be a meaningful improvement.

This flame will burn away the darkness, burn you the way to paradise

Code that can easily be toggled between eager and lazy loading by use-case is certainly desirable. And having all the complexity of caching handled for me would be a dream come true.  That’s one of the more exciting differentiating features of NHibernate. But its not quite clear to me how well that works and whether we’d be able to reach a point where enough of the data access is managed by NHibernate to take advantage of Level 2 caching.

That is strength, boy! That is power!

Since 2005 I've been fascinated with CSLA.NET (although admittedly not fascinated enough to actually build something with it) and more so with each successive version. Its not an O/RM but it has some overlap in terms of the facilities it provides, like validation and change tracking. Additionally, it provides one specific advantage that I find particularly intriguing. The concept of mobile objects and with it a measure of portability. The ability to use the same business logic on both sides of RIA application, whether that be Silverlight, WPF, WP7, or even Monotouch and Monodroid is especially compelling. With the explosion of devices and platforms this is sounding more and more powerful.

Contemplate this on the tree of woe.

These frameworks are not mutually exclusive. We could use EF or NHibernate as both our data access and domain layer, or we could CLSA with EF or NHibernate, and SharpArchitecture already leverages NHibernate.  All this must be weighed against the time, effort, approach, and risk of switching. If the application(s) were to be built from scratch today, we'd clearly be using one or more of these, but alas I'm architecting after the fact. I have to deal with the knowledge that a migration is fraught with peril.

Tuesday, March 20, 2012

Cuckoo for POCO Puffs: Week 1

I completed converting my moderately complex sub-model (24 entities) from the database-first style to code-first style this week.  The results are pretty encouraging.

I find your lack of faith disturbing

The use of foreign key associations, POCO’s and the Fluent API vastly simplified the resulting code.  Not that I didn’t encounter my share of learning curve struggles.  The fluent api syntax for declaring relationships is a bit clumsy, and the difference between declaring properties as virtual or not and its resulting impact on lazy load behavior confused me on at least one occasion.  Notwithstanding those, the bulk of the effort was simply the brute typing involved in re-implementing all the class, repositories and relationships, as well as refactoring tests to use the new code. The EF 4.x POCO Entity Generator gave me a really good starting point, although it didn’t allow me to pick which tables, instead generating the whole database.

All too easy

What I found most compelling with POCOs, is that I no longer needed to jump through all the hoops to detach and re-attach object graphs.  You may recall we used this approach, Reattaching Entity Graphs with the Entity Framework, to handle this problem in EF v1.  Apparently, and it kind of makes sense with POCOs, that detaching and re-attaching is less of an issue.  My code-first repositories are so much simpler that I’m suspicious that either I’ve missed something or my chosen sub-model wasn’t sufficiently complex enough to surface the still lurking issues of detaching objects.  Then again, one thing I haven’t attempted yet is persisting a complex object graph as one unit, relationships and all.

Asteroids do not concern me

Once the object leaves the context its essentially detached.  One odd side-effect of this and lazy loading is that accessing a lazy property (even to check if its null) outside of the context results in an exception.  There doesn’t appear to be a good way to check if properties are loaded outside of the context.  To handle this I disabled lazy loading, which might seem extreme, except that I was already being explicit using Include(), and having EF attempt to lazy load detached objects doesn’t serve much purpose.

Another oddity I haven’t quite grokked is that foreign key associations don’t appear to stay in sync with their ID properties even after a SaveChanges() (e.g. if I update the StateId on an Address, the Address.State association doesn’t seem have the correct State after the persistence).  Its not a feature I’ve thus relied on so its not an issue in this case but something I’ll want to understand better if I go forward.

Impressive, most impressive

Overall, I’m very impressed with combination of code-first, POCO and the fluent api in EF 4.3.   I don’t see any reason not to convert our existing EF code to this approach, other than the time, effort and risk.  It may not be flawless, but it’s a considerable improvement over the contorted v1 style approach currently employed. There are two additional points of inspiration gleaned from this effort:

  1. The POCOs appear as if like they could just as easily work with NHibernate as with EF.  They have no dependency on or direct relationship with EF (thanks the the fluent api) and all the properties are virtual.
  2. There is a striking similarity between the POCO objects and our Domain objects in many cases,  In some cases, the POCO object is nearly identical to the Domain object that its hydrating.  This suggests to me that we may be able to collapse the two layers. 

Perhaps I can find new ways to motivate them

Next up I’m going to try to implement this same sub-model using the same POCO objects, but this time using Fluent NHibernate.  NHibernate is more mature, supports more databases, and has Level 2 cache support baked in.  All else being equal, NHibernate might be the better choice, so I want to see how equal all else is.

At some point I should probably explain what I’m trying to accomplish with all of this.

Friday, March 16, 2012

RIF Notes #15

“If you don’t know why it works, it probably doesn’t. You just don’t know it yet.” – Steve McConnell

“I need serenity
In a place where I can hide”

Wednesday, March 14, 2012

Cuckoo for POCO Puffs: Day 1

Now that we’re firmly on .NET 4.0, and EF 4.x has arrived I’ve begun revisiting our EF v1 style repository pattern looking to see if the new code-first and POCO support will help mitigate some of the frustration we’ve encountered with Entity Framework. The pain we’ve experienced over the past couple of years primarily falls into two buckets:

  1. The models (EDMX and designer files) can become large, unwieldy and idiosyncratic.  Refreshing it with underlying changes to the database can often result in cryptic errors, and source control merges of these files can be painful.
  2. Independent Association relationships are tricky and complicated when working in a disconnected mode.  Properly navigating these relationships often involves loading entities from two or more contexts, associating them together, and then getting them to properly persist as a complete graph when reattached to a new context.

Models, Inc.

Over the years we’ve mitigated some of the giant model problems (our primary database contains just shy of 400 tables)by creating sub-models.  We’ve created these sub-models by clustering groups of related tables in their own model.  This has helped quite a bit, but has also led to a bit of code duplication in the form or repetitive boilerplate code.  Its likely that could also be mitigated with some T4 code generation, but we never got there.  Additionally, some tables belong to more than one model, and querying across models isn’t possible.  Lastly we had to employ some tricks to make sure we don’t have to have a connection string for each sub-model.

Relationship advice

We don’t have a good answer for the contortions we have to go through to manage disconnected relationships.  Up to this point we’ve treated EF entities as DTOs, following Rocky’s Lohtka’s advice, rather than using EF entities to replace our domain objects.  On the other hand, Ayende disagrees with this approach, at least in the case of NHibernate.  His recent posts about limiting abstractions seems like its directed right at us and our repository over EF solution.

If you are writing an application and you find yourself writing abstractions on top of CUD operations, stop, you are doing it wrong.

But I digress…

We have a considerable amount of code invested in this approach, domain objects mapped to EF entities (as well as straight up ADO.NET), so before we throw away both our domain and data access layers in one fell swoop, I figured the first step was to see whether code-first POCO objects could assuage our model problem and if Foreign Key associations could sooth our relationship pains. In the process I hope to get a better sense of whether EF or another O/RM might eventually serve in the domain.

Keeping it real

Naturally, I chose our simplest case to start with, a sub-model with only two entities.  I began to convert our database-first sub-model to code-first, picking the Fluent API over Data Annotations for a few reasons:

  1. I’m not fond of polluting the objects with persistence attributes if I can avoid it.
  2. The Fluent API is a superset of data annotations.  I’m unlikely to be able to do everything with Data Annotations alone, and rather than have configuration information in two places, I decided to centralize it in EntityConfiguration classes.
  3. If I was going to consider EF as a replacement for my domain objects, Fluent API offered me the best opportunity to do that with the least amount of change to the domain objects.  Keeping the POCO’s pure, might even lend itself to more easily attempt a similar approach with a NHibernate (which I have next to no experience with, so this may be a pipe dream).

Trivial pursuit

One of the first things I discovered is that I couldn’t mix code-first in the same assembly with my existing database-first models.  After one day, I can’t say that this is necessarily impossible but I found some indications that it wouldn’t work.  Rather than fight it, I created a separate assembly for the code-first data access and began building my two POCO entities, my new DbContext (which I found much easier to leverage existing connection strings), and a repository with the same signature as the existing database-first repository. 

By the end of the day I had swapped out my unit tests and domain classes to use the new code-first repository instead of the database-first and had all my unit tests passing.  I then deleted the model, designer, and the rest of my database-first artifacts, just to be sure.  The code-first version definitely feels cleaner, fewer files with less code.  One feature of database-first that is both a benefit and a curse is its ability to refresh itself from the database.  I don’t see an obvious way to mimic that with code-first, but then again maybe I won’t need to.  Overall, day 1 went pretty smoothly, albeit with a trivial example. 

Real World/Road Rules Challenge

Next I’m going to pick a more complex sub-model, one with 20+ tables with complex relationships, and begin the conversion. I expect this will take more than a day, but will give me a much better idea of the advantages, feasibility and effort of doing a full conversion to code-first.

Wednesday, February 29, 2012

More Cache please

In my last post on this topic we had sailed through our peak load season only to start experiencing frequent NCrashes which brought down our web farm on a fairly regular basis.  After many conversations with Alachisoft support (and licensing) we agreed to attempt a hail Mary. 

Pretend like everything’s gonna be alright, although you know it won’t be

We decided to upgrade from NCache Professional 3.8 to NCache Enterprise 4.1.  It wasn’t clear whether the issues we were having were version related 3.8 vs. 4.1 or edition related.  By upgrading to 4.1 Enterprise it gave us the opportunity not only to hope that the new version was more stable but more importantly gave us the opportunity to try different topologies.  The professional edition only allowed for one topology, Replicated Cache (synchronous).  Originally, Alachisoft gave us every assurance that at our scale and load this topology should work fine (as it did for many other clients).  They insisted that our troubles with NCache were due to an unstable network rather than NCache itself.  Even if that were true, NCache’s inability to handle or at least pinpoint this ‘instability’, in my mind, is on them. 

Unstable…it’s hard to be the one that’s strong

Regardless, the Enterprise version also gave use several topology choices, the recommended one being Partitioned Replica (asynchronous).  With this topology we were assured that due to its asynchronous nature, our “network instability” if it were to occur would not cause a disruption to NCache that would simultaneously affect both nodes.  That at least would be an improvement, I guess.  The other major feature, which is surprising that we needed an enterprise version for this, is the ability to configure alerts.  The enterprise edition enabled us to receive alerts when nodes left or joined the cluster or other potentially disruptive events occurred.  While not ideal, at least being alerted instantly to an identified problem was better than troubleshooting web farm issues blind.  Our expectations were low, we weren’t expecting the upgrade to fix the problem, but with the proper alerting and a few more options available, we at least had a chance to capture the event early, take memory dumps and other diagnostics that might eventually lead to the true source.

And all I really want to know is, if she’s gonna be alright

We upgraded to NCache Enterprise 4.1 and configured the asynchronous Partioned Replica cache.  Overall it went very smoothly.  And then we watched and watched …and watched.  For days we watched, poised.  But no alerts came, no web farm lockups due to NCache funks.  Its been months now, and still no issue. I have to give Alachisoft credit, for working this thing through despite our increasing disillusionment with them.


Maybe the version fixed it, maybe the topology, maybe we have an unstable network and this version handles it better or maybe we don’t and the old version simply had some sort of bug.  Maybe I’m happy to get past this and move on.

The only time we’ve seen any kind of funkiness with NCache since the upgrade is when we’ve rebooted one of the nodes.  Anytime that happens, the up node’s web application hangs for a good two minutes (merging the caches), which is about the same time it takes the node to reboot.  So as far as NCache providing high availability there’s a bit of an issue there.  And by the way, never try to reboot both nodes at the same time because that surely did bring on the funk. 

Wednesday, February 22, 2012

RIF Notes #14

“There are only two hard things in Computer Science: cache invalidation and naming things” — Phil Karlton

Forget the lies, the money, we’re in this together.
And through it all, they said nothing’s forever.

Friday, January 13, 2012

RIF Notes the 13th

“I can calculate the motions of heavenly bodies but not the madness of men” – Isaac Newton

“Yeah I'm alright
And I need to know
When I'm dead and gone
Where do I go”