5 Unforgivable Windows 8 RTM Problems

I work with a couple of committees at Microsoft for developing the future “story” of some of their key technologies. So I’ve been able to participate in some fun closed focus groups for things like ASP.NET, WCF, Windows and Visual Studio 2012. The gist of it is that the guts of Windows 8 are superb. It is extremely optimized, and the OS has a much lower memory footprint. This was the core of their focus when developing Windows 8, because they need it to run on tablets. That means battery life is paramount. In fact, Windows 8 will run better than Windows 7 on the exact same computer. That’s a first for MS, and it’s pretty incredible.

Sadly, that’s where the incredible ends. I’ve been using the final RTM bits for a bit now, and there are a few problems really need to be addressed SOON. Some you’ve heard, some you probably haven’t. Here’s my current gripe list:

1. Two Separate Browser Processes

When you launch your browser from the Desktop, it feels like every version of Windows you’ve been familiar with since 1995. When you launch it from the Start Screen (the Modern UI), it launches a separate parent process, which is the full screen “immersive” browser. The 2 parent processes are separate, and it creates many, many problems. For one, now you have 2 sets of tabs. So when you have that article you open, and want to get back to reading later, you have to try and guess which browser it’s in. You can never see the 2 browsers on the same screen at the same time. You can’t move tabs from one to the other. In fact, in the Modern UI, you can’t even pull tabs out of Chrome at all. So if you wanted to pull that YouTube video out and watch it on the 2nd monitor, while you continue to do something else on your primary monitor, have fun copying and pasting both urls, going to your desktop, logging in to both sites (again), and finally pulling the youtube tab out. And I hope you don’t trust your browser to properly restore your tabs, because it fails. Hard. You’ll be lucky if you can restore the tabs from the LAST browser you closed. And since most people don’t Alt + F4, you never really close the Start Screen browser. It just gets suspended. Also, have fun logging in to every site TWICE, once in each browser. Logged into Gmail on the desktop Chrome? Log in again on the Modern Chrome. Oh, and don’t unpin the Start Screen IE from your Start Screen, because you can’t get it back. Ever. If you add IE back, it will only launch in the Desktop (which honestly, is what you probably want anyway).

2. Full Screen Apps

Apps created using the WinRT, designed for the new Modern UI (launched from the Start Screen) launch as a full screen process. I’m sure we’ve all accidentally hit F11 in our browsers, when we’re really trying to hit F12 to pull up the dev tools. WTF? Why is my browser full screen? Oh, F11. Oops. Well get used to it. When you launch an app from the start screen, it’s full screen. No border. You can’t resize it, minimize it, close it, move it, or look at anything else at the same time. You can’t see what other apps are running at a glance, or see your battery life, check the time, or see any tray icons. If you’re on a 3G device, you won’t be able to see how many bars you have, or if you’re even connected. Sure, you can try to drag it and snap the window to the side, so you can try to look at 2 apps at once, and maybe that’s a neat gimmick for tablets, but this is the DESKTOP. And you can’t even decide how wide to make the snap-in sidebar. Oh, and you know the little Gmail chat windows? Go ahead, try to click the “pop out” link, I dare you. It opens the chat in a new tab. And since you can’t pull the tabs out, the chat will now be the only thing you can look at, fullscreen. You can’t even pull the tab out to dock it in the sidebar. Forcing people into a full screen window with no option for arranging their work environment is many, many, many steps back, and overall just a huge insult to desktop users.

3. Sharing

This is a quick one, but it’s still annoying. Open a website in your Start Screen browser. Pull up the “charms” menu (move your mouse to the top right of your screen), and click Share. You can share a link to that site via Facebook, Twitter, Email, whatever. Now go to your Desktop browser. Or really, any app on your Desktop, and click the same Share button. “Nothing can be shared from the desktop, EVARRRR”. Ok, maybe I added the EVARRRR, but that’s what it feels like. So why even show the button? Related to this gripe is the Settings menu item in the Charms menu. It does different things depending on if you’re on the Start Screen or the Desktop. Annoying. Like, you can’t access the Control Panel unless you go to the (oh so outdated…) Desktop first. But who needs the control panel! It’s not like I need to install IIS or anything…

4. Two Taskbars

When you’re on the Desktop, you have your trusty taskbar. Even if the Start Menu is absent, at least you have shortcuts on your taskbar, and you can tell which of them are running, and which haven’t been launched yet. You can access MRU and jump lists by right clicking on them. It’s easy to spawn a second process, close multiple windows, or run something as an administrator. Oh, but it WON’T show you any apps that you’ve launched in the Modern UI. Those have their OWN taskbar. Move your mouse to the top left of your screen, and then pull it straight down. Bam! The sidebar pops out from the left, and shows you the tasks running in your start screen. Oh, and go ahead, try to alt+tab. You’ll see a list of DESKTOP processes, but none of your smart screen processes. You know what would be more convenient than 2 taskbars? Win7 in a VM on a Win7 machine. Hey, then you’d have TWO start menus! Way better than zero of them.

5. Two Operating Systems

Windows 8 just has this over all split personality feel to it. I’ve been using it daily, for both work and play, and it doesn’t get easier. It doesn’t start to feel more natural. It just gets more frustrating every time I have to look around for a while before I can find that browser tab I left open, or discover that it’s gone, because only half my tabs were restored. It gets more frustrating when I try to get back to an app with alt+tab, don’t see it in the list, and then have to check my SECOND task bar (which has nothing in common with the first one). It’s frustrating that I can’t have 3 or 4 windows on my screen arranged in a way that optimizes my workflow. Apps have to be fullscreen, you can’t look at 2 at the same time (sorry, their stupid docking thing is so effing broken and worthless it doesn’t count), and you can’t arrange anything. It’s no joke. This is two operating systems in one, and it SUCKS. HARD.

Posted in windows | 2 Comments

The Cake Is Not A Lie

I know it’s been a while since the last post, and I kind of left things hanging with the “end to end application strategy”, that rolled something like:
  • Sql Server
  • Data Layer — EFv4, with T4 templates for DTOs, and AutoMapper
  • Service Layer (WCF REST)
  • Web UI (jquery)

Well, the stuff I was doing with the data layer, with a few utility classes, and the T4 templates ended up being much more useful than I thought. Useful enough that it warrants more than just a couple of blog posts with some file attachments. Also, the more I worked with WCF REST, the more I hated it. Crappy date format when you serialize to json. No control over serialization. No cycles in your object graphs. No anonymous types. An ungodly amount of configuration. Ridiculous error handling (none of it global that “just works”). So, I started to shop around, and while some people are taking to creating REST services using MVC, I haven’t drank the MVC kool-aid yet, and that’s not really what it was designed for anyway, so I wasn’t overly keen on that route. However, a plain IHttpHandler….now we’re talking. You get whatever advantages you need from the ASP.NET runtime, but all you have to do is implement ProcessRequest(), which gives you an HttpContext, which is all you really need. You have full access to the request, and you create a response. There are excellent tools like Json.NET to help you with the json serialization. You just set the content-type of the response, and away you go.

Code Reuse
Once I started making some REST services with plain old IHttpHandlers, the patterns started to emerge. How to process the query string. Well, I’d really rather have a nice route instead of some path to an ashx file. How do I differentiate between uri template params and query params? How should I generically return errors for bad input? What’s the best way to determine which service method to call in the IHttpHandler class? Like any man who was thinking, I tried to follow the DRY (don’t repeat yourself) principle, and centralize the logic for many of these tasks.

So, as our code and our approaches to problems always do, my “end to end” strategy was evolving into something much, much more. I ended up with 2 incredibly useful libraries: One for getting the Data Access Layer up and running (utility classes and T4 templates), and the other for creating RESTful services based on simple IHttpHandlers (with it’s own basic routing and processing engine). These libraries have been released on CodePlex. They are:
RestCake
and
Loef


I have spent a very good amount of time polishing these libraries and running them through their paces, to make them production ready. I can happily say they are in use in some large, enterprise-scale applications right now. Yes, there are bound to be a few bugs here and there, but these are ready for production use, right now. I’ve also spent a very liberal amount of time creating overview pages that explain the gist of the libraries and how they work, and working examples. The examples and documentation can be found at http://restcake.net.

These are very small, unobtrusive libraries. Nothing enormous and overreaching. Read the RestCake overview page and the Loef overview page to get a feeling for these libraries and what they offer. You’ll see what I mean. If you have any issues or suggestions, please use the CodePlex pages to report them, and I’ll try to get on them right away. For now, I’m going to continue to improve the documentation and working examples, and iron out any bugs as they surface. As my good friend Jerry would say, “I’m just one man”.

Enjoy these new libraries! The cake is not a lie!




Posted in Uncategorized | 3 Comments

Entity Framework v4, End to End Application Strategy (Part 3, Changes to Data Layer)

Initially in my service layer, I was doing all of my serialization with the Json.NET library, for 2 primary reasons
  • Control over how dates get formatting (I don’t like the \/Date(123)\/ MS AJAX style)
  • The ability to serialize object graphs that contain cycles (by telling Json.NET to ignore them)
Unfortunately, this started causing me too many problems with WCF. My service methods couldn’t return DTO types directly — that would automatically cause the use of DataContractJsonSerializer. So initially I just returned JSON strings. But then you end up with stuff like this:
“{\”Message\”:\”Hello World\”}” (double wrapped or double quoted — requires two JSON.parse() calls)
instead of this:
{“Message”:”Hello World”}

For more info regarding that problem, see the StackOverflow question I asked about it. It details how to still use Json.Net, but not have the “double quoted” or “double wrapped” json string. Unfortunately, the solution really only works for sending responses back to the client, and it also forces the use of json, so you can’t have soap endpoints to your service methods. For service methods that take object types as params, you can’t force WCF to use Json.Net to deserialize, and it will automatically use DataContractJsonSerializer. This means if your dates aren’t in MS AJAX date format, you’ll throw exceptions. So I’m ditching Json.Net in this particular app stack. Soon, I want to do an article on creating a fully equipped REST service by creating your own IHttpHandler, instead of using WCF. In that case, I think Json.Net would be the best choice for serialization and deserialization. But that post is for another time.

So, I’m back to using DataContractJsonSerializer. The big advantages to this are:
  • My service methods return DTO types, not the arbitrary “Message” type (System.ServiceModel.Channels.Message), or “double wrapped” json strings.
  • My service methods can take object types (DTO types, entity types, whatever) as input params, as opposed to json strings (personJsonString), which require manual deserialization within the service method
  • My services can now have both REST (json) and SOAP (xml) endpoints
  • With both endpoint types, the service can easily be called either directly from client code, or in code behind, using the proxy classes created with Add Service Reference
But, using DataContractJsonSerializer does bring back the 2 problems I had in the beginning: Dates and Cycles. The MS AJAX style dates I can live with (uhhg). The cycles, however, need to be remedied, or WCF will throw a hissy fit (also known as a runtime exception).

Removing Cycles (Almost) Automatically

Imagine that a Person object has an Address object, and an Address object has a list of Person objects that have that Address.
somePerson.Address; someAddress.People;
This creates a cycle in our object graph.
.
Since json doesn’t support references, cycles would cause the DataContractJsonSerializer to infinitely recurse (since it can’t be simply told to ignore cycles, like a much superior serializer I know about). This just means we have to strip out cycles from our object graphs before we try to serialize them. That shouldn’t be too hard. We’ll start with a root object, and traverse all known related objects, adding each object to a flat list along the way. If we encounter an object that’s already in our running flat list, then we assign parent object’s property to null, or remove it, or assign it to a stub object, just so long as it doesn’t point to the object in question, since it’s already in our graph once. This breaks the cycle. The way I accomplished it is with a set of recursive methods in each of the DTO classes (I’m only interested in stripping cycles from DTO graphs, since that’s all I send to the client).

Here’s a reminder of what our sample model looks like.

Given this model, the code to strip cycles (with Person as the root object) is:
public  partial class PersonDto
{
	public PersonDto StripCycles()
	{
		Dictionary<object, string> graphObjs = new Dictionary<object, string>();
		StripCycles(graphObjs, "");
		return this;
	}

	internal bool StripCycles(Dictionary<object, string> graphObjs, string path)
	{
		if (graphObjs.ContainsKey(this))
			return false;
		graphObjs.Add(this, path);

		// Single navigation property references
		if (Address != null)
			if (!Address.StripCycles(graphObjs, path + ".Address"))
				Address = null;

		// Collections
		if (Projects != null)
			for(int i = 0; i < Projects.Count; ++i)
				if (!Projects[i].StripCycles(graphObjs, path + ".Projects[" + i + "]"))
					Projects.RemoveAt(i--);
		return true;
	}
}

So we’ve got 1 public method which can be used very easily. somePerson.StripCycles(); That can be safely serialized, no matter how crazy your object graph was before. This of course requires that each DTO class also has the same 2 StripCycles() methods implemented. Each implementation is different. Each class will have logic for each navigation property, and each collection of other DTO types. The methods aren’t directly recursive; but they do all call each other. A calls B which calls A which calls B, until your “base case” is reached, which is either you end up at a leaf node of your object graph, or you encounter a cycle, and break it (which is really just turning it into a leaf node), then the calls start popping back up.

Now, since it would be quite tedious to create and maintain these methods for each DTO class (especially during initial development, where new types are often added, and existing ones are constantly modified), we’ll again leverage the T4 templating engine to create these methods for us. We have all the model’s metadata exposed to the T4 runtime, so this is pretty easy. I won’t post the code inline, since the whole template is around 400 lines. But it’s an updated version of the DtoTemplate.tt I previously uploaded. Download the project zip at the end of the post to check out the new templates.

More options when removing cycles

You may not want to just “break” the cycle. You might want some indication that the relationship exists. In the case of the drawing, sam and sam.Address.People[0] refer to the same object in memory. Well, you could assign sam.Address.People[0] to a “stub object” that gives us the info we need (like the primary key value of the real object).
// Break the cycle by replacing the circular reference with a "stub" object
sam.Address.People[0] = new PersonDto { ID = 1 };
This approach does have some additional problems though. In client script, you’ll have objects that only have their PK field populated. And you might have to try to “hunt down” the real object elsewhere in the object graph to get its other properties. Very annoying. So, in the T4 template, I provide some additional options for how cycles will be removed. If the “stub” objects are used, then each DTO type will have some additional properties, shown here.
/// <summary>
/// If true, this is not the original object.  This is a placeholder that only has the ID and FK properties set.
/// The original object was removed, to prevent cycles in the object graph.
/// </summary>
[DataMember] public bool __isCycleStub { get; private set; }

/// <summary>
/// The path to the "real" object that this stub represents, from the root of the original object graph.
/// This can be resolved in javascript like this:
/// 	var realObj = eval("graphRoot" + stubObj.__pathFromRoot);
/// 	alert(realObj.someProperty);
/// </summary>
[DataMember] public string __pathFromRoot { get; private set; }
This makes the stub objects easy to find, and the real objects they represent easy to track down.

Wrapup

All in all, there are quite a few changes I’ve made to the data layer. Also, since this is my foundation strategy for new projects right now, I’ve collected all of the templates, utility classes and extension methods into their own project, so I can easily redistribute them as a group for use in multiple dependent projects. And through the glory of svn:externals, it’s easy for my dependent projects to get the latest version of all these tools. Anyway, rather than post all of the files individually, here’s the whole project:
DataLayerHelpers.zip
And here’s what the project looks like.

The zipped project comes with the latest AutoMapper in the Lib folder, and a custom build of HyperDescriptor that works against .NET 4 (StackOverflow question concerning HyperDescriptor in .NET 4). There are also some handy singleton classes in there. I especially like the PerRequestSingleton, which creates a single instance of a class that is per request (ASP.NET-page-life-cycle-wise), and per user (it uses HttpContext.Current.Items for storage). It’s a great way to make sure you’ve always got an instance of your ObjectContext class on hand, but without having to worry about when and where you create it.

Next time I will try to get to the actual WCF REST service.






Posted in Code | Tagged , , , , , , | 7 Comments

Entity Framework v4, End to End Application Strategy (Part 2, Data Layer)

In part 1, I talked about what the whole app stack will look like. To recap the stack:

SQL Server

Data Layer — EFv4
(Auto generated AutoMapper configs between entities and DTOs)
DTOs (Auto generated with T4 templates, based on the edmx)

Service Layer (WCF REST, based on a template)
(Json.NET for serialization/deserialization)
Web UI (heavy on the jquery)


This article will focus on the Data Layer, which will include my DTOs. I’ll cover specifically:
  • Adding a new ADO.NET Entity Data Model and configuring it
  • Turning your DataContext class into a singleton class that works well for ASP.NET environments (optional)
  • Adding a custom T4 template to auto generate your DTO classes.
  • Adding a custom T4 template to auto generate the [entity to dto] and [dto to entity] AutoMapper configurations.
  • Adding a custom T4 template to auto generate additional code for your entity classes
  • Adding a few custom helper classes for use inside the data layer (UnitOfWorkStore, JsonHelper, StronglyTypedIncludeHelpers)


First, here’s my contrived data model. I have Person objects, with basic info. Each person can have an Address. Each person can also be the owner of Projects.
data model
Earth shattering, I know. I can hear people migrating en masse from MS Project to this new scheme.

Creating the EFv4 Data Model

This isn’t an Entity Framework 101 tutorial, so I won’t hand hold you through the entire process of creating the data model. Add a new ADO.NET Entity Data Model item to your data layer project. If you want to use the contrived model I’ve shown above, here’s the SQL create script. I’m not using the POCO generator templates, I’m just using the default config (so my objects inherit from EntityObject). You can have lazy loading enabled or disabled, it’s just a matter of preference. Initially I had a problem with AutoMapper triggering the lazy loading, but that’s sorted out now, so leave it on if you like it.

Turning your DataContext class into a singleton class

This part is optional. If you like seeing this all over your code:
using (PersonEntities context = new PersonEntities())
{
    // ...
}
Then by all means, skip this section. Especially if you like passing around the “current instance” of your object context to every method that interacts with your DAL. If you just love that, then DEFINITELY skip this section.

I’m not going to argue the best way to create the singleton pattern in C#. I’m setting up a special singleton that is unique “per user, per post back”. I use a class called UnitOfWorkStore, that looks like this:
static class UnitOfWorkStore
{
	public static object GetData(string key)
	{
		if (HttpContext.Current != null)
			return HttpContext.Current.Items[key];
		return CallContext.GetData(key);
	}


	public static void SetData(string key, object data)
	{
		if (HttpContext.Current != null)
			HttpContext.Current.Items[key] = data;
		else
			CallContext.SetData(key, data);
	}

}
It allows you to store any object, by string key. If used in an ASP.NET environment, it uses HttpContext.Current.Items as the storage mechanism. The scope of that Items “bag” is unique to each user on your site (since they each have their own HttpContext), and the lifetime of the Items bag is the lifetime of the request/response lifecycle. It’s a very handy storage spot. This UnitOfWorkStore class will also work if used in a non-ASP.NET environment. It falls back to storing things in CallContext.

So create a partial class in your project to add some custom code to your DataContext class. Here’s how I’ve set up my singleton accessor:
public partial class PersonEntities
{
	private static readonly object s_objSync = new object();
	private static readonly string UOW_DATACONTEXT_KEY = "PersonEntities_UserInstance";

	/// <summary>
	/// Uses a UnitOfWorkStore to return the same instance of PersonEntities, that is unique
	/// per user, per postback (or CallContext, if not in an HttpContext environment, such as unit tests or console apps)
	/// </summary>
	public static PersonEntities Instance
	{
		get
		{
			// Dirty (non thread-safe) check
			if (UnitOfWorkStore.GetData(UOW_DATACONTEXT_KEY) == null)
			{
				lock (s_objSync)
				{
					// Thread-safe check
					if (UnitOfWorkStore.GetData(UOW_DATACONTEXT_KEY) == null)
					{
						PersonEntities context = new PersonEntities();
						UnitOfWorkStore.SetData(UOW_DATACONTEXT_KEY, context);
					}
				}
			}
			return (PersonEntities)UnitOfWorkStore.GetData(UOW_DATACONTEXT_KEY);
		}

		/*
		// This could be used if we wanted to add a method that disposed the current DataContext, and created a new one.
		private set
		{
			UnitOfWorkStore.SetData(UOW_DATACONTEXT_KEY, value);
		}
		*/
	}
}
Now you have an instance of your DataContext class that you can just access. It’s always there. It’s always the same one. But, you could still get rid of it and create a new one if you want, just modify that class to your liking. Now your queries can look like this:
var q =
	from people in PersonEntities.Instance.People
	orderby people.Lname
	select people;

I can’t tell you how much I love being able to just write PersonEntities.Instance in all of the methods in my data layer, knowing that it will “just work”. In console apps, web apps, and unit tests, it “just works”. I don’t know if you know this about me, but I’m a big fan of things that “just work”.

Adding a custom T4 template to auto generate your DTO classes

This is pure gold. With EFv2, your model wasn’t really easily exposed to the T4 templating engine, so auto generating code based on your model was difficult. There was a VS2008 plugin named SFS that sort of worked, but it didn’t have support for everything (like inheritance). I used it a lot for a while, and even emailed the sole maintainer to offer help, but alas, no response. The last release was over a year and a half ago. But that’s no problem now, as the EDMX model is now properly exposed to the T4 runtime with EFv4.

So why DTOs at all? They are small and simple. They serialize well. You can control exactly what data goes into them. You can flatten your model in them, if you wish, to send more specific or aggregated data. They aren’t tied to a framework. There are more pros (and certainly a list of cons somewhere too), but this isn’t the place for that discussion. Maybe read the wikipedia page.

Rather than show you the template (which you can download here if you want it), I’ll show you the output for my PersonDto.
[DataContract]
public  class PersonDto
{
	/// <summary>Parameterless constructor (important for serialization)</summary>
	public PersonDto()
	{}
	
	public string ToJson(bool format = false)
	{
		return JsonHelper.Serialize(this, format);
	}
	
	public Person ToEntity()
	{
		return AutoMapper.Mapper.Map<PersonDto, Person>(this);
	}

	[DataMember] public int ID { get; set; }
	[DataMember] public string Fname { get; set; }
	[DataMember] public string Lname { get; set; }
	[DataMember] public string Email { get; set; }
	[DataMember] public Nullable<int> AddressID { get; set; }
	[DataMember] public System.DateTime DateCreated { get; set; }

	[DataMember] public AddressDto Address { get; set; }
	[DataMember] public List<ProjectDto> Projects { get; set; }
}
Perfect! It even includes the FK properties from the model, which can come in handy. You’ll notice some references to JsonHelper and AutoMapper; we’ll get to that in a minute. Notice that the collection types are just lists of other DTO types. This means you can easily create an entire object graph, represented by DTOs. In fact, the way we’re about to set up AutoMapper, if you were to fetch an entire object graph, you can easily convert the whole graph to DTOs, already all wired up.
// Get the person with ID = 1, with his Address and Projects eagerly loaded
var q =
	from p in PersonEntities.Instance.People
		.Include(p => p.Address)
		.Include(p => p.Projects)
	where p.ID == 1
	select p;

Person person = q.SingleOrDefault();

// Convert to DTO
PersonDto dto = person.ToDto();

// All wired up...
Console.WriteLine(dto.Fname);
Console.WriteLine(dto.Address.City);
Console.WriteLine(dto.Projects[0].Name);
Nice. So now that our DTO classes are all auto-generated, let’s see how this AutoMapper thing works.

Auto generate the [entity to dto] and [dto to entity] AutoMapper configurations

If you haven’t ever heard of AutoMapper, that’s ok. It’s a great little object-to-object mapper. Normally, to convert an entity object to a dto object, I’d have to do this:
PersonDto dto = new PersonDto();
dto.Fname = person.Fname;
dto.Lname = person.Lname;
dto.Email = person.Email;
dto.Address = //....dangit, now I have to do this same line-by-line garbage for an address object.  Boo.
AutoMapper helps you do this automatically. It uses reflection, and if the properties are named the same in the 2 classes you’re mapping, you’re configuration is as eash as:
// configs for entity => dto
Mapper.CreateMap<Person, PersonDto>();
Mapper.CreateMap<Address, AddressDto>();
Mapper.CreateMap<Project, ProjectDto>();

// configs for dto => entity
Mapper.CreateMap<PersonDto, Person>();
Mapper.CreateMap<AddressDto, Address>();
Mapper.CreateMap<ProjectDto, Project>();

// Usage
Person person = q.SingleOrDefault();
PersonDto dto = Mapper.Map<Person, PersonDto>(person);
Nice and easy! Scroll up a bit and take a look at the auto generated PersonDto class. You see that ToEntity() method? It does the AutoMapper call for us, so we can just call ToEntity() on any old DTO object we have lying around. In a second, we’ll be adding ToDto() methods to our entity classes, again with the magic of another T4 template.

While it’s easy to write the configurations for mapping from entities to dtos, and from dtos to entities, why not auto generate them? This could be skipped if you have special mappings you want to set up, or if you want to deliberately exclude certain members, or somehow flatten your model, but if you just want a straight across mapping from entities to dtos, then check out this template. It creates code like this:
// AutoMapper config for Person => PersonDto
AutoMapper.Mapper.CreateMap<Person, PersonDto>()
		.ForMember(dto => dto.Address, options => options.MapFrom(obj => obj.AddressReference.IsLoaded ? obj.Address : null))
		.ForMember(dto => dto.Projects, options => options.MapFrom(obj => obj.Projects.IsLoaded ? obj.Projects : null))
	;
				
// Reverse config (dto => entity)
AutoMapper.Mapper.CreateMap<PersonDto, Person>();
Note the ForMember() calls. This prevents AutoMapper from triggering the lazy loading of navigation properties. For any EntityReference or EntityCollection types, it checks to see if the IsLoaded property is true. If it is, it returns that object. If not, null. If you are using the POCO generator to create your entity classes, then these lines will cause errors, since you won’t have EntityObject and EntityCollection types. You can just comment out the appropriate lines from the template.

Now that we’ve discussed AutoMapper, I’d like to point out that the T4 template that auto generates the DTO classes creates one additional class, named DtoExtensionMethods. It contains some extension methods for each IEnumerable<dto> type, so that you can convert a whole set of DTO objects to entity types all at once. This lets you do stuff like:
Person[] people = personDtos.ToEntities();
Very handy.

Auto generate additional code for your entity classes

There are dozens of tutorials and references on how to do this out there. This one is specific to the stack that I’m setting up here, to create the ToDto() methods for the entity classes, as well as the extension methods to convert an IEnumerable<entity> to DTOs, just like the extension method that did the reverse from the dto template. This template is small enough that I’ll just show it in its entirety here (download here):
<#@ template language="C#" debug="false" hostspecific="true"#>
<#@ include file="EF.Utility.CS.ttinclude"#><#@
 output extension=".cs"#><#

CodeGenerationTools code = new CodeGenerationTools(this);
MetadataLoader loader = new MetadataLoader(this);

string inputFile = @"../PersonEntities.edmx";
EdmItemCollection ItemCollection = loader.CreateEdmItemCollection(inputFile);
string namespaceName = code.VsNamespaceSuggestion();

EntityFrameworkTemplateFileManager fileManager = EntityFrameworkTemplateFileManager.Create(this);


#>
using System.Collections.Generic;
using Test.DataAccess.DTOs;


namespace <#= code.EscapeNamespace(namespaceName) #>
{
	public static class EntityExtensionMethods
	{
<#
	foreach (EntityType entity in ItemCollection.GetItems<EntityType>().OrderBy(e => e.Name))
	{
#>
		/// <summary>
    	/// Easily create DTO objects from actual domain entity objects
    	/// </summary>
    	public static <#=code.Escape(entity)#>Dto[] ToDtos(this IEnumerable<<#=code.Escape(entity)#>> entities)
    	{
    		return AutoMapper.Mapper.Map<IEnumerable<<#=code.Escape(entity)#>>, <#=code.Escape(entity)#>Dto[]>(entities);
    	}
		
		public static string ToJson(this IEnumerable<<#=code.Escape(entity)#>> entities, bool format = false)
		{
			return JsonHelper.Serialize(entities.ToDtos(), format);
		}
		
<#
	}	
#>
	}
}
<#


// ********************************************************************************
// *** Emit Entity Types **********************************************************
// ********************************************************************************
foreach (EntityType entity in ItemCollection.GetItems<EntityType>().OrderBy(e => e.Name))
{
    fileManager.StartNewFile(entity.Name + ".AutoGen.cs");
#>
using Test.DataAccess.DTOs;

namespace <#= code.EscapeNamespace(namespaceName) #>
{
	<#=Accessibility.ForType(entity)#> <#=code.SpaceAfter(code.AbstractOption(entity))#>partial class <#=code.Escape(entity)#> <#=code.StringBefore(" : ", code.Escape(entity.BaseType))#>
	{
		public string ToJson(bool format = false)
		{
			return ToDto().ToJson(format);
		}
		
		
		public <#=code.Escape(entity)#>Dto ToDto()
		{
			return AutoMapper.Mapper.Map<<#=code.Escape(entity)#>, <#=code.Escape(entity)#>Dto>(this);
		}
	}
}	

<#
} // end foreach entity

fileManager.Process();
#>
And the output (for just one of the entity classes, Address):
public partial class Address 
{
	public string ToJson(bool format = false)
	{
		return ToDto().ToJson(format);
	}
		
	public AddressDto ToDto()
	{
		return AutoMapper.Mapper.Map<Address, AddressDto>(this);
	}
}
Two convenient methods to have on each entity object.

Study those T4 templates well. You can add all kinds of useful logic to your entity and DTO classes, and the entire data model is at your disposal.

A few custom helper classes for use inside the data layer

I already talked about the UnitOfWorkStore, when we set up our ObjectContext class to have a singleton instance. So let’s take a look at the JsonHelper class. It’s very basic. It sets up a static readonly JsonSerializer (Json.NET), configures it how I like in the static constructor, and adds 2 methods to serializer and deserialize objects. By containing references to actual Json.Net classes to this single class, our code is a bit easier to read, and if we ever decided to switch serializers (say, to the DataContractJsonSerializer, though I don’t know why you’d want to), we would only have to update this one class. Here’s the code in full, or you can download it.
public static class JsonHelper
{
	private static readonly JsonSerializer s_serializer = new JsonSerializer();


	static JsonHelper()
	{
		s_serializer.Converters.Add(new IsoDateTimeConverter());
		s_serializer.MissingMemberHandling = MissingMemberHandling.Ignore;
		s_serializer.NullValueHandling = NullValueHandling.Include;
		s_serializer.PreserveReferencesHandling = PreserveReferencesHandling.None;
		s_serializer.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
	}


	public static string Serialize(object obj, bool format = false)
	{
		StringWriter stringWriter = new StringWriter();
		JsonWriter jsonWriter = new JsonTextWriter(stringWriter);
		jsonWriter.Formatting = format ? Formatting.Indented : Formatting.None;
		s_serializer.Serialize(jsonWriter, obj);
		return stringWriter.ToString();
	}


	public static T Deserialize<T>(string json)
	{
		StringReader reader = new StringReader(json);
		return (T)s_serializer.Deserialize(reader, typeof(T));
	}
}

Last up is the StronglyTypedIncludeHelpers class. (I like descriptive class names). I can’t take credit for this one, it’s a direct copy/paste, here’s the original. There are various versions of this kind of solution floating around, I like this one the best. Here’s the class, I won’t post the code, instead I’ll show you some examples of what it’s used for.

Entity Framework allows you to do eager fetching, meaning you can fetch related data for the entities that you’ll bringing back in your queries. If I wanted to get a person, but have his Address object already loaded, as well as his Projects collection, here’s how you do it.
var q =
	from p in PersonEntities.Instance.People.Include("Address.People").Include("Projects")
	where p.ID == 1
	select p;
I added one extra eager fetch in there; not only will we get the person’s Address, but that Address object will have its People collection populated with all of the people that are assigned to that address as well. I just did this to illustrate that you can specify more than one level in the association path when you call Include(). Now, as useful as Include() is, it’s brittle. What happens when someone changes the entity set name from People to Persons? Your code still compiles, but it will start throwing all kinds of runtime exceptions. You have to hunt down every reference to People, hiding as strings. Well, these new extension methods can solve that.
var q =
	from p in PersonEntities.Instance.People
		.Include(p => p.Address.Include<Address, Person>(a => a.People))
		.Include(p => p.Projects)
	where p.ID == 1
	select p;
For an association path that’s just 1 level, it’s great: p => p.Projects. Very nice. For paths that go multiple levels deep, it’s not as pretty. It seems that the type inference cannot be automatically deduced, so the generic type parameters have to be specified. It’s a little awkward the way that it has to be chained here, but I still prefer it to the brittle string method. If you changed an entity set name now, you’d get a compile time error. My favorite kind.

Wrapping up

This post ended up being much longer than I anticipated. It’s more of a treatise. I wanted to cover the whole data layer today though, so I can move on to the WCF REST service next time. I hope to get that done in one shot as well. But I’ll leave the automatic generation of the javascript service proxy class till after the WCF REST service (you’re going to love that by the way. It’s very cool…just point the JsProxyCreator.exe to your assembly with your [ServiceContract] classes, and presto! Instance js proxy class to call your services).

Here are all the files I went over in this post:
UPDATE: The files here all have updated, improved versions. Check out the files at the end of part 3 in the series.
CreateScript.sqlCreate the SQL tables for my contrived data model (3 tables, Person, Address and Project)
UnitOfWorkStore.csUsed with the DataContext singleton. This is an easy way to store things in HttpContext.Current.Items
PersonEntities.csThe DataContext singleton
DtosTemplate.ttT4 template to auto generate DTOs from your EDMX model, for all your entity classes
AutoMapperConfig.ttT4 template to auto generate AutoMapper configs to/from entities/dtos
EntityToDto.ttT4 template to create partial classes for your entity types, adding ToDto() and ToJson() methods
JsonHelper.csEncapsulate the usage of Json.NET library to this class. A helper class used all over (in the service layer too)
StronglyTypedIncludeHelpers.csStrongly typed Include() expressions, using lambdas, via these extension methods

These files will likely get updated as I find bugs and make revisions. I’ll try to keep a running change log at the end of this post, if it’s anything significant.

Up next, part 3: The WCF REST service.




Posted in Code | Tagged , , , , , | 6 Comments

Entity Framework v4, End to End Application Strategy (Part 1, Intro)

Sorry I haven’t posted in so long. Been very busy with new projects at work and a bit of side work. Ok, so I think I am finally moving on from NHibernate, at least for new development. Obviously I’m not going to do anything rash like completely rewrite the data layer of large and complex existing projects to use a new strategy…..right?

So rather than list my grievances with NHibernate, and the reasons behind my decision to move on, I just want to drop a quick sketch on how I’ll probably be designing systems in the future. I’m really excited about this. Here are the layers involved.
  • Phsical Database. SQL Server. No surprises here.
  • Data Access Layer — EFv4. Not using POCOs, but not lazy loading either. Lazy loading has been giving me headaches with serialization and AutoMapper. Plus it’s…lazy. You should know what you want, and eagerly fetch it.
  • DTOs (Data Transfer Objects). I’ve written some T4 templates based on the EF POCO templates to auto generate these classes from the edmx file. If you update your model, these are automatically updated too. Also (again with T4 templates), I’ve created AutoMapper configs to convert both ways (EF entity => DTO, DTO => EF entity).
  • Service Layer — WCF REST, with as little config as possible. This is based on the downloadable WCF REST templates in VS2010. Also, I’m using Json.NET to serialize/deserialize all of my DTO types, simply because it can handle cycles, and I like control over how dates are serialized.
  • Web UI. Still some decisions to be made here. I haven’t dived into MVC yet, but I’m not really using Web Forms either. Right now in my example solution, there’s nothing in the code behind files, I have no post backs, no server controls, and no viewstate. It’s all done in client script. I create javascript proxy classes to hit my WCF services, and do everything with jquery. All I’m getting out of ASP.NET is Forms Auth, and maybe some caching.

I’m loving it so far. I’ve been doing pretty serious ASP.NET development for the last 4 or 5 years. I know the page life cycle very well. But it’s time for me to try something new and different. The best part? This new approach is simpler, more direct, and more flexible. “Say what you mean, simply and directly”.

I have a simple app up and running with this stack already, and it’s fantastic. In future posts I’ll be focusing on each individual layer. Expect more soon.
Posted in Code | Tagged , , , , , , | 1 Comment

The Entity Framework (v1 and v4) Deal Breaker: TPT Inheritance

9/10/2010 Update #3: It appears that MS has deleted the bug on Microsoft Connect. Here is a cached version indexed from Google where you can see it has 34 upvotes, and 22 users saying they can reproduce. I blogged about this in January 2010, and the bug was opened on 3/17/2010. MS didn’t respond until 6/29/2010, when they said:
Sorry for not responding here. We are aware of performance issues with TPT hierarchies. We are currently investigating solutions and expect to make improvements in this area in a future release. This is definitely at the top of our query/perf optimizations list. I know this isn’t a great answer, and is not something that helps you today. Will keep you posted as we find out more, however. Hope this helps. Entity Framework Team
Apparently “keep you posted” means “we’ll delete your posts”. I don’t want to jump to any conclusions, but it is certainly very odd. Especially when you consider that they have mentioned the problem on their own blog (though they don’t nearly describe how bad it actually is).


3/17/2010 Update #2: I have submitted this problem as a bug on Microsoft Connect. If you’ve been bitten by this bug, vote it up (click the “I can too” link indicating that you can reproduce the bug), and leave some helpful comments.


3/17/2010 Update: I’ve put together a quick sample program that illustrates this problem, if you want to try it for yourself.
Download EFInheritanceTest.zip.
The data project has the included SQL scripts to create a simple EFTest database, with a base class table, some sub class tables, and a bit of dummy data.


I’ll keep the intro short and sweet. Everyone says you’ll eventually get burned by Microsoft data access strategies, because they eventually get abandoned. Well, my turn finally came. And it’s not that Entity Framework is being abandoned, it’s just that it DOESN’T WORK.
</intro>

Regarding inheritance mapping, EF advertises that it offers you 3 solutions:
  • TPT: Table per type
  • TPH: Table per hierarchy
  • TPC: Table per concrete class
I’ll be focusing on TPT: Table per type.

Let’s say I have a line of business application where users answer a bunch of Yes/No questions, and depending on their answers, fill out a series of forms, each form having a unique set of fields that we want to capture end-user data for.

Q: Did you open a bank account?
Yes
Name of bank:
Date account opened:
Purpose of account:
etc.

EVERY form has a common set of fields as well: The end-user that the particular instance belongs to, the date it was created, whatever. Now, when I have a User object, and I want to see all the forms he’s filled out, I don’t want to have to access 20 different collections of the User object, one collection for each form type:
User user = User.GetByKey(20);
user.NewAccountForms;
user.TaxForms;
user.ReplacedLimbsForms;
user.CentralServicesForms;

I want an IList<FormBase> or something similar. This means I need to use inheritance, and each form needs to be derived from my FormBase class (actually named FormInstance in this case).

If you need this type of design, or think that your software at some point in the future will require this type of design, DO NOT use the Entity Framework. The Table-Per-Type inheritance is 100% unusable. If you feel that one of the other forms of inheritance suits you, well, you may be crazy, but just the same….tread lightly. (And please don’t have 50 different discriminator values in a table that’s 200 columns wide…the SQL gods will strike you down).

So what’s wrong with EF’s TPT inheritance? The SQL that EF generates. With some simple testing (2 or 3 subclasses) everything may appear to be fine. In fact, you may even be impressed that EF manages to fetch all the data you need in a single trip, indeed, in a single query. Wow, their SQL generation must be top notch… But once you start adding some more subclasses, surprise! Your application grinds to a halt. So how bad is it? I decided to find out definitively.

Results

The setup is very basic. Here’s the Linq query I’m executing against my model:
	var q =
		from insts in context.FormInstances
		select insts;
	
	List<FormInstances> instances = q.ToList();
Very basic. Give me the FormInstances. FormInstance is the base class. I started with just a single derived class, ran my test 500 times, averaging the time it took to execute my simple query. Then I added a 2nd derived class, a 3rd one, etc etc. The base class has less than 10 fields. Each derived class (to keep things consistent) is exactly the same, with 3 simple fields. Also, the tables are empty. Yes, empty. No data! So the performance numbers here strictly represent the time it takes EF to generate its abhorrent SQL, and the amount of time it takes SQL Server to grind it out.

Most of these are averaged over 500 executions. Some of the later ones were fewer, because it started to take over an hour to execute.
Num Derived Classes Avg ms for query generation and execution Lines of SQL Generated
1 16 80
2 35 106
3 67 152
4 119 214
5 203 292
6 340 386
7 523 496
8 784 622
9 1137 764
10 1608 922
11 2198 1096
12 2958 1286
13 3900 1492
14 4828 1686
15 5962 1892
20 17412 3302
25 41206 5112
30 108807 7862
So what happens when your application has 30 subclasses? It takes 108 seconds to generate and execute the SQL to retrieve the records in them! That’s 3.6 seconds PER FORM! And that’s when the query returns ZERO RECORDS, because the tables are EMPTY. Why does it take so long? Well, look at the 7,862 lines of SQL it generates, to start with (I won’t post it here; I’ll spare you that).
Here’s what it looks like graphed:

Ouch! That sure doesn’t look linear folks… And guess what, my “I got burned” experience doesn’t have 30 form tables. It has 89. Based on the trend outlined above, it would take 97 minutes to generate and execute the SQL to select the data from those 89 tables. Funny, because 30 SQL statements querying those tables, even with WHERE fk IN(SELECT id FROM BaseTable) clauses on each one only take 132ms on the same computer…And it returns the same data.

Pragmatists vs Idealists

Joel Spolsky has some great words (scroll down to “The Two Forces at Microsoft”) regarding two developer camps at Microsoft; the Pragmatists, and the Idealists. Windows XP was a product of the Pragmatists; Vista, of the Idealists. Enough said. While the SQL generated by EF in cases such as these may call to the inner relational mathematician idealist in us, the result is impossible to read or easily understand, takes a long time to be generated, and even longer to execute. In other words, it’s 100% unusable.

Now, as a matter of research, I have to create this same test in NHibernate, to see what the generated SQL looks like, and to see how well it performs. I’ll update this post when I have that data. Until then, if you decide to use EF’s inheritance “features”, here is a list of movies that are all 97 minutes long, so that your end users will have something to do while your site grates off your SQL Server’s face from the comfort of their browsers.






Posted in Code | Tagged , , , , , , | 18 Comments

Eager fetch multiple child collections in 1 round trip with NHibernate

Oren Eini (Ayende), due to some of the comments on his various blog posts regarding EF vs NH that compare some of the different features, has finally put up a post demonstrating how to load a larger object graph in a single trip to the database. So far, all of the examples have been how to load a single collection of related entities (a one-to-many relationship), or a single related many-to-one entity, but never both at the same time.

In his latest post, he demonstrates how to load a root entity, whilst eagerly fetching TWO one-to-many collections. While NH still issues two separate queries, they are at least sent in a single trip, and honestly, the simpler SQL is both easier to understand, should you be debugging your application and running SQL traces, and the total execution time of the SQL is probably less, since there are no complicated subselects, unions, and other things going on to do the work in a single query.

However, I’m not done fighting the good fight. Ayende’s latest example still does not show how to fetch a large, complex object graph, with multiple branches coming off the root entity, and with multiple levels of collections down each branch. I’ve challenged anyone and everyone to accomplish this in NHibernate since the first “versus” posts were published. After a few timid responses suggesting I use FetchMode.Subselect (which doesn’t work) or Futures (which I haven’t seen accomplish what I’m asking for), everyone started to silently ignore the issue. When InfoQ brought up the same “versus” debate, I posted in the comments there as well, calling for anyone that could load large, complex object graphs in a single trip (hopefully in a single query), with NHibernate.

Don’t get me wrong, I’m not saying “put your money where your mouth is” to the NHibernate guys — on the contrary, I still prefer NHibernate to EF, and while I’m going to give EF4 a fair chance, I want to see NHibernate compete in this specific regard. I want to know how to do it, because I want to use it in my own code!


Posted in Code | Tagged , , , , , | 10 Comments

More EF4 vs NH Discussion

There’s a post on InfoQ following up the the Entity Framework 4 vs NHibernate (EF4 vs NH) discussion that Ayende started. I posted some of my findings regarding loading of large or complex object graphs in the comments. I have done more research in that vein (it turns out you CAN load a whole object graph in NH, as long as you map your collections to <set>’s instead of <bag>’s), but you may not like the performance results. You end up with a cartesian product of the records in every table that you end up joining to. NHibernate joins all of the tables flat, while EF combines subqueries, joins, unions, etc, to get JUST the rows you’re after, with no multiplication. However, unless you’re loading very large, complex graphs, with a lot of related entities and collections, you’ll probably be fine.

I set up the AdventureWorksLT2008 database, and have both an EF and a NH model hitting it, so I can do comparable queries, and look at the SQL generated by running a trace in SQL Server Profiler. When I get more of those results organized and coherent, I’ll post them here.


Posted in Code | Tagged , , , , , | Leave a comment

Eager Loading with Entity Framework and NHibernate

At CB Richard Ellis, Rich Alger and I wrote and maintain an internal ordering system that uses NHibernate. It was our first experience with NHibernate, but it was a huge success. Much of our initial focus was to just get things working, not to optimize it, since NHibernate has a bit of a steep learning curve. Now that we have more experience with it, we’re revisiting some of our strategies to improve them. My approach is this:
  • Set up a SQL Trace on the dev database
  • Load up the dev deployment of the ASP.NET app
  • Load a page I want to optimize
  • Count the number of queries going to the database as a result of loading that page
  • Optimize queries till I’m making the fewest db trips possible
Some pages were sending hundreds of queries! We had a lot of the SELECT N+1 problem going on, and some of those were pretty easy to fix. In general, we are trying to improve our loading strategies, to eagerly load everything we know we’ll need the FIRST time around, and to avoid the use of lazy loading, since it requires additional trips to the database. Lazy loading is great in dynamic scenarios, but if you already know you’re going to need something, don’t wait. Get it the first time around.

Which brings me to my current point. Eager loading with NH versus EF. Ayende recently asked what EFv4 can do that NH can’t. In a subsequent post he replies to the many responses he received. I have an ongoing discussion with him in the comments regarding eager fetching.

Suppose you had these tables:
User <1---*> User_x_Role <*---1> Roles <1---*> Role_x_Permission <*---1> Permission
(The reason for the associative tables is that I hate many-to-many relationships. They are evil).
These tables set up a basic security model. Certain operations and resources require that you have a specific permission. Permissions are assigned to roles, and roles are assigned to users. A user’s permissions, then, are inherited from the roles he is assigned to. Permissions are not directly assignable to users. You must use a Role.

These tables would result in a model like this:
UserThe User class
RoleThe Role class
PermissionThe Permission class
User.RolesAn IList<Role> that contains the Roles assigned to the User
Role.PermissionsAn IList<Permission> that contains the permissions assigned to the Role
Role.UsersAn IList<User> that contains all of the Users that are assigned this Role
Permission.RolesAn IList<Role> that contains all of the Roles are that are assigned this Permission
Checking if a User has a Permission may look like:

someUser.Roles.Permissions.Contains(somePermission);

So when fetching a user from the database, we may want to eagerly fetch his roles, and the permissions of those roles. With EF, I can do this:

using (MyEntities context = new MyEntities())
{
	int userID = 6;
	var qUser =
		from u in context.User.Include("Roles.Permissions")
		where u.ID == userID
		select u;
	User user = qUser.FirstOrDefault();

	// Roles is already populated
	int numRoles = user.Roles.Count;

	// As are the Permissions of each Role
	int numPermissions = user.Roles.First().Permissions.Count;
}

Note the use of ObjectQuery<T>.Include(string path). Note that it’s chainable. If you had some other related entities or collections of User that you wanted to eagerly fetch you could do this:
from u in context.User
	.Include("Roles.Permissions")
	.Include("Orders.OrderDetails")
	.Include("Employer")
	.Include("ContactInfo")
select u;
Basically, you have complete control over what data will be eagerly fetched, and you’ll get back the exact object graph you want, with all of the collections populated, and related entities loaded. If you think that’s impressive, fire up SQL Profiler and do a trace on your database. All of the data for all of those entities and collections is retrieved in one query. It may be full of subselects, joins, unions, etc, but it’s one query. One trip to the database, to issue one query, and you have all the data you want. Because of the ease of use of ObjectQuery<T>Include(string path), it’s easy to make methods on your entity classes that give you a LOT of flexibility when it comes to getting the data you want:
		public  static IList<Affiliate> GetPage(int pageIndex, int pageSize,
			string where, string orderBy, out int rawRecordCount, params string[] includes)
		{
			// Start with a basic ObjectQuery<T> that would just return ALL rows (no criteria or sorting)
			ObjectQuery<Affiliate> query = Jam2DataLayer.Instance.Context.Affiliate;

			// Apply the where-clause if they provided one.
			if (!String.IsNullOrEmpty(where))
				query = query.Where(where);

			// Apply the orderby-clause if they provided one
			if (!String.IsNullOrEmpty(orderBy))
				query = query.OrderBy(orderBy);
			else
				// If no sort-by clause was provided, sort by the ID desc (show newest records first)
				// (There has to be sorting applied for paging to work with Skip() and Take())
				query = query.OrderByDescending(ent => ent.ID);

			// Execute the query at this point, but only to retrieve a count of the records it returns.
			// We want to know the number of records our query WOULD return, if it weren't paged.
			rawRecordCount = query.Count();

			// Set up any eager loading that was specified
			foreach (string include in includes)
				query = query.Include(include);

			// Get results just for the page we're interested in.
			query = query.Skip(pageIndex * pageSize).Take(pageSize);

			// Execute the query, storing the results in a List<T>
			List<Affiliate> list = query.ToList();
			return list;
		}


		// Example usage: Get the first 10 active Affiliate objects, with their User entity eagerly loaded
		int totalRecords;
		IList<Affiliate> activeAffiliates = Affiliate.GetPage(0, 10, "it.Active == true", null, out totalRecords, "User");
Definitely a handy method to have on each of your entity classes (note: in an upcoming post, I’ll be showing my T4 templates that use the SFS Plugin for Visual Studio 2008 to generate just such code for each of your entity classes).

Now, can this be done in NHibernate? Honestly, I don’t know yet. I’ve been trying. My first attempt went something like this:
public virtual IList<T> GetPage<T>(int pageIndex, int pageSize, params string[] eagerFetches)
{
	ICriteria crit = CurrentSession.CreateCriteria<T>();
	crit.SetFirstResult(pageIndex * pageSize);
	if (pageSize > 0)
		crit.SetMaxResults(pageSize);

	// Now set up the eager fetching for whatever the caller specified
	foreach(string entity in eagerFetches)
		crit.SetFetchMode(entity, FetchMode.Eager);

	return crit.List<T>();
}
That seemed to work! GetPage<User>(0, 10, “Roles”) DID indeed fetch 10 users, with their Roles collections already populated. Then I tried “Roles.Permissions”. No dice! The data that comes back is correct, but it issues 10 separate queries. It’s still lazy loading. And, when you start to include multiple collections for the root entity, as in GetPage<User>(0, 10, “Roles”, “Orders”), you start getting Cartesian products, producing millions of rows sometimes. NH doesn’t sort it out. This question on StackOverflow has an answer that quotes from the NHibernate In Action book: From “NHibernate in Action”, page 225: NHibernate currently limits you to fetching just one collection eagerly.

Perhaps that is still the case. I mentioned this on Ayende’s blog, suggesting that perhaps this is an area where EF has the features in tow, and NH is still lacking. He told me that FetchMode.Eager is the same as FetchMode.Join, which would be the reason for the cartesian products. He said to look at FetchMode.Subselect, and Future queries. Well FetchMode.Subselect can only be set in the mapping file, it can’t be set arbitrarily on an ICriteria (the NHibernate.FetchMode enum does not have a field named Subselect), so that’s out. I want to leave my mappings as lazy, and specify eager only when I want it, on a per query basis. As for the Future<T> queries, I haven’t been able to get them to do what I want either. I might be able to get individual lists back, that all contain the related entities I’m interested in, by delaying the query to the database, storing the queries in an IMultiCriteria, but that isn’t going to give me a connected object graph, just disparate lists. I’ll keep poking around with different ways to hopefully accomplish the same thing in NH. For now, hopefully the discussion stays alive in the comments on his blog. I’d love to find a way to do this in NH, as it’s a powerful feature. I have a feeling that even if we conjure something up, it’s not going to be the same as what EF does, issuing only a single query to the database. Time will tell.



Posted in Code | Tagged , , , | 3 Comments

Making Entity Framework (v1) work, Part 1: DataContext lifetime management

EFv1 is sorely lacking in many areas. That doesn’t seem to stop people from using it, however. If EFv2 weren’t on the way (March 2010), I’d probably still be using NHibernate for everything. However, EFv2 (actually named v4 to “align” it with the .NET version number, or as a marketing ploy to make it seem more mature than it is) makes up for most of the deficiencies in v1, such as lazy loading and mapping to POCOs. Whatever the case, if you’re currently using v1, then here are some good ways to “make it work”.

DataContext lifetime management

When and where should you create the DataContext? Rick Strahl has an excellent post that explains the problems, and he offers an excellent solution. Rather than re-explain everything in his post, I’m just going to show you my solution.

A singleton that uses a static backer would be a bad idea, because in an ASP.NET application, you’d end up using the same DataContext for everyone, and you’d run into a host of problems. However, I DO like what the singleton pattern offers me: a global way to uniformly access an object, from anywhere I want. I don’t want to have to pass around a DataContext object everywhere I go. We just need to fix up our singleton so that it isn’t using a static backer. What I want is a DataContext per user, per post back. For this, we can use HttpContext.Current.Items which is a bag where you can store any object by key (string), and it’s unique per user, and it’s lifetime is limited to a single request/response cycle.

However, I don’t want to ONLY use HttpContext.Current.Items, because if I’m running a console application, or unit tests, HttpContext.Current is null, and I’ll end up with a NullReferenceException. So here’s the UnitOfWorkStore class I use to manage things like this:
	/// <summary>
	/// Utility class for storing objects pertinent to a unit of work.
	/// </summary>
	public static class UnitOfWorkStore
	{
		/// <summary>
		/// Retrieve an object from this store via unique key.
		/// Will return null if it doesn't exist in the store.
		/// </summary>
		/// <param name="key"></param>
		/// <returns></returns>
		public static object GetData(string key)
		{
			if (HttpContext.Current != null)
				return HttpContext.Current.Items[key];
			return CallContext.GetData(key);
		}


		/// <summary>
		/// Put an item in this store, by unique key.
		/// </summary>
		/// <param name="key"></param>
		/// <param name="data"></param>
		public static void SetData(string key, object data)
		{
			if (HttpContext.Current != null)
				HttpContext.Current.Items[key] = data;
			else
				CallContext.SetData(key, data);
		}

	}

I like this class because it “just works”, whether it’s an ASP.NET project, console app, or a unit test. However, if you are developing a GUI application, I would do some more research. While this class works, you don’t want to end up storing a ton of things in CallContext. You’ll want to find a more advanced unit of work class that gives you more control over scope and lifetime. My console apps that use my EF model run and terminate quickly, as do my unit tests, so the simple class above is perfect for me.

Now that we have a way to store things per user, per post back, let’s create a DAL or DataLayer class that will act as a singleton for our DataContext.

	/// <summary>
	/// This is the data access layer management class.
	/// </summary>
	public partial class MyDataLayer
	{
		/// <summary>
		/// This is our key to store an instance of this class in the <see cref="UnitOfWorkStore" />.
		/// This is used in the <see cref="Instance" /> property.
		/// </summary>
		private static readonly string UOW_INSTANCE_KEY = "MyDataLayer_Instance";

		/// <summary>
		/// This is used for thread-safety when creating the instance of this class to be stored in
		/// the UnitOfWorkStore.
		/// </summary>
		private static readonly object s_objSync = new object();

		// The DataContext object
		private readonly MyEntities m_context;



		// ********************************************************************************
		// *** Constructor(s) *************************************************************
		// ********************************************************************************

		/// <summary>
		/// Default constructor.  Creates a new MyEntities DataContext object.
		/// This is hidden (private) because the instance creation is managed as a "unit-of-work", via the
		/// <see cref="Instance" /> property.
		/// </summary>
		private MyDataLayer()
		{
			m_context = new MyEntities();
		}



		// ********************************************************************************
		// *** Public properties **********************************************************
		// ********************************************************************************

		/// <summary>
		/// The ObjectContext object that gives us access to our business entities.
		/// Note that this is NOT static.
		/// </summary>
		public MyEntities Context
		{
			get { return m_context; }
		}


		/// <summary>
		/// This will get the "one-and-only" instance of the MyDataLayer that exists for the lifetime of the current "unit of work",
		/// which might be the lifetime of the currently running console application, a Request/Response iteration of an asp.net web app,
		/// an async postback to a web service, etc.
		/// 
		/// This will never return null.  If an instance hasn't been created yet, accessing this property will create one (thread-safe).
		/// This uses the <see cref="UnitOfWorkStore" /> class to store the "one-and-only" instance.
		/// 
		/// This is the instance that is used by all of the DAL's partial entity classes, when they need a reference to a MyEntities context
		/// (MyDataLayer.Instance.Context).
		/// </summary>
		public static MyDataLayer Instance
		{
			get
			{
				object instance = UnitOfWorkStore.GetData(UOW_INSTANCE_KEY);

				// Dirty, non-thread safe check
				if (instance == null)
				{
					lock (s_objSync)
					{
						// Thread-safe check, now that we're locked
						if (instance == null) // Ignore resharper warning that "expression is always true".  It's not considering thread-safety.
						{
							// Create a new instance of the MyDataLayer management class, and store it in the UnitOfWorkStore,
							// using the string literal key defined in this class.
							instance = new MyDataLayer();
							UnitOfWorkStore.SetData(UOW_INSTANCE_KEY, instance);
						}
					}
				}

				return (MyDataLayer)instance;
			}
		}
	}

Note that the constructor is private, so you can’t directly instantiate this class. You have to go through the static “singleton accessor”, MyDataLayer.Instance. Where this differs from most singleton implementations is that we don’t use a private static field to hold our singleton instance, we use the UnitOfWorkStore class. Now anywhere in your code, you can refer to MyDataLayer.Instance.Context, and you’ll have a DataContext object that is unique to the current ASP.NET user, and that will exist only during the current request/response, and is then discarded. Handy!

Up next: code generation with T4 templates that can see your EDMX model (using the SFS plugin for Visual Studio 2008) to get methods in each of your entity classes like:
  • GetById(int id) (or Guid id, or whatever the type of your PK is, the code generation will do it correctly)
  • GetById(int id, params string[] includes)
  • GetPage(int pageIndex, int pageSize, string where, string orderBy, out int rawRecordCount, params string[] includes)
  • And more…



Posted in Code | Tagged , , , , | 2 Comments