Infovark Underground » WCF

Looking for an IIS Alternative

Dean — Tue, 01 Dec 2009 17:43:14 +0000

One year ago it became clear that Infovark had outgrown the Windows Communication Foundation (WCF).

We’d decided to use WCF because we wanted Infovark to provide web services, and we liked the fact that we could deploy WCF to client machines. Since WCF is built directly on top of HttpListener, a core part of the Microsoft .NET Framework, we wouldn’t need to use System.Net or Microsoft IIS.

But we’ve struggled with WCF for a variety of reasons. First, we wanted to use a REST model for our web services, and WCF’s support for REST architectures lags behind its SOAP support.

Second, there’s no easy way to return HTML from WCF. We tried transforming our XML with XSLT and returning the XHTML results as a Stream. This works, but the programming experience is frustrating.

Last, because of the previous two reasons, we were left with a website that was way too rigid and programmer-like. It didn’t feel like an organic website. The tool we’d picked was forcing us to compromise on our website design goals.

Infovark’s primary mission is to help human beings, not other computers. That means that the look and feel of the web interface should be our number one priority. Awesome web services are nice to have, but happy users are more important.

Web server alternatives

So for the past few months, we’ve been hunting for an alternative web server. We can’t use IIS because its footprint is too heavy. Most IT departments won’t allow us to install IIS on client machines.

We could use Apache. It has a nice embeddable version, but interacting with it via C# is tricky. We’d prefer something a little more Microsoft-native.

That basically leaves us with one commercial option and two open source options.

UltiDev Cassini is a commercial product that’s been around for some time. We’re sure it could do the job, but the licensing model is cost prohibitive.
C# WebServer is an open source project on CodePlex. It’s been around for two years, but the pace of development seems slow.
Open Rasta is the brain child of Sebastien Lambda, an open source framework for the development RESTful development of web sites and services. It’s been getting a lot of attention recently.
Kayak is a promising open source project, but it hasn’t reached its first public release yet.

(If you know of other web servers worth investigating, please let us know in the comments!)

Making the switch

More important than picking an alternative web hosting framework for Infovark is the timing of the switch. We don’t want to impede future development.

As a stopgap, we might try plugging in the Spark View Engine to replace our current XML-XSLT-XHTML rendering path. Who knows? If it improves our web development flow, we might be able to keep our WCF base after all.

Visual Studio 2008 and its CopyLocal setting

Dean — Tue, 31 Mar 2009 01:08:38 +0000

Our build server wasn’t producing the same results as our local development machines. The same C# code yielded two different sets of code libraries. After much head-banging and hand-wringing, we finally traced the problem to an obscure error in the way Visual Studio 2008 handles .csproj files.

Basically, the Visual Studio IDE doesn’t handle the CopyLocal setting properly. This post from paraesthesia set us on the right track.

I have no idea what caused the problem. We had the same versions of the .NET Framework and Visual Studio 2008 installed on all our machines. The code was exactly the same. Apart from the build machine being Windows Server 2003 and out development machines being Windows XP, I can’t think of any other differences. But somehow the code libraries weren’t getting copied in the same way.

Just another adventure on our way toward the Beta release…

WCF WebInvoke Body Format Error

Dean — Sun, 08 Mar 2009 20:24:48 +0000

I spent Sunday afternoon battling an odd WCF error.

System.InvalidOperationException: Incoming message for operation [your operation here] contains an unrecognized http body format value ‘Xml’. The expected body format value is ‘Raw’. This can be because a WebContentTypeMapper has not been configured on the binding. See the documentation of WebContentTypeMapper for more details.

Troubleshooting this issue sucked. Here’s the deal: WCF helpfully attempts to parse any incoming “text/xml” requests automatically. If you’ve defined the input parameter on a WebInvoke operation to be a Stream, WCF can’t bind to the method and returns an HTTP 400.

There’s two ways to solve this problem: Change the input parameter from Stream to XmlElement, or configure WCF to treat this request as Raw. I picked the former method. Carlos Figueira explains the latter method.

If folks are interested, I can post some more detail about the problem and the resolution. For now, I have to finish making things work.

Edit: So here’s the rest of the story, since Brad asked.

I wasn’t sure exactly what triggered the issue. I got this behavior with just one WebInvoke and one WebGet operation using the same URI template. What I’d done was to create a generic ObjectService that exposed the same RESTian operations for several different types of objects. The particular operation in question looked something like this:

[OperationContract]
[WebGet(UriTemplate = ItemUris.IndexXml, RequestFormat = WebMessageFormat.Xml, ResponseFormat = WebMessageFormat.Xml)]
Stream GetListAsXml();
[OperationContract]
[WebInvoke(UriTemplate = ItemUris.IndexXml, Method = "POST", BodyStyle = WebMessageBodyStyle.Bare, RequestFormat = WebMessageFormat.Xml, ResponseFormat = WebMessageFormat.Xml)]
Stream PostListAsXml(XmlElement input);

Originally, PostListAsXml had accepted a Stream. That seemed to work in other places, but then I started noticing the InvalidOperationException messages.

I think the right way to solve this problem is to follow Carlos’ advice, and create a new WebContentTypeMapper-derived class. But I didn’t have time to figure out exactly where to plug it in, and I was afraid that I might introduce other problems. I just didn’t know enough about the inner workings of WCF to know whether that was a safe operation.

Since we hadn’t shipped the interface yet, I was free to change the return type from Stream to XmlElement for the one or two WebInvoke operations that were returning errors. Fewer lines of code needed to change, and I knew I wouldn’t break anything else.

Of course, I’m probably just setting myself up for more pain down the road, but sometimes you just need to get things done, y’know?

Anyway, since I’ll likely revisit this decision in a later release, I’d love to hear what other folks did in this situation.

Using WCF to return HTML

Dean — Wed, 18 Feb 2009 15:28:22 +0000

I just answered a WCF question on StackOverflow, and decided it was worth cross-posting here as well.

The question was: What is the best / most flexible way to have WCF output XHTML?

Here’s how we do it at Infovark. While I’m not sure that our approach is the best way, it does the job.

Our approach is to use the DataContractSerilizer to generate XML, then apply a Complied XSLT transform and return the result stream, which should now contain XHTML. Here’s a simplified version of our code:

public Stream GetItemAsHtml(string id) {
Item obj = GetItem(objectId);
Stream xml = GetXmlStream(obj);
return TransformXmlStream(xml, defaultTransform);
}
public static Stream GetXmlStream(IXmlSerializable item) {
MemoryStream stream = new MemoryStream();
using (XmlWriter writer = XmlWriter.Create(stream, new XmlWriterSettings { Encoding = Encoding.UTF8 })) {
if (writer != null) {
DataContractSerializer dcs = new DataContractSerializer(item.GetType());
dcs.WriteObject(writer, item);
writer.Flush();
writer.Close();
}
}
stream.Seek(0, SeekOrigin.Begin);
return stream;
}
public static Stream TransformXmlStream(Stream xml, string xsltFile) {
XmlReader reader = XmlReader.Create(xml);
XslCompiledTransform trans = new XslCompiledTransform();
trans.Load(xsltFile);
MemoryStream stream = new MemoryStream();
using (XmlWriter writer = XmlWriter.Create(stream, trans.OutputSettings)) {
if (writer != null) {
trans.Transform(reader, writer);
writer.Flush();
writer.Close();
}
}
stream.Seek(0, SeekOrigin.Begin);
return stream;
}

It works for us, but if you’ve got other, better ideas, please let me know!

Put the “Web” Back in Web Services

Dean — Thu, 05 Feb 2009 18:52:32 +0000

I know it’s possible to transmit Internet Protocol by carrier pigeon, but I’m not sure I could recommend it to our customers with a straight face. By the same token, I’m continually surprised to hear vendors and consultants insist that Web Services can be done without the web.

Yet I’ve seen that claim repeated all over the Internet. Sure, it’s true in the academic sense. Technically, there’s nothing wrong with “webless” web services, just like IP by avian carrier will eventually get the job done (though latency and packet loss are a challenge). But practically speaking, why would you use anything other than HTTP?

The “webless web services” meme has infected a number of good ideas. The SOAP standard is a case in point. At first, SOAP was a simple XML wrapper around an XML payload. (Remember when the S in SOAP stood for Simple?) Then a variety of industry heavyweights jumped on the bandwagon, and soon SOAP could be used with any protocol.

With, uh, a few changes. Like five or six additional specifications. And schema. Lots of schema. Maybe a diagram would help? No?

Why not use REST instead?

Nicholas Allen recently shared a great presentation on REST and SOA given by Stefan Tilkov at QCon. Stefan makes some great points about how RESTful web service design aligns well with the goals of Service Oriented Architecture (SOA).

Sure, you could use the WS-* stack if you like, but RESTful web architecture is a proven, scalable and truly simple approach, as long as you don’t mind having to use HTTP.

Much of the added plumbing in the WS-* stack is there to help transition older computer-to-computer network communication technologies. It allows applications that depend on CORBA or DCOM to tunnel through firewalls. But unless you’re trying to retrofit a system built before, say, 1998, you can skip all that stuff.

Wasn’t using web protocol the whole point of web services anyway? As Stefan said in his closing comments: “Protocol independence is a bug, not a feature.”

Use a RESTful architecture for new development. Put the web back in web services.

Using WCF for REST, Part 3

Dean — Tue, 27 May 2008 15:44:52 +0000

The easiest way to explain the issues we encountered when implementing REST is to work through the design principles we followed. I think much of our trouble came from the fact that we come from a web applications background, not a SOAP services background. I’m hoping that by laying out our REST design, some of the Microsoft gurus can help us do things “the WCF way.” And perhaps we can help the WCF team out by highlighting a handful of places where we found WCF unintuitive.

The Importance of Being Addressable

Fundamentally, the REST pattern is about making resources available. This means that each item stored within your system can be accessed by someone with the correct permissions. Every one of these items has its own unique address, and its address should not change. This consistency is important, because it allows both people and computer programs to remember and reference items in your system.

Note that we’re talking about resources or items. In contrast with the SOAP model of web services, which allows programmers to invoke procedures on remote computers, REST is about providing data. SOAP is about verbs, while REST is about nouns. A SOAP service might CalculateTotalSale(); a REST service provides CustomerRecieptNo_12345. The kind of web services architecture you use will depend on the kind of application you’re building. The choice has major implications for the other components of your system.

REST imposes restrictions on what sort of things you can do, because it supports only a handful of actions: GET, POST, PUT, and DELETE. (There are a few other HTTP methods, but these four are the most important.) Fortunately, with these four actions, you can accomplish most basic programming tasks. There’s a close parallel to these actions and create/read/update/delete, or CRUD, the building blocks of data storage systems.

UriTemplate

Since the address, or URI, is the primary way to access information in your system, it’s effectively part of your user interface. All the principles of good user interface design apply. So when designing a REST service, you need fine control over the structure of these identifiers.

SOAP, by contrast, typically has just one endpoint. The address itself conveys no information about what services are provided — that’s why SOAP services require a separate WSDL file to tell folks what’s possible. With REST, it should be easy to discover the extent of the system by looking at the URIs alone.

Coming up with good REST URI patterns can be tricky. Using short, descriptive naming conventions for your resources makes them easier to type. But URI patterns must also be distinct and unambiguous.

In the .NET framework, you use the UriTemplate class to define patterns. The UriTemplate implementation that shipped with .NET 3.5 allowed you to define variables that fit into slots in your URI. A typical UriTemplate might look like this: http://restserver/{object}/{id}?view={viewname}.

WCF looks for incoming URIs that match the patterns you define. The pattern defined above would match the following URIs:

http://restserver/customer/5?view=profile
http://restserver/article/how_to_do_stuff?view=print
http://restserver/author/John-Smith?view=1

Once you’ve defined a UriTemplate, you bind it to a method that has the same number of parameters. (I won’t go into the ABCs of WCF here, but you can check out this MSDN Introduction to WCF if you need a refresher.)

In WCF 3.5, you could only define a variable for a whole segment. A segment is basically the bit between the one forward slash and another, or one querystring parameter. A few bloggers requested more flexibility in UriTemplates, and the WCF team answered with the soon-to-be-released 3.5 SP1. The ability to define variables for partial segments was crucial for our URI design.

Representation Matters

Most books about web services, including RESTful Web Services, advocate leaving off file extensions from your URIs. This makes sense for SOAP, where you’re accessing methods and all responses are transmitted in XML. But in REST, you’re serving up items.

In our case, some of these items being served were files and some were records from a database. It seemed inconsistent to have some endpoints that had file extensions and others that didn’t. And we also wanted to be able to serve up different representations for our database records. Our REST service supports both JSON, XML, and HTML. It made sense to use a file extension to distinguish between the different representations.

One workaround would have been to create endpoints like http://restserver/form/1040/xml but that looked funny next to URIs like http://resterver/file/documentation.pdf. True RESTafarians would point out that neither the “/xml” or the “.pdf” are needed, since you can request an appropriate representation using the HTTP ACCEPT header. We decided against the header approach because not all browsers use the ACCEPT HTTP header. Besides, it might be useful for us humans to be able to reach alternate formats by simply changing the URL in the browser address bar.

In WCF 3.5, this required us to create three times the number of endpoints, with a separate method to handle each. We can’t wait for the official release of 3.5 SP1 to make UriTemplates like http://restserver/user/{id}.{ext} possible.

The Final Slash

Another source of endpoint duplication was the need to have two different endpoints for http://restserver/folder and http://restserver/folder/. Because the slash is used as a segment delimiter, the dispatcher in WCF 3.5 saw these two URLs as fundamentally different.

So handling what we thought were fairly trivial cases in URI patterns led us to create FIVE TIMES the number of endpoints we wanted. It’s a maintenance nightmare. SP1 can’t get here soon enough.

WCF Instance Context

Dean — Wed, 21 May 2008 22:31:19 +0000

I finally figured out the source of my HTTP 400 problem. Apparently the Windows Communication Foundation deals with exceptions differently depending on your InstanceContextMode settings. I had been using the Single setting but I should have used the PerCall setting. In PerCall mode, the try/catch block works as expected.

I think it has something to do with the way that WCF distinguishes between channel exceptions and message faults.

Anyway, if you’re building a REST web service, you’ll want to make sure your class is decorated with the following ServiceBehavior attribute.

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]

Using WCF for REST, Part 2

Dean — Sat, 17 May 2008 18:16:53 +0000

In part one of this series, I listed several websites and blogs that had useful information on the Windows Communication Foundation (WCF) and REST. I also mentioned that if I was stating again, I’d probably use something other than WCF. Perhaps deriving my own REST server from System.Net.HTTPListener, for example.

Vish asked for some additional details in his comment to that post. He works on the Microsoft WCF development team and was curious about our experience.

I had just begun putting together my response when I noticed Scott Guthrie’s post about Service Pack 1 for the .NET Framework 3.5 beta release. Steve Maine also posted specifics about the ADO.Net Data Services and WCF changes.

So, Vish, it seems your team’s beaten me to the punch on some of these issues! Many of the difficulties I was having with WCF and REST were addressed by the service pack. Here’s an overview of our key stumbling blocks:

REST requires much greater control over the URI than SOAP does, and the URITemplate class just wasn’t up to the task. We had to hardcode most of our endpoints to compensate. (Fixed in SP1. Hooray!)
Supporting multiple formats, such as serving both XML and JSON, either require you to program against Stream or require twice the number of endpoints.
Existing serializers had trouble with complicated object graphs, forcing us to perform serialization/deserialization ourselves. (This seems greatly improved in SP1.)
WCF allows only one contract/interface per endpoint. This makes it tricky to factor out common contract patterns.
Good REST practice would have you return many kinds of errors as HTTP status messages. SOAP embeds all error information in the returned XML. WCF is closely aligned with the SOAP approach, which means that you’ve got to be very careful distinguishing exceptions from faults when implementing REST in WCF. It was an unpleasant surprise, and we had to do quite a bit of work to deal with that.

I’ll talk about all five of these areas in more detail in upcoming posts in this series. And I’ll be sure check out the SP1 beta once we get our Infovark Alpha release out the door.

Using WCF for REST, Part 1

Dean — Tue, 22 Apr 2008 15:51:56 +0000

Just because you can do something doesn’t mean it’s a good idea.

We decided to use the Windows Communication Foundation to drive our REST-based web service. In hindsight, it was a poor choice. REST support in WCF seems like it was a last-minute addition to .NET 3.5. You can certainly hack something together, but I’ve found few real-world examples on the Internet, and most of those sidestep the tricky issues.

Here’s the short of it: WCF was designed for RPC-SOAP. More importantly, it was designed to SOA-enable legacy services that used older communications channels like DCOM. If you’re starting from scratch, and have full control over the output of your web service and the design of your object model, I’d recommend using a different (and simpler) framework.

We’ve gotten a good bit of blog traffic from people looking for help with Windows Communication Foundation and the REST architectural pattern. (It’s good to know that we’re not the only ones needing advice.) Here are the better sources we’ve found so far.

Windows Communication Foundation documentation on MSDN

Good overview presentation on REST and Syndication using WCF

Microsoft’s Picture Services Sample

Justin Smith’s WCF articles on Cybertopian Chronicles

Nicholas Allen’s Indigo blog

Steve Maine’s blog

Assorted posts on Rick Strahl’s blog

That Indigo Girl

If you find other useful places to look, let us know!

WCF Bad Request

Dean — Wed, 09 Apr 2008 23:27:32 +0000

I’ve just identified a horrible bug in WCF for the .NET Framework 3.5.

A caught exception in a WebInvoke operation will cause WCF to return an HTTP 400 Bad Request status code to the client. Any caught exception. Every time. Regardless of whatever error code you might want to send back.

I found the error by mistake. I’d used “BadGateway” instead of “BadRequest” in my code. If it weren’t for other odd WCF behavior, I wouldn’t have noticed that my status code was being ignored.

Consider the following example:

// Read the Xml into our object and save.
try
{
// The following line triggered the error.
obj.ReadXml(reader);
obj.Save();
// Set HTTP Cache Options and MIME Type.
Utilities.SetCaching(WebOperationContext.Current, obj.DateModified, 60);
Utilities.SetMimeType(Format.Xml);
return Utilities.GetXmlStream(obj);
}
catch (Exception e)
{
// Was it a schema validation error? If so, provide detail.
if (!string.IsNullOrEmpty(_XmlValidationErrors))
{
// I slipped here, using BadGateway 502 instead of Bad Request 400.
// But WCF doesn't care. If you enter the catch block it's _always_ 400.
WebOperationContext.Current.OutgoingResponse.StatusCode = HttpStatusCode.BadGateway;
WebOperationContext.Current.OutgoingResponse.StatusDescription = _XmlValidationErrors;
WebOperationContext.Current.OutgoingResponse.SuppressEntityBody = false;
return null;
}
}

If no error occurs, WCF will return the status code you specify. A try/finally block will work just fine; WCF returns whatever status code you specify. Enter a catch block, though, and it’s nothing but 400 Bad Request.

Hey, if there’s an error, it must be the client’s fault, right?