Archive for the ‘REST’ Category

REST: How to respond to an HTTP POST

Dean 24 Mar, 2009 1 Comment

I ran into a problem yesterday. I’d sent a HTTP POST request to a collection of resources on our RESTful web service. Our server responded with an HTTP 201: Created status code and the URI of the new resource in the Location header.

And then… nothing happened.

This was not what I was expecting. I expected my web browser to follow up with a GET request to the URI I’d provided. But Firefox 3 wasn’t biting. A problem with Firefox? I checked in IE8 and Google Chrome and got the same behavior.

Had I misread the HTTP spec? Did I misunderstand the REST pattern? I grabbed for my worn copy of RESTful Web Services. Nope. HTTP 201 seemed to be the right status for this situation.

Had I blundered into some common error? I checked Stefan Tilkov’s useful list of REST Antipatterns. But I couldn’t find anything that quite matched my situation.

I started Googling, but couldn’t find much apart from this question about HTTP Post on Stack Overflow. There were some cryptic responses (to which I’ve added my own answer now).

Eventually, I discovered what I needed to know from Ben Ramsey’s article on HTTP redirection. It’s part of his series discussing RFC 2616, which describes the HTTP/1.1 protocol.

The answer is that while web service clients will often “take the hint” provided by a HTTP 201: Created response, web browsers won’t. If you actually want a web browser to go somewhere else, you need to send a status code in the 3xx series. In this situation, the status code you want is HTTP 303: See Other.

Once I changed the status code returned by the server, all the web browsers followed up the response with a GET to the new URI.

Using WCF to return HTML

Dean 18 Feb, 2009 1 Comment

I just answered a WCF question on StackOverflow, and decided it was worth cross-posting here as well.

The question was: What is the best / most flexible way to have WCF output XHTML?

Here’s how we do it at Infovark. While I’m not sure that our approach is the best way, it does the job.

Our approach is to use the DataContractSerilizer to generate XML, then apply a Complied XSLT transform and return the result stream, which should now contain XHTML. Here’s a simplified version of our code:

public Stream GetItemAsHtml(string id) {
Item obj = GetItem(objectId);
Stream xml = GetXmlStream(obj);
return TransformXmlStream(xml, defaultTransform);
}
public static Stream GetXmlStream(IXmlSerializable item) {
MemoryStream stream = new MemoryStream();
using (XmlWriter writer = XmlWriter.Create(stream, new XmlWriterSettings { Encoding = Encoding.UTF8 })) {
if (writer != null) {
DataContractSerializer dcs = new DataContractSerializer(item.GetType());
dcs.WriteObject(writer, item);
writer.Flush();
writer.Close();
}
}
stream.Seek(0, SeekOrigin.Begin);
return stream;
}
public static Stream TransformXmlStream(Stream xml, string xsltFile) {
XmlReader reader = XmlReader.Create(xml);
XslCompiledTransform trans = new XslCompiledTransform();
trans.Load(xsltFile);
MemoryStream stream = new MemoryStream();
using (XmlWriter writer = XmlWriter.Create(stream, trans.OutputSettings)) {
if (writer != null) {
trans.Transform(reader, writer);
writer.Flush();
writer.Close();
}
}
stream.Seek(0, SeekOrigin.Begin);
return stream;
}

It works for us, but if you’ve got other, better ideas, please let me know!

Put the “Web” Back in Web Services

Dean 5 Feb, 2009 1 Comment

I know it’s possible to transmit Internet Protocol by carrier pigeon, but I’m not sure I could recommend it to our customers with a straight face. By the same token, I’m continually surprised to hear vendors and consultants insist that Web Services can be done without the web.

Yet I’ve seen that claim repeated all over the Internet. Sure, it’s true in the academic sense. Technically, there’s nothing wrong with “webless” web services, just like IP by avian carrier will eventually get the job done (though latency and packet loss are a challenge). But practically speaking, why would you use anything other than HTTP?

The “webless web services” meme has infected a number of good ideas. The SOAP standard is a case in point. At first, SOAP was a simple XML wrapper around an XML payload. (Remember when the S in SOAP stood for Simple?) Then a variety of industry heavyweights jumped on the bandwagon, and soon SOAP could be used with any protocol.

With, uh, a few changes. Like five or six additional specifications. And schema. Lots of schema. Maybe a diagram would help? No?

Why not use REST instead?

Nicholas Allen recently shared a great presentation on REST and SOA given by Stefan Tilkov at QCon. Stefan makes some great points about how RESTful web service design aligns well with the goals of Service Oriented Architecture (SOA).

Sure, you could use the WS-* stack if you like, but RESTful web architecture is a proven, scalable and truly simple approach, as long as you don’t mind having to use HTTP.

Much of the added plumbing in the WS-* stack is there to help transition older computer-to-computer network communication technologies. It allows applications that depend on CORBA or DCOM to tunnel through firewalls. But unless you’re trying to retrofit a system built before, say, 1998, you can skip all that stuff.

Wasn’t using web protocol the whole point of web services anyway? As Stefan said in his closing comments: “Protocol independence is a bug, not a feature.”

Use a RESTful architecture for new development. Put the web back in web services.

Using WCF for REST, Part 3

Dean 27 May, 2008 3 Comments

The easiest way to explain the issues we encountered when implementing REST is to work through the design principles we followed. I think much of our trouble came from the fact that we come from a web applications background, not a SOAP services background. I’m hoping that by laying out our REST design, some of the Microsoft gurus can help us do things “the WCF way.” And perhaps we can help the WCF team out by highlighting a handful of places where we found WCF unintuitive.

The Importance of Being Addressable

Fundamentally, the REST pattern is about making resources available. This means that each item stored within your system can be accessed by someone with the correct permissions. Every one of these items has its own unique address, and its address should not change. This consistency is important, because it allows both people and computer programs to remember and reference items in your system.

Note that we’re talking about resources or items. In contrast with the SOAP model of web services, which allows programmers to invoke procedures on remote computers, REST is about providing data. SOAP is about verbs, while REST is about nouns. A SOAP service might CalculateTotalSale(); a REST service provides CustomerRecieptNo_12345. The kind of web services architecture you use will depend on the kind of application you’re building. The choice has major implications for the other components of your system.

REST imposes restrictions on what sort of things you can do, because it supports only a handful of actions: GET, POST, PUT, and DELETE. (There are a few other HTTP methods, but these four are the most important.) Fortunately, with these four actions, you can accomplish most basic programming tasks. There’s a close parallel to these actions and create/read/update/delete, or CRUD, the building blocks of data storage systems.

UriTemplate

Since the address, or URI, is the primary way to access information in your system, it’s effectively part of your user interface. All the principles of good user interface design apply. So when designing a REST service, you need fine control over the structure of these identifiers.

SOAP, by contrast, typically has just one endpoint. The address itself conveys no information about what services are provided — that’s why SOAP services require a separate WSDL file to tell folks what’s possible. With REST, it should be easy to discover the extent of the system by looking at the URIs alone.

Coming up with good REST URI patterns can be tricky. Using short, descriptive naming conventions for your resources makes them easier to type. But URI patterns must also be distinct and unambiguous.

In the .NET framework, you use the UriTemplate class to define patterns. The UriTemplate implementation that shipped with .NET 3.5 allowed you to define variables that fit into slots in your URI. A typical UriTemplate might look like this: http://restserver/{object}/{id}?view={viewname}.

WCF looks for incoming URIs that match the patterns you define. The pattern defined above would match the following URIs:

http://restserver/customer/5?view=profile
http://restserver/article/how_to_do_stuff?view=print
http://restserver/author/John-Smith?view=1

Once you’ve defined a UriTemplate, you bind it to a method that has the same number of parameters. (I won’t go into the ABCs of WCF here, but you can check out this MSDN Introduction to WCF if you need a refresher.)

In WCF 3.5, you could only define a variable for a whole segment. A segment is basically the bit between the one forward slash and another, or one querystring parameter. A few bloggers requested more flexibility in UriTemplates, and the WCF team answered with the soon-to-be-released 3.5 SP1. The ability to define variables for partial segments was crucial for our URI design.

Representation Matters

Most books about web services, including RESTful Web Services, advocate leaving off file extensions from your URIs. This makes sense for SOAP, where you’re accessing methods and all responses are transmitted in XML. But in REST, you’re serving up items.

In our case, some of these items being served were files and some were records from a database. It seemed inconsistent to have some endpoints that had file extensions and others that didn’t. And we also wanted to be able to serve up different representations for our database records. Our REST service supports both JSON, XML, and HTML. It made sense to use a file extension to distinguish between the different representations.

One workaround would have been to create endpoints like http://restserver/form/1040/xml but that looked funny next to URIs like http://resterver/file/documentation.pdf. True RESTafarians would point out that neither the “/xml” or the “.pdf” are needed, since you can request an appropriate representation using the HTTP ACCEPT header. We decided against the header approach because not all browsers use the ACCEPT HTTP header. Besides, it might be useful for us humans to be able to reach alternate formats by simply changing the URL in the browser address bar.

In WCF 3.5, this required us to create three times the number of endpoints, with a separate method to handle each. We can’t wait for the official release of 3.5 SP1 to make UriTemplates like http://restserver/user/{id}.{ext} possible.

The Final Slash

Another source of endpoint duplication was the need to have two different endpoints for http://restserver/folder and http://restserver/folder/. Because the slash is used as a segment delimiter, the dispatcher in WCF 3.5 saw these two URLs as fundamentally different.

So handling what we thought were fairly trivial cases in URI patterns led us to create FIVE TIMES the number of endpoints we wanted. It’s a maintenance nightmare. SP1 can’t get here soon enough.

WCF Instance Context

Dean 21 May, 2008 2 Comments

I finally figured out the source of my HTTP 400 problem. Apparently the Windows Communication Foundation deals with exceptions differently depending on your InstanceContextMode settings. I had been using the Single setting but I should have used the PerCall setting. In PerCall mode, the try/catch block works as expected.

I think it has something to do with the way that WCF distinguishes between channel exceptions and message faults.

Anyway, if you’re building a REST web service, you’ll want to make sure your class is decorated with the following ServiceBehavior attribute.

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]

Using WCF for REST, Part 2

Dean 17 May, 2008 4 Comments

In part one of this series, I listed several websites and blogs that had useful information on the Windows Communication Foundation (WCF) and REST. I also mentioned that if I was stating again, I’d probably use something other than WCF. Perhaps deriving my own REST server from System.Net.HTTPListener, for example.

Vish asked for some additional details in his comment to that post. He works on the Microsoft WCF development team and was curious about our experience.

I had just begun putting together my response when I noticed Scott Guthrie’s post about Service Pack 1 for the .NET Framework 3.5 beta release. Steve Maine also posted specifics about the ADO.Net Data Services and WCF changes.

So, Vish, it seems your team’s beaten me to the punch on some of these issues! Many of the difficulties I was having with WCF and REST were addressed by the service pack. Here’s an overview of our key stumbling blocks:

REST requires much greater control over the URI than SOAP does, and the URITemplate class just wasn’t up to the task. We had to hardcode most of our endpoints to compensate. (Fixed in SP1. Hooray!)
Supporting multiple formats, such as serving both XML and JSON, either require you to program against Stream or require twice the number of endpoints.
Existing serializers had trouble with complicated object graphs, forcing us to perform serialization/deserialization ourselves. (This seems greatly improved in SP1.)
WCF allows only one contract/interface per endpoint. This makes it tricky to factor out common contract patterns.
Good REST practice would have you return many kinds of errors as HTTP status messages. SOAP embeds all error information in the returned XML. WCF is closely aligned with the SOAP approach, which means that you’ve got to be very careful distinguishing exceptions from faults when implementing REST in WCF. It was an unpleasant surprise, and we had to do quite a bit of work to deal with that.

I’ll talk about all five of these areas in more detail in upcoming posts in this series. And I’ll be sure check out the SP1 beta once we get our Infovark Alpha release out the door.

Using WCF for REST, Part 1

Dean 22 Apr, 2008 5 Comments

Just because you can do something doesn’t mean it’s a good idea.

We decided to use the Windows Communication Foundation to drive our REST-based web service. In hindsight, it was a poor choice. REST support in WCF seems like it was a last-minute addition to .NET 3.5. You can certainly hack something together, but I’ve found few real-world examples on the Internet, and most of those sidestep the tricky issues.

Here’s the short of it: WCF was designed for RPC-SOAP. More importantly, it was designed to SOA-enable legacy services that used older communications channels like DCOM. If you’re starting from scratch, and have full control over the output of your web service and the design of your object model, I’d recommend using a different (and simpler) framework.

We’ve gotten a good bit of blog traffic from people looking for help with Windows Communication Foundation and the REST architectural pattern. (It’s good to know that we’re not the only ones needing advice.) Here are the better sources we’ve found so far.

Windows Communication Foundation documentation on MSDN

Good overview presentation on REST and Syndication using WCF

Microsoft’s Picture Services Sample

Justin Smith’s WCF articles on Cybertopian Chronicles

Nicholas Allen’s Indigo blog

Steve Maine’s blog

Assorted posts on Rick Strahl’s blog

That Indigo Girl

If you find other useful places to look, let us know!

Review: Restful Web Services

Dean 10 Apr, 2008 3 Comments

As we mentioned recently, we’re building the infovark server using the REST pattern. Since REST is more a loose set of guidelines than a strict series of rules, it’s hard for implementers to know where to begin.

OK, you could go to the source, chapter 5 of Roy Fielding’s dissertation. Or you could check out the somewhat academic discussions on the REST wiki, though there hasn’t been much activity lately. You can occasionally find good advice from the odd blog post, like the REST for the Rest of Us article at Open Garden. But ironically, there’s not a whole lot of material about implementing REST web services available on the web yet. (If you know of good links, leave a comment.)

RESTful Web Services

For the practical, gritty details of how it’s actually done, you’ll need the Restful Web Services book by Sam Ruby and Leonard Richardson. They describe the principles that inform REST-ian (RESTafarian?) design in detail, taking you step-by-step through two different sample applications. If you’re a Ruby programmer using Rails, you’ll find the book especially valuable, since that’s the language and framework in which most of the examples are done. For those of us using different technology, it’s the thought process behind the examples that is most illuminating.

This is because the key challenge of the REST paradigm is the fact that it can’t really be implemented on today’s web without some workarounds. REST will come into its own with HTML 5. The book steers an interesting course between how REST web services might be done in HTML 5 with how they must be done today. I think the authors get the balance right, but at times it can make for a frustrating read for someone wanting practical advice about building a REST service right now.

But that’s less a criticism of the book than of the openness of the REST concept itself. The occasional what-if digression the authors make is a small price to pay for the amount of sound guidance you get. The appendixes alone, which discuss things like which HTTP status codes and headers are worth implementing and which are worth forgetting, will save you far more time than you’ll lose in reading how great things will be when HTML forms finally support the PUT method.

Until that day comes, keep this book handy.

WCF Bad Request

Dean 9 Apr, 2008 5 Comments

I’ve just identified a horrible bug in WCF for the .NET Framework 3.5.

A caught exception in a WebInvoke operation will cause WCF to return an HTTP 400 Bad Request status code to the client. Any caught exception. Every time. Regardless of whatever error code you might want to send back.

I found the error by mistake. I’d used “BadGateway” instead of “BadRequest” in my code. If it weren’t for other odd WCF behavior, I wouldn’t have noticed that my status code was being ignored.

Consider the following example:

// Read the Xml into our object and save.
try
{
// The following line triggered the error.
obj.ReadXml(reader);
obj.Save();
// Set HTTP Cache Options and MIME Type.
Utilities.SetCaching(WebOperationContext.Current, obj.DateModified, 60);
Utilities.SetMimeType(Format.Xml);
return Utilities.GetXmlStream(obj);
}
catch (Exception e)
{
// Was it a schema validation error? If so, provide detail.
if (!string.IsNullOrEmpty(_XmlValidationErrors))
{
// I slipped here, using BadGateway 502 instead of Bad Request 400.
// But WCF doesn't care. If you enter the catch block it's _always_ 400.
WebOperationContext.Current.OutgoingResponse.StatusCode = HttpStatusCode.BadGateway;
WebOperationContext.Current.OutgoingResponse.StatusDescription = _XmlValidationErrors;
WebOperationContext.Current.OutgoingResponse.SuppressEntityBody = false;
return null;
}
}

If no error occurs, WCF will return the status code you specify. A try/finally block will work just fine; WCF returns whatever status code you specify. Enter a catch block, though, and it’s nothing but 400 Bad Request.

Hey, if there’s an error, it must be the client’s fault, right?

The Address Bar as the New Command Line

Dean 1 Apr, 2008 No Comment

It’s been a common meme over the past few years that command-line interfaces (CLIs) are making a resurgence. Jeff Atwood noted in 2005 that the Google search box is a command line of sorts. In early 2007, Lifehacker’s Gina Trapani listed several examples of CLIs showing up in a variety of applications.

The return of the command-line interface is striking. It was pronounced dead, replaced by the graphical user interface (GUI) more than two decades ago, after usability research discovered that novice users were much more comfortable with GUIs. But a CLI has three big advantages over a GUI.

The first advantage is speed. Expert users find that command-line interfaces are significantly faster than GUIs. People can type much faster than they can identify a link or button, move the mouse to it, and click. This is especially true if the user’s hands are already positioned above the keyboard. People that do a significant amount of word processing (or writing code) often memorize keyboard shortcuts to avoid having to grab the mouse. It might save only a fraction of second, but if you do it many, many times over the course of the day, those split-second differences add up.

The second advantage is that a command line interface is a linear interface. Neither the user nor the application have to worry about the exact position of the pointing device over a two-dimensional surface. This makes command-line interfaces ideal for devices that have tiny screens, or whose pointing devices aren’t very accurate. It’s much easier to text a few commands on a cell phone or PDA than it is to use a stylus or manipulate a touch screen.

These first two advantages of CLIs are are well-known. The third advantage is less obvious: It turns out that CLIs are also much easier for computers to use.

Why do we care about making things easier on our silicon-based friends? Well, one reason is that it indirectly makes things easier on those of us humans that give instructions to computers. (As a programmer myself, that’s a big selling point.) Another reason is that if it’s easy enough for a computer to figure out, then it ought to be brain-dead simple for a human to understand. As I’ve pointed out before, computers just aren’t very bright.

So the advantage of having a command-line interface is that if you, a human, get bored of typing the same silly commands over and over, you can easily write a small program that instructs a computer to do it for you. This is the reason why every office suite has some kind of macro language. It allows humans to hand off repetitive tasks to a machine so that people can get on with the important creative stuff.

So what’s this about using a browser’s address bar as a command line? With AJAX technologies, the URI is no longer merely something that a human types into the address bar to get a web page. It’s also an interface that computers use to interact with data. This means the humble web URL shares most of the key characteristics and advantages of a command line interface.

It’s this insight that’s led to increased interest in the REST architectural style. REST has begun moving into the mainstream because the patterns it promotes allow the URI to serve an audience of computers as well as an audience of humans.