Thoughts on a RESTful Transaction Process in HTTP

By: Seairth Jacobs
Status: Draft, Revision 0
Date: February 10, 2004

NOTE(S): I realize that this may not be a new idea. Much of what I have learned (and continue to learn) about REST is directly due to the community that congregates in the rest-discuss yahoo groups. I wont even be surprised if some of them correct me on this document. More to the point, I credit them with giving me the knowledge to write this. At their discretion, they can take the credit or pretend that they don't know me.

This idea is actually a "fleshing out" of a suggestion I made on the xml-dev mailing list. I have put a bit more thought into it and think the overall concept has value. If you have any comments, feel free to post them at http://www.seairth.com/blog/industry/rest/83 or e-mail me.

The Problem

You want to update multiple resources at the same time. You also want the capabilities that are common in RDBMS, specifically "transactions". Transactions allow for an "all or nothing" mechanism to update multiple datasources in a single step. In truth, the requests are performed in multiple steps, only the commit process at the end being the "single step" that does the actual changes.

However, this is somewhat difficult to do with web servers. There are certainly mechanisms to update single resources at a time, both in terms of the HTTP methods available and the HTTP headers that can be used to conditionally commit changes. However, HTTP does not directly support the ability to update multiple resources at the same time (in a single step). This makes HTTP a difficult protocol to use when resources have dependencies that must be carefully managed.

The Solution In A Nutshell

So that multiple resources can be updated, a transaction resource is created and maintained to allow control over the process. Fundamentally, it maintains the current state of the updates. The transaction resource has several functions which will be covered below.

The Transaction URI

The first thing we need to do is create a transaction resource. While this document does not dictate how that must happen, here is a suggest (for illustrative purposes):

Client submits a POST request to the "/t" resource on the Server. The request contains an initial transaction document that specifies certain settings:
```
<transaction type="optimistic"/>
```
If the document is accepted, the Server returns (using 201 Created) a representation of the official Transaction Resource with the HTTP "Location" header containing the new URI. For instance, the new URI might be "http://example.com/t/123". The returned representation would look like:
```
<transaction type="optimistic" status="active" uri="http://example.com/t/123"/>
```

Before we get into the specifics of the transaction resource, let's take a moment to talk about the new URI (http://example.com/t/123). From this point on, while the transaction is still active, any resource updates that are to be included in the transaction must be referenced with the transaction URI as the base. For instance, if you want to update "http://example.com/path/resource1" inside the transaction, you would use "http://example.com/t/123/path/resource1".

The Transaction Resource

The transaction resource is basically the internal state of the updated resources related to the transaction. Some of this information is exposed via the transaction URI as a representation of the resource. The format of the representation is an XML document. You can GET, PUT, and DELETE the resource through the transaction URI.

GET the Transaction Resource

If you GET the transaction resource, an XML document will be returned that looks like:

<transaction type="optimistic" status="active" uri="http://example.com/t/123"/>

There are three attributes:

type: This can have a value of either "optimistic" or "pessimistic". The difference between these two values are discussed below. This value can only be set when creating the transaction resource. Once the resource is created, this value cannot change.
status: This can have a value of either "active" or "commit". When a transaction resource is created, the value always starts as "active". The client manually sets the value to "commit" at the end of the transaction, which is discussed in detail below.
uri: This is the transaction resource's URI. This allows the transaction resource to ensure that an update (from a PUT) is being applied to the correct transaction resource. This also allows the XML document to be passed between systems (via e-mail, for instance) and still allow the recipient system to know what the URI is. Similarly, the URI itself could be passed, since a GET request would return the same XML document.

The "transaction" element may contain additional elements, which is covered in detail below.

PUT the Transaction Resource

To make changes to the transaction, you can PUT a new XML document with the changes. For the portion of the document covered so far, all that can be changed is the value of the "status" attribute. By setting the value to "commit", the transaction is completed and all updates are appropriately applied. I cover errors/exceptions for this process below. Also, there is one other way that the document can be modified, which is also covered below.

DELETE the Transaction Resource

To cancel a transaction, you simply DELETE it. All updates are lost. You are back to where you began.

Updates

When you want to update (a PUT, for example) the resource at "http://example.com/path/resource1" within the transaction, you would perform the update at "http://example.com/t/123/path/resource1". For each successful update, a "receipt" of the action is added to the transaction resource. At any time, the updates performed up to that point can be determined by GETting the transaction resource, which lists each update (in the order performed) in the body of the "transaction" element.

The exact contents of the receipt depend on the HTTP method used. They are as follows:

DELETE:
```
<delete uri="" etag=""/>
```
The "uri" attribute indicates which resource was deleted. The "etag" attribute indicates what the resource's ETag was at the time of the deletion request.
POST:
```
<post uri="" etag="" new-etag="" new-uri="" new-uri-etag=""/>
```
The "uri" attribute indicates which resource was POSTed to. The "etag" attribute indicates what the resource's ETag was at the time of the POST request. The "new-etag" attribute indicates the value of the ETag after the request is completed (which may still be the same value as "etag"). If the POST cause the creation of a new resource, then two additional attributes would be available. "new-uri" indicates the URI of the newly created resource and "new-uri-etag" indicates the initial ETag of that resource.
PUT:
```
<put uri="" etag="" new-etag=""/>
```
The "uri" attribute indicates which resource was PUT to. The "etag" attribute indicates what the resource's ETag was at the time of the PUT request. The "new-etag" attribute indicates the value of the ETag after the request is completed.

Only successful updates (2xx responses) are recorded. If an error occurs during an update inside a transaction, the HTTP server would return a response as would have if the same action were taken outside of a transaction. This rule has a couple purposes. First, there is no need to record the failure since the response itself will always be more descriptive about what the problem is. Second, suppose that you had a network failure during an update request. By checking the transaction resource, you would know whether the update was completed or not. If it was not, there would not be a "receipt" for your last request. This could be due to either an error or a failure of the request to have made it to the HTTP server. The request can then be re-submitted.

These elements are the only other place that an updated transaction document (via PUT) is allowed. You may undo updates in the reverse order that they were performed by PUTting an updated document that does not contain the receipt elements you want to undo. For instance, if you have performed four updates:

<transaction type="optimistic" status="active" uri="http://example.com/t/123"/>
    <put .../>
    <put .../>
    <delete .../>
    <delete .../>
</transaction>

You could undo the last two statements by PUTting the following document:

<transaction type="optimistic" status="active" uri="http://example.com/t/123"/>
    <put .../>
    <put .../>
</transaction>

Pessimistic Updates

Like the RDBMS counterpart, "pessimistic" updates means that once a resource has been "updated" in the scope of a transaction, then it is "locked" and no other transaction can update until the transaction is completed (or cancelled). The resource can still respond to a GET (if supported), which would return the current state of the resource, not the "updated" state. Generally, pessimistic transactions should not be active for long periods of time. The exact duration depends on the requirements of each application. Just remember that the longer the transaction holds an updated resource, the more likely someone else will encounter a problem when trying to update the same resource.

When an error occurs for a specific update, all prior updates must be implicitly undone. This requirement will make more sense below when discussing optimistic updates. But for now, just remember that pessimistic transactions should not be active for long periods of time. Since it is likely that an error will not be immediately corrected, the entire transaction is undone to release the other resources. The transaction resource still exists and can still be used, but it will contain no updates at that point.

Optimistic Updates

Also like the RDBMS counterpart, "optimistic" updates means that the resource is not "locked" until the transaction is completed, at which point all updated resources are locked at the same time. Internally, this process can use a pessimistic transaction, since this is little more than a "delayed pessimistic" update anyhow.

However, since there is a period of time between the original update and the actual locking of the resource, it is possible that conflicts could occur. For instance, another transaction may update (and commit) the same resource before your transaction is completed. When this occurs, it is not possible to directly return the HTTP error response given. This is because the process is potentially attempting to commit multiple resources at the same time and there is no concept in HTTP of a multi-request. Instead, the error is recorded in the transaction resource within the associated update receipt, which can then be retrieved via a GET. An error is indicated by the fact that the PUT of the transaction document (with the status attribute set to "commit") is rejected and the server returns an appropriate error.

NOTE: This is why the pessimistic update has a draconian response to errors. If it did not, then the optimistic transaction would be left in a state somewhere between optimistic and pessimistic (with only a portion of the resources being "committed").

An error element looks like:

<transaction type="optimistic" status="active" uri="http://example.com/t/123"/>
    <put ...>
        <error code="" text="" uri=""/>
    </put>
</transaction>

The "code" attribute contains the HTTP response code. The "text" attribute contains an optional textual description of "code". The optional "uri" attribute points to a resource that contains the contents of the the error response (if there was one).

Also, it is still possible for an error to be returned in response to an update request. In other words, it is not necessary to wait until the transaction is completed to detect problems with update request. In fact, it is far more desirable to detect the problem at the time of the update so that corrective action can be taken before any additional updates are applied within the same transaction.

Odds and Ends

Until I figure out an appropriate place to insert the following bits, they are listed here (in no particular order):

Obviously, all resources must have an ETag in order to be used here. The reason is simple the (internal) update process can use conditional updates to ensure that conflicts are handled gracefully. When using optimistic transactions, it is not necessary for the client to deal with headers like If-Match to determine conflicts. On the other hand, if using pessimistic transactions, the client should use conditional updates to ensure that data is not accidentally overwritten or lost. Explicit use of ETags ensures that this is possible.
When applying the updates to the actual resource, the updates should be performed in the same order that the original updates were performed. However, there are possible exceptions to the rule. For instance, if a later update supercedes a prior update of the same resource, it may be possible to skip the prior update altogether. This sort of optimization, however, is beyond the scope of this article.