Posts tagged with “REST”

Units of Work

Friday, 20 March, 2009

In regard to cross-resource transactions, I’m a member of the camp that wonders whether distributed transactions are strictly necessary in the REST/HTTP world; indeed I wonder whether they represent a design failure in modeling the granularity of your resources.

That said, we don’t live in a perfect world — and even if I can’t envisage why a properly designed RESTful application might require access to a distributed transaction, I can certainly envisage the environment where such an application might evolve. I’ve worked in a few of them. Places where interdepartmental barriers are as solid as the Great Wall; bastions of archaic technology where “one might provide a pseudo-RESTful interface, but one certainly won’t be re-architecting one’s legacy system in the buzzword language of the day”.

But, I think there’s a certain amount of smoke and mirrors in the JBossTS article “Transactional support for JAX RS based applications“:

Certainly it is worth pointing out that if a system cannot be made reliable then it can be of only limited utility. That said it is a worthwhile exercise to show how a REST based system can be made reliable.

Lack of distributed transactions would hardly seem to make a REST based system “unreliable” and, as a consequence, of “only limited utility”. Imagine a hotel booking facility — perhaps a booking resource, which internally might be constructed from a number of components, all governed (again internally) by transaction demarcation. Does the fact that the booking resource is coarse-grained and does not require an external transaction make it less reliable than a number of fine-grained resources which do? On the contrary. The latter sounds more like a WS-* api than a RESTful architecture, nothing to do with reliability.

So… hopefully it’s obvious I think it’s a bad idea. But if I were to write such an API, I think the 8-year old spec mentioned in the article falls short of the mark. Here’s my first-cut attempt at an alternative (which I still think falls a bit short of the mark, but is possibly an improvement):

Resource: tc
Method URL Content Description Statuses
GET /tc Returns HTML containing a summary of all transactions (status), plus an href to the transaction detail 200 – ok
/tc/{txid} Returns HTML containing detail for the transaction with id {txid}, including the status and a list of hrefs to the each of the participants. For example:


<html>
<body>
	<dl>
		<dt>Transaction ID</dt>
			<dd id="transaction-id">12345a</dd>
		<dt>Status</dt>
			<dd id="transaction-status">ACTIVE</dd>
		<dt>Timeout</dt>
			<dd id="transaction-timeout">5000</dd>
	</dl>
	<ul id="participants">
		<li>
			<a href="/tc/12345a/participants/1">
				Participant 1
			</a>
		</li>
		<li>
			<a href="/tc/12345a/participants/2">
				Order
			</a>
		</li>
	<ul>
</body>
</html>
200 – ok

404 – if txid is not found
409 – if the transaction has been deleted
/tc?status={status-type} Return HTML containing a summary of transactions with a specific status, with href to the transaction detail
For example: /tc?status=recovering or /tc?status=active
200 – ok
/tc/{txid}/participants Return a list of participants in the transaction (list of hrefs) 200 – ok
404 – if the transaction does not exist
/tc/{txid}/partipants/{rec-coord-id} Return HTML containing the detail of a participant. For example:


<html>
<title>Participant #2</title>
<body>
<dl>
    <dt>ID</dt>
        <dd>2</dd>
    <dt>Name</dt>
        <dd>Order</dd>
    <dt>URL</dt>
        <dd>
            <a href="http://internal.mydomain.com/someresource/123">Order</a>
        </dd>
</dl>
</body>
</html>
200 – ok
404 – if the transaction or participant does not exist
POST /tc [timeout={timeout}] Start a transaction (with default timeout) returning the url /tc/{txid} — which is deleted after the timeout or after completion (any HTTP method relating to {txid} thereafter returns 404). Use timeout={timeout} to override the default timeout period. 201 – created
DELETE /tc/{txid} Rollback and stop a transaction 204 – ok
404 – if the transaction does not exist
/tc/{txid}?commit Commit and stop a transaction 204 – ok
404 – if the transaction does not exist
POST /tc/{txid}/participants url={url}&[name={name}] Enlist {url} in the transaction, returning a unique resource for that participant of the form /tc/{txid}/participants/{rec-coord-id}. If name exists, record against the participant detail 201 – created
404 – if the transaction does not exist
PUT /tc/{txid}/participants/{rec-coord-id} url={url} Replace the participant url 200 – ok
404 – if the transaction or participant does not exist

The resource identified by a participant URL will have the following semantics:

Method URL Content Description Statuses
PUT URL/tx/{rec-coord-id} action=prepare The participant prepares any work done in the context of the transaction. The Warning header will contain additional info about the state of the prepare (either readonly, or notok). 200 – ok
200 – ok (+ Warning: readonly)
200 – ok (+ Warning: notok)
404 – participant has rolled back
action=commit The participant commits any work done in the context of the transaction. 200 – ok
200 – ok (+ Warning: heuristic)
404 – participant has rolled back
action=rollback The participant commits any work done in the context of the transaction. 200 – ok
200 – ok (+ Warning: heuristic)
404 – participant has already rolled back

Basic usage might look something like the following:

1. Create a new transaction resource
POST /tc
Host: somedomain.com

timeout=5000
HTTP/1.1 201 Created
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
Location: http://somedomain.com/tc/10a23v991
X-Powered-By: TransactionServer/0.1
2. Create a new resource of some kind (notify the resource that it will operate with a distributed transaction)
POST /res1?tx
Host: internaldept1.somedomain.com

<xml>some xml describing the resource</xml>
HTTP1.1 201 Created
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
Location: http://internaldept1.somedomain.com/res1/100
3. Update another resource (again notify that it will be operating within a transaction)
PUT /res2/somename?tx
Host: internaldept2.somedomain.com

<xml>some xml describing the update,
perhaps including a reference to the previously
created resource</xml>
HTTP1.1 200 Ok
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
Location: http://internaldept2.somedomain.com/res2/somename
4. Enlist the url for each resource in the transaction
POST /tc/10a23v991/participants
Host: somedomain.com

name=resource1&url=http://internaldept1.somedomain.com/res1/100
HTTP/1.1 201 Created
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
Location: http://somedomain.com/tc/10a23v991/participants/a01abgv21
X-Powered-By: TransactionServer/0.1
POST /tc/10a23v991/participants
Host: somedomain.com

name=resource2&url=http://internaldept2.somedomain.com/res2/somename
HTTP/1.1 201 Created
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
Location: http://somedomain.com/tc/10a23v991/participants/a01abgv22
X-Powered-By: TransactionServer/0.1
5. Commit the transaction
DELETE /tc/10a23v991?action=commit
Host: somedomain.com
HTTP/1.1 204 Committed and deleted
Connection: close
Date: Thu, 19 Mar 2009 21:01:56 GMT
X-Powered-By: TransactionServer/0.1
6. ‘Behind the scenes’, the commit results in the following (2-phase commit at this point)…
PUT /res1/100/tx/a01abgv21
Host: internaldept1.somedomain.com

action=prepare
PUT /res2/somename/tx/a01abgv22
Host: internaldept2.somedomain.com

action=prepare

and

PUT /res1/100/tx/a01abgv21
Host: internaldept1.somedomain.com

action=commit
PUT /res2/somename/tx/a01abgv22
Host: internaldept2.somedomain.com

action=commit

when does REST become complex?

Saturday, 17 February, 2007

According to this:

The great marketing coup that REST folks are trying to pull off is that WS-* is complex and REST is simple. That’s just nonsense – if you really try to build the type of systems that people build with WS-* using REST, then you’d end up just as complex.

Who’s talking nonsense? So, you take something that is inherently complex and build a system with it — then you take something that is fundamentally simple and build a similar system with that — and the final results are the same in terms of complexity?

Mais non. I don’t think so.

simple REST testing tools

Sunday, 6 August, 2006

After a bit of googling, I have yet to find a command-line tool that simply handles HTTP requests. There’s wget for retrieving files, curl for PUT and POST, but what to use for DELETE (or other HTTP methods)? For that matter, I’d prefer to have a single tool, rather than 2 or 3.

I wrote a simple python script a while back — the last time I couldn’t find such a tool — and so now that I need to use it again, it seems like a good time to polish it up and release it (here). Not before chopping out a bunch of specific hacks that really need tidying up before being included in released code, of course.

So if you need a basic tool for handling HTTP requests (GET, PUT, POST, DELETE) you may find it useful… rather than having to write the few lines of code yourself.

apples and oranges

Saturday, 1 July, 2006

My own experience suggests that REST and Soap/WS is an apple and oranges comparison. Actually that’s being unfair to oranges. It’s more like an apple and strange-looking-fruit-bearing-no-resemblance-to-anything-palatable comparison.

When I’ve had the opportunity to design a system around REST principles (resource-oriented, POX, basic HTTP, etc), it has had a simplifying effect on the overall architecture — not dumbing things down, just a reduction in complexity. I’ve had to think differently about fundamental design issues — what are my resources, and how should they be interacted with… rather than RPC API details — in a way it’s a little like the difference between thinking OO terms and thinking procedurally.

The same can not be said for the SOAP/WS systems I’ve seen, and worked with, where complexity seems to be an accepted byproduct. BPEL, choreography, discovery, and the myriad other WS-* specifications are, to me, a good reason to sell expensive product, and justify business power lunches, but a bad way to design software. SOAP at its heart was a relatively simple idea, but there’s enough other crap bolted-on around it now to sink even the most bouyant of ships. There will be plenty of proponents who’ll argue the whole tooling angle — that their (obscenely expensive) tools will hide the implementation details from you. Well I’ve used some of their tools, and I’m definitely not sold. Just what the world needs: the equivalent of a Renault Alliance, with worse fuel economy than a Hummer, slower than a Simca 1000 running on 3 cylinders, and a Ferrari price tag to boot.
Not to forget that hiding away implementation details is a good way to cause a mass of problems when the client asks you to do something that fall just outside the capabilities of the tool.

All of which is to say I believe that positions which appear to argue the differences between REST and SOAP down to mere implementation details, are less than convincing. Okay, to be absolutely honest, I have argued something vaguely similar in the past… but I’m wavering on the cusp of thinking I should’ve explained myself a bit better. I still believe that a resource-oriented application could expose some SOAP-style services, but it certainly wouldn’t make sense for an entire application to expose dual interfacing. And I certainly don’t think that a WS-* design can then expose its methods as HTTP verbs and somehow magically satisfy the goals of a REST architecture (usage of HTTP verbs, alone, doth not a REST app make). A bit like an elephant trying to squeeze into a pair of hot-pants and then announcing to the world that it’s a prima ballerina.

Related footnote: I hadn’t read this before, but it’s a brilliant read.

REST illustrations

Friday, 30 June, 2006

Stefan has just posted a nice, simple illustration of the differences between service-oriented and resource-oriented (i.e. REST) design. Something to bookmark (which I’m doing right here… ;-) ) for the next time someone asks me what the hell REST is…

unconvinced by WADL

Wednesday, 3 May, 2006

I remain thoroughly unconvinced by WADL. I accept there’s a (current) corporate need to over-describe everything in XML, but quite frankly I’d rather describe my REST resources with HTTP OPTIONS plus extended http headers and/or, preferably, human-readable (and also machine parseable) xhtml, ala microformats.

Interestingly, I find no mention of WADL on rest-discuss. Crappy yahoo search algorithm? Or just “phasers set to ignore”?

Resource versus Activity

Saturday, 7 January, 2006

In an article on IBM’s developerworks:

Proponents of the REST-based pattern often point to its architectural simplicity compared to the complexities inherent in the broad collection of SOAP-related Web service specifications as being the key to why REST is the superior approach. This argument has flaws in that, as I have illustrated, the two approaches are looking to solve different types of problems. Web-based application services, such as Bloglines, Amazon, flikr, del.icio.us, and so forth, face a different set of issues than say, a hospital looking to automate their internal processes for order procedures and writing prescriptions and therefore demand a different architectural approach. That’s why design patterns exist — different approaches to solve different problems.

I don’t believe he’s successfully illustrated any flaws at all, particularly not with those two examples. Both order procedures and writing prescriptions can fit just as comfortably within a REST paradigm as they can SOAP.

For example, it’s easy to envisage POSTing to a prescription resource such as:

/health/services/prescription

which creates a new prescription identifier (for example, P1RA43) and adds the items in the prescription request. The identifier is returned to the caller by way of the Location header:

Location: http://somehealthprovider.com/health/services/prescription/P1RA43

The prescription could then be viewed in a web browser by calling that URI and specifying a content type of text/html, or used by a pharmacy computer during the fulfillment stage, by requesting text/xml instead. An HTTP DELETE against the URI would get rid of the prescription, a PUT would replace the contents, and subsequent POSTs add to the contents.

It seems to me this looks hellaciously (love that word) more elegant than the equivalent CreatePrescription, GetPrescription, AddToPrescription, ReplacePrescription, DeletePrescription, etc that I assume would be the SOAP equivalents.

TDD, Jython & REST (Part 3)

Sunday, 2 October, 2005

With the addition of xml validation and value checking, the testing scripts actually become somewhat useful:


> PUT http://localhost:8080/ws/user ''
    'username=testuser1&email_address=testuser1@test.com'
> GET http://localhost:8080/ws/user/testuser1
< assert status 200
< assert header Content-Type text/xml
< assert xmlschema ../web/user.xsd
< assert xmlbody /user/username=testuser1
< assert xmlbody /user/email_address=testuser1@test.com
> DELETE http://localhost:8080/ws/user/testuser1

Depak Vohra’s article at ONJava, shows how to perform schema validation using xerces/jaxp. Adapting his code to jython is straightforward, by adding the following imports to the test script from part 2:


from java.io import StringReader
from java.lang import System
from javax.xml.parsers import DocumentBuilderFactory
from javax.xml.parsers import DocumentBuilder
from org.xml.sax.helpers import DefaultHandler
from org.xml.sax import InputSource

System.setProperty('javax.xml.parsers.DocumentBuilderFactory',
        'org.apache.xerces.jaxp.DocumentBuilderFactoryImpl')

Depak’s Validator becomes:


class Validator(DefaultHandler):
    def __init__(self):
        self.validationError = 0
        self.saxParseException = None

    def error(self, exception):
        self.validationError = 1
        self.saxParseException = exception

    def fatalError(self, exception):
        self.validationError = 1
        self.saxParseException = exception      

    def warning(self, exception):
        pass

Finally, a new assertion method is added to the Test class:


    #
    # assert that an xml document validates against an xml schema
    #
    def assert_xmlschema(self, linearr, req, res):
        xml = res.getContent()

        factory = DocumentBuilderFactory.newInstance()
        factory.setNamespaceAware(1)
        factory.setValidating(1)
        factory.setAttribute('http://java.sun.com/xml/jaxp/properties/schemaLanguage',
                'http://www.w3.org/2001/XMLSchema')
        errhandler = Validator()
        factory.setAttribute('http://java.sun.com/xml/jaxp/properties/schemaSource',
            linearr[0])
        builder = factory.newDocumentBuilder()
        builder.setErrorHandler(errhandler)

        dom = builder.parse(InputSource(StringReader(xml)))

        if errhandler.validationError == 1:
            errmsg = '%s, %s' % (str(errhandler.validationError),
                    errhandler.saxParseException.getMessage())
        else:
            errmsg = ''

        assert errhandler.validationError != 1, errmsg

Value checking is accomplished with the following method:


    #
    # assert that a particular xml element contains a particular value
    #
    def assert_xmlbody(self, linearr, req, res):
        xml = res.getContent()
        if not self.xml or self.xml != xml:
            factory = DocumentBuilderFactory.newInstance()
            factory.setNamespaceAware(1)
            builder = factory.newDocumentBuilder()
            self.dom = builder.parse(InputSource(StringReader(xml)))
            self.xml = xml

        # get rid of the starting '/'
        # (could leave this out in the test, but it looks better in to me)
        if linearr[0].startswith('/'):
            linearr[0] = linearr[0][1:]

        # split into xml element list and a value based on the first '=' position
        equalspos = linearr[0].find('=')
        tmp = linearr[0][0:equalspos]
        value = linearr[0][equalspos+1:]
        elements = tmp.split('/')

        # loop through the elements in the list and retrieve each named node
        node = self.dom
        for x in xrange(0, len(elements)):
            nl = node.getElementsByTagName(elements[x])
            if not nl:
                raise AssertionError, 'sub element "%s" is missing' % elements[x]
            else:
                node = nl.item(0)
            if x == len(elements)-1:
                eval = node.getFirstChild().getNodeValue()
                assert eval == value, 'element %s (value "%s") does not match expected %s'
                            % (elements[x], eval, value)
                return
        raise AssertionError, 'missing element %s' % tmp

Which takes an assertion line such as “assert xmlbody /user/username=testuser1″ and looks up elements in the dom, where “user” is the root node, and “username” is a child node of user. At the moment this is very basic value checking which won’t work well for more complicated documents — but I’m trying to avoid complication as much as possible anyway, so it’s difficult to justify (to myself at least) going to greater lengths to make something more flexible. The main enhancement I’m looking to make initially is the ability to specify the nth child of a node. For example, something like the string “/list/user[2]/username” (in case it’s not painfully obvious, the second “user” element in “list”).

But for the moment, this will hopefully get me going.

TDD, Jython & REST (Part 2)

Wednesday, 28 September, 2005

In part one of this rumination, I discussed test-driven-development of a REST-style application written using Jython. An example of a test method would be something like:


def test_getvaliduser():
    req = MockHttpServletRequest('GET', 'user', 'http://localhost:8080/ws/user/testuser1', '', '')
    req.setContentType('text/html')

    res = MockHttpServletResponse()

    user.doGet(req, res)

    assert res.getStatus() == 200, 'incorrect status %s (%s)' % (res.getStatus(), res.getReason())
    assert res.getContentType().startswith('text/html'), 'expected text/html, got %s' % res.getContentType()

The main problem I have with this idea, is the amount of coding (read: effort) required for each test method, and therefore the amount of duplicated effort testing each resource (servlet). Testing other components in this manner, or testing non-standard functionality, is another matter, of course.

I’ve come up with what I believe is a more workable solution. Reducing the previous example to something like the following:


== Get Valid User
> GET http://localhost:8080/ws/user/testuser1
< assert status 200

Which is a serious reduction in the amount of typing required for a test, particularly when you add the setup functions (missing from the 'old' test example):


== Get Valid User
> PUT http://localhost:8080/ws/user '' \
    'username=testuser1&email_address=testuser1@test.com&level=1&password=password'
> GET http://localhost:8080/ws/user/testuser1
< assert status 200
< assert header Content-Type text/xml
> DELETE http://localhost:8080/ws/user/testuser1

Hopefully the script is relatively straightforward to understand: '==' signifies a new test, '>' is an http call, and 'here.

The next step is to add handlers for checking the content in the response (particularly xml), both in form and content.

(If anyone is interested in a working example, let me know and I'll endeavour to post something useable)

TDD, Jython & REST (Part 1)

Friday, 16 September, 2005

I’ve been working on a personal project for a few months now (begun while living in Thailand for a few months), combining a number of different technologies I currently find interesting. I’ve chosen Jython as the programming language in recognition of how powerful I’ve found developing in Python — the reasons for not going completely down the Python route (much as I would like to be using Py 2.4 modules) are mainly because of library issues. In particular I haven’t found a templating library as good as Velocity in the Pythonic world. Of course there is Cheetah, but while it is as close to Velocity as it’s possible to be without ’sleeping in the same bed’, there is still something about Cheetah that rubs me the wrong way. I suspect it’s simply because, as a long time Velocity user, I have become so accustomed to the techniques we use with that library, that slight changes/quirks in the other become larger issues, completely out of proportion to their actual magnitude.

REST is less a technology than a way of thinking which I’m endeavouring to apply to both personal and professional projects. How successful that application is remains to be seen, but thinking about web services in terms of a resource with a simple set of verbs, has certainly resulted in a re-think about how one approaches component development. It’s difficult to justify a full J2EE deployment (EJBs and so on) when your operations are reduced to a basic four (GET, POST, PUT, DELETE) and the business logic fits comfortably within the framework provided by a servlet (in my case a Jython-Servlet).

Finally TDD (which technically should’ve been mentioned first, considering where it falls in the development cycle), comes from the understanding that developing a project, which I hope to eventually turn into something commercially viable, requires more than just hacking together a bunch of parts in the hope that my frankenstein will live when electricity is applied to its extremities. I have to also consider the future of this system — have confidence in both my design and in any changes/refactoring I might make to it after it has a community of users.

TDD is at the crux of the code sample below. This is just the beginnings of a testing system for my rest+jython servlets.

It may make more sense, to some, to reuse what’s already available (JUnit for example, or the unit-testing modules already included in Python/Jython), but this is as much a learning exercise, as serious development, since I haven’t yet had to opportunity to apply TDD in any professional projects.

Read the rest of this entry »