Geronimo is a mess.

To put it politely.

After looking at what’s missing from their web console, a brief experiment with trying to add a jar to the repository, and a few other false starts, the law of diminishing returns kicked in, and I decided to give up before wasting too much time.

Glassfish seems quite polished by comparison (which is a surprise considering I was never that enthused by iPlanet… but then, the last time I looked at that was about a million years ago).

JBoss hasn’t changed a lot since I last looked at it. Still has the hideous applet-driven console app — but then configuring JBoss has always been a case of direct editing of config files (which is part of its attraction really).

Odd difference between the two. When accessing an InitialContext from an external app, in Glassfish you can get access to a configured data source, but in JBoss it doesn’t seem to be accessible. I’m a bit hazy now about how JBoss’ JNDI works, so I can’t recall if this is expected behaviour or not.

In any case, for testing purposes, these Jython excerpts are useful:

1. To access a Glassfish context, set the classpath to:

glassfish/lib/appserv-rt.jar:glassfish/lib/javaee.jar:glassfish/lib/appserv-admin.jar:glassfish/lib/webservices-rt.jar:glassfish/lib/install/applications/jmsra/imqjmsra.jar

(plus whatever jars you need for the resources you’re accessing), then invoke the following in Jython:

from java.util import *
from javax.naming import *
p = Properties()
p.put("java.naming.factory.initial", "com.sun.enterprise.naming.SerialInitContextFactory")
p.put("java.naming.factory.url.pkgs", "com.sun.enterprise.naming")
p.put("java.naming.factory.state", "com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl")
ic = InitialContext(p)

2. To access a JBoss context, set the classpath to:

jboss/client/jnp-client.jar:jboss/server/all/lib/jnpserver.jar:jboss/client/logkit.jar:jboss/lib/jboss-common.jar

…then invoke the following in Jython:

from java.util import *
from javax.naming import *
p = Properties()
p.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory")
p.put("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces")
p.put("java.naming.provider.url", "jnp://localhost:1099")
ic = InitialContext(p)

Hope that’s useful for somebody.

It is, by no means, an endorsement of hibernate; rather a reflection on the fact that I can’t find any tools to access the domain which don’t require the use of Eclipse (<sarcasm-alert>because, of course, everyone uses eclipse</sarcasm-alert>).

In any case, the script needs to be executed with a classpath referencing the hibernate jars, jython jars, your domain classes, and anything else hibernate & jython might need to access — I run it from within ant, but it could quite easily be executed from a shellscript.

First impressions, after not having looked a the Eclipse 3 platform for a while:

1. NetBeans panelling is -way- more slick than Eclipse. Eclipse seems clunky by comparison — perhaps there is a way to set a “FastView” panel to open on mouse-over like NetBeans, but it wasn’t immediately obvious…?

2. Also because the panelling isn’t as good, the Eclipse screen layout doesn’t work as well unless you have a large screen, I think. I’m running a small widescreen on a laptop (1280 x 800), and the editor interior seems vertically challenged unless you minimise the bottom panel — which wouldn’t be an issue, if it wasn’t for the fact that the UI is designed by touchpad bigots (a majority of window managers are able to put the close, minimise and maximise buttons in a group on a window title-bar, so why does Eclipse have to be different?), so it seems as if more touchpad use is required to perform the same action on Eclipse when compared with NetBeans.

3. The method to add new file filters isn’t obvious to me either

4. Fonts are a lot (in large flashing letters on a billboard somewhere) nicer on Eclipse than on NetBeans. NetBeans is hard to look at after Eclipse. JEdit is generally ghastly as well from the font perspective — both suffering from Java’s traditionally wonderful font management (that’s sarcasm in case it’s not apparent).

5. It seems that the blame for the chunky behaviour of WSAD/RAD (+Linux) should fall squarely on the heads of the IBM teams in charge of corrupting (I mean) enhancing Eclipse, because alone, it’s a lot more responsive, and memory usage is considerably better than NetBeans. Doing nothing, NetBeans sits at about 17-19% memory usage. Eclipse is currently (doing nothing) sitting on 9.7%.

6. I still don’t like the way that most (all?) IDEs force you to work to their project formats — i.e. create a web project, create a simple java project, etc. A simple text editor with plugins you can twist around, to your own liking, seems much more attractive — but I’m probably in the minority.

> PUT http://localhost:8080/ws/user ''
    'username=testuser1&email_address=testuser1@test.com'
> GET http://localhost:8080/ws/user/testuser1
< assert status 200
< assert header Content-Type text/xml
< assert xmlschema ../web/user.xsd
< assert xmlbody /user/username=testuser1
< assert xmlbody /user/email_address=testuser1@test.com
> DELETE http://localhost:8080/ws/user/testuser1

Depak Vohra’s article at ONJava, shows how to perform schema validation using xerces/jaxp. Adapting his code to jython is straightforward, by adding the following imports to the test script from part 2:

from java.io import StringReader
from java.lang import System
from javax.xml.parsers import DocumentBuilderFactory
from javax.xml.parsers import DocumentBuilder
from org.xml.sax.helpers import DefaultHandler
from org.xml.sax import InputSource

System.setProperty('javax.xml.parsers.DocumentBuilderFactory',
        'org.apache.xerces.jaxp.DocumentBuilderFactoryImpl')

Depak’s Validator becomes:

class Validator(DefaultHandler):
    def __init__(self):
        self.validationError = 0
        self.saxParseException = None

    def error(self, exception):
        self.validationError = 1
        self.saxParseException = exception

    def fatalError(self, exception):
        self.validationError = 1
        self.saxParseException = exception      

    def warning(self, exception):
        pass

Finally, a new assertion method is added to the Test class:

    #
    # assert that an xml document validates against an xml schema
    #
    def assert_xmlschema(self, linearr, req, res):
        xml = res.getContent()

        factory = DocumentBuilderFactory.newInstance()
        factory.setNamespaceAware(1)
        factory.setValidating(1)
        factory.setAttribute('http://java.sun.com/xml/jaxp/properties/schemaLanguage',
                'http://www.w3.org/2001/XMLSchema')
        errhandler = Validator()
        factory.setAttribute('http://java.sun.com/xml/jaxp/properties/schemaSource',
            linearr[0])
        builder = factory.newDocumentBuilder()
        builder.setErrorHandler(errhandler)

        dom = builder.parse(InputSource(StringReader(xml)))

        if errhandler.validationError == 1:
            errmsg = '%s, %s' % (str(errhandler.validationError),
                    errhandler.saxParseException.getMessage())
        else:
            errmsg = ''

        assert errhandler.validationError != 1, errmsg

Value checking is accomplished with the following method:

    #
    # assert that a particular xml element contains a particular value
    #
    def assert_xmlbody(self, linearr, req, res):
        xml = res.getContent()
        if not self.xml or self.xml != xml:
            factory = DocumentBuilderFactory.newInstance()
            factory.setNamespaceAware(1)
            builder = factory.newDocumentBuilder()
            self.dom = builder.parse(InputSource(StringReader(xml)))
            self.xml = xml

        # get rid of the starting '/'
        # (could leave this out in the test, but it looks better in to me)
        if linearr[0].startswith('/'):
            linearr[0] = linearr[0][1:]

        # split into xml element list and a value based on the first '=' position
        equalspos = linearr[0].find('=')
        tmp = linearr[0][0:equalspos]
        value = linearr[0][equalspos+1:]
        elements = tmp.split('/')

        # loop through the elements in the list and retrieve each named node
        node = self.dom
        for x in xrange(0, len(elements)):
            nl = node.getElementsByTagName(elements[x])
            if not nl:
                raise AssertionError, 'sub element "%s" is missing' % elements[x]
            else:
                node = nl.item(0)
            if x == len(elements)-1:
                eval = node.getFirstChild().getNodeValue()
                assert eval == value, 'element %s (value "%s") does not match expected %s'
                            % (elements[x], eval, value)
                return
        raise AssertionError, 'missing element %s' % tmp

Which takes an assertion line such as “assert xmlbody /user/username=testuser1″ and looks up elements in the dom, where “user” is the root node, and “username” is a child node of user. At the moment this is very basic value checking which won’t work well for more complicated documents — but I’m trying to avoid complication as much as possible anyway, so it’s difficult to justify (to myself at least) going to greater lengths to make something more flexible. The main enhancement I’m looking to make initially is the ability to specify the nth child of a node. For example, something like the string “/list/user[2]/username” (in case it’s not painfully obvious, the second “user” element in “list”).

But for the moment, this will hopefully get me going.

def test_getvaliduser():
    req = MockHttpServletRequest('GET', 'user', 'http://localhost:8080/ws/user/testuser1', '', '')
    req.setContentType('text/html')

    res = MockHttpServletResponse()

    user.doGet(req, res)

    assert res.getStatus() == 200, 'incorrect status %s (%s)' % (res.getStatus(), res.getReason())
    assert res.getContentType().startswith('text/html'), 'expected text/html, got %s' % res.getContentType()

The main problem I have with this idea, is the amount of coding (read: effort) required for each test method, and therefore the amount of duplicated effort testing each resource (servlet). Testing other components in this manner, or testing non-standard functionality, is another matter, of course.

I’ve come up with what I believe is a more workable solution. Reducing the previous example to something like the following:

== Get Valid User
> GET http://localhost:8080/ws/user/testuser1
< assert status 200

Which is a serious reduction in the amount of typing required for a test, particularly when you add the setup functions (missing from the 'old' test example):

== Get Valid User
> PUT http://localhost:8080/ws/user '' \
    'username=testuser1&email_address=testuser1@test.com&level=1&password=password'
> GET http://localhost:8080/ws/user/testuser1
< assert status 200
< assert header Content-Type text/xml
> DELETE http://localhost:8080/ws/user/testuser1

Hopefully the script is relatively straightforward to understand: '==' signifies a new test, '>' is an http call, and 'here.

The next step is to add handlers for checking the content in the response (particularly xml), both in form and content.

(If anyone is interested in a working example, let me know and I'll endeavour to post something useable)

REST is less a technology than a way of thinking which I’m endeavouring to apply to both personal and professional projects. How successful that application is remains to be seen, but thinking about web services in terms of a resource with a simple set of verbs, has certainly resulted in a re-think about how one approaches component development. It’s difficult to justify a full J2EE deployment (EJBs and so on) when your operations are reduced to a basic four (GET, POST, PUT, DELETE) and the business logic fits comfortably within the framework provided by a servlet (in my case a Jython-Servlet).

Finally TDD (which technically should’ve been mentioned first, considering where it falls in the development cycle), comes from the understanding that developing a project, which I hope to eventually turn into something commercially viable, requires more than just hacking together a bunch of parts in the hope that my frankenstein will live when electricity is applied to its extremities. I have to also consider the future of this system — have confidence in both my design and in any changes/refactoring I might make to it after it has a community of users.

TDD is at the crux of the code sample below. This is just the beginnings of a testing system for my rest+jython servlets.

It may make more sense, to some, to reuse what’s already available (JUnit for example, or the unit-testing modules already included in Python/Jython), but this is as much a learning exercise, as serious development, since I haven’t yet had to opportunity to apply TDD in any professional projects.

#! /usr/bin/env ../bin/jython

import base64
import sys

def authbasic(username, password):
    return 'Basic %s' % base64.encodestring('%s:%s' % (username, password))

global successtests
successtests = 0
global failedtests
failedtests = 0
global errortests
errortests = 0

def contains(srclist, matchlist):
   for i in matchlist:
       if i not in srclist:
           return false
   return true

#
# given a module name and a list of tests, run each test in the module
#
def runTests(mod, testnames=None):
   global successtests
   global failedtests
   global errortests

   exec 'import %s as testmod' % mod
   if hasattr(testmod, 'setup'):
       getattr(testmod, 'setup')()

   for name in dir(testmod):
       if name.startswith('test_'):

           if testnames:
               chkname = name[5:]
               if chkname not in testnames:
                   continue

           try:
               try:
                   # get the method from the module object
                   testmethod = getattr(testmod, name)

                   # run the test, and if successful increment our counter
                   testmethod()
                   successtests = successtests + 1
               except AssertionError, ae:
                   # handle assertion failures
                   failedtests = failedtests + 1
                   print '[%s] failed because of "%s"' % (name, str(ae))
           except Exception, e:
               # handle errors that fall outside of the test boundaries
               errortests = errortests + 1
               print '[%s] failed because of "%s"' % (name, str(e))
               (type, value, traceback) = sys.exc_info()
               print traceback.dumpStack()

   print '''%s tests succeeded
%s tests failed assertions
%s tests failed because of errors''' % (successtests, failedtests, errortests)

if __name__ == '__main__':
   if len(sys.argv) == 2:
       runTests(sys.argv[1])
   elif len(sys.argv) == 3:
       runTests(sys.argv[1], sys.argv[2].split(','))

NB: I’ve hacked out some of the functionality required to set up that particular test (such as creating and deleting the user record), but you get the idea.

A sample scripting testing a valid user of a user management web service would look like this:

from wstesting import MockServletContext, MockServletConfig, MockHttpServletRequest, MockHttpServletResponse

import miscutils
import test

import props

global user
user = None

def setup():
   # change the database connect string to use the test database
   props.db_connect_string = 'jdbc:postgresql:testdb'

   # dynamically load the module
   usermod = miscutils.loadmod('../web','user.py')

   # create a servlet context for testing with
   ctx = MockServletContext.getInstance().setRootPath('../web')

   # load the servlet and then initialise with the config object
   global user
   user = usermod.user()
   user.init(MockServletConfig('user'))

def test_getvaliduser():
   req = MockHttpServletRequest('GET', 'user', 'http://localhost:8080/ws/user/testuser1', '', '')
   req.setContentType('text/html')

   res = MockHttpServletResponse()

   user.doGet(req, res)

   assert res.getStatus() == 200, 'incorrect status %s (%s)' % (res.getStatus(), res.getReason())
   assert res.getContentType().startswith('text/html'), 'expected text/html, got %s' % res.getContentType()

So that’s the basic, lots of typing approach to TDD for my REST app. Next step is to completely scrap that structure, and do it the right way (i.e. a method that doesn’t involve writing the similar code over and over again for each service). More on that later…

NB. pseudofork will not work if called from the console, it must be called from a script.

The source for pseudofork can be found below:

import copy
import os
import re
import string
import sys
import threading

from java.io import InputStream
from java.lang import Thread
from org.python.util import PythonInterpreter
from org.python.core import Py

pat = re.compile(r'(s*).*')
repl_re = re.compile(r'myos.(forkpseudofork)()')

# not a real pid (obviously)
global pid
pid = 1000

lock = threading.Lock()

#
# a pretend fork routine, which basically creates a new thread,
# starts up a python interpreter in that thread, reads the source
# code from the calling script and then attempts to continue execution
# from that point in the new thread.
#
def pseudofork():
   frame = sys._getframe(1)
   if hasattr(threading.currentThread(), '__forkedcode__'):
       s = threading.currentThread().__forkedcode__
   else:
       f = file(frame.f_code.co_filename)
       s = f.read()

   split = s.split('n')
   callingline = split[frame.getline()-1]

   mat = pat.match(callingline)
   if mat:
       indent = len(mat.group(1))
   else:
       indent = 0
   sindent = ('%' + str(indent) + 's') % ' '

   lock.acquire()
   global pid
   pid = pid + 1
   lock.release()

   lines = [ repl_re.sub(str(pid), callingline) ]
   for x in xrange(frame.getline(), len(split)):
       if indent == 0 or split[x] == '' or split[x].startswith(sindent):
           lines.append(split[x][indent:])
       else:
           break
   code = 'n'.join(lines)

   localvars = copy.copy(frame.f_locals)
   if localvars.has_key('frame'):
       del localvars['frame']

   for key in frame.f_globals.keys():
       if not localvars.has_key(key):
           localvars[key] = frame.f_globals[key]

   thr = thread(code, localvars)
   thr.start()

   return 0

#
# we'll pretend to be a fork as well
#
fork = pseudofork

#
# thin wrapper over a java.lang.Thread to pseudo-fork some code
# (why didn't I use jython 'threading'?  I forget)
#
class thread(Thread):
   def __init__(self, execstr, localvars):
       Thread.__init__(self)
       self.execstr = execstr
       self.localvars = localvars

   def run(self):
       interp = PythonInterpreter(self.localvars, Py.getSystemState())
       threading.currentThread().__forkedcode__ = self.execstr
       interp.exec(self.execstr)