internet

A recent release of Do My Invoice was a minor update to add more documentation for the various resources used in the application. DMI is a REST application — well, at the very least, that was the initial design goal — so there are some basic access points for most system resources.

For example, you may call GET to request your user profile:

    http://www.domyinvoice.com/resources/[username]/profile

Using CURL the request:

curl -X GET -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/profile

…will return something like…

<profile>
    <username>joebloggs</username>
	<email_address>joe@bloggs.com</email_address>
	<first_name>Joe</first_name>
	<last_name>Bloggs</last_name>
</profile>

Resources in DMI are self-describing, thus with the addition of a single parameter, you can view the documentation for that resource. Thus the url…

http://www.domyinvoice.com/resources/joebloggs/profile?help

…returns a list of the HTTP verbs accepted by the resource, basic description and more detailed information about each verb. If you don’t have an account (it’s free for the first month, by the way), you can see a static version of this help information here.

Another example, is the clients resource. You can GET a list of clients:

curl -X GET -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients

<clients>
	<client id="76">
		<client-name>test company</client-name>
		<address1>address line 1</address1>
		<address2>address line 2</address2>
		<city>London</city>
		<postcode>sa1 sa2</postcode>
		<country>United Kingdom</country>
	</client>
</clients>

A client can be added with the use of an HTTP PUT:

curl -X PUT -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients/test+company+2 -d @-

address_1=23+Wrights+Lane&address_2=Kensington&city=London&postcode=W8+6TA&country=GB&currency_symbol=£&currency_format=1&sales_tax_label=VAT&sales_tax=0.175

Which also returns the created client (assuming nothing went wrong):

<clients>
	<client id="80">
		<client-name>test+company+2</client-name>
		<address1>23 Wrights Lane</address1>
		<address2>Kensington</address2>
		<city>London</city>
		<postcode>W8 6TA</postcode>
		<country>United Kingdom</country>
	</client>
</clients>

Said client could then be deleted with the use of the DELETE verb:

curl -X DELETE -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients/test+company+2

The obvious advantage of using RESTful resources internally is that I’m effectively providing an external API for free — in other words, without requiring any additional development effort.

Is it just me, or do ISPs in New Zealand tend to operate on the south-side of the “moral tracks”? At the moment, I’m trying to decide whether it’s worth having an argument with Orcon over the advertising for their “Zero Shock” plan (see “The Yogurt Container Broadband Plan“) — supposedly, this plan is “traffic managed” to keep email and web access fast, and the speed of P2P applications are reduced as a consequence (by accepting this, you get the piece of mind that if you go over your datacap, you won’t get charged extra… as long as you don’t keep going over every month). For me, traffic management actually meant I had the average speed of a 28K modem most evenings for any site out of the country. Of course, if you look at their Terms & Conditions, there’s just enough caveats there to make them liable for nothing. Legally they’re fine. I quibble, however, on the dubious morals that allow a company to sell a “broadband” plan that they know barely meets the definition at the best of times.

To be honest, I really have no argument with Orcon otherwise. It just seems like false advertising to call something “Zero Shock”, when I find myself damn shocked at how slow it is.

The other ISP I have experience with is IHUG, who, if they reduce their prices for some reason, will keep quiet about it… “it’s your responsibility to check our website”. I’ve experienced this and so, more recently, has my father. Seems you should focus on keeping your current customers happy, rather than just attracting new ones.

That’s two of NZ’s larger ISPs, and I know the largest (Xtra) has had problems in the past.

I wonder if the issue is industry wide?

It’s an impressive feat.

When my daughter was about 2 and a half years old, we got a long piece of string, poked it through holes in a pair of yogurt containers, pulled the string tight and used it as a telephone. It was difficult to get the concept across to a 2 year old; of talking into the container, then listening for the response. It was also difficult keep the string taut enough, so that sound waves were transmitted properly.

Somehow, Orcon have managed not only to keep a piece of string tight over thousands of kilometres, they have also managed to connect the yogurt container to the local internet loop. An extremely impressive piece of engineering. Even if it does mean my international traffic is slower than a 28K modem. Just think of the science!

So it’s now almost a month since I first noticed the problem, and over 3 weeks since I reported it — and there is still no resolution in sight. At least a couple of emails have gone unanswered in the last few weeks, although the initial response from their support department (instructions on how to provide all the data their technicians would need) was positive. Last night I finally gave up waiting and decided to sit on hold for 10 minutes or so. Their helpdesk, while nice enough (and local), were unable to shed any light and would need to pass on the details to technicians. But…

“…we have had reports from other customers of similar problems… it may have something to do with the traffic shaping…”

Yes. I’m blimmin’ well aware that it’s traffic shaping. I could’ve told you that a month ago.

“…a quick fix might be to upgrade your plan…”

Where’s the sound of screeching tyres when you want it?

Upgrade my plan?

I must remember that if I ever get a complaint about software I’ve developed…

“I’m sorry you’re having problems sir. Other people have also experienced problems with that version of our software. I invite you to upgrade to a more expensive version, in which we have fixed that issue.”

PS. Unsurprisingly, still no response from technical support.

If you want to preload images before displaying them in the browser, a convenient way to do so is to use image.onload.

For example, the following code shows the onload in action:

img = new Image();

var onloadFunc = function() {
    alert("got here... you really should do something now");
};

img.onload = onloadFunc;
img.src = url;
if (img.complete) {
    onloadFunc();
}

That works in most situations, but I’ve just discovered IE6 is seriously buggy (so what’s new?) in its onload event handling. The specific problem is preloading an image from the body’s onload event (or any other method that fires after the page has loaded, such as a script at the bottom of the page, etc).

A morning of googling failed to find a solution, so for future-me’s reference, basically the best way to ensure images are preloaded at the end of a pageload is something like the following:

var timer = null;
var images = new Array(); 

// trigger this from the body's onload
function onPageLoad() {
	images[0] = new Image();
	images[0].src = 'url to some image';
	images[1] = new Image();
	images[1].src = 'url to another image';
	timer = window.setInterval("onPageCheck()", 250);
} 

function onPageCheck() {
	for(var i = 0; i < images.length; i++) {
		if (!images[i].complete) {
			return;
		}
	}

	window.clearInterval(timer);
	images = null;

	// now use the images
	...
}

Basically, create an array of images (where the .src property triggers the preload), then use a timeout to check if the load is completed.

This appears to get around the problem. Note that the onload seems to work fine when triggered outside of the page onload (a button click and so on).

One of my earliest posts (back when I was using blogger.com), was regarding a reverse logging proxy — a simple Python script to transparently log requests between a client and a server. Very useful for debugging web service calls, to see what exactly what is being sent back and forth.

I’ve just added logproxy.py to my projects page, after updating it to handle virtual hosts (in other words, when using Apache to serve more than one domain).

This is not quite as impressive a change as it might sound. It basically means correctly setting the Host HTTP header, rather than defaulting to localhost…

Anyway, a few people appear to have found it useful, judging from my access logs, so this update might help someone else (it certainly helped me figure out why my wsgi app was failing to properly handle multipart form content).

I’ve just discovered that my implementation of a Digest Authentication middleware for WSGI, which had been working perfectly with Firefox, fails miserably when I try it with IE.

A bit of googling finds the following…

http://www.eweek.com/article2/0%2C1895%2C1500432%2C00.asp
http://www.extremetech.com/article2/0,1697,20373,00.asp

…making me think that I’m basically stuffed, despite the fact that I followed the RFC. Excellent work Microsoft! (Sarcasm alert!)

Should’ve done some reading first I think.

So now a decision. Persevere with digest auth and upgrade (irreversibly?) to IE7 in the vain hope that it works, or roll back to Basic auth which will obviously support more browsers.

Much as I’d like to stick with digest (and see if my implementation works in IE7), I think Basic auth is the safer option (at least coupled with SSL), particularly when you look at browser market share.

Every time I have a requirement for a basic CSS popup box of some kind, I end up going through the same process. Hunt through old code looking for straightforward examples, then trawl through the googleweb trying to find something that isn’t an example of a CSS popup menu.

I’m sure pure CSS popup menus are all well and good, but it’s painful picking out the useful bits.

So, more for my own future reference, a simple CSS popup can be accomplished using the following CSS:

a span {
	display: none;
	text-decoration: none;
}

a:hover {
/** fix for IE6 popup bug.  nice one Microsoft! */
	overflow: hidden;
	text-decoration: none;
}

a:hover span {
	display: inline;
	border: 1px solid black;
	position: absolute;
	background-color: white;
	padding: 5px;
	margin-left: 5px;
	overflow: hidden;
}

The first reference hides the popup, the second redisplays, using margin-left to move the popup a little to the right of the href it’s linked from.

Then create HTML as follows:

<p>this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test this is a test
<a href="#">popup<span>test message 1<br />test message 2</span></a>
this is a test this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test </p>

Check this link for an example.

Every time I sit down to do something with mod_python, I have problems. Usually they can be traced to forgetting to RTFM.

It doesn’t help that I tend to put experimental code in my apache www dir on occasion — and forget to back it up — so that after a system reinstall, I then have to dredge the dim, dark recesses of my memory to remember how to do simple stuff.

So for my own future reference…

1. a WSGI handler (wsgi_handler.py) for mod_python can be found here
2. the handler needs to be accessible on the python path, so copying it to /usr/lib/python2.4/site-packages means it’s accessible (there’s a better way, but it momentarily escapes me)
3. the apache config required is:

   <Directory /var/www/test>
   PythonHandler wsgi_handler
   PythonOption WSGI.Application hello::simple_app
   AddHandler python-program .py
   </Directory>
   

   Where:

   • /var/www/test is an absolute path to a directory I’m serving on the web
   • hello is a python script (hello.py — see below)
   • simple_app is a function in the script

   In the case of my kubuntu laptop, I just created a mod_python.conf file containing the above in /etc/apache2/mods-enabled

4. hello.py, a simple WSGI app, looks like this (snaffled from the python PEP for WSGI):

   def simple_app(environ, start_response):
       status = '200 OK'
       response_headers = [('Content-type','text/plain')]
       start_response(status, response_headers)
       return ['Hello world!\n']
   

5. restart apache, and hopefully we’re away…

I’ve also added an extension to make wsgi_handler slightly more useful for my purposes; the ability to serve more than one ‘app’ from the same script. The extended version can be found here.

The apache configuration can then be modified to:

<Directory /var/www/test>
PythonHandler wsgi_handler
PythonOption WSGI.Application hello
AddHandler python-program .py
</Directory>

And hello.py now looks like this:

def simple_app(environ, start_response):
    status = '200 OK'
    print str(environ)
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello WSGI world!\n']

def test(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-Type','text/plain')]
    start_response(status, response_headers)
    return ['test test test\n']

Meaning that you can now point your browser to:

http://localhost/test/test.py/simple_app

or:

http://localhost/test/test.py/test

Which is useful for experimentation.

I’ve done a little more work on ‘my’ approach to templating. One thing I’ve only mentioned in passing, is that there are other templating engines out there that work in this fashion — both in Python and Java. HTMLTemplate, is a python module I’ve already mentioned previously. Tapestry is a Java framework, which I haven’t personally used, but seem to recall uses an attribute language for its templates. And there are others, so what I’m experimenting with is hardly anything new. But I have yet to see one that I think goes far enough with the removal of markup from view components. Not to harp on about it too much (not half).

Read the rest of this entry »

I finally took an hour to revisit the templating code mentioned back here, using minidom instead of rparsexml. As expected, refactoring produces better code, but ‘better’ is a relative term — it’s still a major hack-job. However, in this case, I’m not bothered as long as it proves a point with as little effort as possible.

So, to recap my prior thoughts:

1. I don’t like the complexity, and general inelegance, of the templates required to produce dynamic content with facelets & JSF. Faces on its own is bad enough; facelets (while admittedly helping to fix a lot of the nastiness inherent in JSF) adds its own ugliness factor.
2. I don’t think it would be hard to design something better. Actually, let me rephrase that:

   An electric charge applied to a few single-celled organisms contained in a small bucket of slime could design something better.

My intention is, therefore, to hack together a quick prototype — hopefully coming up with something a little more elegant in the process. And since (I hope) I have a few brain cells to rub together, I might just provide reasonable competition to the bucket of slime.

Read the rest of this entry »

No sooner have I set up a brand-spanking new site, than I am suddenly too busy to do anything with it. Not only that, but I also gave myself a challenge to come up with an example of something better than the butt-ugly facelets/JSF templating, and I’ve had no time to do anything much about that either. I have, however, decided that trying to hack together Amazon page would be a good test of the layout concepts, especially taking into consideration how busy the Amazon front page is. So I’ll document the process as I go… thinking aloud, so to speak.

I have hit one minor snag already. I’ve used rparsexml.py in my hacked together template engine (a piece of python code that will admittedly never win any code-elegance competitions), only to discover (RTFM) it dumps html comment blocks — which is perhaps only a pain if you happen to wrap your embedded styles in comments. So I haven’t entirely convinced myself that it’s not worth going back and revisiting the code with a better xml parser, motivation being in general short supply at the moment…