1. removed the ‘lower()’ on header keywords to support case-sensitive headers (patch submitted by Eugene S).

2. added the facility to wait until the socket is connected before issuing a “CONNECT”

3. removed the space between header and value (i.e. header: value becomes header:value) which fixes an incompatibility with RabbitMQ

4. added the missing connect headers (user/pass) to the “CONNECT” command (again fixes an incompatibility with Rabbit).

As usual changes are committed and you can get the latest version from my Mercurial repo here.

The problem is fixed now, and the zip files should be accessible again.

There are a couple of reasons why this is more of a challenge than the other versions of the book:

1. I don’t have an OLPC
2. The version of Python installed on the OLPC includes neither Tkinter nor, more importantly, the turtle module.

The first point is easy to solve. I’ve been running OLPC on VirtualBox (occasionally) for a few months now, so at least I have a vague idea of how to interact with the Sugar interface.

The second point is slightly more problematic. As far as I’m aware, the OLPC runs GTK, and so the pygtk module is available — but I have yet to come across a turtle implementation for gtk (ignoring, for the moment, the TurtleArt activity already installed on the OLPC, which I don’t think is particularly useful for my purposes).

It seems like quite a fun project — implementing the turtle module using pygtk — until you consider the arcane pygtk API, the (IMHO) lack of reasonable documentation (I’m not particularly impressed by the pygtk reference manual… particularly the lack of index), and simple examples to expand from.

Technical issues aside, a week later, I’m considering the semantics of an OLPC edition of the book. SWFK for Windows, Mac and Linux, all target the same fundamental audience. An OLPC edition has a completely different potential audience. The Western+English market is, no doubt, vanishingly small — so while, to date, I’ve had over 6500 downloads, I’m guessing an OLPC edition might garner less than 1-3% more. The real market would be translated versions (assuming the interest in translating actually results in translations) — but that begs another question: will kids in non-Western countries actually understand some of the references? Is talk of DVD players, in-car computers, Nintendos, etc (i.e. some of the references in chapter one, for example), at all meaningful in the developing world? I’m doubtful.

Which leads me to posit the question, is it worth the not-so-insignificant effort?

What do you think?

Genius or madness?

This proved rather more challenging than expected (rather characteristic of LaTeX as a whole actually), and I haven’t fully proofed the final result yet. Those interested can check out the new editions here:

SWFK - Linux Edition
SWFK - Mac Edition
SWFK - Windows Edition

The Mercurial repository (here) containing the LaTeX source has also been updated with the latest changes.

SWFK is still a work in progress — although that progress has been rather slow since we moved to the UK — but I can’t see any reason why I should put off releasing the LaTex source until some sort of mythical completion date, particularly not when there are willing participants out there.

So for those who are interested in translating SWFK to another language, the latex “source” can be found in the following Mercurial repository:

http://www.briggs.net.nz/hg/swfk

Note that it isn’t “buildable” in its current state. I haven’t added the image files yet because some of them are rather large (the cover alone is over 2MB) — the wonders of the EPS format. Mercurial doesn’t seem to handle excessively large image files that well (at least not on my web host it doesn’t), so if anyone has ideas on that front, let me know.

Interestingly though, I’ve somehow made the second page of Google results (currently sitting at position 16) for the keywords “web based invoicing”.
However, for the term “web based invoice”, the site is either so far down the list it can’t be found, or it doesn’t appear at all.

Also interesting: 3 weeks ago, DMI was at position 83 for the search term “invoices”, yet I can’t find it anywhere in the first 100 pages now — obviously we hit a broken rung in the Google ladder and fell down a few hundred feet.

Completely off-topic, but one thing this has highlighted as a gap in Google’s toolset is “Search Within”. It would be really useful to enter a few search terms, then find exactly where another term sat in those results. In other words being able to enter something like: “domyinvoice within: web based invoicing“, which gave you the results found within a set of results, along with the index positions. Seems like a killer Google feature for web developers, marketers and the like.

The only major change is to add a few exercises to the end of some of the chapters, plus a new appendix for the answers. This is a work in progress. Exercises are currently missing from Chapter 9, and possibly aren’t as detailed (nor fun) as they should be.

But they’re a start…

Automating the testing of example code proved more challenging than expected. In the end, I failed miserably to get doctest working they way I wanted, and had to roll my own testing code which works for a majority of the examples. It’s not perfect, but picked up a few problems, so I’m reasonably happy with the end result… even if it is a complete hackjob.

At University, we had papers that required learning a functional language and assembler, but neither of these were like the screaming brick wall that I hit in my second year when I decided to learn C.

Actually, since then, I can’t think of a single technology where I’ve experienced a similarly steep learning curve.

Until LaTeX.

I knew what LaTeX was, of course. But neither had the need, nor in fact, the interest, in learning it before. But after deciding I needed to move SWFK out of the atrocious (and, from an automation perspective, useless) word processing format it’s in at the moment; and after a few aborted attempts to get some reasonable output from various docbook toolsets, I’ve been provided with just the incentive.

But the learning curve is excrutiating.

Get one thing working, and the next thing stops. Change this and effect that. It was all me, of course. There’s nothing wrong with LaTeX after you get used to how it works. But, oh that learning curve. I haven’t felt such a need to scream and yell at my computer in a decade.

This is not to say things are perfect now. I still can’t get the front cover to look centered (advice from a LaTeX guru would be greatly appreciated) and, at times, LaTeX seems to think it knows, better than I do, where to put figures. Working out how to put visible spaces in a \verbatim block is also proving a little challenging. But the overall result does look, in my opinion, much nicer.

Anyone interested in proofreading the new version, please let me know.

A google search at the time turned up WriteRoom, which is Mac-only. But not much else.

So, of course, I developed my own alternative.

It didn’t quite fulfill the vision, so I dumped it in the “deprecated projects” list after it languished, unused. But I’ve been coming back to the idea lately, and finally did a bit more research on the various buttons you can push in PyGTK.

So, the Vanilla text editor is now -slightly- more supported than it was before (i.e. I’m probably going to use it myself), and is a bit closer to the vision. It’s completely configurable (background and foreground colours, font, key map, etc are all set from an ini file), and now runs full screen (as it should’ve from the beginning).

Only the source is available at the moment (Python2.5 and PyGTK2.0 required), but I’ll eventually make a commercially supported package available if there’s any interest.

So, after coming across Bill de HÓra’s “Mercurial First Impressions” and subsequent “Setting up Mercurial on TextDrive” a few weeks ago, it seemed a good time to do something about the situation.

Installation went reasonably well, apart from the obligatory, and rather obscure, Apache error: “Premature end of script headers” — where it took me an embarrassingly long amount of time to figure out that the hg directory needed to be chmodded to 755 as well as the hgwebdir.cgi file (despite the fact I’d checked the file and directory permissions a couple of times during the process… sigh).

The URL for the stomp.py Mercurial repository is now:

http://www.briggs.net.nz/hg/stomp.py/

To checkout the latest copy you can use the following command:

hg clone http://www.briggs.net.nz/hg/stomp.py stomppy

After which hg update will keep you in sync with the (all too infrequent) changes.

Now have to decide whether I want to move YAK off sourceforge and into its own Mercurial repository. The control-freak, tech-geek part of me, gleefully says yes. The lazy, I’ve-got-too-much-other-stuff-to-do part says “pass”.
Time will tell… hopefully before I start arguing with myself.

I must say, I prefer Mercurial, though I would be hard-pressed to quantify that into some kind of this-feature-is-better-than-that-feature list (apart from the most obvious advantage that it’s cross platform, of course). For some reason, it just feels right. Which is weird, I know.

This is not to say Mercurial is perfect. Because there are a few niggles. The one that jumps immediately to mind is the fact that no matter where you are in a repository tree, hg status presents the list of changed files from the repository root — not the current context directory. So if you then copy and paste a file name (with path) from that status list, to commit a single change, you get an error. Because subdir/filename does not exist when you’re already in subdir. Small niggle, but there it is.

Second problem I’ve had recently: hg rm file, where file is the last in the directory. Mercurial appears to remove the directory. If you then try to commit, it throws its hands up in horror. Again, only a minor annoyance.

Apart from that, I like Mercurial enough to migrate all my other personal projects off subversion. I’d also be really interested to see how it handled a really large enterprise project, since subversion (at work) is groaning under the strain of handling that project. Chances of that happening are minimal, and would require replicating almost 2 years worth of commits, to really test it, but it would be interesting to see the results…..

Sigh.

On a completely unrelated note, a special thanks to the various people who’ve recently linked to this site (specifically to the YAK page). I was never that bothered by an abysmally low technorati ranking, but now find myself absurdly pleased when I’ve climbed up by about half-a-million.

Well… I’m still in the 500,000s anyway, it’s an improvement. A-list? Can you say Z-list?

I’ve just added logproxy.py to my projects page, after updating it to handle virtual hosts (in other words, when using Apache to serve more than one domain).

This is not quite as impressive a change as it might sound. It basically means correctly setting the Host HTTP header, rather than defaulting to localhost…

Anyway, a few people appear to have found it useful, judging from my access logs, so this update might help someone else (it certainly helped me figure out why my wsgi app was failing to properly handle multipart form content).

My premise was that none of the templating engines currently available (you name the language/platform), come close to what I feel should be the fundamental goal of xml/xhtml templating — complete separation of markup from the code that populates it with dynamic data. There are a few templating systems that come close (Tapestry’s templating in the Java world and HTMLTemplate in Python, for example), but none that pushed all the buttons I thought needed to be pushed.
So, to this lofty ideal, I offered the idea of using a simple ID on elements and then putting all complexity in code to handle repeating of child elements, setting of attributes, setting node values, hiding elements, etc.

All in all it seemed to work for the (admittedly limited) cases I tried, but that’s the problem with grandiose schemes that are mainly based in the theoretical. Come time to apply the ideas in a practical sense, and you suddenly discover you haven’t thought it through well enough. Or to be more precise, I suddenly discovered I hadn’t thought it through well enough.

Case in point: I believe that in some situations it should be easy to use a different template without having to make modifications to the generating code. Which is where my simple IDs fall over in a quivering heap. Here’s an example (contrived) which immediately breaks it:

Example 1:
<users>
	<user name="Joe Bloggs">
		<address>10 Test St</address>
		<city>somewheresville</city>
	</user>
</user>

Example 2:
<html>
<body>
	<ul>
		<li>Joe Bloggs
			<ul>
				<li>10 Test St</li>
				<li>somewheresville</li>
			</ul>
		</li>
	</ul>
</body>
</html>

The xml in example 1 uses an attribute for the user’s name. The html in example 2 uses a textual element for the name, followed by another unorder list for the address elements. I could make the examples more complicated to show even more disparities, but you hopefully get the idea: if we’re just using an ID to distinguish elements, how do you say “set the attribute name to this value” for example 1 and “set the element value to this” for example 2, without having massively complicated code that either allows for both cases (and uses some kind of logic to figure out which should be applied), or has different code for each?

It’s naff. To put it politely. Not to mention that littering a template with unnecessary IDs is just as ugly as littering it with code.

So, in no way have I managed to reach templating nirvana. I have, however, demonstrated that if you’re not using this stuff in anger, there’s no way you’re going to come up with a good solution. Certainly some of the templating systems I’ve used (JSF springing immediately to mind) tend to suggest that whomever came up with the concepts didn’t follow through and use them either. So at least I’m in good company.

Enter the latest vain attempt.

My current thinking is that the template needs 3 ID attributes, one for repeating an element (rid), one for setting attributes (aid), and one for setting node values (eid). This gives a reasonable amount of flexibility and allows for the same code to populate different templates.

This code is very much a work in progress… and very much a completely inelegant mess. But it does satisfy a couple of requirements: 1) I am actually using it, and 2) the same code can be used to populate a variety of templates without requiring code changes.
It uses 4suite’s xpath to set values, and the use of xpath no doubt means performance will be adversely affected, but it’s a start.

In terms of the above examples, you might use this engine as follows:

<users rid="user-list">
	<user aid="user-name" name="">
		<address eid="user-address"></address>
		<city eid="user-city"></city>
	</user>
</user>

<html>
<body>
	<ul>
		<li rid="user-list" eid="user-name">
			<ul>
				<li eid="user-address"></li>
				<li eid="user-city"></li>
			</ul>
		</li>
	</ul>
</body>
</html>

tmp = Templates()['users.xml']
tmp.repeat('user-list', 2)
tmp.setelement('user-name', 'Joe Bloggs', 1)
tmp.setattribute('user-name', 'name', 'Joe Bloggs', 1)
tmp.setelement('user-address', '10 Test St', 1)
...etc

In the case of the first template, the attribute name is set with the value “Joe Bloggs”, and for the second template, the element value is used. The idea is, don’t use the relevant ID (rid, eid, aid) if you don’t need to apply that value.

More to come as I iron out the wrinkles…

Note: the template engine uses the Borg pattern.

A bit of googling finds the following…

http://www.eweek.com/article2/0%2C1895%2C1500432%2C00.asp
http://www.extremetech.com/article2/0,1697,20373,00.asp

…making me think that I’m basically stuffed, despite the fact that I followed the RFC. Excellent work Microsoft! (Sarcasm alert!)

Should’ve done some reading first I think.

So now a decision. Persevere with digest auth and upgrade (irreversibly?) to IE7 in the vain hope that it works, or roll back to Basic auth which will obviously support more browsers.

Much as I’d like to stick with digest (and see if my implementation works in IE7), I think Basic auth is the safer option (at least coupled with SSL), particularly when you look at browser market share.

UPDATE: Of course, the easier alternative, is just to upgrade to Kubuntu Feisty…

More for my own reference than anything else….

To get mod_python working with python2.5 on kubuntu:

1. Install apache2, if you haven’t already.

2. Install python2.5:

sudo apt-get install python2.5
sudo apt-get install python2.5-dev

Don’t get rid of python2.4, since it’s still used by a number of things.

3. Change the symlink for /usr/bin/python to point at the new version:

sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.5 /usr/bin/python

4. Install apache apxs2:

sudo apt-get install apache2-threaded-dev

5. Download and extract the latest dist of mod_python. cd to that directory, configure and build:

./configure
make

6. Possibly a controversial step: install modpython as normal (apt-get), then replace the shared-objects with the .so’s you’ve just created:

sudo apt-get install libapache2-mod-python

From the downloaded modpython directory (i.e. where you ran configure & make), copy the shared-object files:

sudo cp src/mod_python.so /usr/lib/apache2/modules/
sudo cp dist/build/lib.linux-x86_64-2.5/mod_python/_psp.so /usr/lib/python2.4/site-packages/mod_python/

Copy the mod-python directory from python2.4 site-packages to 2.5:

sudo cp -R /usr/lib/python2.4/site-packages/mod_python/ /usr/lib/python2.5/site-packages/

Restart apache and use modpython as normal…

Actually, it’s not the main reason for not posting, since I wrote the piece back in July, but I’m still using it as an excuse until the next couple of projects come to fruition…

I wrote a simple python script a while back — the last time I couldn’t find such a tool — and so now that I need to use it again, it seems like a good time to polish it up and release it (here). Not before chopping out a bunch of specific hacks that really need tidying up before being included in released code, of course.

So if you need a basic tool for handling HTTP requests (GET, PUT, POST, DELETE) you may find it useful… rather than having to write the few lines of code yourself.