development

You are currently browsing the archive for the development category.

I’ve finally released the first version of the Installation Handbook for YAK (see here for more information). While this is just the first release, if you decide to purchase the handbook, you needn’t worry that it’ll soon become out-of-date — whenever I release a new version, I’ll be sending out copies to everyone who previously bought a copy.

The Handbook covers basic installation, setup for gateways such as PayPal and Google Checkout, downloadable products, multiple types of products, etc. If you’re having difficulty with the basic instructions on the YAK page, this is hopefully the best place to start.

I’ve just released YAK version 1.1.1 (although it won’t show up on the WordPress site for a while).

This release includes the following changes:

  • “Hard-code” the PayPal IPN page. This means you don’t need to create it manually. Instead just specify: http://yourdomain.com/wordpress/wp-content/plugins/yak-for-wordpress/yak-paypal-ipn.php
  • Add Italian translation, provided by Roberto Mogliotti.
  • Change the download message to use the tag [downloads] rather than a code.
  • Change the email confirmation message to use tags rather than codes.
  • Fix a nasty problem zeroing price and alternate title when commenting on a product.
  • Fix a problem with the random order number.

I’ve also removed support for the old tags (the html comment form such as <!−−yak_buy−−>), so you’ll need to change your posts to use square brackets if you upgrade (e.g.

YAK shopping cart plugin for WordPress
).

As usual, this release has only been fully tested by… well, me… so there may be some bugs I’ve missed — if you find something, please post on the Bugs page or email me directly.

I’m experiencing a rather annoying technical problem, which is holding up the YAK release — and consequently the release of the YAK Handbook.

Google Checkout integration requires https, but when testing I’m currently getting the dreaded error:

Protocol https not supported or disabled in libcurl

This despite the fact that curl -V gives:

curl 7.18.2 (i386-apple-darwin9.4.0) libcurl/7.18.2 OpenSSL/0.9.8h zlib/1.2.3
Protocols: tftp ftp telnet dict http file https ftps
Features: Largefile NTLM SSL libz

And port installed gives the following output (excerpted):

curl @7.18.2_0+ssl (active)
curl-ca-bundle @7.18.2_0 (active)
lighttpd @1.4.19_0 (active)
mysql5 @5.0.51a_0+server (active)
php5 @5.2.6_1+fastcgi+macosx+mysql5 (active)

I’m obviously missing something, but no idea what. Hoping that any Mac gurus who happen to come across this might have a suggestion or two…?

I’ve just released a bug fix (version 1.0.3e) for the stable version of YAK. This fixes the bug where prices are cleared if a customer submits a review (by commenting on a product).

If you’re using 1.0.3d, I recommend you upgrade immediately.

The experimental release (1.1.0-exp) is still unpatched, and will be replaced by version 1.1.1 in a few days.

There’s a nasty bug in the latest version(s) of YAK. It’s definitely present in 1.1.0-experimental and probably in 1.0.3d as well. Basically if you have comments enabled on products, and someone adds a comment, it can wipe the prices.

It’s fixed in trunk (in source control), and I’ll be releasing an updated 1.1.1 version shortly which includes the fix. I’ll also be releasing a patched version of 1.0.3d just in case people don’t want to upgrade to the latest and greatest.

In the meantime, if you’re running either of those versions, I recommend you disable comments on those products for the moment.

I’ve decided to release 1.1.0 of YAK for Wordpress as an “experimental” version. It hasn’t been fully tested yet, but enough people are hanging out for some of the features that I hope a few early adopters might kindly help out with the process.

1.1.0-exp includes features from the not (publically) released 1.0.4:

  1. Add new non-unique order id
  2. Move javascript on cart off page and into separate script. split javascript into generic and admin specific files — to stop problems with (missing) jquery on the wp pages.
  3. Add warning message for people with session.auto_start turned on.
  4. Integrate YAK back into the WordPress menu structure. The top-level menu was a nice idea, but it’s inconsistent with all other plugins (plus isn’t recommended, according to the WP devs). You can now get to YAK’s options through Settings->Yak, and to Orders/Products/Sales Reports from the Manage menu.
  5. Fix PHP4 problem with the view products screen

Along with a few additional features:

  1. Add https support
  2. Add first cut of Google Checkout integration.
  3. Tidy up setting payment types (drop downs now rather than input boxes). Add setting of a credit card landing page.
  4. Add support for tags with square brackets [ ]. The html-comment version of tags will be removed in an upcoming release

Google support is minimal so far. It’s one-way integration at the moment (allowing customers to submit an order through Google Checkout) — with no automated return path for handling purchase. That will come with a later release.

These updates hopefully make configuration a lot more user-friendly — particularly the changes to the Payment Types. Note that if you decide to upgrade, please make sure you backup your installation, and you double-check all your configuration options after upgrading.

YAK version 1.1.0-experimental can be downloaded from the WordPress Extend site.

A recent file rearrangement on my server resulted in all the links for “Snake Wrangling for Kids” breaking.

The problem is fixed now, and the zip files should be accessible again.

YAK 1.0.3d has been released (the WordPress plugin page hasn’t updated yet, but the files are currently available from Sourceforge).

Please backup your existing installation before upgrading. Deactivate, remove the current yak-for-wordpress directory, then copy the new version into the plugins dir and re-activate. You’ll need to double-check the configuration settings, because many input boxes have changed to drop-down selects (for category and page selection for example… see below). This upgrade will also require you to go through your products and set the price in the “YAK Product Details” tab (see point 5 in the changes below) — the custom fields are no longer required.

This release includes the following bug fixes:

  1. Problem with table creation.
  2. Country options not saving properly.
  3. Fix an odd problem with globals.
  4. The return URL appears to be broken with PayPal IPN. PDT works fine, but IPN seems to wind up back at the home page of the blog. Added a fix to cleanup the order from the shopping cart when this happens.

And the following changes:

  1. Frank Malina’s new look-and-feel.
  2. French translation (provided by Charles Dixon-Spain).
  3. Change money format and currency format to be simpler drop down boxes.
  4. Change the download url from the settings screen so it is now an override (i.e. optional).
  5. Add new data input on the post/page edit screen for yak price and title (replacing the old method of using custom fields).
  6. Remove the “Auto Set Price” option (doesn’t make a huge amount of sense now).
  7. Fix the Auto Set Quantity so it works with selected categories.
  8. Tidy up the stylesheet.
  9. Add a hidden link to the checkout and to the buy button form — vain hope that this might drive more search engine traffic to the YAK plugin page — if you don’t want the link included, it can be switched off on the Basic settings page.
  10. Change the product category (settings page) from an input box to a dropdown.
  11. Change the redirect on buy (setting page) from an input box to a dropdown.
  12. Change the return and cancel urls on the PayPal settings page to a dropdown so you can select a page rather than having to type a url.

The latest YAK release is slightly delayed because I’ve, thus far, been completely unable to test PayPal integration.

First problem: The return URL doesn’t appear to work (cancel return URL works fine). PayPal keeps sending me back to the index page of my test site.

Second problem: When trying to save the Website Payment Preferences (in PayPal Profile) I get a “Page Not Found” error. So there’s no way to change between IPN and PDT. This has been broken now for at least 5 days.

If anyone is using YAK and PayPal and is willing (and able) to test both IPN and PDT integration, please let me know. Worst case scenario, I’ll release this version as unstable, and wait for PayPal to fix their problems…

UPDATE: Problem solved. PayPal have updated their site, but this has somehow corrupted old test accounts. Deleting the old account and creating a new one fixes the problem.

Interestingly, there doesn’t seem to be a WordPress Community group on LinkedIn.

Or, possibly, there are dozens. There’s no way to tell since LinkedIn’s Group directory isn’t open and also isn’t searchable. Unless you’re lucky enough to find the relevant group on someone’s profile, you’re stuck with creating your own.

So that’s what I’ve done. If anyone is interested in joining, the invitation link is in the groups section of my profile.

And let’s hope LinkedIn develops a really good ratification process for merging millions of one-member groups… ;-)

If anyone is interested in previewing YAK 1.0.3, please get in contact. This release adds a new (admin) UI design (provided by Frank Malina), and fixes a couple of nasty bugs that emerged with the latest version of WP (tables not being created, globals not working as they used to, etc).

Apart from the new UI, there are no major feature improvements in this release, however there are a couple of “big ticket items” I’m thinking about adding in version 1.0.4.

I’ve just fixed (rather hurriedly) an annoying bug in domyinvoice.com. Invoices generated for a project with a daily rate (rather than hourly) were calculating the days incorrectly — basing the calculation on the number of tasks rather than the distinct list of days. Easy to fix, but since I haven’t been doing any Python for the last couple of months, it took a few hours longer than it should have.

A recent release of Do My Invoice was a minor update to add more documentation for the various resources used in the application. DMI is a REST application — well, at the very least, that was the initial design goal — so there are some basic access points for most system resources.

For example, you may call GET to request your user profile:

    http://www.domyinvoice.com/resources/[username]/profile

Using CURL the request:

curl -X GET -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/profile

…will return something like…

<profile>
    <username>joebloggs</username>
	<email_address>joe@bloggs.com</email_address>
	<first_name>Joe</first_name>
	<last_name>Bloggs</last_name>
</profile>

Resources in DMI are self-describing, thus with the addition of a single parameter, you can view the documentation for that resource. Thus the url…

http://www.domyinvoice.com/resources/joebloggs/profile?help

…returns a list of the HTTP verbs accepted by the resource, basic description and more detailed information about each verb. If you don’t have an account (it’s free for the first month, by the way), you can see a static version of this help information here.

Another example, is the clients resource. You can GET a list of clients:

curl -X GET -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients
<clients>
	<client id="76">
		<client-name>test company</client-name>
		<address1>address line 1</address1>
		<address2>address line 2</address2>
		<city>London</city>
		<postcode>sa1 sa2</postcode>
		<country>United Kingdom</country>
	</client>
</clients>

A client can be added with the use of an HTTP PUT:

curl -X PUT -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients/test+company+2 -d @-

address_1=23+Wrights+Lane&address_2=Kensington&city=London&postcode=W8+6TA&country=GB&currency_symbol=£&currency_format=1&sales_tax_label=VAT&sales_tax=0.175

Which also returns the created client (assuming nothing went wrong):

<clients>
	<client id="80">
		<client-name>test+company+2</client-name>
		<address1>23 Wrights Lane</address1>
		<address2>Kensington</address2>
		<city>London</city>
		<postcode>W8 6TA</postcode>
		<country>United Kingdom</country>
	</client>
</clients>

Said client could then be deleted with the use of the DELETE verb:

curl -X DELETE -u joebloggs:password http://www.domyinvoice.com/resources/joebloggs/clients/test+company+2

The obvious advantage of using RESTful resources internally is that I’m effectively providing an external API for free — in other words, without requiring any additional development effort.

The latest version of “Snake Wrangling for Kids” has been uploaded, and is now available on the main page. This is the LaTeX conversion I mentioned in a previous post — but I’ve now applied a Python code checker to 99% of the example source code. Hence I’ve fixed up a few omissions, bugs, and so on.

Automating the testing of example code proved more challenging than expected. In the end, I failed miserably to get doctest working they way I wanted, and had to roll my own testing code which works for a majority of the examples. It’s not perfect, but picked up a few problems, so I’m reasonably happy with the end result… even if it is a complete hackjob.

I started learning C programming in, or around, 1991. Up until that point, I had been exposed to Basic, Pascal, Modula-2, and (I think) Prolog. C was… somewhat of an eye-opener. Anyone who has had to come to grips with pointers and memory management in C, will know what I mean (and thank goodness for the Borland C IDE help system - unparalleled as a learning tool).

At University, we had papers that required learning a functional language and assembler, but neither of these were like the screaming brick wall that I hit in my second year when I decided to learn C.

Actually, since then, I can’t think of a single technology where I’ve experienced a similarly steep learning curve.

Until LaTeX.

I knew what LaTeX was, of course. But neither had the need, nor in fact, the interest, in learning it before. But after deciding I needed to move SWFK out of the atrocious (and, from an automation perspective, useless) word processing format it’s in at the moment; and after a few aborted attempts to get some reasonable output from various docbook toolsets, I’ve been provided with just the incentive.

But the learning curve is excrutiating.

Get one thing working, and the next thing stops. Change this and effect that. It was all me, of course. There’s nothing wrong with LaTeX after you get used to how it works. But, oh that learning curve. I haven’t felt such a need to scream and yell at my computer in a decade.

This is not to say things are perfect now. I still can’t get the front cover to look centered (advice from a LaTeX guru would be greatly appreciated) and, at times, LaTeX seems to think it knows, better than I do, where to put figures. Working out how to put visible spaces in a \verbatim block is also proving a little challenging. But the overall result does look, in my opinion, much nicer.

Anyone interested in proofreading the new version, please let me know.

Spring has a rather nice scripting framework that I haven’t paid much attention to, more from a lack of need than any real lack of interest.

One thing I recently noticed, is that there are only 3 supported scripting languages: Beanshell, Groovy and JRuby. Now Beanshell (for myself at least) was interesting about 4 or 5 years ago (maybe longer… since I can’t quite recall when it was first released), and I have little interest in Groovy (read “little interest” as “no interest”).

A bit of googling only uncovered a brief discussion about adding Jython support back in 2004 (I haven’t been able to find any code to go with that discussion), and Rhino-in-Spring which, as far as I can tell, is aimed at the web tier. Interesting that 2 of the most popular (IMO) scripting languages are rather conspicious by their absence from the core Spring distribution. Or if not absent, then certainly not very visible.

I’ve been looking for a reason to hack around with Spring extensions for a while, so this seemed like the opportune moment. Following one of the comments in that 2004 discussion, I started with cloning GroovyScriptFactory and a few other classes from the scripting package, and then hacked away the bits I didn’t want or need.

GenericScriptFactory is my abstract base class, and contains most of the code from GroovyScriptFactory. Subclass JythonScriptFactory is the workhorse for Jython scripting, while RhinoScriptFactory is obviously for Javascript code.

The cool part — I’ve implemented a NamespaceHandler for both, so bean configuration looks like this for Jython:

<script:jython id="test1" script-source="test.py"
               bean-name="mytest" return-type="test.TestInterface"
               interpreter="pythonInterpreter">
    <script:property name="test" value="jython prop from spring" />
</script:jython>

and for Javascript (Rhino):

<script:rhino id="test2" script-source="test.js"
              bean-name="mytest" return-type="test.TestInterface"
              scope="jsScope">
    <script:property name="test" value="javascript prop from spring" />
</script:rhino>

You’ll note that in both cases the attributes are passed to the factory, but properties are injected after the bean (in whatever language) has been created. A distinction which is rather useful when you think about it. This is the same as the other scripting languages already built-in to Spring.

You can see the full configuration file here, which includes setting up a PythonInterpreter for Jython, and a global scope for Rhino.

The sweet spot is, of course, this configuration:

<script:jython id="test3" script-source="test.py" bean-name="mytest" return-type="test.TestInterface" interpreter="pythonInterpreter">
    <script:property name="testInterface" ref="test2" />
</script:jython>

test2 is the object created by the Javascript code, and you’ll note that it’s being injected into the object created (in this case) by Jython code. The opposite works as well, meaning you can call Jython code from Javascript and Javascript code from Jython (well, at least through Spring injection). Exactly why you’d want to do this, I don’t know, but it’s a fun feature.

In either case, my classes implement a simple Java interface, meaning the beans created by both scripts are useable from Java as well — as you can see in the Test code:

public static final void main(String[] args) throws Exception {
    ApplicationContext context = new ClassPathXmlApplicationContext(new String[] { "beans.xml" });
    BeanFactory factory = (BeanFactory) context;

    TestInterface ti1 = (TestInterface) factory.getBean("test1");
    System.out.println("jython test: " + ti1.getTest());

    TestInterface ti2 = (TestInterface) factory.getBean("test2");
    System.out.println("rhino test: " + ti2.getTest());
}

Anyone interested can find the whole thing (very much a work in progress) in a Mercurial repository here.

Some time ago I decided I needed a text editor with minimal distractions. I can’t stand the sheer volume of buttons on the average word processor. Even AbiWord, which is skeletal in comparison with OpenOffice or Word, is still too busy at times (I miss Q&A Write actually — the Word Processor I used to use back in the days of MS-DOS).

A google search at the time turned up WriteRoom, which is Mac-only. But not much else.

So, of course, I developed my own alternative.

It didn’t quite fulfill the vision, so I dumped it in the “deprecated projects” list after it languished, unused. But I’ve been coming back to the idea lately, and finally did a bit more research on the various buttons you can push in PyGTK.

So, the Vanilla text editor is now -slightly- more supported than it was before (i.e. I’m probably going to use it myself), and is a bit closer to the vision. It’s completely configurable (background and foreground colours, font, key map, etc are all set from an ini file), and now runs full screen (as it should’ve from the beginning).

Only the source is available at the moment (Python2.5 and PyGTK2.0 required), but I’ll eventually make a commercially supported package available if there’s any interest.

YAK 1.0.1

YAK version 1.0.1 has now been released. Downloadable from either the WordPress plugins page, or from Sourceforge. This updates YAK to be compatible with WordPress 2.3, but unfortunately breaks backwards compatibility with prior versions of WP. If you’re using an older WordPress, I’m afraid you’ll have to stick with 1.0.

As well as WP2.3 compatibility, 1.0.1 adds paging to the “products” page, adds a facility to select the countries you support (see the options page), and adds a new quantity tag () to display product availability to your customers.

Disappointingly, but perhaps unsurprisingly, there are no plugins for WordPress which provide bug-tracking capabilities. Well, at least none that I’ve been able to find. I did consider the possibility of developing one myself, but I have enough to do in my spare time as it is. The family would probably kick me out of the house, if I started another project…

However, I did want a basic bug tracking system for Do My Invoice, and it makes sense to also use it for YAK (and perhaps some of my other low volume projects).

After a bit of searching, I came across Mantis, a PHP bug tracking system which looks simple enough for my needs, and perhaps approachable enough for other users to easily post their issues and feature requests. It was certainly easy to install, which makes it one of the more attractive candidates… (that said, I wouldn’t have minded using FogBugz, but don’t have the budget for the moment).

So if you have features or bugs for Do My Invoice, or for the YAK shopping cart plugin, please sign up for a new account at:

http://www.briggs.net.nz/bugtrack/

In the case of YAK, the comments on the project page is still the place for support issues and, in the case of DMI, the email for support requests can be found on the dashboard after you’ve logged in.

YAK, for Wordpress 2.3, is on its way.

There are structural database changes required for WP2.3, and while I don’t necessarily need to change the YAK tables as a consequence, I do need to change a few (relatively complicated) SQL statements. So if anyone has a test environment and would like to help, it would be great if someone (other than me) can check whether or not the changes have been completely successful.
I can package up the current tip of the repository and email out to those interested.

Please note that I’ve made the decision not to support older versions of WP (i.e. versions less than 2.3) from this point on, as supporting different versions of a database makes development inordinately complicated. That said, if you won’t (or can’t) upgrade to the latest version of Wordpress, and have some essential feature that you want in YAK, feel free to cross my palms with (quantities of) silver and I’d be happy to help. ;-)

The only other change that goes into this release is the support for paging in the YAK products page. This is the product list page (using the tag <!–yak_product_page–> tag) where your users can see all your current products (in order of newest to oldest). I’m only aware of one person actually using this feature, so for most, the only thing of interest in this coming release is the 2.3 support. More to come when I have the time.

I’ve just updated Do My Invoice with a few minor bug fixes, and some new functionality in the form of a download facility for project tasks (download for expenses is coming soon). Download in XML format took no time at all, since the functionality was basically already there, but a few changes were needed for supporting CSV output. Also included in this release were “presets” for the invoice date input fields (because, yes, I am that lazy), and a bug fix for the daily email service (which wasn’t working quite as well as expected). Behind the scenes, I’ve also finally -completed- the automated test scripts (thanks to Selenium), that will hopefully notify me when I’ve broken something. Nothing like a bit of a safety net to make you feel a little more comfortable.

Next task on the list is to fully document the (REST) API, which will actually also require some minor code updates, because of the way I want to do it. Not to be unnecessarily cryptic, but more on that later…

UPDATE: it helps if you check the menu options properly… blushsmiley

I don’t tend to waste my time investigating all the nooks and crannies of an IDE (Integrated Development Environment, for the non-developer-types). I typically just use the basics. Some might say I’m wasting time by not using those features, but it works for me.

So when the Eclipse & IntelliJ users on the team opened a dialog box and jumped straight to a source file by typing the first few letters of the filename, I just shrugged and went back to navigating through the project tree…. that is, until someone pointed out that Netbeans had the exact same feature — “Go To File” (so obviously a little investigation of menu options wouldn’t have gone astray).

Then, lo and behold, I discovered shortly after, that Netbeans’ “Go To File” function actually did more than Eclipse (and was at least equivalent to Intelli). You could type the capital letters of the class and it would list the matching classes. For two glorious months (or so), I had a feature that was at least as good as my colleagues. The months of enduring taunts, because of my obscure choice in IDEs, faded from memory as I jumped from file to file with a flick of a few (less) fingers… at least, until the latest version of Eclipse arrived with that feature.

Now Netbeans 6 beta 1 has been released. I’ve been burnt in the past by NB early releases — but this is a brand-spanking-new beta. Spit-polished with the glean of months of development effort, since then. Gosling is talking about it. The general consensus elsewhere appears to be that it’s good.

And, after upgrading, it does look good. Not a huge amount of visible difference from NB5. A few burrs here and there have been filed off the sharp edges. There are some things I don’t find attractive, but it’s bearable. Integration with source control doesn’t seem to thrash the CPU, which is a major improvement.

But then… Alt+Shift+O

My most-used feature.

The only thing I could (until recently) hold above the Eclipse users on the team, and say confidently (and maturely), “Nyah nyah nyah nyah nyah”.

One thousand curses on the perpetrator, for violating the perfection of the Alt+Shift+O! Yes, they’ve indeed extended it to search all filenames instead of just classes, but at the same time they’ve rendered it unuseable. Not only is it slower, it is s…l……o………w……………e……………………r.

It is grindingly slow.

If the phrase that immediately pops to mind to describe the fastest thing you’ve ever seen is “blazingly fast”, then a snail would be described as “blazingly fast” by this feature. A tortoise would have the opportunity to perform a full circuit around my desk, and execute a perfect pirouette, before this abysmal piece of code decided that it should return some results.

Slowness I could’ve forgiven if they hadn’t removed the only thing that (until recently — and yes, I know I keep saying that) made me better than the Eclipsers. The feature-in-the-feature that made me as good as the Intellis. Searching, via the capital letters of a class.

MCGF would find MyCrazyGeneratorFactory.java and MyCrazyGeneratorFactoryImpl.java. WDMCHSLN would find WhyDoMyClassesHaveSuchLongNames.java. As it is, I now have to type My (then wait an inordinately long amount of time) to find all files starting with “My”. Carry on typing if I want to narrow the results. With a bit more waiting…

Netbeans 6.

The upgrade that is actually a downgrade.

Now I have to endure the snickers of both the IntelliJ AND the Eclipse users.

Sigh.

Footnote: Welcome back NB5.5.1!

We’ve thrown a few ideas around at the office, for development projects we could do after hours. The idea was to come up with something interesting (i.e. techie) that’s different enough from the contract we’re working on that we’d be able to keep motivated.

Most of the ideas were far too ambitious (i.e. would require the resources of a Google, or significantly more development effort than we could commit), until someone came up with the idea of designing something we ourselves would find useful. Which is one of those simple, rather obvious ideas which we probably should’ve started with from the beginning.

That opened up a whole different “can of discussion worms”, with the end result being that a few of the guys went in one direction, and I went in another. Which is a roundabout way of bringing me to: “Do My Invoice“.

What I decided I really needed was a simple way to record timesheet information (outside of the client’s timesheet system), which could then be converted into an invoice at the end of the month. To me “record timesheet information” means a simple email interface, because if I leave this stuff until the end of the week, it’s invariably a time-wasting struggle to remember everything I’ve been doing (of course, that’s not all there is to it… but it is the fundamentals of the idea).

Now, you might argue that any number of packages can be used to accomplish the invoicing side of things: MYOB, Quickbooks, and so on. But since the number of invoices I’m generating in a year numbers in the teens, my accountant has me working from a spreadsheet-based cashbook. His argument is why waste the money and effort on an expensive package I’ll hardly be using — I couldn’t agree more.

I’m equally sure there are other web-based, invoicing systems out there (I already came across one three-quarters of the way through the development), but I have yet to see something cheap and simple enough to satisfy my purposes.

Hence Do My Invoice is a cheap and simple (I hope, user-friendly) web-based invoicing system, with hopefully enough features to make it attractive to spend the US$50 subscription per year. In true eat-your-own-dog-food fashion, I’m using my own application, so new features will be added as, and where, I discover the lack — and so feature requests are welcome. At some point, over the next few weeks, I’ll get some sort of bug tracking software installed so requests can go there (in the meantime info ‘at’ domyinvoice.com will go to the right place).

Feedback and critique welcome — except if it’s regarding the web design (which will be fixed as soon as I have more than just myself as a paying subscriber…)

Well, I haven’t had much time to work on YAK lately, but still have a list of things I want to add when I get a chance.

However, in the meantime, I have decided I like Mercurial enough to shift YAK off subversion at SourceForge, and into its own mercurial repository hosted on my website. The subversion repository won’t go away, since I haven’t bothered to replay the tags into the Hg repos (just the individual revisions).

So if you want access to YAK from source control, the new checkout command is:

hg clone http://www.briggs.net.nz/hg/yak yak

…and then…

hg update

to keep it up to date.

You will, of course, need to install both Python and Mercurial to do so — but the repository itself is also directly accessible from the URL: http://www.briggs.net.nz/hg/yak.

While I have pretty much all my code in source control of some kind (originally subversion, now mainly mercurial), the only repos that’s externally accessible is YAK for WordPress (currently a subversion repos hosted at Sourceforge). For something like my quick (read hack-job) attempt at the Stomp protocol, that’s somewhat less than optimal, since I do occasionally (i.e. like today) get updates — so either I have to upload multiple versions, or just overwrite the current version. Which isn’t much use if someone wants to track back to previous revisions.

So, after coming across Bill de HÓra’s “Mercurial First Impressions” and subsequent “Setting up Mercurial on TextDrive” a few weeks ago, it seemed a good time to do something about the situation.

Installation went reasonably well, apart from the obligatory, and rather obscure, Apache error: “Premature end of script headers” — where it took me an embarrassingly long amount of time to figure out that the hg directory needed to be chmodded to 755 as well as the hgwebdir.cgi file (despite the fact I’d checked the file and directory permissions a couple of times during the process… sigh).

The URL for the stomp.py Mercurial repository is now:

http://www.briggs.net.nz/hg/stomp.py/

To checkout the latest copy you can use the following command:

hg clone http://www.briggs.net.nz/hg/stomp.py stomppy

After which hg update will keep you in sync with the (all too infrequent) changes.

Now have to decide whether I want to move YAK off sourceforge and into its own Mercurial repository. The control-freak, tech-geek part of me, gleefully says yes. The lazy, I’ve-got-too-much-other-stuff-to-do part says “pass”.
Time will tell… hopefully before I start arguing with myself.

It’s one thing to play around with a source control tool on smaller-scale, personal projects — it’s another to decide to use it on a large-scale project with confidence (something I understand Sun have been doing with OpenSolaris). There’s evidence out there (if you go hunting for it), but personal experience counts for more, particularly if you hope to convince those with plenty of experience with other tools.

The obvious answer, is to take a large-scale project project and load it into Mercurial. The java project I’m leading at work has evolved over the last 18 months into a rather excessive 2.2GB (for trunk alone). I hate to think how much disk space would be required if I actually checked out every tag and branch.

There are tools (such as Tailor) to migrate from one SCM to another, but tailor, at least, appears to have some issues. So, after a bit of fruitless googling, I decided to roll my own script (of course). Luckily, I’m not particularly bothered by a less than accurate migration — all I want is to reproduce the volume of commits to see that 1) a mercurial repository *really* is smaller than the equivalent in subversion, and 2) performance is better than the abysmal response times I’m getting from subversion (which isn’t going to be hard when a majority of hg commits are local, with the occasional remote push).

The results…?

Well, after about 5 days of intermittent running, I’m about 80% of the way through the migration (mainly because of the aforementioned snail-paced subversion response), but Mercurial appears to be holding up well. At the very least, hg operations are responsive. I’ve excluded files like library jars, since Mercurial warns you about adding files to the repository that are over 10MB in size, so this is a bit of an apples-and-oranges comparison (I can’t really compare the overall repository sizes if I’m excluding files from one, but not the other, so I’m going to fail testing point 1), but that’s not really my fundamental goal anyway. I can say, with more than a reasonable level of confidence, that Mercurial looks viable for larger scale projects.

I’ve been playing with Mercurial lately, after having previously tried Git out for a few weeks.

I must say, I prefer Mercurial, though I would be hard-pressed to quantify that into some kind of this-feature-is-better-than-that-feature list (apart from the most obvious advantage that it’s cross platform, of course). For some reason, it just feels right. Which is weird, I know.

This is not to say Mercurial is perfect. Because there are a few niggles. The one that jumps immediately to mind is the fact that no matter where you are in a repository tree, hg status presents the list of changed files from the repository root — not the current context directory. So if you then copy and paste a file name (with path) from that status list, to commit a single change, you get an error. Because subdir/filename does not exist when you’re already in subdir. Small niggle, but there it is.

Second problem I’ve had recently: hg rm file, where file is the last in the directory. Mercurial appears to remove the directory. If you then try to commit, it throws its hands up in horror. Again, only a minor annoyance.

Apart from that, I like Mercurial enough to migrate all my other personal projects off subversion. I’d also be really interested to see how it handled a really large enterprise project, since subversion (at work) is groaning under the strain of handling that project. Chances of that happening are minimal, and would require replicating almost 2 years worth of commits, to really test it, but it would be interesting to see the results…..

I thought I’d already noted this somewhere, but a quick search failed to find the answer.

A new Postgres database installation always catches me out, when it comes to creating a user. I usually do the following:

1. update permissions in pg_hba.conf (to allow text password access, for example)

2. createuser -W &lt;username&gt;
(prompted for a password)

Then scratch my head when I try to create a database and get the following error:

createdb: could not connect to database postgres: FATAL: password authentication failed for user "&lt;username&gt;"

The answer is:

~$ sudo su - postgres
~$ psql
postgres=# alter user &lt;username&gt; with password '&lt;password&gt;';
ALTER ROLE

One of those annoying little details that is extremely easy to forget.

Spent the best part of a couple of hours investigating the possibility of Google Checkout integration in YAK.

It’s not looking positive, to put it politely.

Google Checkout only accepts POSTed content, which means I can’t just add a simple function to include the functionality (like I did with PayPal). I’ll have to modify the final confirmation page in YAK’s checkout to include the google parameters, and then (because I use a Location header to perform the redirect to the final payment page) I have to send an HTTP 307 before redirecting to Google, to stop PHP from converting the HTTP POST to an HTTP GET. Result –> an ugly confirmation dialog (at least from Firefox), to let the user know they’re about to be redirected.

More problems: Checkout only appears to support GBP and USD. I can’t figure out how to configure it to allow shipping to other countries, so I couldn’t even submit the order for my initial request. In addition, I couldn’t even get the shipping to appear in the checkout screen. No clue as to why that was happening.

Next problem: Checkout only allows confirmation of orders to a site supporting https. That’s fine for production, but no good for my development efforts (particularly since I can’t be bothered setting up SSL on my test apache server).

So I’m putting Google Checkout on the backburner for a few days. Things might start to work if I come back to it fresh… My worry is, that it’ll require a complete rework of the process in order to support it. Considering I have precious little time to waste at the moment, I’m not convinced the outcome is worth the effort.

I know!! Google should pay me a few hundred thousand dollars to sort it out…. ;-)

On a related (YAK) note, thanks to habibbijan for the pointer to a blog article about YAK, at InterestingMoney.com.

A week or so ago I, rather grumpily, concluded that git wouldn’t work the way I wanted it to, because of differences between the push and pull mechanisms:

…I liked the way Git handles branches, but I think the differences between the push and pull mechanisms are a pain to deal with…

Turns out, if I’d RTFM properly, I might’ve discovered that I should’ve been looking at the “git for CVS users” documentation (which is surprisingly difficult to find from the home page), rather than the “git for SVN users” page…

…despite the fact that I’m an svn user.

The sections on “Developing against a shared repository” and “Setting Up a shared repository” appear to particulary relevant to the way I want to work (at least on one project). The main point is, I can use ssh for both push and pull access to a central repository (despite the fact a centralised repository is not the optimal way to use a distributed source control tool).

If you want to preload images before displaying them in the browser, a convenient way to do so is to use image.onload.

For example, the following code shows the onload in action:

img = new Image();

var onloadFunc = function() {
    alert("got here... you really should do something now");
};

img.onload = onloadFunc;
img.src = url;
if (img.complete) {
    onloadFunc();
}

That works in most situations, but I’ve just discovered IE6 is seriously buggy (so what’s new?) in its onload event handling. The specific problem is preloading an image from the body’s onload event (or any other method that fires after the page has loaded, such as a script at the bottom of the page, etc).

A morning of googling failed to find a solution, so for future-me’s reference, basically the best way to ensure images are preloaded at the end of a pageload is something like the following:

var timer = null;
var images = new Array(); 

// trigger this from the body's onload
function onPageLoad() {
	images[0] = new Image();
	images[0].src = 'url to some image';
	images[1] = new Image();
	images[1].src = 'url to another image';
	timer = window.setInterval("onPageCheck()", 250);
} 

function onPageCheck() {
	for(var i = 0; i &lt; images.length; i++) {
		if (!images[i].complete) {
			return;
		}
	}

	window.clearInterval(timer);
	images = null;

	// now use the images
	...
}

Basically, create an array of images (where the .src property triggers the preload), then use a timeout to check if the load is completed.

This appears to get around the problem. Note that the onload seems to work fine when triggered outside of the page onload (a button click and so on).

One of my earliest posts (back when I was using blogger.com), was regarding a reverse logging proxy — a simple Python script to transparently log requests between a client and a server. Very useful for debugging web service calls, to see what exactly what is being sent back and forth.

I’ve just added logproxy.py to my projects page, after updating it to handle virtual hosts (in other words, when using Apache to serve more than one domain).

This is not quite as impressive a change as it might sound. It basically means correctly setting the Host HTTP header, rather than defaulting to localhost…

Anyway, a few people appear to have found it useful, judging from my access logs, so this update might help someone else (it certainly helped me figure out why my wsgi app was failing to properly handle multipart form content).

I’ve been playing around with distributed source control systems over the last few days. I like the idea of disconnected commits, with occasional pushes onto a central server. This is particularly useful on the occasions when I don’t need my development server running.

So far I’ve tried Git, Mercurial and Bazaar.

I haven’t come to any definite conclusions yet about any of them, other than that they are all extremely easy to setup and start using in a local environment, but rather more difficult when it comes to shared environments.

In addition, I liked the way Git handles branches, but I think the differences between the push and pull mechanisms are a pain to deal with. I didn’t like the fact that Mercurial pops up a GUI tool to deal with conflicts, although that is more likely to be a lack of understanding how to configure it correctly.

Upon reflection, the documentation for each tool was fundamentally lacking in some way, that made it difficult to figure out how to accomplish what I wanted. The much maligned Subversion was infinitely easy to get running and start using, in comparison.

So back to the drawing board, I think. There are certainly plenty of other tools to choose from (Arch, monotone, etc).

I’ve documented my quest for templating nirvana before (here, here, here and here… oh and also here), and thought I’d come to a reasonably satisfactory conclusion in some of my latter experiments.

My premise was that none of the templating engines currently available (you name the language/platform), come close to what I feel should be the fundamental goal of xml/xhtml templating — complete separation of markup from the code that populates it with dynamic data. There are a few templating systems that come close (Tapestry’s templating in the Java world and HTMLTemplate in Python, for example), but none that pushed all the buttons I thought needed to be pushed.
So, to this lofty ideal, I offered the idea of using a simple ID on elements and then putting all complexity in code to handle repeating of child elements, setting of attributes, setting node values, hiding elements, etc.

All in all it seemed to work for the (admittedly limited) cases I tried, but that’s the problem with grandiose schemes that are mainly based in the theoretical. Come time to apply the ideas in a practical sense, and you suddenly discover you haven’t thought it through well enough. Or to be more precise, I suddenly discovered I hadn’t thought it through well enough.

Case in point: I believe that in some situations it should be easy to use a different template without having to make modifications to the generating code. Which is where my simple IDs fall over in a quivering heap. Here’s an example (contrived) which immediately breaks it:

Example 1:
<users>
	<user name="Joe Bloggs">
		<address>10 Test St</address>
		<city>somewheresville</city>
	</user>
</user>

Example 2:
<html>
<body>
	<ul>
		<li>Joe Bloggs
			<ul>
				<li>10 Test St</li>
				<li>somewheresville</li>
			</ul>
		</li>
	</ul>
</body>
</html>

The xml in example 1 uses an attribute for the user’s name. The html in example 2 uses a textual element for the name, followed by another unorder list for the address elements. I could make the examples more complicated to show even more disparities, but you hopefully get the idea: if we’re just using an ID to distinguish elements, how do you say “set the attribute name to this value” for example 1 and “set the element value to this” for example 2, without having massively complicated code that either allows for both cases (and uses some kind of logic to figure out which should be applied), or has different code for each?

It’s naff. To put it politely. Not to mention that littering a template with unnecessary IDs is just as ugly as littering it with code.

So, in no way have I managed to reach templating nirvana. I have, however, demonstrated that if you’re not using this stuff in anger, there’s no way you’re going to come up with a good solution. Certainly some of the templating systems I’ve used (JSF springing immediately to mind) tend to suggest that whomever came up with the concepts didn’t follow through and use them either. So at least I’m in good company.

Enter the latest vain attempt.

My current thinking is that the template needs 3 ID attributes, one for repeating an element (rid), one for setting attributes (aid), and one for setting node values (eid). This gives a reasonable amount of flexibility and allows for the same code to populate different templates.

This code is very much a work in progress… and very much a completely inelegant mess. But it does satisfy a couple of requirements: 1) I am actually using it, and 2) the same code can be used to populate a variety of templates without requiring code changes.
It uses 4suite’s xpath to set values, and the use of xpath no doubt means performance will be adversely affected, but it’s a start.

In terms of the above examples, you might use this engine as follows:

<users rid="user-list">
	<user aid="user-name" name="">
		<address eid="user-address"></address>
		<city eid="user-city"></city>
	</user>
</user>

<html>
<body>
	<ul>
		<li rid="user-list" eid="user-name">
			<ul>
				<li eid="user-address"></li>
				<li eid="user-city"></li>
			</ul>
		</li>
	</ul>
</body>
</html>

tmp = Templates()['users.xml']
tmp.repeat('user-list', 2)
tmp.setelement('user-name', 'Joe Bloggs', 1)
tmp.setattribute('user-name', 'name', 'Joe Bloggs', 1)
tmp.setelement('user-address', '10 Test St', 1)
...etc

In the case of the first template, the attribute name is set with the value “Joe Bloggs”, and for the second template, the element value is used. The idea is, don’t use the relevant ID (rid, eid, aid) if you don’t need to apply that value.

More to come as I iron out the wrinkles…

Note: the template engine uses the Borg pattern.

I’ve just discovered that my implementation of a Digest Authentication middleware for WSGI, which had been working perfectly with Firefox, fails miserably when I try it with IE.

A bit of googling finds the following…

http://www.eweek.com/article2/0%2C1895%2C1500432%2C00.asp
http://www.extremetech.com/article2/0,1697,20373,00.asp

…making me think that I’m basically stuffed, despite the fact that I followed the RFC. Excellent work Microsoft! (Sarcasm alert!)

Should’ve done some reading first I think.

So now a decision. Persevere with digest auth and upgrade (irreversibly?) to IE7 in the vain hope that it works, or roll back to Basic auth which will obviously support more browsers.

Much as I’d like to stick with digest (and see if my implementation works in IE7), I think Basic auth is the safer option (at least coupled with SSL), particularly when you look at browser market share.

Had a question from a YAK user wondering if anyone is using it with a credit-card payment service like DPS?

I didn’t think so, figuring if someone had hacked an additional payment option, they would’ve probably let me know already… but just in case, if you are, please tell me and I can put you in contact.

UPDATE: Of course, the easier alternative, is just to upgrade to Kubuntu Feisty… ;-)

More for my own reference than anything else….

To get mod_python working with python2.5 on kubuntu:

1. Install apache2, if you haven’t already.

2. Install python2.5:

sudo apt-get install python2.5
sudo apt-get install python2.5-dev

Don’t get rid of python2.4, since it’s still used by a number of things.

3. Change the symlink for /usr/bin/python to point at the new version:

sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.5 /usr/bin/python

4. Install apache apxs2:

sudo apt-get install apache2-threaded-dev

5. Download and extract the latest dist of mod_python. cd to that directory, configure and build:

./configure
make

6. Possibly a controversial step: install modpython as normal (apt-get), then replace the shared-objects with the .so’s you’ve just created:

sudo apt-get install libapache2-mod-python

From the downloaded modpython directory (i.e. where you ran configure & make), copy the shared-object files:

sudo cp src/mod_python.so /usr/lib/apache2/modules/
sudo cp dist/build/lib.linux-x86_64-2.5/mod_python/_psp.so /usr/lib/python2.4/site-packages/mod_python/

Copy the mod-python directory from python2.4 site-packages to 2.5:

sudo cp -R /usr/lib/python2.4/site-packages/mod_python/ /usr/lib/python2.5/site-packages/

Restart apache and use modpython as normal…

Every time I have a requirement for a basic CSS popup box of some kind, I end up going through the same process. Hunt through old code looking for straightforward examples, then trawl through the googleweb trying to find something that isn’t an example of a CSS popup menu.

I’m sure pure CSS popup menus are all well and good, but it’s painful picking out the useful bits.

So, more for my own future reference, a simple CSS popup can be accomplished using the following CSS:

a span {
	display: none;
	text-decoration: none;
}

a:hover {
/** fix for IE6 popup bug.  nice one Microsoft! */
	overflow: hidden;
	text-decoration: none;
}

a:hover span {
	display: inline;
	border: 1px solid black;
	position: absolute;
	background-color: white;
	padding: 5px;
	margin-left: 5px;
	overflow: hidden;
}

The first reference hides the popup, the second redisplays, using margin-left to move the popup a little to the right of the href it’s linked from.

Then create HTML as follows:

<p>this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test this is a test
<a href="#">popup<span>test message 1<br />test message 2</span></a>
this is a test this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test this is a test this is a test
this is a test this is a test this is a test this is a test this is a test this is a test </p>

Check this link for an example.

A few things I’m finding a minor annoyance in Netbeans at the moment…

1. Netbeans is not an island.

I’m the only one using Netbeans on our project. I’m the small deformed child, in the corner of the class, that none of the other kids wants to play with.
Our projects are all driven off ant build scripts, so the logical way to use NB (when everyone else is using Eclipse or Intelli) is a free-form project (i.e. “Java Project with Existing Ant Script”). Okay, that works fine, but when you want to compile a single file from within the IDE, the ant script ide-file-targets.xml is created — that’s still okay, I can hack so that the internal classpath in that file points at the classpath in our main build script. But what about code completion, and the other IDE tools that rely on knowing the classpath?
First, right-click on the Project, select Properties, then Java Sources Classpath, then reproduce the classpath you’ve already created in the build script….?

Hmmm. That’s a word that starts with A and rhymes with “pass”.

A related note: I hate the ant icon on freeform projects in NB6. Go back to the old one.

 

2. A Project is not an island.

If NetBeans is not an island, neither are the projects in it. If you hit Ctrl and move your mouse pointer over a class name you can jump to the implementation… but not if that class is defined in another project. There might be a way to do this — but it ain’t obvious.

 

3. Why can’t I turn off Editor Hints?

I don’t like them. I don’t need them. And if I do, I’ll ask. But the problem is, there’s no obvious option that says “Use Editor Hints” to turn them off, and I’ve been through the advanced options — a number of times.

Besides, checkstyle covers all the stuff we actually want to check for.

 

4. “Help” is rubbish.

Stick it in a proper browser, so I can at least hit CTRL+F and find something in the current page. Oh yeah, and my up and down arrows don’t work in the help window either. Might be a Linux issue. But maybe not.

 

5. Where’s “Jump to Resource”?

Alt+Shift+O opens a dialog to jump to a “type” (class/interface/etc), which is nice. But Eclipse will open any resource: Python scripts, build scripts, whatever. Gimme that instead.

 

6. Fix Imports doesn’t remember what I chose last time.

Chances are if I select commons LogFactory one time, I’m going to want to select it the next time. I do not want to select it from a drop-down menu every time.

 

7 Minimal installation wanted.

I want to specify a minimal installation. All non-essential modules switched off. Everything that doesn’t stop the IDE from falling over in a steaming heap. Code completion yes — built-in Tomcat, app server integration, HTTP monitor, J2EE blueprints, integration, etc, etc, etc… absolutely not. I’ll switch on the bits I want, thanks all the same. In fact, don’t even bother including them in the distribution. Download on demand.

 

8. Reformat

If you say reformat xml, surely it means fully reformat? Not just reformat a bit. Either that, or relabel the menu option to “Half-assed Reformat Code”.

 

Okay, so that’s only eight. But Top Ten rolls off the tongue better than Top Eight… :-D