App reviews: APN/3G/GPRS/EDGE kill switch

With a smartphone like the Android based devices a lot of programs continue running in the background, even when the screen is off. Email and calendar sync, Google Latitude, etc. So it is important to be able to kill the data connections for these services when they are not needed, to save battery power and money.

Of course you could just use flight mode, but that really defeats the purpose of a phone, doesn’t it.

The two apps under consideration are APNdroid and Toggle Data. Both are free from the Android Market.

APNdroid works by changing the name of your APN entries, invalidating them, making cellular data connections unavailable. The interface is an app which you can place as a shortcut on your home screen. Run the app once it disables the connection and places a message in your notification. Run it a second time and it enables the connection again.

Toggle Data is similar in interface, again with an app shortcut, but using a toast, or on screen message to indicate enabled or disabled. Also, Toggle Data does uses an alternative approach to disabling the data connection, perhaps the same as the flight mode does?

With both apps you need to make sure you enable your data connection again before removing them, else you might be in a little bit of trouble.

For the record, I’ve stuck with Toggle Data because I prefer the toast to the notification (I have enough notifications as it is), and I have the feeling its method of disabling the connection is cleaner (but I can’t really back that up)

Lenny missing out

As a long time Debian user I have really come to appreciate the Debian repository system. With stable, testing, unstable and experimental points in the release cycle to choose from. Typically I run testing, with occasional packages from unstable and experimental. Or, put another way, I want to use the newest “stable” releases of software which are going to be in the next proper release of Debian.

However, as the package freeze for the next Debian release, Lenny, has been in place for some time it has prevented new packages from making their way into testing. I’m not sure if Lenny is actually over due or not, but what I am sure of is that it will put Debian behind the times with three very important pieces of desktop software: Openoffice, the kernel, and network management.

Openoffice.org (OOo) in Lenny is going to be set at 2.4, only missing out on Openoffice 3.0 by some weeks. The hassle here is that OOo 3.0 is significantly nicer to use than OOo 2.4. Firstly, because it supports a multipage view in writer meaning I can take advantage of a larger desktop screen. Secondly, because the multihead view actually works in Presenter, putting it back in the ring against Power Point, etc. As well it is faster and smoother to use.

The kernel will be 2.6.26. 2.6.28 has been in the wild for a few months now, and supports devices like the Atheros wireless network card which is found in MacBooks and many other laptops. This is significant because the previous option to get these cards working was to use MadWifi, which uses a closed source HAL object file, which isn’t in keeping with the free and open Debian ethos.

And finally, the network manager for Lenny will be 0.6.6. 0.7 is in experimental, and it supports GSM modems, and has better support for VPNs, which is very significant for the travelling business applications.

All of these newer packages will be available in the next release of Ubuntu, and are already in the latest Fedora Core, which leaves Debian behind the curve in this market. Of course people could, like me, use a mix of release packages, but a line should be drawn somewhere to prevent packages in a frozen-for-release repository from getting so out of date. Given the quality of Debian packages have become very good, and reliable, it would be better to unfreeze a release after a month to allow newer, tested, software in. This assumes that this won’t affect the “blockers” which have so far prevented the release.

In summary, experimental isn’t as scary as it sounds, and perhaps we need a different philosophy on the release cycle in this rapidly changing world of software.

Projects for the new year

These are some project idea’s I’ve been sitting on all year and have not yet started, but hope to in the new year.

1. Python CMS framework in the vein of Drupal, based on Repoze BFG

So Drupal is quite a popular community building framework, with good plugin system, and a data model which seems to work well. But I don’t think PHP is the way of the future, at least for me. I use Python for many other projects, I understand how to debug, unit test, etc. And BFG brings the really nice framework features from Zope and Plone, which makes it an ideal starting point. I’ve got a project plan for this which I’ll reveal when I get around to cutting some code.

2. Google Gadget for Bugzilla

We use Bugzilla to do service and software task tracking. It would be really swell if I had a quick list of outstanding tasks, and the ability to quickly add new one. GG works on all platforms now, so I think it is the way to go. I’d prefer this over another FF plugin too, just because it fits more naturally into  this level of things.

3. Signing of public pages which you endorse.

Wikipedia isn’t the only place “anonymous” people post their ideas and understandings, but it’ll work for this example. There can be issues with trusting online source, but what if I could sign a version of a page (the content, not the entire layout). I might be an expert in the field for instance. Then someone browsing to the page can see that I endore it. If they don’t know who I am they can also see what else I endorse, and find out more about me. Then, through social networking, their friends who trust their judgement, perhaps just one a particular subject, can judge the quality of a post through the GPG/PGP web of trust.

ZSI -> CXF: Parameters coming in as NULL

Recounting a strange little compatibility issue I had between ZSI 2.0 and CXF 2.0.x. I was using CXF as the server, running from Maven using Jetty, and ZSI as the client. The parameters from the ZSI were arriving at the service implementation as null. With the web services logging turned on I could see that the SOAP packet was arriving OK, and looked good.

A little bit of digging round the web turned up this message.

The issue was that ZSI wasn’t explicitly namespacing the elements, and so CXF was not seeing them. The solution was to add elementFormDefault=”qualified” to my WSDL definition, and rebuild the ZSI stubs.

<wsdl:definitions xmlns:soap=”http://schemas.xmlsoap.org/wsdl/soap/”
    …
    xmlns:xsd=”http://www.w3.org/2001/XMLSchema”
    xsd:elementFormDefault=”qualified”
    xsd:attributeFormDefault=”unqualified”>

Trac taking a hammering

At work we have one VM which hosts all our project management software like Git, SVN, Trac and Bugzilla. However, recently it has been taking a hammering and essentially crashing. The issue was it was running out of RAM, and swapping like crazy. A little investigation into the situation uncovered that there were some Trac 0.10 CGI processes which were using >500MB of memory.

Using the ps ewww -p <pid> command to look at the enviornment variables of the processes we determined that each of them was serving the same kind of request, SVN changesets. And essentially all of them were being hit by spiders.

Essentially the issue was that the spiders were managing to create a large number of overlapping changeset requests, which were chewing up all the RAM on the VM. Increasing the amount of RAM wouldn’t only offset the problem. So, we now just block the spiders from the changeset areas of our tracs (robots.txt):

User-agent: *
Request-rate: 1/5
Disallow: /projects/data-activities/changeset
...

In the weeks since then we’ve had no issues. Besides, who wants their gaffs showing up on google

Interestingly we’ve not seen the same thing happen with Trac 0.11, so perhaps the issue is resolved now.

Zope3 Component Architecture (CA) style Adapters for Java

After programming for Zope3/Plone for the past year I’ve come to really
admire the flexibility and elegance that their implementation of the
adapter pattern gives us.
And, after Martin Aspeli put the call out almost a year ago, and it has not yet been answered, I thought it was time to give it a go.
How hard could it be.

So, what I’ve developed is a very simple, no dependency, maven available library which should greatly improve the flexibility of your code.

The project page is here
and the repository is at Git Hub.

From the desk in 60 seconds

Quick roundup of whats what:
1) Wireless in UK hotels.

It’s bad. I went to Edinburgh, Manchester and London recently and stayed in some ok 3 and 4 star hotels which all advertised wifi/internet access. Here’s the rundown:
 a) Grassmarket hotel in Edinburgh: Free wireless only available in the pub/common rooms
 b) Fountain Court – Harris in Edinburgh: Really fast and reliable free wireless in the room
 c) The Palace Hotel in Manchester: Fast, expensive wired access in the room.
 d) St Mark Hotel, Earls Court in London: Didn’t actually advertise wifi, and didn’t have it

2) Inheriting templates with chameleon.zpt (1.0a1) and repoze.bfg (0.3.8)

This is only noteworthy because it isn’t documented. I’m assuming the reader knows how to do metal/tal in zope.

Create a master.pt in the usual way, using <html metal:define-macro=”master” and metal:define-slot=”main”.

Create a template pt with <html metal:use-macro=”main.macros['master']“ and metal:fill-slot=”main”.

The trick is to manually load the master and pass it to the render_to_response. In the Django PT renderer they scan the templates dir and load all the templates, making this unnecessary.

from repoze.bfg.chameleon_zpt import render_template_to_response, get_template

def my_view(context, request):
    main = get_template(‘templates/master.pt’)
    return render_template_to_response(‘templates/mytemplate.pt’, project = ‘test’, main=main)

3) Text to speech hotkey in Gnome

First, install festival and xclip.
Next, get the script off the gentoo wiki, and make it executable.
Now, fire up gconf-editor, browse to app/metacity/keybinding_commands
Find a spare command and put in the path to the script.
Finally browse to app/metacity/global_keybindings, find the run_command_# key and enter the hotkey you want.

Now test it out. It should work on selected text, ie no need to copy it explicitly. (Gnome gets notified when you change things in gconf so there is no service restarting).

Writting a DiigoDelicious plugin for Conduit

Tonight I started trying to write a bookmark syncing tool for my Delicious and Diigo bookmarks. Being of the Python persuasion I gave Conduit 0.3.11 a spin as the framework. This turned out to be a good choice, except for the incorrect documentation which suggests that custom modules should be placed in ~/.conduit/modules. In fact they should go in ~/.config/conduit/modules.

I got as far as writing a Bookmark data type (from the marketing I would have thought this should have already existed), and a Diigo data source. When I turned to the Delicious side of the equation I discovered that:

1. The API has just changed; and
2. The library I wanted to use (DeliciousAPI) doesn’t seem to support adding/updating entries.

So the project is now on hold until the Delicious API is updated to the latest API. Either way, Conduit seems like a very flexible framework and I hope it has long-term prosperity. Perhaps in due course it could natively support bluetooth SyncML

MyProxy server segfaulting

Setting up a new MyProxy (v3.9 12 Jun 2007 PAM OCSP) from VDT 1.8.1 I ran across an annoying segfault issue when running the server to run as a CA, in debug mode.

# myproxy-server -dmax_proxy_lifetime: 43200 seconds
PAM enabled, policy requiredCA enabledmax certificate lifetime: 43200 seconds
using storage directory /var/myproxy
Starting myproxy-server on localhost:7512...
Connection from 127.0.0.1
using trusted certificates directory /opt/vdt/globus/TRUSTED_CA
Authenticated client <anonymous>
applying trusted_retrievers policy
trusted retrievers policy matched
applying authorized_retrievers policy
applying authorized_renewers policy
Program received signal SIGSEGV, Segmentation fault.

Note, the client can connect and only fails once the client responds with their credentials. Using GDB I got the stack trace of the issue:

#0  0x008f3950 in strip_newlines (string=0x901d13 "unknown error\n")    at myproxy_log.c:72
#1  0x008f3a95 in myproxy_log_verror () at myproxy_log.c:141
#2  0x0804ca27 in myproxy_authorize_accept (context=0x9fcc020,     attrs=0x9fcc008, client_request=0x9fd70c0, client=0xbfd892e0)    at myproxy_server.c:1445
#3  0x0804ae5a in handle_client (attrs=0x9fcc008, context=0x9fcc020)    at myproxy_server.c:465
#4  0x0804a932 in main (argc=2, argv=0xbfd897e4) at myproxy_server.c:308

Noting that the exception is in the myproxy_log_verror method, and looking at the myproxy code I found this method is only activated when in debug mode. So, assuming all my config was good I ran the myproxy-server proper and all was good.

Some thoughts on a dynamic (lazy) data access layer for web services

The problem I’ve been address in my most recent work block has been to develop an interface to a relational data store which can be used either as a local DB, or via webservices. The concept is quite straight forward, we want CRUD operations on a set of data objects. The schema is fairly straight forward too, with a core hierarchy of elements with a few enumerated lists. As well as a few cuts across the hierarchy.

We want to use this with webservices, as well wanting to be able to serialise to an XML document, so we layed out the schema in XSD, and applied HyperJAXB3 to is using a JAXB annotations document. Other than a few tweaks to the HJ3 code to add in some extra features we needed, everything this far is vanilla.

We then laid out a generic DAO interface, and implemented a JPA and webservices client. The webservices server actually just wraps the JPA client with a security layer.

All is well so far, with the base functionality passing all the unit tests. But, this is a fairly clumsy, class by class CRUD interface. With JPA/Hibernate you can lazily fetch parents and children of an object, and replicating this functionality would be really useful. But the webservices link really kills things.

One approach we tried was to override the getter methods on the beans to allow dynamic retrieval of data if the stored attribute is null. But we needed the webservices client to create these new (extended) beans. This is actually harder than it sounds, as CXF w/ JAXB does not allow you to replace the context factory. (see https://jax-ws.dev.java.net/issues/show_bug.cgi?id=282 )

We also tried writing our own proxies, with some success. But we ran into troubles when proxying single elements (as opposed to lists) as you loose the annotations which CXF and JPA require.

The obvious answer, which I haven’t mentioned yet is to put this logic in the getters proper. This would be nice, but because we are generating the classes from a schema we don’t want to change the generated code. There are many reasons, none less than because the schema is evolving and it hurts to merge changes. We attempted to write our own JAXB plugin to change the methods which were already generated, but XJC/the-java-code-model do not allow you to remove code from a method, only append to it. Similarly, you can’t get the annotations on a method, nor modify existing annotations. This is a huge problem for us, as it would save us having to modify the JAXB plugins which create this code, rather we could simply apply our changes to the produced code.

Perhaps there is a way around this, but it certainly isn’t obvious, or indexed by Google.

The closing remarks are these. With a bit of backfilling JAXB/XJC will be an immensely powerful tool, able to develop a wide range of data models. (I’m working on the tickets now). Dynamic getter access to attributes via web services would be nice, but it is too much hard work really, and if it is required then creating a second set of domain objects is your only real hope, IMHO. When this project wraps up I’ll release the code here so we can all think about how this approach really faired.