There are some problems that you can only really solve, let alone understand, by diving into code. Lucky enough the Open Source movement has spread like wildfire these days, also for developer tools. So besides that it’s fun and very educating to read someone else’s source code, in come cases it’s also really, really helpful to find out how a program is doing its tricks internally to really understand how to best use it. In particular this problem case that I’ve been investigating including a combination of SCons, SWIG and Python.
It’s a pretty straightforward setup. You’ve got this C++ library which you want to export to Python. You’ve already setup an interface to Lua, using SWIG. Now the cool thing about SWIG is, that it’s really easy to export the same interface to different scripting languages, so you want to use SWIG’s power to use your existing setup and extend that from Lua to Python as well. You take your existing .i file, divide it up into two separate files, a general one and a Lua specific one. Say common.i and lua.i where you include common.i in lua.i. Then you can create your new python.i with the Python specific instructions for SWIG, again including common.i. Problem solved.
Well, not if you’re using SCons as your build tool and you put the %module statement in common.i. That does work for Lua, but for Python strange things will happen with SCons’ build dependencies on the .py wrapper file generated by SWIG. I had to go all the way into SCons’ Tool/swig.py to find out what the problem was. While SCons scans for %module statements, which, if found SCons uses to define the .py file build dependencies for Python extensions, %include statements aren’t honored as such. That’s when I undestood that the problem was my Python specific python.i which didn’t have a %module statement! I put that into common.i, reusing it for Lua and Python, thinking that would be ok. However, that setup will mislead SCons to thinking that there are no %module statements at all. So build dependencies will be incorrect, leading to build errors.
Lesson learned: when using SCons, always put the %module statements for SWIG in the .i file that is used to directly generate the scripting language extension from.
I wanted to start out with some wxPython GUI testing and was trying to get dogtail installed on my Ubuntu 10.04 system. The current dogtail version that you can install with Synaptic is 0.6.1, while I want to try out 0.7.0. So I grabbed the source from dogtail’s website, unpacked it into /usr/local/src and ran:
$ cd /usr/local/src/dogtail-0.7.0
$ sudo setup.py install
However, when I tried to run sniff for example, it can’t find it’s image files, because it’s looking explicitly in /usr/share instead of /usr/local/share. Ok, so let’s try again:
$ sudo setup.py install --prefix=/usr
This time, sniff start complaining that there’s no module named dogtail.config. A bit of Googling tells me that Ubuntu doesn’t have /usr/lib/pythonx.y/site-packages in its path by default. Only site-packages inside /usr/local/lib/pythonx.y and dist-packages in /usr/lib/pythonx.y are added to Python’s path by default. This is to clearly separate the place for packages that are included in the Ubuntu distribution (dist-packages) and those you install yourself (site-packages).
Now there’s a special argument for distutils if you want to install a “distribution” package manually, namely --install-layout=deb. Passing this argument will put the package using /usr as prefix and using dist-packages instead of site-packages, as if you were installing a deb package. So:
$ sudo setup.py install --install-layout=deb
That did the trick for me. Now back to what I was originally wanting to figure out: wxPython GUI scripting and testing with dogtail.
Last month I visited silicon valley again. Always nice to be there. Not only for the weather and the surroundings, but also always really good to catch up and work together with my overseas colleagues face to face again. Only the flight from Amsterdam to San Francisco is not a short one, though that leaves you a lot of time for reading. So I took my chance and took with me some various writings, most notably Coders at Work, which I can definitely recommend.
Anyways, all that reading got me to pen down some random but interesting quotes I encountered:
There are two ways to design a system: “One way is to make it so simple that there are obviously no deficiencies and the other ways is to make it so complicated that there are no obvious deficiencies” — C.A.R. Hoare.
“When documents are mostly to enable handoffs, they are evil. When they capture a record that is best not forgotten, they are valuable” — Tom Poppendieck.
“Its easier to optimize correct code than to correct optimized code” — Joshua Bloch.
“There’s a grand myth about requirements: If you write them down, users will get exactly what they want. That’s not true. At best, users will get exactly what was written down, which may or may not be anything like what they really want.” — Mike Cohn
It’s interesting that only such a small number of words can express such a great deal of experience.