MatBurt

Lately I’ve been working on a version of MobileOrg for the Android platform. If you are an iPhone user then check out the main MobileOrg site or the store for it. MobileOrg is a free and open source application for storing, searching, viewing and editing your Org-mode files. If you are an emacs user and you aren’t using Org-mode then you should definitely check it out. From the website it is: “an Emacs Mode for Notes, Project Planning, and Authoring”.

Org-mode has become an integrated part of my life, I use it mostly for notetaking and task planning so having it on my Droid would be extremely powerful for me.

This early alpha version is slim on features and has a few bugs but it is a great start for putting the contents of your Org files in the palm of your hand. My goal is to reach feature parity with the iPhone version and extend the application to make use of some things available to Android like Widgets.

You can find release notes and access to the source code and download links on the Wiki page on Github: http://wiki.github.com/matburt/mobileorg-android

Spent this past weekend at PyCon in Atlanta and had a great time.

Most of the talks were pretty good, though there were some duds. You can find videos of the talks online but I thought I would post my notes from a few talks that I went to. These notes are mostly things that I found interesting or noteworthy and I didn’t take notes for every talk I went to. This file is in org-mode plaintext format.

PyCon 2010 Notes

A couple of days ago I discovered a bug in a popular piece of software that we use at work… it took a few back and forth conversations with one of the developers on the mailing list but I was able to work past it and find a solution that put my system back on track. This little nugget came in today as a follow up to that conversation after another person (not a developer of this product) had asked me if I thought it could be bad memory… I assured him that we had looked at that as a possibility and ruled it out. Then there was this:

You lost the contents of one file didn’t you? Hits on memory from
cosmic rays cause random single bit flips at a small but measurable
rate–though ECC memory should help. These could cause something in
Linux’s page buffer to go awry and hence ending up with an empty file
when unexpected. The bug rate of normal software means that that
these sorts of errors are almost completely masked, however when code
matures these sorts of errors start to become important.
…
Higher assurance software tends to include checksums and other simple
invariants that are checked at various places in code to make sure that
errors like this aren’t propagated too far.

That is the most ridiculous thing I have ever read… never mind that I had already ruled out any hardware problems. I’ve been writing code for a long time and in the infrequent times where memory in a production system has gone bad I have never chalked it up to cosmic rays.

Come to think of it… I think I’ll open a bug against myself in all of my projects marked “Cosmic rays cause unpredictable behavior” and any time I come across something that’s unreproducible I’ll just mark it as a dupe of that one.

Every year around this time I find myself faced with a problem. I really enjoy football and playing fantasy football but I’m pretty much locked into Yahoo’s terrible web interface.

Each year I tell myself I’m going to throw something together so I can access my Yahoo Fantasy leagues in Python and somehow bypass needing a web browser altogether.

Since I’m managing a team for my employer in a multi-company league I thought it would be nice to be able to generate an RSS feed so that everyone in the company could see how we are doing without needing to access the team or league pages directly.

This small Python utility uses Beautiful Soup, Mechanize and PyRSS2Gen to login to Yahoo and scrape the main league page for the appropriate details.

fantasy-tracker project on git.matburt.net

Wouldn’t it be nice if Yahoo actually provided an API so that we didn’t have to do stupid stuff like this?

It should be easy to take and extend this to do other things and make it more featureful. Like dbus notifications and whatnot.

For my birthday this year I was given a brand new Asus EeePC 1000HE, one of those fancy new netbooks.  It came pre-loaded with Windows XP which didn’t last too long.  I was aware going into it that OpenBSD wouldn’t recognize the wireless card so I thought I’d give FreeBSD a shot.

The challenge to getting FreeBSD on this system is that it doesn’t have an optical drive and FreeBSD doesn’t have a really easy way to load the system from a USB disk.  You’ll need to bootstrap your own following instructions you find at the FreeBSD EeePC Wiki.

I installed FreeBSD 7.2 from the bootonly image.  The ethernet device was recognized but not marked ‘Up’ for some reason so I had to use the emergency shell to bring it up but after that I was able to perform a network install and then upgrade to CURRENT.

It looks as though the Atheros 9280 is supported pretty well, though you need to follow the man pages for creating the wlan0 device to be able to use it, I’m also using WPA2 on my home wireless network.  powerd also works, here’s my rc.conf that, along with wpa_supplicant will bring up the wireless device and enable powerd.

wlans_ath0="wlan0"
ifconfig_wlan0="WPA DHCP"
powerd_enable="YES"

and the relevant entries in /boot/loader.conf:

snd_hda_load="YES"
acpi_asus_load="YES"
acpi_video_load="YES"
hw.acpi.reset_video="1"
hw.acpi.sleep_button_state="S3"
hw.psm.synaptics_support="1"

There are a few things that I have not tested… like the camera, and I can confirm that as of right now suspend does not work.   Power usage is fantastic, however.

It did take a little configuring to get Xorg to work but it does nicely, with direct rendering here is my xorg:

xorg.freebsd.eeepc.conf

And my dmesg:

dmesg.eeepc.freebsd

During the day I like to listen to various audio streams so I find myself switching back and forth between music and news.  Wouldn’t it be nice if you could play streams for a given amount of time?  I couldn’t find anything that did it automatically so I wrote this quick Python program to do it (requires pymad):

Musplice Project Page

You can download the latest version here:

http://git.matburt.net/cgi-bin/gitweb.cgi?p=musplice.git;a=snapshot;h=HEAD;sf=tgz

It’s pretty simple and straightforward setup script that installs the module, a musplice bin launcher, and a config file in /etc.  The example script plays Groove Salad for an hour and then switches over to WUNC, a public radio station in Raleigh, NC for 15 minutes.  You can add as many sections as you want to play different streams at different times.  If there is a stream that is a fixed length and you want to play all of it just set it’s time to: all

If /dev/dsp is not your sound device then you should set it in the ‘device’ section.  My sound device in OpenBSD is /dev/sound.  There is some initial ALSA support that doesn’t play to well if you set the device to: alsa

There is a simple cli interface, that takes a few commands… see the README for more details.

It’s a pretty simple 200 line program that I’ll probably enhance as I see fit since I tend to use it pretty frequently.

So I realized today that my database has been down for two days without me realizing it. It appears that it just shut itself down at some point and never came back.

I’ve decided to give Monit a try for monitoring and notification of bad things like this happening. Do you use something similar with good results? If so let me know.

I like to tell stories about poorly written code.  As a programmer you come across it all the time, and good programmers work really hard to make sure everything they write is both clean and as free from bugs as you can make it.  There is a reason, though, that we spend so much time fixing bugs and that’s because it can be extremely difficult to write bug free code.  This problem that I encountered recently at work is one of those bugs that just sneaks up on you and punches you in the face… see if you can find the bug:

def createProc(acmd):
    dn = open("/dev/null")
    return subprocess.call(acmd, stdout=dn, stderr=dn)

I honestly did not see it at first and debugging it was a real pain, but if you study it and maybe take a quick refresh of the Python documentation you’ll see it.   In this case the program being spawned was crashing infrequently, and it would only fail if the program it was calling had an error.  Here’s how this function should have been written:

def createProc(acmd):
    dn = open("/dev/null", "w")
    return subprocess.call(acmd, stdout=dn, stderr=dn)

By now it should be obvious.  Python’s default open() mode is “r”ead.  This program was normally silent, but whenever it had a problem it would write an error message stderr and then quit.   The first time it tried to write to stderr it would throw an exception within the subprocess context, without being raised into the calling process.  If you tried to just open a file in Python without setting “w”rite mode and write to it the interpreter would throw an IOError telling you that the file handle is invalid but because of the call to fork() you would never see that exception in the calling function… just a return code of 1.

I actually found this problem while trying to track down another issue:

def createProc_bad(acmd):
    return subprocess.call(acmd, stdout=subprocess.PIPE,
                                 stderr=subprocess.PIPE)

This also seems rather benign.  Well, as benign as something can be when you are forking another process but in this case the process would hang!   This time the problem was a little easier to track down.   I’ve done a lot of work on pipes and I know enough about Python’s implementation to know that it was probably blocking on the pipe because we were never reading from it.  Once that pipe buffer fills up the process will block waiting to write to it and the application wouldn’t be able to make progress.   Here’s a good solution if you may need the data that would be written to those pipes and don’t want to worry about reading from it as the process progresses:

def createProc_better(acmd):
    proc = subprocess.Popen(acmd, stdout=PIPE, stderr=PIPE)
    (sout, serr) = proc.communicate()
    rc = proc.wait()
    return (rc, (sout, serr))

One has to be careful with this implementation, though.   Python will store this information in heap and it will quickly blow up the interpreter’s memory usage.   If you didn’t care about the output you could just redirect it to /dev/null as shown above.

For a very long time I have been floating back and forth between a handfull of windows managers trying to find something that I like and doesn’t get in my way.  The one I have always gravitated to is Window Maker, simply because it is minimal and has a ridiculously small footprint.  I’ve also used Gnome and KDE from time to time but they just try to do too much and I need something that will support my terminal driven usage habbits.

Since I’ve been doing a lot of functional programming lately, a lot of which being in Haskell, I thought it was time to re-evaluate my window manager situation.  This is where xmonad comes in.

To be honest, I’ve never given much though to tiling window managers and I’m not sure why.  I’ve been using xmonad for the last 6 months and it is simply beautiful.  It can be driven entirely from the keyboard with some dead simple keybindings that are customizable.  I found myself changing the meta key used in most keybindings to the silly windows key on the keyboard… for a long time this key was worthless to me but xmonad likes to use Alt by default for most commands which conflicts with Emacs in some situations.  Configuration can be a little difficult for anyone who is not familiar with Haskell but there are plenty of examples and the code itself doesn’t require any advanced programming skill.

Another thing that impresses me is how well it handles multiple displays and multiple desktops.  Each screen’s desktop is capable of displaying any one of my 10 virtual desktops and they can be selected independently.  This may seem very rudimentary but the way that xmonad handles this is very elegant.  Per-desktop layouts are also excellent, and it’s a quick keystroke to switch between layouts.

It almost goes without saying that the aim of the project is to provide a crash free experience and I can back this up.  Since I have been using it it has crashed exactly zero times.  It also has a massive extension library that includes dozens of layouts, support for docks, panels, prompts and more.   It even has extensions to use it with Gnome and KDE.

The only problem I could imagine someone having is in writing a configuration file.  Luckily there is an archive of user submitted configuration files demonstrating pretty much everything that it can possibly do.  As is the case for the Haskell community in general there is also a wonderful user base to support it and any one who wants to use it.

I’ve been using VPSLink for almost a year now, and from a hosting standpoint they have been fantastic. Having my own Linux VPS has been nice but what I really wanted was one that would let me run FreeBSD and after a year of searching I finally found one: RootBSD

I have fully switched everything over and migration and set up was painless so we shall see how well everything runs.

Another nicety, for me anyway, is that they are actually local to me, one city over in Cary, NC