An Experience with NSData, base64, and 3rd-Party Code

One modernization made in v4.0 was the conversion to NSData’s new out-of-the-box base64 encoding and decoding capabilities. Trail Tracker uses base64 to store and export waypoint images. Trail Tracker had previously used Matt Gallagher’s NSData base64 category for base64 encoding and decoding.

Satisfying as it was to get rid of now-obsolete 3rd-party code, I faced an annoying surprise in the form of – initWithBase64EncodedString:options:, particularly the ‘options:’ part. Somewhat inconsistently, decoding base64-encoded images saved with Gallagher’s NSData category gave nil output. Throwing me off the scent was the fact that the decoder was working with several of our test trails, which initially convinced me that the transition had gone smoothly, and barred closer inspection. This led to an uncomfortably long time much later in the development process digging through the XML parsing in Trail Tracker’s import manager, tracing the data through and export-import cycle, comparing exports from v3.5 and v4.0, and examining TTMD (Trail Tracker Map Data) as it sat on the device. The fifth piece of the Joel Test talks about fixing bugs, and emphasizes fixing bugs as soon as possible.

When you have a bug in your code that you see the first time you try to run it, you will be able to fix it in no time at all, because all the code is still fresh in your mind.

If you find a bug in some code that you wrote a few days ago, it will take you a while to hunt it down, but when you reread the code you wrote, you’ll remember everything and you’ll be able to fix the bug in a reasonable amount of time.

But if you find a bug in code that you wrote a few months ago, you’ll probably have forgotten a lot of things about that code, and it’s much harder to fix. By that time you may be fixing somebody else’s code, and they may be in Aruba on vacation, in which case, fixing the bug is like science: you have to be slow, methodical, and meticulous, and you can’t be sure how long it will take to discover the cure.

Unfortunately, in this case, I faced something like the third situation (though no one had gone to Aruba). A bug that could have been fixed in seconds when the code was written took over an hour, because I had to take a breadth-first approach to fixing the bug. As time goes forward, the intuition about a piece of code that allows for insightful depth-first bug-fixing falters.

My first instinct was maybe that the base64 hadn’t been padded correctly, which was confirmed by the fact that the lengths of the base64 data weren’t all multiples of 4. Adding explicit padding-to-multiple-of-4-length mysteriously didn’t solve the problem (and actually broke the decoder for the few instances in which it was working).

Ultimately, all that was missing was the option NSDataBase64DecodingIgnoreUnknownCharacters (the only available option for the method). The anomalous lengths I observed were coming from these unknown characters, not improper end padding. Perhaps I should have realized from the beginning that passing this option should be the default, as opposed to passing an empty flag, but alas. A lesson learned in modernizing code with similar-but-not-quite-the-same APIs. Pay attention, even when you think you’re already paying attention.

Looking through Gallagher’s own explanation of his category, ignoring unknown characters is the decoder’s default behavior. The benefit of having access to a base64 decoder that enforces strict adherence to base64 characters only is… strict adherence to the base64 standard. This is probably useful in situations where data is coming from unknown sources, where any non-zero number of non-base64 characters could indicate that the data had been corrupted and not to be trusted as valid. Okay, I buy it.

One thought on “An Experience with NSData, base64, and 3rd-Party Code

  1. Hi Eric, congratulations for this app, it is great!! Question, any plans to support topo maps, when hiking? Would be great…!! Thanks, best wishes,


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s