Automating iOS app builds

I believe an important part of the development process is getting a working version into the hands of others. As part of this, being able to install the latest and greatest on the fly is paramount. It took me a lot of time to find a reliable way to build, sign, and distribute apps as part of this internal continuous delivery.

I do not recommend running your own build machines. Wasted days with Jenkins failing and Xcode crashing have taught me this is the most fragile part—and worth outsourcing. Fortunately, there are now companies dedicated to it. CircleCI has served me well: their support is fantastic and their service, despite the beta label, is very reliable. You might need to send them a quick message to request access.

Below is the outline of creating a script to generate builds. My goal isn’t to create a file you can copy-and-paste, rather to describe the methodology. If you want a full solution, I hear good things about fastlane.

Configure deployment

To instruct CircleCI to run a distribution script, add a deployment step like the following to your circle.yml file. You can find more information in the CircleCI iOS docs.

machine:
  xcode:
    version: 6.3.1 # We want nullability, etc.
deployment:
  s3:
    branch: /.*/ # All branches
    commands:
      - ./your-directory/your-script.sh | tee $CIRCLE_ARTIFACTS/distribute.log:
          timeout: 600

I like to log the build verbosely, so I save the entire transcript as an artifact. This makes it a lot easier to find compile issues. Note the timeout key is indented another level below the script command, which is a key itself. YAML syntax is strange.

Create a keychain

The most complicated part is handling your signing credentials. I’ve found that the simplest and most secure solution is a keychain checked into the repository. To store your private key and Apple-signed certificate, let’s create one:

  1. Launch Keychain Access (inside your Utilities folder).
  2. In the “File” menu, choose “New Keychain” and note the secure password you use.
  3. From your “login” keychain, locate and select both:
    1. Your distribution certificate—the one from Apple.
    2. Your private key, which you generated, likely a sub-item.
  4. Copy the two items to your new keychain by holding Option (⌥) and dragging them into the keychain.

Using a keychain directly allows Xcode’s tools access to your credentials. It’s also easy to update: just repeat the above steps with your existing keychain after renewing your certificate. You can include as many signing identities as you wish in the same keychain: enterprise, ad-hoc, App Store, etc.

Use the keychain to codesign builds

Create an environmental variable (here named IOS_CI_KEYCHAIN_PASSWORD) with the password you created above. Don’t store this within your repository.

Command-line operations require absolute paths to the keychain file, so we need to save that into a variable. This example assumes you’re executing a script in the same directory as the keychain:

# Make any command erroring fail the whole script
set -e

# Get the absolute keychain path - `security` requires absolute paths
pushd $(dirname $0)
TOOLS_PATH=$(pwd)
popd

KEYCHAIN_NAME="$TOOLS_PATH/distribute.keychain"

With our keychain’s absolute location, we can now:

  1. Tell the system to use our keychain, and
  2. Use the password environmental variable to unlock our keychain.

Note: If you run the following locally, you will lose access to your keychain. To fix it, instead use the absolute path to $HOME/Library/Keychains/login.keychain.

# Set our keychain as the search path. Don't run this locally.
security list-keychains -s "$KEYCHAIN_NAME"
# Unlock it using the environmental variable.
security unlock-keychain -p "$IOS_CI_KEYCHAIN_PASSWORD" "$KEYCHAIN_NAME"
# Make sure we have a long timeout window.
security set-keychain-settings -t 3600 -u "$KEYCHAIN_NAME"

function cleanup {
  # Clean up on any kind of exit, just in case something bad happens.
  echo "Cleaning up keychain due to exit"
  security lock-keychain -a
}
trap cleanup EXIT

When run, your commands will have access to your certificates and keys until the script exits cleanly or otherwise.

Copy provisioning profiles

You will also need the provisioning profiles you’ve configured in Xcode. For simplicity’s sake, don’t bother figuring out which; just copy everything into your repository.

PROVISIONING_SYSTEM="$HOME/Library/MobileDevice/Provisioning Profiles/"
PROVISIONING_LOCAL="$TOOLS_PATH/provisioning-profiles/"

mkdir -p "$PROVISIONING_SYSTEM"
find "$PROVISIONING_LOCAL" -type f -exec cp '{}' "$PROVISIONING_SYSTEM" \;

You can add an rm -rf "$PROVISIONING_SYSTEM" to the cleanup function as well.

Create and upload the build

Building .ipa files is unnecessarily hard. I don’t know why. I use Nomad, which cleanly wraps the somewhat-private Xcode commands.

Nomad supports many upload destinations and services; for file storage, I like to use S3. Its library uses AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environmental variables, which you should create as a new limited-access IAM user.

# Where to store the IPA. CircleCI has an artifacts folder you can stuff things into.
# Some other providers seem to lack this, so perhaps use /tmp/.
IPA_NAME="$CIRCLE_ARTIFACTS/$CIRCLE_BRANCH.ipa"

# Disable archive because otherwise it will build twice, wasting time.
# If you aren't using Bundler (you should, though), drop the `bundle exec` part.
bundle exec ipa build --no-archive --verbose --ipa "$IPA_NAME"
bundle exec ipa distribute:s3 -b "your-bucket-name" -f "$IPA_NAME"

You may also use Nomad to upload additional files like the dSYM to your crash reporting service, metadata like the latest build number from agvtool, or certain branches directly to iTunes Connect.

Install on device

Name files by branch. This makes testing somebody’s idea or feature extremely easy. The iOS Deployment Reference gives you all you need to get started. Basically, you need an itms:// link pointing to a .plist file pointing to the .ipa file. It’s quite easy.

Localizing attributed strings on iOS

In an iOS app, localization can be especially difficult when dealing with attributed strings. Fairly often, designers request something like:

Read our Terms of Service, Privacy Policy, or contact us with any questions.

or like:

Searching for burgers in SOMA, San Francisco, CA:

The golden rule of localized strings is to treat them as atomic units:

  • Never concatenate strings to form sentences. Many languages have different sentence structure or gender rules than English and you cannot just substitute a single word or phrase in for any other.
  • Never use substrings for searching in an original string. If you depend on the standalone translation of “Privacy Policy” matching the sentence version, you will likely find translators do not understand this intent. You may also find the same search term multiple times (imagine if the user searched for “Search”).

Often these complexities are cited as reasons to avoid localization. But, unless you have geographic constraints, you will find a substantially larger audience with a localized application.

ZSWTappableLabel and ZSWTaggedString are two open-source libraries I have released to help solve these problems.

ZSWTappableLabel makes links inside your attributed strings tappable, as the name suggests. It’s a UILabel subclass which does not do any drawing itself, making it fast and easy.

ZSWTaggedString is the powerhouse. It transforms an HTML-like syntax into an attributed string. You can read more about the syntax and advanced usage on its GitHub page, but here’s how you might use it for the examples above:

Read our <i><tos>Terms of Service</tos></i>, <i><privacy>Privacy Policy</privacy></i>, or <i><contact>contact us</contact></i> with any questions.

Searching for <term>%@</term> in <location>%@</location>:

In my experience, localizers1 are familiar enough with HTML to have no issues with localizing these strings. By marking the regions you intend to be visually distinct, they can more easily understand your intent, producing better localizations.

While on the subject, here are a few best practices for localization in iOS:

  • To handle the current locale changing, or the dynamic type setting changing, reload your UI when observing:
    • NSCurrentLocaleDidChangeNotification
    • UIContentSizeCategoryDidChangeNotification
  • To represent dates, durations, distances, lengths, etc., use an appropriate formatter.
  • To create your own date formats, use +dateFormatFromTemplate:options:locale: on NSDateFormatter. Remember that these need recreating if the locale changes.
  • To combine a first and last name, use ABPersonGetCompositeNameFormatForRecord with a temporary ABPersonRef, or use the new NSPersonNameComponentsFormatter.
  • For sending non-user-facing data to a server, use en_US_POSIX as your locale.

Read more tips and tricks at NSHipster about NSLocalizedString and NSLocale.


  1. If you’re looking for recommendations on localization, working with Applingua has always been wonderful. ↩︎

T-Mobile: good ideas, bad experience

When T-Mobile entered the wireless scene as the “Uncarrier” I was impressed. Their greatest contribution to the carrier ecosystem is consistently adding features, forcing other carriers to keep up. Instead of rationed text and voice, we’re in a world where data is king.

However, if you are considering T-Mobile service, I suggest reconsidering. The plans and features look appetizing, but their execution leaves a lot to be desired. I would not count on their unique features; just the now-basics with worse coverage.

I wrote off my initial experiences as anecdotal, but it became cumulatively enough for me to leave their service.

Free data for life on tablets

My first experience with the new T-Mobile was their offer of free monthly 200mb of data. It’s a compelling reason to go with them.

After a month of lifetime free data, it ended and I got “could not connect to data service” errors. The first rep I spoke with ran me through the usual steps culminating in toggling something on his side and waiting 48 hours to see if it worked.

The next rep a couple days later insisted I had to reformat without restoring a backup. This was a huge pain, and unsurprisingly didn’t fix anything. He filed a report with “the engineers.”

A few more days later, service started working again, and later the rep called to make sure. The follow-up made me happy, despite the length of time from start to finish.

Update 2014-12-29: After moving my cell phone from T-Mobile to AT&T, I started receiving a bill for $10/mo for my “free data for life” device. I called and received a credit the first time, but dug into it more the second time.

Their retention department informed me that the “free data for life” offer requires having another device with T-Mobile service. According to their press release (backup) no such qualifier existed when I signed up, and according to their second press release (backup) “there is no $10.00 per month fee for the 200MB of free data.”

I see no qualifiers on their support page so I can only assume their screw-the-customer attitude is behind-the-scenes policy.

Refer-a-friend bonus

There’s no zealot like a convert, and I convinced a few friends to switch their service over. When T-Mobile announced a new referral system, wherein both sides get unlimited data for a year, my latest referral and I signed up.

I received only a $25 in-store gift card, and waited a few weeks to call the refer-a-friend support to figure out why. I was immediately told it was a mistake, and the rep said he would add unlimited data to my plan.

I asked for clarification on the change, and he confirmed it was “free for 12 months,” and suggested my friend call, too. Another rep added unlimited data to my friend’s account.

Not-so-bonus

My bill went up by $50, or $20 more than just adding unlimited data would cost. My friend’s bill went up, too.

When I called support, the first rep clarified that my referral would only give me a $10 discount on unlimited service. After expressing confusion, she placed me on hold to ask refer-a-friend support.

The phone rang, and I was greeted with, “T-Mobile refer-a-friend support, how can I help you?” I was blindly transferred, and had to explain everything for the second time.

This rep said a few things, and I got the impression he believed I added unlimited data myself:

  1. I had 2 referrals, one processed (the gift card) and one pending (a few days old).
  2. Referral #1 was activated a few days before the data offer started, and didn’t qualify.
  3. Referral #2 would give me a $10 discount on my existing unlimited data.
  4. If I ended my existing unlimited data, referral #2 would be nullified.
  5. No refer-a-friend rep would have added unlimited data to my account.

The net result is I would have to pay $20 extra monthly to get unlimited data, which should have been free twice over. I asked him to take off the unlimited data charge, and he placed me on hold.

And then I heard, “all of our representatives are busy, please hold,” and got to explain everything again to a third rep I was blindly transferred to. This final rep was nice enough to credit the plan and remove unlimited data from my account, at least, even though she accidentally hung up on me towards the end.

I complained about this on Twitter and although their Twitter presence contacted me, they never responded days after asking for my phone number.1

Free texting in the air

I had the opportunity to give another new benefit a try: on Gogo in-flight internet, devices already set up for WiFi calling can connect to the network for SMS service. This did not work.

When it came time to “get started” on the T-Mobile side of things, I received errors repeatedly and I gave up. Gogo support was nice enough to credit a free hour of internet for my troubles, which is a fine example of what I’m really looking for: solutions.

Goodbye

T-Mobile is a noble experiment executed with good intentions but badly in need of restructuring. Every time I try a perk, it backfires, and I don’t care to spend hours fighting.

I am switching to AT&T to pay the same amount. If there’s one good thing T-Mobile is doing, it’s pushing the other carriers to offer competitive plans. However, issues with implementation make their poor coverage feel even worse. Hopefully this changes in the years to come.


  1. I dislike that Twitter direct messages require following the sender. I gave up after a few days because I was tired of seeing their tweets, so perhaps they tried to respond some time later. I doubt it. ↩︎

Enabling bridge mode on AT&T U-verse

These are instructions to configure a U-verse gateway to send all of its incoming traffic to your own router without impacting its normal networking services.

The gateway

The gateway model that I am using is a 2Wire/Pace 3800HGV-B, but these instructions likely work for similar models.

You can connect to it at 192.168.1.254. This service is abysmally slow and fairly hard to navigate as a result. The gateway will on occasion prompt for the password labeled “system password” on its serial number sticker.

Turning on DMZplus

First, we need to forward all of the traffic to the router. This can be done as follows:

  1. From the gateway, navigate to: Settings > Firewall > Applications, Pinholes and DMZ.
  2. Choose the router from the list. It will appear as a link named “Choose <name>.”
  3. Change its setting below to “Allow all applications (DMZplus mode).”

Assigning a public IP

Finally, we need to give the router the knowledge that it’s a public-facing device. UPnP and NAT-PMP rely on determining the public IP, and having the correct WAN IP is required for many routers.

We can achieve this by assigning the router the public IP address as follows:

  1. Navigate to Settings > LAN > IP Address Allocation.
  2. Locate the settings box for the router.
  3. Change its “Address Assignment” from “Private” to “Public.”

All done

After the router renews its DHCP lease, it will be assigned the public IP address. This is generally a huge button labeled “Renew DHCP Lease” or a power cable plugged into the device.

U-verse IP addresses are dynamic, but are deterministic from hardware configurations on their end. Thankfully, this means the addresses very rarely change. I’ve gotten away with just adding an A record to a random domain name to make it easy to access my VPN.

Saving optimal JPEGs on iOS

Conventional wisdom for creating a JPEG version of a UIImage is first to turn it into an NSData and immediately write it to disk like so:

NSData *jpegRepresentation = UIImageJPEGRepresentation(image, 0.94);
[jpegRepresentation writeToFile:outputURL.path
                     atomically:NO];

Most of the time this is exactly right. However, if file size is important, Image IO is a great alternative. It is a powerful system framework to read and write images, and produces smaller files at the same compression level.

Why Image IO?

A project I am working on requires uploading photos en masse. Low upload bandwidth makes file size a limiting factor, so I sought out ways to reduce it.

I put together a test project to find the differences between the two methods. The results are pretty interesting:

  • Image IO files are on average 20% (but up to 30%) smaller1.
  • Image IO takes about 2x longer.

The only discernible visual difference is the grain in the images, but even that is minor. Here’s a diff between two versions of the same original photo:

Image diff

Using Image IO

First, you’ll need to add two new framework dependencies:

@import ImageIO; // to do the actual work
@import MobileCoreServices; // for the type defines

When creating your JPEG file, the first step is to create a CGImageDestinationRef specifying where to write the result:

CGImageDestinationRef destinationRef =
CGImageDestinationCreateWithURL((__bridge CFURLRef)outputURL,
                                /* file type */ kUTTypeJPEG,
                                /* number of images */ 1,
                                /* reserved */ NULL);

Image IO is able to produce files of a few different types2 but my focus here is JPEGs. Next, we set up the properties of the output file, specifying a constant compression factor:

NSDictionary *properties = @{
  (__bridge NSString *)kCGImageDestinationLossyCompressionQuality: @(0.94)
};

CGImageDestinationSetProperties(destinationRef,
                                (__bridge CFDictionaryRef)properties);

And, importantly, we specify what is to be written out:

CGImageDestinationAddImage(destinationRef,
                           /* image */ image.CGImage,
                           /* properties */ NULL);

And finally, we write it to disk and clean up the reference:

CGImageDestinationFinalize(destinationRef);
CFRelease(destinationRef);

  1. The UIImage version has a color profile, while the Image IO version does not. However, running both files through Image Optim produces a 7% reduction on both, so I am choosing to ignore this difference. Afterall, you can’t remove the color profile anyway! ↩︎

  2. The following are possible types you can use, from the documentation:

    Constant UTI type
    kUTTypeImage public.image
    kUTTypePNG public.png
    kUTTypeJPEG public.jpeg
    kUTTypeJPEG2000 public.jpeg-2000 (OS X only)
    kUTTypeTIFF public.tiff
    kUTTypePICT com.apple.pict (OS X only)
    kUTTypeGIF com.compuserve.gif
     ↩︎

Using the Xcode Structure menu

Xcode’s Editor > Structure menu has a few great actions:

Screenshot demonstrating where to find the menu item in Xcode

These actions all act on either your cursor position or selection.

Balance Delimiter

Normally you can double-click quotes, brackets, or parenthesis to select the matching character and all text in-between. The Balance Delimiter behaves similarly: it looks at your cursor position (or selection), finds the nearest pair and selects in-between.

This doesn’t have a default keyboard shortcut, but you can set one up in Xcode’s Key Bindings preferences. I set it to ⇧⌃I since I use it in similar ways to Re-Indent.

Re-Indent ⌃I

Objective-C is a fairly indentation-heavy language, and Xcode generally does a good job at indenting while you’re typing. However, if you’re pasting text or refactoring things can get pretty hairy, so let Xcode fix up your code for you with Re-Indent.

Shift Left ⌘[, Shift Right ⌘]

You can also indent manually using the Shift Left and Shift Right actions, which unindent or indent by one tab, respectively. They do exactly what they say on the tin.

Move Line Up ⌥⌘[, Move Line Down ⌥⌘]

These actions move your current line (or selection) up or down by one line. Simple, right? What’s really useful is that it is context-aware: they understand going in and out of control flow or blocks. Much faster than cutting, pasting and re-indenting each time.

Comment Selection ⌘/

Rather than wrapping your code in /* … */ and dealing with the conflicting multi-line comments you probably already have, simply select what you want to temporarily eliminate and hit the shortcut. Each line selected is then prefixed by //.

Error arguments in Objective-C

From the Programming with Objective-C (backup) overview from Apple:

When dealing with errors passed by reference, it’s important to test the return value of the method to see whether an error occurred, as shown above. Don’t just test to see whether the error pointer was set to point to an error.

and from the Error Handling Programming Guide (backup):

Success or failure is indicated by the return value of the method. Although Cocoa methods that indirectly return error objects in the Cocoa error domain are guaranteed to return such objects if the method indicates failure by directly returning nil or NO, you should always check that the return value is nil or NO before attempting to do anything with the NSError object.

For example, let’s say we’re executing a fetch request:

NSError *error = nil;
NSFetchRequest *request = /* … */;
NSArray *objects = [context executeFetchRequest:request
                                          error:&error];

To test if the fetch was successful, we must do:

if (objects) {
    // hooray!
} else {
    NSLog(@"Got an error: %@", error);
}

A method taking an NSError ** does not guarantee how it is used when successful. In this case, even a successful method call may end up with error set to something other than nil.

You might need to know the characters to truly appreciate, but in Scott Lynch’s Gentleman Bastard series, these quotes made me chuckle.

From The Lies of Locke Lamora:

“Bug,” Calo said, “Locke is like a brother to us, and our love for him has no bounds. But the four most fatal words in the Therin language are ‘Locke would appreciate it.’”

“Rivaled only by ‘Locke taught me a new trick,’” added Galdo.

And from Red Seas Under Red Skies:

“What? How dare I contemplate what you were going to do to me? You self-righteous strutting cock, I’ll–”

“What?” shouted Jean.

“I’ll throw myself at you, and you’ll beat the shit out of me,” said Locke. “And then you’ll feel awful! How about that, huh?”

Cleaning up old git branches

My local git repository is full of branches which have long-since been merged: code review, one-off features, and quick bug fixes add up really quickly. Deleting these branches is cumbersome.

Complaining on Twitter proved really fruitful:

There’s also git branch --merged, which you could pipe to xargs (assuming you can remember the syntax, which I never can.) – @bjhomer

Of course, he’s right, so finding and deleting them is easy. The end result is this wonderful snippet, which deletes all merged branches:

git branch --merged origin/master | grep -v master | xargs git branch -d

The text spam after it executes can only be described as cathartic.

Portable encrypted backups

To keep track of the ever-growing array of passwords and private data, I use 1Password, which I could not recommend more.

There’s a problem though: what if I lose access to my database? I’ve toyed with a few options in the past, including uploading it to Dropbox or Google Drive, but these are services for which I’ve enabled multi-factor authentication, and require internet access.

For a few years now, Macbook Pros have included an SD slot for photographers. Or, as it turns out, everybody!

I bought a Transcend SD card with a decently-useful 16GB capacity for about $10. As of OS X 10.8, you can format any disk to be encrypted. Match made in heaven.

Going from the stock card to an encrypted version is easy, but does involve making the disk Mac-only:

  1. Launch Disk Utility from /Applications/Utilities
  2. Select the SD card from the source list and choose “Partition.”
  3. When repartitioning the drive, choose “Options…” and format using GPT.
  4. Erase the newly-created partition with the “Mac OS Extended (Journaled, Encrypted)” option.

With a tiny, portable, encrypted disk, there’s a lot of cool things you can back up. I currently save:

  • My 1Password and system keychains.
  • The database for Authy, which is like Google Authenticator, backed up via iExplorer.
  • Copies of my source code repositories.
  • Public and private SSH keys.

I’ll find more uses as time goes on. Right now it’s a manual process to update, but this is more of an emergency backup: I use Arq for my normal backup needs.

Getting into fantasy

Science fiction has always been my favorite genre. I enjoy imagining possible futures and I’ve never stopped wondering what changes may happen in my lifetime.

The mysterious worlds of the future always held such potential that I rarely strayed from the genre. I started reading simple novels by Isaac Asimov and grew into reading hard sci-fi like Alastair Reynolds where the thick story and technology are enrapturing.

Fantasy is similar, but different. Instead of imagining the future, you’re compelled to imagine something potentially in parallel. It could be our past, our future, or another world entirely, with any kind of supernatural circumstances.

Earlier this year I was introduced to fantasy by David: his careful but persistent prodding toward reading Name of the Wind was eventually successful beyond his wildest imaginations.

Below are a few of the unforgettable series I have read since then and recommend without hesitation. The links point to the Kindle editions on Amazon, and feature an affiliate marker.

Fantasy is an amazing genre: much of the books are not only available digitally, but also DRM-free, negating previous hacks I’ve had to use. I am incredibly happy about that.

The Kingkiller Chronicle (1, 2)

This series by Patrick Rothfuss is still my favorite, and displays an incredibly powerful piece of storytelling. The depth of the characters and plots combine with the history and lore of the world to produce what I have come to expect in many fantasy novels.

The first novel is Name of the Wind. I was initially put off by the name–of all things!–but I’ve come to realize just how wrong was that assumption. The second novel follows a similar model in which the protagonist walks us through a long series of flashbacks.

It was impossible to put down.

I entered expecting something interesting, and I exited fully in love with the idea of the genre and the execution of the novels.

I strongly recommend picking up Name of the Wind and starting from there. You will not be disappointed.

Mistborn (1, 2, 3, 3.5) & other Brandon Sanderson (1, 2, 3, 4, 5, 6)

If Patrick Rothfuss was a great introduction to fantasy, Brandon Sanderson’s Mistborn trilogy was a wonderful bit of bliss to keep me coming back for more. From his blog entries on the novels, I’ve since learned he had all three written before final publication and he was able to form an amazingly cohesive universe.

Mistborn starts us in a tyrannical world ruled by a single god-king full of brown and dying plants and an entire race of people subjugated by nobility.

Sanderson coined Sanderson’s Laws, a set of idealized goals for fantasy novels wherein the magic system is concretely defined, additive and the novels don’t require last-second, unexpected uses of magic to bring about victory.

Logic, as you may guess from being in software development, is something I eagerly enjoy.

It turns out this is really important to making fantasy novels engaging. Since reading Mistborn I’ve used it as a metric in defining what I do or do not like, and nothing by Sanderson has disappointed.

He is an impossibly prolific writer. I have nearly read everything the man has written and everything has been amazing and worth every minute. He has very easily become my favorite author.

The Gentleman Bastard (1, 2, 3)

This series by Scott Lynch was a first for me as it didn’t feature a strong magic component.

It’s an interesting approach: it’s a world of rich history, of which we see mostly the grimy bits. It’s a world of magic, in which we see but a glimmer.

Contained within is a truly interesting premise: what if thievery wasn’t for personal gain, but for the art of it? What happens when skill of the game becomes more important than even living in luxury?

The long con, and really thorough introspection by characters makes this series a really different place to visit.

Codex Alera (1, 2, 3, 4, 5, 6)

When I was in college, I loved the show The Dresden Files. When I came across The Furies of Calederon, I grabbed the sample on name recognition alone, but didn’t follow up for some time.

What a mistake! It’s an excellent series, and the writing is really easy to absorb. I read the entire series in about two weeks, and I would loved to have continued on forever.

I really enjoyed the many races and cultures featured in the novels. I think Jim Butcher did a wonderful job bringing to life the history and superstitions each.

Ad infinitum

At the time of writing this, I’ve read 50 fantasy novels totalling more than 26,000 pages since Name of the Wind, and there’s no end in sight. Here are a few other amazing pieces of literature that I can’t help but identify:

  • Riyria Revelations by Michael J. Sullivan (1-2, 3-4, 5-6)
  • Abhorsen by Garth Nix (1, 2, 3)
  • The Demon Cycle by Peter V. Brett (1, 2, 3)
  • Imager by L.E. Modesitt (1, 2, 3)
  • The Wheel of Time by Robert Jordan (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) and Brandon Sanderson (12, 13, 14)
  • Farseer by Robin Hobb (1, 2, 3)

My Reading page is a good place to find more novels I’ve enjoyed. I hope this list is helpful to at least someone out there! I cannot believe I went so long without indulging in fantasy.

Samuel Taylor Coleridge (via Contact):

It is not impossible that to some infinitely superior being the whole universe may be as one plain, the distance between planet and planet being only the pores in a grain of sand, and the spaces between system and system no greater than the intervals between one grain and the grain adjacent.

Morality clauses in domain registration

GoDaddy has received a lot of bad press for their support of the Stop Online Privacy Act (SOPA), which they helped write and from whom they are exempt. However, GoDaddy and many other registrars should be avoided for more simple reasons.

From the GoDaddy Legal Agreement:

Go Daddy may also cancel the registration of a domain name, after thirty (30) days, if that name is being used, as determined by Go Daddy in its sole discretion, in association with spam or morally objectionable activities. Morally objectionable activities will include, but not be limited to:

  • Activities designed to defame, embarrass, harm, abuse, threaten, slander or harass third parties;
  • Activities prohibited by the laws of the United States and/or foreign territories in which you conduct business;
  • Activities designed to encourage unlawful behavior by others, such as hate crimes, terrorism and child pornography;
  • Activities that are tortious, vulgar, obscene, invasive of the privacy of a third party, racially, ethnically, or otherwise objectionable;
  • Activities designed to impersonate the identity of a third party; and
  • Activities designed to harm or use unethically minors in any way.

In response, many are switching to Namecheap. This registrar tends to play contrarian with GoDaddy so fortunately they are against SOPA. However, from the Namecheap Registration Agreement: (Namecheap has removed their morality clause, see below.)

Namecheap may also cancel the registration of a domain name, after thirty (30) days, if that name is being used, as determined by Namecheap in its sole discretion, in association with spam or morally objectionable activities (as well as any activities set forth in Section 4 above). Morally objectionable activities will include, but not be limited to

  • activities designed to defame, embarrass, harm, abuse, threaten, slander or harass third parties;
  • activities prohibited by the laws of the United States and/or foreign territories in which you conduct business;
  • activities designed to encourage unlawful behavior by others, such as hate crimes, terrorism and child pornography;
  • activities that are tortious, vulgar, obscene, invasive of the privacy of a third party, racially, ethnically, or otherwise objectionable; activities designed to impersonate the identity of a third party;
  • and activities designed to harm or use unethically minors in any way.

Both have nearly identical language. These clauses leave a bitter taste in my mouth.

Should I really be held to the backwards laws of “foreign territories in which you conduct business?” Software sales are global, I interact with many customers in very dangerous nations.

Who determines what is “objectionable?” Certainly the test should be legality not questionability. The internet is the backbone of the economy and vital for freedom of expression. Depending upon corporations like this, especially those in violent support of SOPA, leaves wide a huge vulnerability in communication.

Let’s not get started on obscene.

What can you do about it? Support a better registrar. I use Gandi whose motto is “no bullshit.” No morality clause, and easy-to-read legal agreements.

Addendum (2011-12-25): As pointed out in this Hacker News thread, Gandi has a similar clause. Hover (Tucows) doesn’t feature any of these crazy terms. A statement from the GM of Hover discusses why these clauses exist and why they don’t have them:

Generally, most of the power a registrar requires to prevent the bad guys from doing bad things comes from national laws and not all these extra clauses. We (Hover/Tucows) find that all these extra conditions just make it harder for our customers to do business with us and so we’ve left out as much as we can and rely mostly on national laws to get what we need done.

Addendum (2011-12-29): Namecheap has removed their morality clause. I am happy to have effected change on this issue, and it is a move in the right direction. :)

Migrating from Kindle to iBooks

I started off reading eBooks from the Amazon Kindle store. As time has progressed, I’ve found myself using my physical Kindle less and less. Partly because I forget to charge it, and partly because I can never get the lighting in my favorite reading locations quite right. And at night? Forget about it.

So I’ve converted all of my Kindle purchases to ePubs for use in iBooks. I could use the Kindle app for iOS but I’ve found iBooks to be faster at syncing read position, and it feels a lot more natural to use.

Since I went through the effort of converting all of my Kindle documents, I figured I’d write a mini guide to getting it done. I’m specifically focusing on the Mac since that’s all I know; other users will have to venture elsewhere.

Setting up Calibre

Calibre is a Java application which is an eBook management suite. It’s a bit ugly but it does what it says and works well. Since the Kindle’s eBooks are encrypted, we need to install a decryption plugin to do the heavy lifting there.

  1. Download and install Calibre.1
  2. Download and decompress the DeDRM archive.
  3. Open Calibre’s preferences (⌘,).
  4. Go to “Plugins.”
  5. Click “Load plugin from file.”
  6. Choose the K4MobileDeDRM plugin’s zip file (zip within the main zip; don’t extract).

The encryption used on Kindle books is fairly basic. The decryption key is a product of the serial number for your device, so it’s not difficult to determine.

Getting books from Amazon

The DeDRM scripts work by understanding Kindle for Mac’s settings files, so to get the eBooks we need use the application. There are scripts to do conversions directly from Kindle hardware’s eBooks, but it’s more effort.

  1. Download Kindle for Mac.
  2. Register it with your account.
  3. Download all of your eBooks (open them from the “Archived Items”).

I really wish this application were retina.

Converting books to ePub format

Kindle eBooks are in the MobiPocket format. However, iBooks requires ePub, so we need to both decrypt and convert the files. At this point, we’ve got all we need, so we can use Calibre to do the conversion.

  1. Navigate to either of the following (likely the second):
    • ~/Library/Application Support/Kindle/My Kindle Content/
    • ~/Library/Containers/com.amazon.Kindle/Data/Library/Application Support/Kindle/My Kindle Content/
  2. Drag all of the .azw files into the Calibre window.
  3. Select the books in the Calibre window which you wish to export.
  4. Click the “Convert books” toolbar item.
  5. Choose “ePub” as the output format in the top-right of the convert window.
  6. Go to the “Page Setup” item in the left list, and configure the conversion:
    • Select “Kindle” as the input profile.
    • Select “iPad” as the output profile.
  7. Hit the “OK” button to begin the conversion. It will take a while.

If you have any issues at the decryption step, you should delete your ~/Library/Application Support/Kindle folder and start again.

Exporting and enjoyment

You can now “Save to disk” from the toolbar item to save the books which you’ve converted, and import them into iTunes for use in iBooks. Calibre is powerful enough to do many other formats if you want to use other devices as well.

Remember, don’t post any of the unencrypted documents anywhere. Just because the DRM is gone doesn’t mean you’re legally authorized to do so. They’re for your personal use only.


  1. This is a link to the project’s GitHub page because their main website does not support HTTPS. You should not install software over an unencrypted connection. ↩︎

Incrementing with a bitmask

Bitmasks are fun. There’s lots of little tricks you can do with them. A common situation is checking for the presence of a flag among elements in a linked list, or some similar data structure. I came across a trick a few years ago that makes it drop-dead simple.

Let’s say we needed to check for AUsefulFlag in the flags element of each node, and total how many elements in the linked list had the flag.

uint64_t count = 0;
for(Node *iter = head; iter != NULL; iter = iter->next)
{
    count += !!(iter->flags & AUsefulFlag);
}

After execution, count is the number of items which have AUsefulFlag set.

Double-not (!!) is one of those useful operations which are especially useful with bitmasks. It may require a double-take at first, but it behaves exactly how you’d think.

!! of 1 is 1. !! of 0 is 0. In fact, !! of any true value evaluates to 1, so we can use it to transform something like 0b00001000 to simply 1 and increment by that value.

From Atlas Shrugged by Ayn Rand:

James: What are you after?
Francisco: Money.
James: Don’t you have enough?
Francisco: In his lifetime, every one of my ancestors raised the production of d’Anconia Copper by about ten per cent. I intend to raise it by one hundred.
James: What for?
Francisco: When I die, I hope to go to heaven–whatever the hell that is–and I want to be able to afford the price of admission.
James: Virtue is the price of admission.
Francisco: That’s what I mean, James. So I want to be prepared to claim the greatest virtue of all–that I was a man who made money.
James: Any grafter can make money.
Francisco: James, you ought to discover some day that words have an exact meaning.