Clock Work: The True Story of the Backdoor in the BB10 Clock

We didn’t think anyone would ever find the needle in our haystack. But someone did.

The lesson: never underestimate the power of incentives.

Introduction

I’m Eric Peterson, and I’m a software engineer. I’ve been living in South Florida and writing code for various tech companies since 1990, including Motorola, General Dynamics, and Ford. (Disclaimer: I don’t speak for any of them, and my opinions here are all my own.)

I joined BlackBerry in March of 2011, and by the end of 2011, the entire company ceased all work on the legacy BlackBerry devices and pivoted to constructing a modern computing platform.

We worked like mad, going all-in with every resource at our disposal, trying to crank out a line of mobile devices that could compete with iOS and Android. That line of products was BlackBerry 10, and I was privileged to be part of the team tasked with building the clock app.

I’ve always wanted to tell this story from an insider’s perspective, but I felt that I couldn’t as long as I was working for BlackBerry and so long as BB10 was a viable consumer product. Since I’ve moved on from BlackBerry and since BlackBerry appears to have moved on from BB10 to Android, the time seems right to tell the tale before I forget the details.

This story takes place in the two-year period from January 2012 to December 2013, when the bulk of the BB10 engineering work took place. It’s the story of the creation, growth, and demise of a hidden feature of the BB10 clock app: the backdoor.

Our Legacy

You all remember these things, right?

Almost without exception, traditional BlackBerry phones had:

  • A high-res touch-screen display
  • A “tool belt” with a call button, a menu button, a track pad/ball, a back button, and an end call button
  • A physical keyboard

There’s a lot more going on inside a phone than most people realize, and phones are generally pretty good at hiding that complexity from the user. While great for user, it is difficult for developers to figure out what’s going on when something goes wrong.

To solve this problem, engineers added hidden alt-codes that displayed special screens so they could look “under the hood” to see what was really going on inside a device. Just hold the ALT key while typing a four-letter code, and *poof!* hidden information would appear.

The Backdoor

The early BB10 devices had no physical keyboard. This made it impossible to create any alt-codes.

To solve this problem, we added a backdoor menu item to the swipe-down menu. Touching that item would open a new set of screens that would let the user see internal app status and control various features. The backdoor item would be locked (invisible) by default.

But how to unlock the backdoor? Since we couldn’t add alt-codes, we came up with another method: go to the world clock feature, select the search box, and type a secret code known only to the developers. This code would silently add the backdoor to the swipe-down menu.

But what code should we use? We wanted something that we could all remember, that didn’t have any direct connection to the app, and that normal users would be extremely unlikely to type. We finally decided to use the Twitter username of one of the clock developers (who had sadly left the company for another career opportunity).

Experimental Features

In addition to using the backdoor to get debugging information, we also used it to prototype new ideas. It allowed us to get them into people’s hands internally to see if they were worth making a permanent feature of the app.

The two most easy-to-see features that could be enabled by the backdoor were:

World Clock Flags

National flags were shown alongside the city search results as well as in the city detail screen. It helped to spruce up the mostly monochrome color palette with a bit of localization and colorful icons.

Dynamic Timer Trail

The trail of the timer would cycle through hue values of the color cylinder as the timer setting advanced. The trail color then gave you a easier-to-see visual indication of how far away you were from zero.

Just Because We Could

Part of the reason we left all of this stuff in the shipping app was that we’re tech nerds, too.  We love finding Easter eggs buried in apps, so when we finally got the chance to hide our own, we took it. Here’s what else we stuck in:

Alarm Dongle Enhancements

We were avid readers of CrackBerry even before we began work on BB10. As an homage to a site that had done so much to promote our company’s products and build the community, we threw in a switch to replace the standard alarm dongle with the CrackBerry logo.

BlackBerry Cities

As a nod to all of the people working for BlackBerry, we added all cities in which BlackBerry had a facility of any kind into the world clock database.

Alternate Stopwatch Display

This was supposed to use the Porsche Design font on non-PD hardware instead of the LED-style display.

Unfortunately, it didn’t work quite right. As the UI layout engine evolved, some of the changes in padding and margins caused the numbers in the display to wrap off of the label in which they were supposed to appear.

Team Members

We couldn’t let a work of art go unsigned, could we? So we included a list of everyone who had input over the definition, implementation, and testing of the clock app.

Also, since the majority of the clock app developers were located in Sunrise, Florida, we included a photo of everyone there working on BB10 apps and features taken on January 30, 2013 – the day of the public launch of the BB10 platform.

(I’m in the front row on the left.)

The Problem

Porsche Design had worked with BlackBerry to create its own variant of the Bold 9900, which was released as the P’9981:

When BB10 was released, PD wanted a new phone spun off of the Z10 platform, to be called the P’9982. That phone would have unique features exclusive to PD, such as a tachometer-style display face for the Clock. Functionality would be the same, but fonts and graphics would reflect the user experience that PD desired.

Since the only differences between Z10 and the P’9982 were mechanical, it was very easy to support both within a single application. If the app read the hardware ID at launch, it could determine if it was running on a PD device or not, then display the appropriate set of UI elements.

Unfortunately, we were expected to have the feature working before we had enough PD hardware available on which to develop and test the feature. Any PD devices we had tended to be heavily in use by our product testing team, making development time difficult to come by.

The Solution

Since we had plenty of Z10s available for development, we added a switch to the backdoor that set a flag in the application persistent memory which caused the app to ignore the hardware ID and pretend it was a PD device. This allowed us to develop and test the PD features on a plain, ordinary Z10.

With the flip of a switch, the problem of hardware scarcity was solved. Plus, it was a cool thing to use for ourselves, if we wanted it.

We knew all along that this was something that could potentially be problematic if discovered. But we felt that the odds of tripping over it by accident were ridiculously low since the keyword was a nonsense string buried within an executable binary file. There was virtually no way anybody would ever bother to take the time to figure it out the hard way.

We were very, very, ridiculously wrong.

The Leak

December 3, 2013, the following post appeared on CrackBerry:

Berryleaks is proud to present instructions for getting the Porsche Design Clock on the Z10. Parts of this will work on Z30 and Q, but not the Porsche Design option. Appears to require OS 10.2.0.1047 or higher (confirmed through 10.3.2) to get the Porsche Design option.

This was the final act of the BerryLeaks Team, done so everyone could enjoy it.

I walked in on the morning of December 4 completely unaware of the leak. I was immediately accosted by my coworkers in person, in email, and via BBM, who all directed me to the CrackBerry thread.

Long story short: the BerryLeaks Team reverse-engineered the app, found the secret string, made some educated and lucky guesses, unlocked the backdoor, and shared that secret with the world.

The Aftermath

Some of the best rewards we got were from users who posted comments like this:

Very cool. Also great to see that they included CB into the official app as an “unofficial” option hidden away. I like that kind of stuff. A real connection to your fan base.

If we had taken out the feature to turn on the Porsche Design UI before we released that build to the public, our management probably would have let the backdoor remain unlocked. Unfortunately, the PD emulation feature effectively turned a $500 phone into a $2,000 phone with the flip of a switch.

As a result (and after much apologizing to PD), the senior management told us to close the backdoor for good. Fortunately for us, there were no negative repercussions for anyone involved (at least, none that I know of).

Epilogue

One last thing that the leakers discovered was this QR code that I made:

TWELVE EASTER EGG MODELS.
SEVEN ARE WONDERFUL.
FOUR LIVE IN THE BACKDOOR.
ONE WILL BE REVEALED.

No, it’s not a reference to the One Ring. It’s an allusion to the opening credits of the reboot of Battlestar Galactica used during season 4. (Warning: MAJOR SPOILER ALERT!)

SEVEN ARE WONDERFUL

While working on BB10, the developers in my group discovered a mutual love of board games. In particular, we started playing 7 Wonders during our lunch breaks. (We played it so much that we had the card frequencies and costs memorized.)

So we put the cities in which the Seven Wonders of the Ancient World were found into the World Clock database.

FOUR LIVE IN THE BACKDOOR

This refers to the special elements in each of the four features of the Clock app:

  • Alarm Clock: The CrackBerry dongle
  • World Clock: The country flags
  • Stopwatch: The alternate font
  • Timer: The dynamic color trail

ONE WILL BE REVEALED

I really thought someone would have stumbled across this by now (Feburary 1, 2017). As far as I know, nobody has found it. Or, if they have, nobody’s said so (at least on CrackBerry and the like).

So, here’s the One That Is Now Revealed:

Ph’nglui mglw’nafh Cthulhu R’lyeh wgah’nagl fhtagn.

Yes, the fictional city of R’lyeh (from the story “The Call of Cthulhu” by H. P. Lovecraft) is in the World Clock. Its presence was never dependent upon the backdoor, so it appears in every single release of the BB10 clock app.

(Fun fact: when I added the current weather conditions to the world clock, opening the detail screen for R’lyeh would cause the app to freeze. I first thought that we had angered the Great Old Ones, but it turns out that our weather data provider didn’t have any data for R’lyeh’s coordinates.)

Epilogue

Unfortunately, the market never responded well to BB10. Sales were weak, and few developers ported their apps to the platform. By the end of 2016, virtually everyone who was involved in building BB10 had moved on (both voluntarily and involuntarily) to career opportunities at other companies.

But the time I spent working on the team that built the BB10 clock app will remain one of my most enjoyable and memorable periods of my professional career.