iphone gui testing
baby steps toward cucumber
Ah, the iPhone. Darling of the geeks. All these shiny new applications running on it. How will we test them all? What software test framework is simple enough, elegant enough, trendy enough to suit the device?
Why, Cucumber, of course.
the state of the art
The art of automated testing of iPhone apps is still in its infancy. Not to say folks haven’t been busy. There’s the Google Toolbox for Mac, which includes an iPhone test section. And Dr. Nic has paved the way for code-level unit testing of Cocoa classes, albeit on the Mac.
Most promisingly, Matt Gallagher has put together the three crucial pieces of automated GUI testing for iPhone apps: identifying pieces of the UI, simulating a human finger poking at the screen, and finding out what happened.
And in just a few short steps, we’re going to hook up Matt’s demo app to Cucumber, the Ruby-based framework that understands tests in plain English. What you’re about to see is a toy. It’s the minimum amount of glue to get Ruby pointed at Matt’s code. Plenty of time to make it into something for grownups later.
First, a brief word on Matt’s approach is in order. He basically
monkey-patched
the UIView class that underlies the iPhone’s on-screen
gizmos, so that you can interrogate them at runtime: “What are you?
Where are you? Do you have any text or gizmos inside you?” Next,
he reverse-engineered how to simulate screen taps. And he tied it
all together with an XML-based scripting language that lets you
define simple steps like, “scroll to the 10th row in the list,” or,
“Press the button that says, ‘Click me.’”
For this technique to work on an iPhone app, you have to add a little piece of custom code to read the test script and spider your UI. And you have to bundle the XML test script itself into your app—well, into a special test version of your app.
hooking it up to cucumber
I thought of just building a translator from Cucumber’s subset of plain English (affectionately called “Gherkin”) into the XML property list format, and then just using that generated code as before. But I wanted Cucumber to be actually talking to the iPhone app.
Remember, Apple likes iPhone apps to pull data from the rest of the world, rather than having data pushed into it. And one of the simplest, dumbest ways you can make data available for the iPhone is to use a web server.
So this project uses a slight modification of Matt’s app. Instead
of reading its instructions all at once at the beginning of the
program, we want to fetch each step one at a time from a web server.
The app runs the step—pushing a button, or comparing a
caption, or what have you—and then reports its results by
hitting a /pass or /fail URL on the web
server. (Yes, I know that’s not what HTTP
GET is for, but until
Apple provides a one-liner for HTTP POST instead of a ten-line
novella,
GET will remain the refuge of scoundrels writing quickie blog
posts.)
Another web server listens to requests from Cucumber and…. Wait a sec. We’re going to need better names for these than “the first server” and “the other one.” In honor of the Big Important Handegg Game that was going on during this madcap experiment, we’ll call them the quarterback and the coach. The quarterback talks to the iPhone, and the coach talks to Cucumber. The coach signals plays to the quarterback via the silliest set of hand signals you can think of: the file system (told you it was a toy).
trying it out
You can actually try out these pieces in isolation, using curl to pretend to be an iPhone asking for instructions or an instance of Cucumber issuing them. But of course, it’ll be more fun to put it all together.
More detailed instructions and code are available using the distributed revision control system of your choice. Here’s the short, short version.
- Apply a patch to Matt’s project to fetch the list of test instructions from a web server instead of a disk file.
- Launch the coach and quarterback web servers.
- Start the Cucumber test.
- Kick off the test version of the app.
You won’t see the iPhone simulator pop up, but the code is running and duly reporting its results to the quarterback.
Where do we go from here? Lots of directions.
- The test script should have an option to kick off the simulator app, instead of waiting for you to launch it manually.
- The Ruby side should expand to do things like string comparisons and pass/fail decisions, so that we can move some of that stuff out of the iPhone.
- The cheesy one-liners I added to the app to fetch data should be replaced by event-driven network I/O.
- We should get this thing going on a real iPhone.
What we’ve got so far is a just a baby step. But take it from the father of a toddler, those can be entertaining.