I was recently given some direction by an iPhone developer on an existing application:
You need to have unit tests. I know there aren’t any unit tests in the code now, but you need to have them. When the original code was written, it was done over weekends and I didn’t have time to write tests.
Not having time is certainly understandable (but aren’t unit tests supposed to *save* time?) – but in reality there are many reasons why a lot of iPhone applications you see won’t have solid unit test coverage:
- The iPhone IDE (yes, you Xcode!) does not encourage TDD.
- Apple provides very little guidance around unit testing iPhone applications. And the guidance they provide can be confusing to folks coming from other platforms.
- Setting up the environment can be tricky!
- There are multiple competing iPhone unit testing frameworks that improve upon the stock OCUnit (SenTestingKit) framework that comes with the iPhone SDK. So it seems that multiple people must have thought OCUnit could be improved upon!
This post will get you started with the “default” unit test framework, OCUnit (a.k.a. SenTestingKit), provided by Apple. Our goal is to understand what is available out of the box with the iPhone SDK and get our first test to execute!
Let’s get started!
Setting up our project.
By far the most confusing part in setting up tests for the iPhone is configuring the project and the build environment to execute tests. Once the project structure has been laid out correctly and we get the first test executing, there is no more infrastructure to prevent us from writing hundreds and hundreds of tests.
Unfortunately when compared to frameworks like ruby on rails – which rightly puts an emphasis on TDD – the iPhone SDK and Xcode look rather silly. In rails, tests are highly encouraged. Not so in Xcode. There is a little setup required outside of the default template. It’s not difficult, but it’s not out of the box either.
Step 1 : Add a new target
I’ve created a project called UnitTestingExample. Add a new Unit Test Bundle target to the project. This target will execute all of our unit tests, so we’ll name it something related to that. I’ve called this target LogicTests.
Notice in the last screenshot we have two targets – our main application (UnitTestingExample) and our unit test bundle (LogicTests). Behind the scenes, Xcode has configured our target to find OCUnit, which quite confusingly is called SenTestingKit – based on the name of the open source project in which the code was written.
Step 2 : Create Test Cases
Now that we have a target for our tests, let’s create a new class that will contain our tests. Create a class (deriving from NSObject in Xcode). I created one called LogicTests.m. Be sure to add it to the LogicTests target only.
Open LogicTests.h and make a few changes. First, we need to import SenTestingKit. SenTestingKit is the unit testing framework which is also referred to as OCUnit (rather obviously short for “objective-c unit” – to keep in line with the xUnit pattern). We also derive our class from SenTestCase. At this point, the header is done. It’s time to write our tests.
In LogicTests.m – create (void) functions with no parameters that start with test. SenTestingKit will automatically execute all functions that have this signature. In the following case, we will let the test fail to understand how SenTestingKit will report errors.
Step 3 : Build The Test Target
Now let’s run the tests. Ensure the LogicTests target is setup to run under the Simulator. The tests will not be executed using Device. This is quite unfortunate – the Device build actually succeeds without running any tests! We’ll keep this little fact in mind when we look at other unit testing frameworks.
Unlike most all other unit testing frameworks, the tests are run *after* the project has successfully built. Not so with SenTestingKit. The tests are actually ran as part of the build process. The build will fail if any tests fail, which is good. However it is not typical workflow in how unit test frameworks tend to work. Typically tests run *after* the build step! (Really these are running after the application is built, but the workflow is a bit different than the workflow in other platforms). Looking at the build output we will see where the failures occurred. Xcode does a rather nice job of pointing you at the errors.
Congratulations! If you have made is this far you have unit tests up and running! At this point we can create more tests and more test classes. In the following example, we test a Calculator class and a simple test using NSURLConnection to illustrate that we can test against the Foundation library.
Limitations
Let’s step back a second. There are a few limitations that are going on here that we need to call out:
- We can’t debug tests. Because these tests run during build time (in a shell script build step), we can’t set breakpoints. This will be a deal breaker as the number and complexity of our tests increase.
- The tests are executed in what Apple refers to as a “clean room”. i.e., not the real world. We need to be aware that we do not have access to a UIApplication object or the rest of the iPhone application environment. Apple refers to these type of tests as “Logic Tests” (which is why I named my test class LogicTests.m). Logic Tests test your lower level code that is independent of the iPhone application context (run loop, touch events, etc..)
- Controlling execution / output. Getting build errors in Xcode helps us solve the problems and fix the tests. That does the job – but it barely scrapes the surface on features when compared to other platforms. We are sorely lacking other helpful features that most all other environments have. How about a GUI runner where we can see log output per test? How about the ability to run one test or a group of tests? A few more simple features would certainly be nice to have.
Next steps
This has been a good first step. We have unit tests running – which puts us ahead of many iPhone applications currently on iTunes! We’ve learned how to create a target specifically for testing, add files to our project that will only be contained in one target, and create tests.
With these fundamentals out of the way, it’s time to get out of our “clean room” and start cranking out tests that actually stress our application – tests that hit UIApplication, our view controllers, and our UI code. We also need to tackle the other limitations – like better debugging and better control over our test execution! We also need to go behind the scenes and see how SenTestingKit really works and how our bundle is created. We didn’t add a reference to SenTestingKit in our application – but hey, we have tests!
In the next post we will address testing UI functionality and getting tests running on a device. Once we have this, we will be able to test cover our entire iPhone application. We’ll then move on to more advanced features that 3rd party toolsets provide – like being able to selectively run tests and control test execution a bit better.
