Testing and Debugging
This is the fourth post of our five-part series on Android vs. iOS development from our microProduct Lead, Mark Oldytowski.
You’ve made a great app, so now you just have to make sure it actually works on the majority of devices out there (don’t skimp out and only test your own device, it will bite you quickly). Luckily, both environments provide you with the tools you need to debug your code and each have ecosystems for pushing your software out to local and remote testers. Testing your app with external users will be key to finding issues that everyday users might run into, but you wouldn’t notice as a developer (users do some really, really strange things you would never expect).
Within Android Studio, you will find an easy to use debugging environment with all the standard features including breakpoints, console output, memory analyzer, and hover over variable inspection. As mentioned earlier, Android Studio has an emulator you can use for debugging, but anyone besides a master of patience will find it fairly useless to use on a regular basis. When you are ready to enter the real world and use a regular device for testing, just tap the build number 7 times (weird, I know), enable debug mode, and you are ready to go. Runtime debugging is straightforward with Android, but there are a few issues. Right away, you will likely need to hunt down the USB driver for your specific device before Android Studio will even recognize it, which can be difficult with off-brand devices. After that, you may randomly drop the connection between the debugger and your device, or it might just stop recognizing the device completely, so you may need to restart Android Studio / your phone to get things back to normal. Other than those few problems, the experience with Android Studio is enjoyable with great support for catching runtime exceptions and crashes, sans the few segmentation faults that can occur when using specific device features (camera, device storage, etc.) and require serious research to figure out what is causing the crash (although this happens on iOS as well).
On the Xcode side, you will find similar features to Android Studio, but with three key differences: usable emulators (as mentioned before), paid device debugging, and provisioning profiles. Apple requires you to join the developer network before you can deploy your app to a physical device, so if you don’t pay the buddy money (plus signing up for the store, more on that in the next post), you won’t be able to deploy your app to any physical device (unless you jailbreak it). This may not impact most developers who plan on deploying their app to the store anyways, but it could affect new developers who are trying to learn iOS development and aren’t sure if they want to pay the money and stick with it. Xcode also adds the complication of requiring the use of a “Provisioning Profile” for features such as push notification, In-App purchased, etc., and any program using these features requires the device ID to be registered into the profile before deploying to the device. This causes a back-and-forth situation between the Apple Member Center and Xcode to be able to load new local devices. At the end of the day, it doesn’t prevent you from deploying the app, but it can be a major cause of frustration if the Provisioning Profile is not in sync with the device being used or the deployment features.
Once you are ready to send your app out to external testers, Android keeps things simple by not restricting the way you send the app out to the users. If you want to email the app to the user, just send it and all the tester has to do is enable debugging mode, drag-and-drop the app onto the device, and they are ready to go (no device IDs required). If you are looking for a more formal and traceable way of deploying the app, there are multiple external services you can use to send your app out to users on a version-by-version basis (HockeyApp, Apphance, etc). Most of these services will provide you with a way to get feedback and crash reports from the users testing the app, which can be extremely useful given the wide spread of device types the testers will be using.
Apple has taken a more integrated approach to testing recently (it’s about time) with its purchase of TestFlight in 2014. As of iOS 8, Apple has incorporated it directly into its development environment, for better or for worse. The advantage of this is that you can now deploy beta versions of your app directly from Xcode, and you no longer have to worry about getting the Device IDs from each user and adding them to the provisioning profiles (Apple handles this for you). Of course, Apple has managed to convolute multiple aspects of the setup process and test deployment (as it tends to do). The initial setup process of adding users to the testing process has moved to a hidden area of the Developer center, which now requires configuring the app and each user you would like to invite to the beta. Internal and external testers are now broken up into separate sections, and deploying to external testers requires approval from the Apple review board (more on this in the next blog post). Once you pass the initial headache of figuring out how it actually works and get everything up and running, it does run very smoothly and allow quick deployments straight from Xcode (a huge plus). There are external testing platforms still available on the iOS side, but with the way Apple typically is, I can see those starting to disappear (or just stop working) soon now that there is an official way of doing things.
|+ Plenty of options for internal and external testing||+ Integrated testing system
|+ No Provisioning profiles / Device IDs required for testing||+ Stable testing on devices|
|– USB Driver hunt for devices||– Provisioning profile issues, even when debugging|
|– Debugging device disconnect issues||– Convoluted initial setup for adding users to TestFlight|
|– Pay wall for device debugging|
Stay tuned for Part 5: App Store Deployment