Mobile apps are not just ‘software for smaller screens’. You are creating an app for a device that’s structurally and logically different from your workstation, on your workstation. Resizing your browser window isn’t enough to make a web app built on Chrome-on-macOS compatible with Chrome-on-Android.
Any app built for mobile devices needs to be tested-early and often-within mobile environments for context and compatibility. Early in the app development lifecycles, emulators and simulators are used for rapid prototyping and unit testing.
We’re taking a deeper look at emulators and simulators to understand how they work and what types of testing they are suitable for.
What are Emulators and Simulators?
The terms ’emulators’ and ‘simulators’ are often used interchangeably, but they have very different sets of capabilities. Let’s look at each to make the difference clearer.
Emulators mimic your target device’s hardware and software on your workstation. Android Emulator (by Android Developer Studio) is a popular example.
In a previous post, we covered how Android emulators (both SDK and any third-party emulator online) work. All the same, here’s a recap.
How the Emulator Works: All computers–including mobile devices–work on an ISA, that is, Instruction Set Architecture. This is a set of instructions written in machine-language that your processor understands. Different processor families (think Intel, AMD, ARM, etc.) have their own instruction set architectures, which they implement in their own ways.
The emulator mimics the target/mobile device processor. Then it translates its ISA into the one used by your computer, through a process called binary translation. This binary (ABI–or Application Binary Interface) can be equipped with a compatible operating system and APIs.
Capabilities: The emulator can give you virtual device instances with near-native capabilities and extended controls to adjust the devices’ physical sensors, battery state, geolocation, and more.
Limitations: These near-native capabilities come with a significant performance overhead, mostly due to binary translation.
It is possible to speed up the ABI translation through hardware-assisted virtualization (also known as hardware acceleration). If the mobile device you’re emulating has the same ISA as your computer, the emulator can skip the translation and run the virtual device directly on your workstation’s hardware.
There are two problems with this. First, there’s the precondition–the ISAs of your computer and target mobile device need to match. Most commercially available mobile devices run on ARM’s architecture. Almost all computers work on Intel x86. Their ISAs are categorically different from each other.
You also need a sizable list of hypervisor components just to enable hardware acceleration. These are tricky to set up, even for the more experienced programmers.
Simulators let you run programs that were not made for your computer’s OS. In the context of this post, ‘simulator’ refers to the iPhone and iPad simulator in XCode.
Note: There are no ‘Android simulators’, largely because it’s simpler to emulate Android devices with freely available tools like Quick Emulator (QEMU).
How iOS Simulators Work: The iOS simulator sits on top of your operating system. From there, it mimics iOS and runs your app inside it. This process is viewable in an iPhone or iPad-like window.
You can interact with this device window via keyboard or touchpad/mouse.
Capabilities: The iOS simulator is significantly faster than Android emulator, purely because there’s no machine-language translation involved.
Limitations: The simulator cannot mimic battery states or cellular interrupts, unlike the Android emulator.
You also cannot use the iOS simulator on platforms other than macOS. This is because the simulator needs Apple’s native Cocoa API–a massive library of frameworks–to handle GUI, runtime, and more.
Porting Cocoa to a different platform is far too much trouble. Instead, developers virtualize macOS on their computer hardware–or procure a MacBook.
Note: As of 2019, there are no true ‘iPhone or iPad emulators’. There are only iOS simulators dressed up as such. This is because Apple uses proprietary chipsets and custom binaries in all its devices. You will need to reverse engineer the Apple device to virtualize it.
There’s the Cider APK, which positions itself an ‘iOS emulator for Android’ for gamers. It works by porting some iOS apps to Android.
There was an open-source project called iEMU which was supposed to be an emulator for iOS devices. The project was discontinued in 2014.
The only true iOS emulator that exists today is owned by Corellium–and it’s not available to everyone.
For these reasons, we refer to the mobile emulator in Android SDK as simply the ’emulator’.
Emulators and Simulators: Key Differences
The Android emulator abstracts hardware differences. It emulates everything from the higher-level operating system right down to the lower-level I/O (input-output model), memory cells, instruction sets, et al. This gives you a very close imitation of a real device environment, despite the enormous performance overhead.
The iOS simulator abstracts platform differences. Since both iOS and macOS are based on Cocoa API and can run the same apps (written conditionally in Objective-C), there’s little-to-no compiling involved. However, it can not virtualize real iOS device hardware or its instruction set architecture. The best it can do is use ARM’s instruction length for running commands on your Mac’s processor. This provides a limited real-device hardware context.
This is important. Hardware abstraction and fragmentation are the primary roadblocks to building stable cross-device compatible apps. By now, it should be clear how it is nearly impossible–in practice–to virtualize Android or iOS device hardware.
But emulators and simulators are not entirely unreliable.
Emulators and Simulators in Mobile App Testing
Emulators and simulators are popular for their ease of access. They are available within the SDK (Android as well as XCode). In a few clicks, you can test your current project inside a virtual device environment.
UI engineers and designers sometimes use online emulators and simulators for rapid prototyping and quick mockups.
In production, developers use emulators and simulators for running basic unit tests on app code in a platform and (limited) device context. This reduces the likelihood of having to make major changes in the app’s logic after integration. An example would be testing whether the UI accepts input as it should, how the code uses third-party and platform-specific APIs and so on.
It applies to web apps too. Chrome Developer Tools, for instance, has a mobile browser emulator which gives you a device-viewport context. This lets you test your web app’s UI during development in a few clicks.
Pre-Release Testing: Virtual vs. Real Devices
Compatibility and performance tests cannot give conclusive results on virtual iOS and Android devices. There are several reasons for this.
In the iOS simulator, your app is running with unrestricted access to your Mac’s resources. Say you’re testing an iOS app feature that stores large media files in iPhone’s main memory. Your test will always pass on the simulator, but not on a physical iPhone.
A simulated iOS device will run faster or slower depending on the number of computing resources that are online on your MacBook.
Similarly, virtual Android devices–when created with non-x86 ABIs–will always run slower than real Android devices, regardless of your computer’s clock cycle.
Let’s go back to the iPhone simulator example from before. Assume that you take the false positive at face value and continue to code an entire feature–say, a voice modifier–on top of it. If your CI engine tests your builds on real devices, the error will be caught in time. If not, you will end up deploying a buggy feature to beta.
Remember that most mobile apps are deleted in a week after being installed for reasons like battery and memory drain and janky UI. You cannot catch these issues without testing on mobile device hardware—which needs real mobile devices.
This is why modern product teams run automated tests across a carefully curated matrix of real mobile devices to find UI, system, and performance bugs in mobile apps before release. This helps pinpoint error sources to the exact line of code for quick debugs.
Using mobile simulators and emulators helps you develop a habit of testing code in the mobile context. But hardware differences–in a fragmented and ever-growing mobile device market–are impossible to fully abstract, even with the best of virtualizing tools. That’s why both Apple and Google insist upon testing your apps on real mobile devices before deployment.
Ready to take your native, hybrid or web app through a real testing mill? Take a look at App Automate, which runs your Appium, Espresso, XCUITest, and EarlGrey test suites on 3000+ real iOS and Android devices, concurrently.