Emulators and Simulators
Mobile apps are not just ‘software for smaller screens’. You are creating an app for a device that’s structurally and logically different from your workstation, on your workstation. Resizing your browser window isn’t enough to make a web app built on Chrome-on-macOS compatible with Chrome-on-Android.
Any app built for mobile devices needs to be tested-early and often-within mobile environments for context and compatibility. Early in the app development lifecycles, emulators and simulators are used for rapid prototyping and unit testing.
We’re taking a deeper look at emulators and simulators to understand how they work and what types of testing they are suitable for.
Note: Bear in mind that emulators and simulators are only fit for use in the initial stages of testing. Website emulators, browser simulators. mobile phone simulators, online virtual devices, and the like are not reliable enough to run tests on for public release of apps and sites. The best practice, without exception, is to test on real devices.
What are Emulators and Simulators?
The terms ’emulators’ and ‘simulators’ are often used interchangeably, but they have very different sets of capabilities. Let’s look at each to make the difference clearer.
Emulators
Emulators mimic your target device’s hardware and software on your workstation. Android Emulator (by Android Developer Studio) is a popular example.
In a previous post, we covered how Android emulators (both SDK and any third-party emulator online) work. All the same, here’s a recap.
How the Emulator Works: All computers–including mobile devices–work on an ISA, that is, Instruction Set Architecture. This is a set of instructions written in machine-language that your processor understands. Different processor families (think Intel, AMD, ARM, etc.) have their own instruction set architectures, which they implement in their own ways.
The emulator mimics the target/mobile device processor. Then it translates its ISA into the one used by your computer, through a process called binary translation. This binary (ABI–or Application Binary Interface) can be equipped with a compatible operating system and APIs.
Capabilities: The emulator can give you virtual device instances with near-native capabilities and extended controls to adjust the devices’ physical sensors, battery state, geolocation, and more.
Limitations: These near-native capabilities come with a significant performance overhead, mostly due to binary translation.
It is possible to speed up the ABI translation through hardware-assisted virtualization (also known as hardware acceleration). If the mobile device you’re emulating has the same ISA as your computer, the emulator can skip the translation and run the virtual device directly on your workstation’s hardware.
There are two problems with this. First, there’s the precondition–the ISAs of your computer and target mobile device need to match. Most commercially available mobile devices run on ARM’s architecture. Almost all computers work on Intel x86. Their ISAs are categorically different from each other.
You also need a sizable list of hypervisor components just to enable hardware acceleration. These are tricky to set up, even for the more experienced programmers.
In honesty, virtual mobile device emulators can be rather unreliable when it comes to testing Android apps and websites. Given their various limitations, they simply cannot replicate real-world conditions, thus leading to inconclusive and inaccurate results. They cannot be trusted to test apps and websites for public release.
It is far easier and more trustworthy to test on real Android devices. That way, you know exactly how the site or app will operate in the hands of end-users.
Additionally, if you’re wondering “how to test a mobile app on desktop”, simply avail of our cloud-based infrastructure with 3000+ real browsers and devices. Just sign up for free, and test your app or site on thousands of real Android devices, including the latest ones.
Simulators
Simulators let you run programs that were not made for your computer’s OS. Simulators for mobile app testing are commonly used by testers to run their code, especially in the early stages. In the context of this post, ‘simulator’ refers to the iPhone and iPad simulator in XCode. You can also think of them as mobile simulators (especially for iPhones).
How iOS Simulators Work: The iOS simulator sits on top of your operating system. From there, it mimics iOS and runs your app inside it. This process is viewable in an iPhone or iPad-like window.
You can interact with this device window via keyboard or touchpad/mouse.
Capabilities: The iOS simulator is significantly faster than Android emulator, purely because there’s no machine-language translation involved.
Limitations: The simulator cannot mimic battery states or cellular interrupts, unlike the Android emulator.
You also cannot use the iOS simulator on platforms other than macOS. This is because the simulator needs Apple’s native Cocoa API–a massive library of frameworks–to handle GUI, runtime, and more.
Porting Cocoa to a different platform is far too much trouble. Instead, developers virtualize macOS on their computer hardware–or procure a MacBook.
There’s the Cider APK, which positions itself an ‘iOS emulator for Android’ for gamers. It works by porting some iOS apps to Android.
There was an open-source project called iEMU which was supposed to be an emulator for iOS devices. The project was discontinued in 2014.
The only true iOS emulator that exists today is owned by Corellium–and it’s not available to everyone.
It is tempting and common for testers to be looking for a “mobile simulator for testing”, but as this piece demonstrates, existing iOS simulators, especially available ones are incapable of replicating real user conditions effectively. That means any tests run on mobile simulators will be inconclusive at best and inaccurate at worst. In other words, these tests cannot be trusted. Testing on real iOS devices is the only way to guarantee completely accurate results, every time.
For these reasons, we refer to the mobile emulator in Android SDK as simply the ’emulator’.
Emulators and Simulators: Key Differences
The Android emulator abstracts hardware differences. It emulates everything from the higher-level operating system right down to the lower-level I/O (input-output model), memory cells, instruction sets, et al. This gives you a very close imitation of a real device environment, despite the enormous performance overhead.
The iOS simulator abstracts platform differences. Since both iOS and macOS are based on Cocoa API and can run the same apps (written conditionally in Objective-C), there’s little-to-no compiling involved. However, it can not virtualize real iOS device hardware or its instruction set architecture. The best it can do is use ARM’s instruction length for running commands on your Mac’s processor. This provides a limited real-device hardware context.
This is important. Hardware abstraction and fragmentation are the primary roadblocks to building stable cross-device compatible apps. By now, it should be clear how it is nearly impossible–in practice–to virtualize Android or iOS device hardware.
But emulators and simulators are not entirely unreliable.
Free Test on Real Device Cloud
Emulators and Simulators in Mobile App Testing
Emulators and simulators are popular for their ease of access. They are available within the SDK (Android as well as XCode). In a few clicks, you can test your current project inside a virtual device environment.
UI engineers and designers sometimes use online emulators and simulators for rapid prototyping and quick mockups.
In production, developers use emulators and simulators for running basic unit tests on app code in a platform and (limited) device context. This reduces the likelihood of having to make major changes in the app’s logic after integration. An example would be testing whether the UI accepts input as it should, how the code uses third-party and platform-specific APIs and so on.
It applies to web apps too. Chrome Developer Tools, for instance, has a mobile browser emulator which gives you a device-viewport context. This lets you test your web app’s UI during development in a few clicks.
Pre-Release Testing: Virtual vs. Real Devices
Compatibility and performance tests cannot give conclusive results on virtual iOS and Android devices. There are several reasons for this.
In the iOS simulator, your app is running with unrestricted access to your Mac’s resources. Say you’re testing an iOS app feature that stores large media files in iPhone’s main memory. Your test will always pass on the simulator, but not on a physical iPhone.
A simulated iOS device will run faster or slower depending on the number of computing resources that are online on your MacBook.
Similarly, virtual Android devices–when created with non-x86 ABIs–will always run slower than real Android devices, regardless of your computer’s clock cycle.
Let’s go back to the iPhone simulator example from before. Assume that you take the false positive at face value and continue to code an entire feature–say, a voice modifier–on top of it. If your CI engine tests your builds on real devices, the error will be caught in time. If not, you will end up deploying a buggy feature to beta.
Remember that most mobile apps are deleted in a week after being installed for reasons like battery and memory drain and janky UI. You cannot catch these issues without testing on mobile device hardware—which needs real mobile devices.
This is why modern product teams run automated tests across a carefully curated matrix of real mobile devices to find UI, system, and performance bugs in mobile apps before release. This helps pinpoint error sources to the exact line of code for quick debugs.
Using mobile simulators and emulators helps you develop a habit of testing code in the mobile context. But hardware differences–in a fragmented and ever-growing mobile device market–are impossible to fully abstract, even with the best of virtualizing tools. That’s why both Apple and Google insist upon testing your apps on real mobile devices before deployment. Don’t depend on online mobile emulators for website testing or mobile phone simulators for app testing.
Ready to take your native, hybrid, or web app through a real testing mill? Take a look at App Automate, which runs your Appium, Espresso, XCUITest, and EarlGrey test suites on 3000+ real iOS and Android devices, concurrently.
