Do you want a guide on how to set up automated ui testing for your NativeScript apps? Welp, here it is!
This post outlines my experiences of working with the
nativescript-dev-appium plugin and writing my tests manually.
I have received a few messages requesting help with e2e testing so I wanted to get this post out there quicky. I will update it as I have more information to share, as well as write a few more blog posts on things like Using the Appium Client and Using the appium-doctor to Diagnose Installation Problems.
Note: This post contains my finds and gotchas along the way, I started by following the NativeScript docs on e2e testing NativeScript E2E Guide
I did my setup on a Mac, so if you are on Windows or Linux, your installation process might be different. Please refer to the NativeScript guide above if you have issues or reach out in the NS community Slack and I’ll try to help you!
npm install -g appium
You might have to run
sudo npm install -g appium
brew install carthage
brew install libimobiledevice --HEAD
brew install ideviceinstaller
brew install ios-webkit-debug-proxy
npm install -g ios-deploy
brew install telnet
npm install -D nativescript-dev-appium
During the installation process you will be asked a few questions
- What framework you are using (JS/TS, Angular, Vue) - I chose Angular
- What is your preferred testing framework (Mocha, Jasmine, etc.) - I chose Mocha
Note: After installing I recommend you go to your
tsconfig.json and add your new
e2e folder to your
This brings us to our first gotcha!
Gotcha #1 - Running the Default Tests
The default tests generated when you install the
nativescript-dev-appium plugin are for the hello-world-ts template.
If you used a different template the generated tests will likely fail
- Make changes to your app
- Build your app
tns build ios|android
- Write your test
- Run your test
npm run e2e -- --runType <capabilityName>
<capabilityName> is the corresponding key to the simulator/device you want to run the tests against inside of your new
npm run e2e -- --runType sim.iPhone7
Note: The first
-- is not a typo, I believe it is a flag to set
fullReset: true and
noReset: false in the appium.capabilities on the fly
If you ran
npm run e2e -- --runType sim.iPhone7 like me without having an iPhone7 simulator running, it should fallback to the simulator you have running, or atleast it did for me :)
If that command runs without any errors after a little bit a web browser tab should open, and the terminal should alert you of the results (in this case 0 passed 2 failed with information on why, see gotcha #1)
Gotcha #2 - App Permission Requests
My app has a location permission that immediately pops up, the tests will hang/fail but I manually accepted the permission alert from iOS and then the tests ran successfully
You can set up auto accept alerts by adding this line in the JSON body for the capabilityName you are using when running your tests, found in your
You have many other options you can use like
“autoDismissAlerts” : false
For a full list of the different capabilities of both iOS and Android check the Appium Docs on Capabilities
Gotcha #3 - Find Element By Image
With Appium you have many ways to find elements, one of them is
findElementByImage(base64String, threshold) where you pass in a base64 string of your image and Appium will search for that image in the page using the supplied threshold.
Intellisense was telling me that method takes the
path not the
base64 encoded string.
I added the image to the
e2e/resources/images/projectName/iPhone 6 folder.
Then I updated the test to try
But then got the following error in the mocha reporter:
Error response status: 13, , UnknownError - An unknown server-side error occurred while processing the command. Selenium error: An unknown server-side error occurred while processing the command. Original error: opencv4nodejs module is required to use OpenCV features. Please install it first (npm i -g opencv4nodejs) and restart Appium. Read https://github.com/justadudewhohacks/opencv4nodejs#how-to-install for more details on this topic.
I tried to run
npm I -g opencv4nodejs and got this error:
Error: failed to execute cmake --version, cmake is required to build opencv, error is: Error: Command failed: cmake --version /bin/sh: cmake: command not found
To get around this error I ran these commands
brew install cmake- to install cmake
cmake —version- to verify cmake installation
Then I tried to run
npm I -g opencv4nodejs again, after installing what felt like 5 million files/dependencies/etc the installation succeeded
Note: You might want to make sure your laptop is plugged in, I lost about 20% battery during this step 😂
All that leads me to our next gotcha
Gotcha #4 - What does findElementByImage() expect? Path or base64??
findElementByImage() method is an experimental feature so you might want to avoid trying to use it if you can, but if you must, here is my experience. I hope it helps you! 🤞
I looked at the source code for
findElementByImage() method and found that it is expecting the image fileName without the extension
Then it builds out a path to that file, and converts it to a base64encoded string before passing it to the Appium driver
So I tried using
driver.findElementByImage(imageName); again but it didn’t work.
I got the error
Error: The provided image does not exist!!!
findElementByImage() method tries to create the
filePath for the fileName you passed it, it looks in the following directory
After verifying that the
generatedPath by the
findElementsByImage() method was correct I got the following error even though the image was on the screen at the time:
Error response status: 7, NoSuchElement - An element could not be located on the page using the given search parameters
I switched to search for an image in the ActionBar to test and see if I could get Appium findElementByImage() method to succeed but it didn’t.
Gotcha #5 - findElementByImage() - ImageThresholdMatch
The image I was search for was a pin using the Google Maps SDK, so maybe that’s why the Appium driver couldn’t find it
I switched to search for an image in the ActionBar to test and see if I could get Appium’s
findElementByImage() method to succeed but it didn’t…
The 2nd parameter for
findElementByImage() is a
imageThresholdMatch, which is a setting for how close the images must match to be returned by the driver
imageThresholdMatch parameter is optional, and defaults to 0.4 if not passed
The range for the
imageThresholdMatch parameter is 0 (no comparison) - 1 (pixel by pixel comparison)
When I passed in
findElementByImage(fileName, 0); the test succeeded finally!!
After doing some more research I found some information regarding this
imageThresholdMatch on the Appium Docs on Finding Elements by Image
Under the Related Settings section, I read more into the
imageThresholdMatch, here is what it said:
The OpenCV match threshold below which to consider the find a failure.
Basically the range of possibilities is between 0 (which means no threshold should be used) and 1 (which means that the reference image must be an exact pixel-for-pixel match).
The exact values in between have no absolute meaning
For example a match that requires drastic resizing of a reference image will come out as a lower match strength than otherwise
It’s recommended you try the default setting
Then incrementally lower the threshold if you’re not finding matching elements
If you’re matching the wrong element, try increasing the threshold value
Note: If you use 0 as the threshold, it appears to always pass the
Gotcha #6 - findElementByClassName()
findElementByClassName() does not search for css class names like you might expect, but rather className like
Gotcha # 7 - findElementByAutomationText()
I tried setting automationText on an element in my view, and find it using the
findElementByAutomationText() method but it did not work…
After looking at the source code, it looks like
findElementByAccessibilityId() (this might be a bug)
This is where I switched and went to using the Appium client to help me with the rest of my testing. I will post another blog post about that shortly! Stand by! 😉
See you next time! ✌️