Situation

Our client handles applications services for one of the world's leading automakers. A manual ad hoc process for testing how new vehicles sent data to driver's apps — once a luxury but now commonplace in new many new vehicles — caused unreliable information to appear. Owner complaints led to poor ratings of the associated vehicles.

After our work, J.D. Powers ranked the client’s app third in a 2023 report covering the entire industry.

With no formal testing procedures, the quality assurance process was costly. Testing cycles were long – up to quarterly. There was no vehicle integration testing or confirmation that smart phone application (SPA) commands arrived at the apps appropriately.

Action

We created and implemented a next-generation testing strategy and approach for the two devices affected. This included ensuring the process could run on multiple mobile OS and conducting in-vehicle testing of the devices' functionalities.
Highlights included:

  • Test runs with the Appium framework (Selenium Eclipse code) using Java scripts to test a physical device connected via USB cable.

  • Installing Android SDK and XCode for accelerated device testing.

  • Employing customized Appium framework for increased script reusability in mobile app testing and overall reduced maintenance.

To ensure that the software met quality standards, we conducted validation testing, including functional, manual, and automated regression testing.


We used Microsoft Visual Studio App Center/TestFlight to install and monitor the latest test build and initiated commands from the Smart Phone Application (SPA) for in-vehicle testing.


The devices then reported tested, reliable information for drivers, including service tracking, software version updates, fuel/energy miles, and the status of climate control, engine, tires, and airbags displayed on a diagnostic dashboard.


Results

After automating 95 percent of the test scripts, the client saw significant savings, and test regression cycle time dropped from five days to one. Test release cycle times dropped from three months to one.

Automated testing also led to the ability to validate more data for more vehicles and overall improved experience. Drivers could access vital information on Chrome, Edge, and Safari browsers and Android and iOS mobile platforms.

J.D. Powers ranked the client’s app third in a 2023 industry report.

1/3

Drivers receive vital information, including engine, tire, and airbag status.

2/3

After automating 95 percent of the test scripts, the client saw significant savings.

3/3

Drivers' ability to access real-time data about their vehicles is a selling point.

  • 95%

    Test scripts automated

  • 1

    1 Day (down from five) for test regression cycle

  • 3rd

    Ranking of app in industry