Why apps fail – and what you can do to prevent it

The mobile app market continues to grow and shows no signs of letting up. Major brands such as Nintendo and Warner Bros are making big investments in mobile. Nintendo recently announced their U-turn to take their games mobile for first time, and Warner Bros has said that it expects its new Batman and Game of Thrones mobile games to be its biggest ever.

According to Gartner, the number of app downloads will reach nearly 269 billion by 2017 and this shift into mobile is impacting companies of all shapes and sizes, making it one of the most competitive ventures the company will face.

But, while stories of major brands pushing into mobile talk up the opportunities and rewards of developing mobile strategies, the risks remain.

The reality is that mobile app strategies continue to come unstuck because of app development and quality control issues. It is estimated that that 44 percent of the defects are found by the user, 24 percent from direct feedback, and another 20 percent from public user reviews in app stores (Source: Perfecto Mobile 2014).

Too many apps of low or high value are released without sufficient quality testing and control, making it important to tighten up testing and certification processes if a brand is to avoid the financial or reputational damage of seriously disappointed and angry end users.

In this article, we outline the main challenges publishers face in mobile testing and what they can do to improve overall app quality.

Challenges for today’s developer

Mobile apps have dramatically changed the way users interact with brands. But with the rapid pace of mobile hardware and software innovation, many organisations struggle to meet user expectations. Enterprises spend months investing, planning and building an app to coordinate with the company’s mobile strategy, and when it launches they want it to be successful and have the same quality experience across all mobile platforms for every unique user.

The top challenges faced by quality assurance teams and the developer community can be summarised as insufficient device coverage, lack of reliable automation, and needing more coverage to execute test scenarios than time permits. Certainly, time is not on the side of today’s development and QA teams, as they must adapt to a number new dynamics emerging in real time.

The reality is that mobile app strategies continue to come unstuck because of app development and quality control issues.

Below are more of the most common challenges:

Rapid Innovation
The fast pace of change in the mobile market is putting pressure on development and operations teams to adopt rapid development and deployment practices. New devices, OS enhancements, wearables, sensors and third-party plug-ins are continuously being released into the market. Mobile app development teams must constantly update their release plans to ensure compatibility with a growing number of variables. These external pressures have compressed release schedules to weeks instead of months.

Increased Complexity
The mobile experience is dependent on a combination of factors including battery, memory, CPU / GPU, network connectivity, screen size, sensors and even app style. Ensuring quality in mobile environments requires enhanced testing, complex analysis and client side visibility. 

Skyrocketing User Expectations
Typically, commerce websites will fully load pages in about six to ten seconds. Mobile users expect faster results. Although few enterprises are achieving Google’s often cited one-second content load target, studies show that users will quickly abandon applications with response delays.

Overcoming Testing Challenges
To ensure that mobile apps launch successfully, more testing is needed. Survey respondents believe that the most critical measures to reduce mobile app defects are more functional testing, more device/OS coverage and more performance testing. 

The user experience is different in almost every scenario, each one contributing to an endless possibility of use-case issues. The testing needs to consider how the app will perform on different operating systems (Android, iOS, etc.), OS versions (Jellybean, KitKat, iOS 7, etc.), carrier networks (Vodafone, EE, O2 etc.) and devices (Samsung Galaxy S4, S4, S5 iPhone 5, 5C, 5S).

Broad market statistics about mobile adoption are important to consider when formulating a device coverage strategy, but paying attention to market trends is not enough.

Typically, commerce websites will fully load pages in about six to ten seconds. Mobile users expect faster results.

The most important question to answer in order to ensure sufficient coverage is “What devices are my target users using?” Analysing user traffic should help formulate a real-device strategy and give the most comprehensive coverage. 

Building an app that doesn’t fail

In order to ensure application quality in the modern age of mobile, a new approach is required.

As a mobile developer, there are many components in building an app that needs to be considered to ensure its success. The worst nightmare for a developer to launch an app they spent hours, days week and months with bugs and errors that taint the user’s experience. The important thing is to prevent failure from the beginning.

It is critical that an app is launched with the same quality experience across all mobile platforms for every unique user. Mobile apps need to effectively perform well across a broad range of mobile devices, operating systems and networks. Pre-production testing is necessary to avoid common pit falls, including UI glitches, cross-platforms inconsistencies, poor performance and excessive resource consumption. A positive end-user experience is only possible when testing strategies can effectively navigate these common pit falls. 

The worst nightmare for a developer to launch an app they spent hours, days week and months with bugs and errors that taint the user’s experience. The important thing is to prevent failure from the beginning.

The findings from Perfecto Mobile’s survey show some quick fixes that can easily be implemented by any developer team:

Stop having your users as part of the QA team
One of the most alarming stats from the survey conveyed how users have become part of the testing team – and that’s not good! According to the data, nearly half of errors (44 percent) are uncovered by users. After all the work in building and testing, it is quickly diminished when a user is the one who is impacted and communicating the problem.

Users are taking to public reviews (i.e. Apple’s App Store, Google’s Play Store and Twitter) and sending direct feedback to the organisation about these errors. The types of errors that were most frequently reported included user interface issues (58 percent); performance (52 percent); functionality (50 percent) and device compatibility (45 percent). DevOps teams need to focus on a strategy to improve mobile app quality; using end users as testers cannot be part of the solution.

Improving Overall App Quality
In order to improve mobile app quality from the start of the app building process, we recommend a three-phased approach:

1. Use a cloud-based Device-as-a-Service platform. This will help to assure sufficient device and OS coverage, and can help teams focus on testing, not managing devices
2. Implement a tool that supports continuous, unattended testing, Automating device testing enables developers to test more with fewer resources and greater consistency. It’s also important to select an automation tool that can develop cross-device, keyword-based automation scripts and run them across real devices in parallel, while connected to live networks. The tool should be a scalable and dynamic solution which integrates with existing tools
3. Deploy a monitoring system providing early detection of performance issues before they impact users will help to improve quality for users. 

Once the mobile strategy is set and the app is being developed, the key is to ensure its ability to perform and meet user expectations. Not accounting for the rapid changes in this tremendously fast-paced, fragmented market or failing to implement the proper automation tools that will ensure quality applications will guarantee user abandonment and app failure. 

According to the data, nearly half of errors (44 percent) are uncovered by users – and that’s not good!

Developing an enterprise testing strategy

When developing a mobile strategy, a company’s application should not only be at the core but everyone from the developer team all the way up to executive leadership should have a very clear picture of what goes into building a consumer-facing application and be aware of the worst case scenarios that could arise during the process.

Testing is the key to ensuring mobile app success, and enterprises need to decide which devices to test their mobile apps by establishing the right mix to match their customers’ needs.

Here are some of the most important questions to ask when devising your strategy:

What devices are used the most?
In order to determine the right mix of devices for testing, enterprises need to research and understand what devices are in the market, what devices are being used by their customers and what new devices are coming to the market.

We find that currently enterprises are analysing their employee’s device pool for internal productivity apps as well as their end-users device pool to generate a representation of where mobile traffic will come from. As a result, data shows 49 percent of enterprise testing hours are dedicated to Apple. This percentage is higher than the overall UK market representation and much higher than the global mobile representation, but as mobile strategies improve and enterprises listen to their end users, device test selections could quite possibly vary from industry norms. 

The Samsung Galaxy S2 is not even available in mobile retail stores and stopped receiving Android OS updates over a year ago. However, it is still being used by many, making it important from a quality assurance perspective.

In addition, enterprises must be a step ahead of the market and keep continued tabs on relevant market analysis. If the newest phone or tablet, like the new Samsung Galaxy 6 smartphone, enters the market and finds its way into the company, apps must be supported it. Combining these two analyses are important in creating an optimal device mix for testing.

What about the legacy devices?
Issues such as network coverage, screen size, legacy devices and operation systems all need to be considered. According to our data collected, iPhone 5S/C and Google Nexus 5 are the fastest growing operating system versions in use. And, despite nearing three years old, the Samsung Galaxy S2 continues to be used by many customers. This device is not even available in mobile retail stores and stopped receiving Android OS updates over a year ago. However, it is still being used by many, making it important from a quality assurance perspective.

Will my app work in other countries?
As your market expands, your reach into different regions also grows and with that, your enterprise mobile strategy needs to ensure its mobile apps are being tested in all regions, with all carriers and networks. For instance, a function on your application may work in the United Kingdom but not in the USA. Knowing how to fix your app by testing locally will prevent the app from failing.

Does size matter?
There are a growing number of screen sizes and form factors to consider when building out the overall enterprise testing strategy. Screen resolution will impact how the app will be viewed by the end user on the various devices. We find the leading resolution of the market today is 640 x 1134 (4”), the popular android devices. Next was 640 x 960 (3.5”) resolution—the iPhone. There were over seven other types of resolutions that testers were using in their mobile testing strategy.

Apple vs. Android Battle: the surprising leader
Finally, despite the latest studies that show Android leading the consumer market with more than 80 percent market share globally, Perfecto Mobile’s data shows that enterprises in Europe and North America actually spend close to 50 percent of their time testing on iOS devices. Consumers may be leading the pack with their Android devices outside of work but enterprises often take longer to adapt to market trends.

Why do apps fail? The same reason most projects do: lack of planning and strategy.

Planning is the key to success

So, why do apps fail? The same reason most projects fail: lack of planning and strategy. One of the most critical protocols that should top the list but is often overlooked is mobile app testing prior, during and after launch. 

A company’s mobile application can have all the bells and whistles to blow away the competition, but if it’s not available on all the devices their customers are using, doesn’t perform well or isn’t reliable they will face a number of implications, including app abandonment.