Before a butterfly can take its first flight, it must push and fight its way out of a cocoon. Without this challenging obstacle, it would be too weak to survive in the outside world. What does this have to do with game testing? Well, like a butterfly, every game should undergo a serious test that strengthens it before release to the world.
Game testing is a vital step in the life cycle of games large and small, and skipping it can cause disaster. Even if you have absolute faith in the quality of the work done by your developers, choosing to engage your testers or hire a game testing company can spare you loads of time and hassle. In this guide, we will examine game testing from all angles and share everything you need to get started with this process.
What is Game Testing?
Game testing, sometimes referred to as QA (quality assurance) review, is a process of quality control over a game in development. It usually includes a series of tests that encompass every aspect of the application, from its design and structure to performance and user experience. The data and feedback received from the tests are used to apply bug fixes and improvements, and are often iterative, with testing being applied multiple times until the game is in a polished state.
What is game testing automation?
Automation is one of two approaches in game QA, with the other being manual testing. With automation, certain performance tests can be performed many times in a row without the tester’s participation. This saves time and makes it much more likely to catch rare bugs.
Though this approach is more convenient than asking testers to complete the same levels many times manually, it is quite difficult to create automation tools tailored to an individual application. According to a survey of game development companies, most prefer to manually test their games, citing a lack of resources and time to create automation software.
What do game testers do?
Many young people dream of becoming game testers because they can play games all day, but this only scratches the surface of the position’s responsibilities. Apart from reviewing the game mechanics, they have less romantic and dynamic duties like:
- Navigating and sampling all menu options
- Running CPU and performance analysis
- Sampling all customization options
- Intentionally failing at the game and testing obstacles
- Checking and analyzing system specs
- Compiling reports and filling in feedback forms
- Running test scripts and test cases for game applications
As you can see, there is a lot of repetition involved in game functionality testing, and experts have to devote much of their time to other tasks like making reports and working with code and scripts. For anyone considering this sort of job, you have to not just love games, but be ready to view them from an analytical and constructive point of view.
How Does Game Testing Work?
As we have mentioned above, there is more to game testing than just launching a game and playing it from start to finish to make sure it runs smoothly. There are dozens of intricacies to the game testing process, so let’s take a closer look.
Stages of game testing
The overall structure of the QA process is simple:
But where does QA review fit in the grand scheme of development? During the initial phases of development (planning and production), QA engineers tend to have very little input. These stages are mostly handled by project managers, business development experts, and developers. Still, they may be asked for feedback on things like game logic and flow, UX design, and interaction design.
Most of the work done by engineers starts when a playable build of the application is ready. This is sometimes referred to as video game alpha testing. The review takes place within the confines of one company, with the goal of fixing most issues and delivering a high-quality, fully-fledged gaming experience.
Some companies opt to also add beta testing games to the docket. With this approach, people from outside the company are asked to preview and beta test PC games and releases on other platforms (consoles are also common). This gives a much better understanding of the overall impression to expect from players than a company could get from a few QA engineers.
Key metrics of QA functional testing
Feedback on game quality is not something formulated broadly like “The game runs smoothly”. It includes concrete figures, statistics, and terminology that offers much more useful information. Let’s go over some of the key game QA metrics:
- Test effectiveness. This metric is collected based off dozens of other measurements, including the number of tests conducted, how many bugs found/fixed, and how many issues persisted post-fix.
- Economic variables. This metric pertains to the overall cost of testing and cost per bug fix, variance of planned and actual cost, and difference between planned time spent and actual time.
- Team metrics. In this case, the work of the testing team is measured, both as a single unit and individually.
- Change metrics. The changes applied during testing are analyzed to determine how many issues were fixed, which new issues popped up and how many, and other statistically significant changes over time.
- And many others. Depending on who you ask, there may be as many as hundreds of metrics that a company records and analyzes.
Bug reports
Bug reports are an indelible part of QA review and represent the information/feedback collected based on the engineers’ tests which can be used for further improvements. A traditional bug report will feature the following details:
- Summary – A brief description that identifies the issue and its impact.
- Background information – data about the platform and build number, along with other relevant contextual info.
- Steps to reproduce – instructions for developers to check the issue from their own side.
- Actual and expected results – a short text pointing out how the issue currently affects the player experience and how it should work without the issue.
- Other relevant information – any other details that can help developers/designers understand and solve the problem.
Main Types of Game Testing Methods
Each company has its own perceptions about which tests are necessary for a game to undergo, and the list may even vary from project to project. Still, we can outline the most common tests, dividing them into functional (game functionality testing) ones and non-functional ones (associated with overall performance and UX):
| Functional testing | Non-functional testing |
| Component/module (checking the performance of smallest software units) | Performance (speed of game loading in real-time) |
| Integration (finding defects in interfaces and component interactions) | Load/Stress (checking performance under heavy user activity/traffic) |
| Smoke (determining build stability) | Installation (how well the game saves on different devices) |
| Regression (verifying bug fixes) | Usability (convenience of gameplay mechanisms) |
| Localization (checking the consistency of translated content) | Recovery (how the application works after crashing) |
| Security and access control (identifying vulnerabilities and verifying user permissions) | Compliance (ensuring coding standards are consistent across builds and platforms) |
| Configuration (reviewing the interaction between software and its environment) |
How Testing Works on Different Platforms
Games are often released on several platforms at once, and even within the scope of one platform, testers have to make sure that everything works on different devices. Thus, there are plenty of peculiarities when it comes to QA review on different platforms:
Desktop devices tend to have more power and more flexibility in terms of screen resolution/graphics. Thus, if a game is built with various customization and optimization options in mind, much of the PC game test work goes into making sure that each player with the hardware to support the game is able to pick the customization options (or have them configured automatically) to get the most of the experience.
Additionally, desktop games often take advantage of large keyboards by supporting dozens of key combinations. It is the job of a computer games tester to make sure these commands work consistently across computers with different key names and compositions (e.g. macOS, Linux, and Windows devices).
The two biggest unique aspects of review by a game console tester are: 1) ensuring compliance and compatibility with console maker standards; 2) providing a uniform gaming experience across platforms and console generations. In the first case, Xbox, Sony, and Nintendo have strict guidelines about the parameters and content of the games that can run on their platform, so testers have to make sure that the product ticks all of the right boxes.
In regards to cross-platform and multi-generation gameplay, we have recently seen an example of poor work by a PlayStation video game tester with the release of Cyberpunk 2077 on PS4. Developers have a very difficult job in making sure that their application runs both on weaker previous-gen consoles and next-gen, not to mention cross-platform releases which are several times more complex. In turn, a functional game tester may spend months mopping up the bugs and performance issues that arise pre- and post-release.
One of the biggest difficulties in developing for mobile devices is making use of limited screen space, so anyone AB testing mobile games must work hard to check that players are able to access all of the game’s features even on the smallest smartphone screens.
Another peculiarity of these apps is that they often have multiplayer and social networking features, which are notoriously difficult to properly develop. Thus, mobile game load testing and cross-platform evaluation are integral to the traditional mobile QA experience.
Finally, mobile apps are more likely to use open-source code and elements than other platforms, which leaves them vulnerable to hackers and malicious actors. Developers are often unaware of these issues, but the mobile game QA tester must be vigilant in finding and covering any vulnerabilities.
Immersive apps present unique challenges to testers, but ones that can be overcome. For example, the movement element of most VR games can cause motion sickness and dizziness in some players, which developers often fail to account for. In addition to that, those who test VR games have to make sure that the game properly conveys safety recommendations, such as clearing the area where the user is playing. Otherwise, there is a chance of them injuring themselves.
There is also a significant physical aspect to a VR games test. The specialist could be just fine with spending 8-10 hours in the game on their computer screen, but once they stand up, put a VR headset on, and move around, they can get tired quite easily. Thus, test sessions are often broken up into 1-3 hour intervals scattered throughout the week, which can ultimately prolong development.
Testing Example – Hexagonium Game
Hexagonium is a game developed by Game-Ace and also tested by our specialists. It was designed for mobile devices and features multiple game worlds, strategy-focused gameplay, and even multiplayer competitions. Our developers were thorough and studious, but there were still some bugs and issues identified by our QA specialists that had to be ironed out prior to release.
For example, our Unity game tester found that certain menu and UI options were truncated or disappeared completely when the game was played on very small mobile devices. Thus, we had to come up with several solutions to scale and optimize the experience for everyone’s enjoyment.
When it came to graphics, we also discovered that older and less powerful devices had difficulty rendering some of the 3D action, so we optimized for the devices that we could and removed support for those that were beyond our scope.
We can help you overcome testing challenges
Game-Ace successfully polished and released Hexagonium to the delight of thousands of players, and we can do the same for your project. With numerous QA engineers, developers, and project managers on our staff, we can help you with any stage of game development, or just lead you to the finish line with a thorough and professional batch of tests.
We have been building and testing games for over a decade now, and have acquired in-depth expertise that few other studios can match. If you wish, we are ready to demonstrate this experience and push your project forward. All it takes is for you to contact us!
It will be an honor for us to help you break new ground and deliver top quality with your game project.
