Game testing

Game testing, a subset of game development, is a software testing process for quality control of video games.[1][2][3] The primary function of game testing is the discovery and documentation of software defects (aka bugs). Interactive entertainment software testing is a highly technical field requiring computing expertise, analytic competence, critical evaluation skills, and endurance.[4][5] In recent years the field of game testing has come under fire for being excessively strenuous and unrewarding, both financially and emotionally.[6]

History

In the early days of computer and video games, the developer was in charge of all the testing. No more than one or two testers were required due to the limited scope of the games. In some cases, the programmers could handle all the testing.

As games become more complex, a larger pool of QA resources, called "Quality Assessment" or "Quality Assurance" is necessary. Most publishers employ a large QA staff for testing various games from different developers. Despite the large QA infrastructure most publishers have, many developers retain a small group of testers to provide on-the-spot QA.

Now most game developers rely on their highly technical and game savvy testers to find glitches and 'bugs' in either the programming code or graphic layers. Game testers usually have a background playing a variety of different games on a multitude of platforms. They must be able to notate and reference any problems they find in detailed reports, meet deadlines with assignments and have the skill level to complete the game titles on their most difficult settings. Most of the time the position of game tester is a highly stressful and competitive position with little pay yet is highly sought after for it serves as a doorway into a rapidly growing industry.[7]

A common misconception is that all game testers enjoy alpha or beta version of the game and report occasionally found bugs.[5] In contrast, game testing is highly focused on finding bugs using established and often tedious methodologies before alpha version.

Overview

Quality assurance is a critical component in game development, though the video game industry does not have a standard methodology. Instead developers and publishers have their own methods. Small developers do not generally have QA staff; however, large companies may employ QA teams full-time. High-profile commercial games are professionally and efficiently tested by publisher QA department.[8]

Testing starts as soon as first code is written and increases as the game progresses towards completion.[9][10] The main QA team will monitor the game from its first submission to QA until as late as post-production.[10] Early in the game development process the testing team is small and focuses on daily feedback for new code. As the game approaches alpha stage, more team members are employed and test plans are written. Sometimes features that are not bugs are reported as bugs and sometimes programming team fails to fix issues first time around.[11] A good bug-reporting system may help the programmers work efficiently. As the projects enters beta stage, the testing team will have clear assignments for each day. Tester feedback may determine final decisions of exclusion or inclusion of final features. Introducing testers with fresh perspectives may help identify new bugs.[10][12] At this point the lead tester communicates with the producer and department heads daily.[13] If the developer has an external publisher, then coordination with publisher's QA team starts. For console games, a build for the console company QA team is sent. Beta testing may involve volunteers, for example, if the game is multiplayer.[12]

Testers receive scheduled uniquely identifiable game builds[12] from the developers. The game is play-tested and testers note any uncovered errors. These may range from bugs to art glitches to logic errors and level bugs. Testing requires creative gameplay to discover often subtle bugs. Some bugs are easy to document, but many require detailed description so a developer can replicate or find the bug. Testers implement concurrency control to avoid logging bugs multiple times. Many video game companies separate technical requirement testing from functionality testing altogether since a different testing skillset is required.[5]

If a video game development enters crunch time before a deadline, the game-test team is required to test late-added features and content without delay. During this period staff from other departments may contribute to the testing—especially in multiplayer games.

Most companies rank bugs according to an estimate of their severity:[14]

Game tester

A game tester is a member of a development team who performs game testing.

Roles

The organization of staff differs between organizations; a typical company may employ the following roles associated with testing disciplines:

Employment

Game QA is less technical than general software QA. Game testers most often require experience however occasionally only a high school diploma and no technical expertise will suffice. Game testing is normally a full-time job for experienced testers;[18] however, many employees are hired as temporary staff,[2][19] such as beta testers. In some cases, testers employed by a publisher may be sent to work at the developer's site. The most aggressive recruiting season is late summer/early autumn, as this is the start of the crunch period for games to be finished and shipped in time for the holiday season.

Some games studios are starting to take a more technical approach to game QA that is more inline with traditional software testing. Technical Test positions are still fairly rare throughout the industry but these jobs are often full-time positions with long term career paths and require a 4-year computer science degree and significant experience with test automation.

Some testers use the job as a stepping stone in the game industry.[3][20] QA résumés, which display non-technical skill sets, tend towards management, than to marketing or production. Applicants for programming, art, or design positions need to demonstrate technical skills in these areas.[21]

Compensation

Game testing personnel are usually paid hourly (around US$10–12 an hour). Testing management is usually more lucrative, and requires experience and often a college education. An annual survey found that testers earn an average of $39k annually. Testers with less than three years' experience earn an average of US$25k while testers with over three years' experience earn US$43k. Testing leads, with over six years' experience, earn on an average of US$71k a year.[22] Typically, they will make $35–45k with less experience. Some employers offer bonuses for the number of bugs found.

Process

A typical bug report progression of testing process is seen below:

Methodology

Main article: Software testing

There is no standard method for game testing, and most methodologies are developed by individual video game developers and publishers. Methodologies are continuously refined and may differ for different types of games (for example, the methodology for testing an MMORPG will be different from testing a casual game). Many methods, such as unit testing, are borrowed directly from general software testing techniques. Outlined below are the most important methodologies, specific to video games.

The requirements are proprietary documents released to developers and publishers under confidentiality agreements. They are not available for the general public to review, although familiarity with these standards is considered a valuable skill to have as a tester.
Compliance may also refer to regulatory bodies such as the ESRB and PEGI, if the game targets a particular content rating. Testers must report objectionable content that may be inappropriate for the desired rating. Similar to licensing, games that do not receive the desired rating must be re-edited, retested, and resubmitted at additional cost.
Compatibility testing ensures that the game runs on different configurations of hardware and software. The hardware encompasses brands of different manufacturers and assorted input peripherals such as gamepads and joysticks.
The testers also evaluate performance and results are used for game's advertised minimum system requirements. Compatibility or performance issues may be either fixed by the developer or, in case of legacy hardware and software, support may be dropped.

Test automation

Implementing test automation for mobile application development process is the best way to gain agile, effective use of resources and time for testing.

Frameworks

The most popular test automation frameworks for Android testing:

The most popular test automation frameworks for iOS testing:

Console hardware

For consoles, the majority of testing is not performed on a normal system or consumer unit. Special test equipment is provided to developers and publishers. The most significant tools are the test or debug kits, and the dev kits. The main difference between from consumer units is the ability to load games from a burned disc or from a hard drive, as well as being able to set the console for any publishing region. This allows game developers to produce copies for testing. This functionality is not present in consumer units to combat software piracy and grey-market imports.

See also

Notes

  1. ↑ Bates 2004, pp. 176-180
  2. 1 2 3 4 Moore, Novak 2010, p. 95
  3. 1 2 Oxland 2004, p. 301-302
  4. ↑ Bates 2004, pp. 178, 180
  5. 1 2 3 Oxland 2004, p. 301
  6. ↑ "The Tough Life of a Games Tester" from IGN
  7. ↑ Plunkett, John. "Scary Maze Game", Didi Games, London, 27 October 2005. Retrieved on 27 October 2014.
  8. 1 2 3 4 5 Bethke 2003, p. 52
  9. ↑ Bates 2004, p. 176
  10. 1 2 3 4 5 6 Bethke 2003, p. 53
  11. 1 2 3 4 Bates 2004, p. 177
  12. 1 2 3 4 5 6 7 Bates 2004, p. 178
  13. 1 2 3 4 Bates 2004, p. 179
  14. 1 2 Bates 2004, pp. 178-179
  15. ↑ Moore, Novak 2010, p. 72
  16. ↑ Bob Johnstone. "Didi Games". Research on Video Games. Didi Games. Retrieved 2009-04-01.
  17. ↑ Bates 2004, p. 180
  18. ↑ Moore, Novak 2010, p. 25
  19. ↑ Moore, Novak 2010, p. 2
  20. ↑ Bates 2004, p. 261
  21. ↑ Moore, Novak 2010, pp. 84, 237-238
  22. ↑ Fleming, Jeffrey (April 2008). "7th Annual Salary Survey". Game Developer. United Business Media. 15 (4): 8.
  23. ↑ Adams, Rollings 2003, p. 17

References

Research

External links

This article is issued from Wikipedia - version of the 11/2/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.