Test type・Function/Large amount of data/High load/Usability/Security/Performance/Storage area
System test category replacement and selection
In this article, I will introduce Father Gutara’s thinking about the replacement and selection required to use the 15 types of system test categories listed in “The Art of Software Testing" as test categories for embedded software. This article introduces functional testing , high volume data testing , high load testing , usability testing , security testing , performance testing , and storage area testing .
Facility test
Riding in the specifications and instruction manuals function is the most to make sure that is installed because it is also a basic test categories, use and Chi basic test items as a test category of embedded software. Sometimes called a Function test, but in “The Art of Software Testing", the Function test is used to name a single-function test that is performed before the system test, so the name Facility test is used so as not to be confused. I am. So my dad Gutara also used the name Facility test as a category of system tests. Since it is a confirmation that the functions are installed, we carry out a normal system test to confirm that the functions and services described in the specifications and instruction manuals are provided without problems when normal operation is performed. increase.
Volume test
The Volume test checks whether a large amount of data can be processed, but Gutara’s father did not use it as a system test category for embedded software. In the case of server software, a category called Volume test is indispensable to check whether it can handle a large amount of input data. Even in the case of embedded software, it is possible to test from the perspective of whether it can be processed when large amounts of data are input via a communication path. However, in embedded systems, large data is often input continuously over a long period of time. And the confirmation of operation for continuous input data is often included in the test performed in the confirmation of normal function. As you can see, in embedded systems, the Volume test is often included as part of the Facility test, so my father, Multi-Gutara, did not use this category in the system test.
Stress test
In Japanese, the stress test is called the high load test or overload test. While the Volume test confirms whether a large amount of data can be handled, the Stress test is a test for confirming the operation under peak load of data and processing . Taking document creation as an example, the Volume test confirms the ability to process a large number of pages of text, but the Stress test confirms that a large number of pages of a document can be processed in one minute.
For embedded software, which is often responsible for device control and communication, it is important to operate reliably even during peak loads, so this Stress test must be performed firmly. In embedded system tests, it is necessary to identify the types of loads and then check the operation when the maximum load is applied to each load. Since these test cases must be designed and implemented in a test category called Stress test, it is also essential as a system test category.
Usability test
The Usability test , which confirms usability from the viewpoint of whether it is easy to use for the person who uses the device, is highly necessary even for embedded software that often has a Web GUI . In recent devices, browser-based GUIs are often adopted as a user interface for end users to check and set various settings. It is also important for embedded software to check the ease of use in various aspects, such as the ease of viewing the screen, the ease of entering setting values, and the ease of understanding the error display when an incorrect value is entered. It’s becoming important. Usability test is not necessary for devices that are embedded software and have no user interface at all, but since many devices have some kind of GUI or CUI these days, Usability test is a system test category for embedded software. Is required.
Security test
As I wrote in another article , security testing has become a very important test for embedded software these days. Embedded software rarely has a large amount of important data in it, so from the perspective of information leakage , embedded systems do not pose much security risk. However, in the current situation where the spread function that is always connected to the Internet in the IoT has increased it to be mounted, device hijacked and to others used in the attack base has been growing risk that Ru is. To ensure that you have the ability to respond to these cyberattacks, a test category called Security Testing is essential for system testing.
Performance test
The Performance test is a required test because it is the most common test item of the system test along with the Facility test that confirms the function . In particular, embedded software often has not only a human interface but also an interface with other computers and an interface with devices to be controlled and monitored .
Since the interface with these non-human opponents must be balanced with the response speed and performance of the opponent, it is very important to perform a Performance test to confirm that the performance and responsiveness are as specified by the beard. When checking the performance, it is important not to forget to check not only the processing amount per unit time (communication speed and traffic processing speed) but also the response performance .
In hard real-time (real-time systems that need to guarantee that processing is performed within a certain period of time) such as control systems, processing performance and responsiveness confirmation are especially important. For example, control software that controls anti-lock braking in automobiles must perform the necessary brake control with the necessary responsiveness in any situation. Since the Performance test confirms such a thing, this test category is indispensable as a system test for embedded software.
Storage test
The Storage test is a test to check whether the storage area of the storage device installed in the computer system can be controlled . Computers have primary storage (a storage area where the contents disappear when the power is turned off) and secondary storage (a storage area where the contents are maintained even when the power is turned off), both of which have a finite storage capacity. Since the storage capacity is finite, the software always has a built- in function to control the storage capacity used .
In many cases, personal computers are equipped with a storage device with a sufficient capacity, and since the OS controls the storage capacity, there is not much awareness about the control of the storage capacity. However, in embedded systems, it is necessary to efficiently use a storage device with a limited capacity to proceed with processing, so it is necessary to control various storage capacities, so confirmation is important.
For example, a simple Storage test for main memory , which is a typical primary storage device, is to confirm the remaining amount management function of dynamic memory . The dynamic memory remaining amount management function includes the remaining amount confirmation function, memory acquisition / release function, confirmation of the upper / lower limit of the remaining memory realized by combining these functions, and processing when the remaining amount is exceeded. It is implemented in software. The Storage test for dynamic memory on the main memory is to confirm that these are working properly. The memory leak test, which checks for dynamic memory leaks, is also one of the items under test for this dynamic memory.
In the case of embedded systems, flash memory is often installed as a secondary recording device . Some kind of file system is installed here to record the program itself or record an error log. Storage tests for flash memory include, for example , writing tests for large log files . The Storage test for flash memory can confirm the correct operation of the software when a file whose size exceeds the current free space of the flash memory is written.
The Storage test was originally intended to confirm the information recording function for the hard disk, which is the external storage device of a large computer, but in the embedded system , the focus is on the storage capacity control function of the temporary storage device and the secondary storage device. By narrowing down the consideration, it is treated as a test category required for system testing.