Test type・Configuration/compatibility/installation/reliability
System test category replacement and selection
In this article, Father Gutara’s thinking about the replacement and selection required to use the 15 system test categories listed in “The Art of Software Testing" as test categories for embedded software following the previous article. I’d like to introduce_______ This article introduces configuration testing , compatibility testing , installation testing , and reliability testing .
Configuration test
In Japanese, it is a configuration test . In the case of a large computer system, various system configurations can be taken by combining hardware and middleware software, so the purpose of this configuration test is to confirm that any configuration described in the specifications works. In the case of embedded systems, the configuration of the equipment is almost fixed, so the need for configuration testing is not that great. However, recent group systems do not provide services independently, but are increasingly providing services by connecting to various lower-level devices and higher-level clouds .
This way it becomes the important in the case, such is, with the lower unit and the upper cloud to connect interoperability ( interoperability is a confirmation of). For example, there are more and more situations where the original service is provided by connecting an external hard disk via USB as a lower device or communicating with a specific server as a higher cloud. In such a case, if you do not thoroughly test the interconnection with the lower device to be connected and the upper cloud, the actual operation will not work.
Therefore, Father Gutara used this Configuration test as a system test category, replacing it with a test for interoperability with lower-level devices and higher-level clouds .
Compatibility test
The compatibility test confirms compatibility . Originally, in the case of a large computer system, it was considered as a test to confirm that it operates in the same way as before when the version of the middleware software or application software that composes the system changes. Since a large computer system is equipped with a lot of software, the version of each software often changes, and the compatibility test is to confirm the operation in that case.
In the case of embedded software, the version of the middleware installed inside does not change very often, so checking compatibility for the original purpose is rather less important. But for embedded systems, another importance of compatibility comes into play. It is the compatibility of the embedded device itself .
The embedded device itself may be replaced by a newer version of the device for a variety of reasons. For example, because the main parts that was selling a device called AAA communication equipment has become obsolete, that the New-AAA communication device using those of another manufacturer in the main parts new equipment to develop a successor model as Imagine selling it. In this case, it is necessary to clarify what level of compatibility the New-AAA communication device has with the original AAA communication device. There is no clear definition for the level of compatibility, but Father Gutara divided it into three categories: full compatibility , upward compatibility, and incompatibility.
Full compatibility is defined as being completely the same as an AAA communication device , with various settings , basic performance, and operation when an error occurs . In this case, the customer uses the AAA communication device for New-AAA communication. There is no problem even if you simply replace it with a device and operate it. In the case of full compatibility, the purpose of the compatibility test is to confirm that it is the same as the AAA communication device of the previous model, including performance such as responsiveness and handling in case of error.
Upward compatibility is, but he same thing is mounted as a function, a little difference in how to set there or is improved performance not in front of the model or not function had increased is the definition of a case to benefit. In this case, the customer needs to change or confirm the settings on the operation side before replacing the AAA communication device with the New-AAA communication device . If the setting method has changed, you may have to change the setting manual according to the new setting method, and if the performance has improved, the timeout set with the AAA communication device will occur . You may need to review the values for your New-AAA communication device. In the case of upward compatibility, what has changed and what has not changed is very important, so the purpose of compatibility testing is to check what has changed and what has not changed.
If there is no compatibility, compatibility testing is not necessary in the first place, and system testing of other categories as a new New-AAA communication device should be done firmly. So, Compatibility testing is needed as a category of system testing.
Installation test
The installation test is a test for the operation of installing software on the system with a large computer. It is necessary to install software in various situations such as when upgrading existing software or adding new software, but confirm that those installation work works properly with various system configurations. The installation test is done.
In the case of embedded systems, the necessary software is pre-written inside the equipment and devices when they are manufactured at the factory, so installation work is rarely performed at the operation site. So my dad Gutara didn’t use the test category of the Installation test.
Reliability test
Reliability testing is a test category that requires a little attention . In the first place, most tests including system tests are conducted to confirm the reliability of software, so if you create a test category for the purpose of confirming reliability again, you can see the difference from other tests. It can be lost and confuse the test design . The purpose of this test category, Reliability Testing, is to ensure that when there is another requirement for reliability , that requirement is met.
So what are the special requirements? If reliability is clearly included in the required specifications, for example, “it must operate 24 hours a day, 365 days a year " , this is a special requirement for reliability. To achieve this, at least duplication at various design levels is required. A dual system is designed to continue operation using another system even if hardware, software, or communication failure occurs at various levels such as power supply system, backplane, rack, board, software system, and communication path. It should be. By confirming that this dual system is properly constructed and operating as designed, it is necessary to confirm whether the special reliability requirements are met by the Reliability test.
Even if it is not being sought high reliability that far, for example, one downtime system down at 1 year more than once, as shake 10 minutes or less, specifically reliability requirements defined is If so, this is a special requirement for reliability. From a software point of view, the stability is confirmed by various acceleration tests so that hangs and resets do not occur more than once a year , and even if hangs occur, the watch dock function detects this and restarts. It is necessary to perform a test that guarantees reliability , such as recovering with software or returning to a state where service can be provided within 10 minutes when a restart occurs . Reliability testing is a category of system testing that confirms these special reliability requirements.
Reliability testing is a necessary test as a category of system testing, with the condition that it be considered only when there are special reliability requirements.