Test type・Abnormal recovery/maintainability/documentation/runbook
System test category replacement and selection
In this article, Father Gutara’s thinking about the replacement and selection required to use the 15 system test categories listed in “The Art of Software Testing" as test categories for embedded software following the previous article. I’d like to introduce_______ In this article, abnormal recovery testing , maintenance testing , document test , procedure manual test introduces you to.
Recovery test
The Recovery test confirms the ability to recover from an abnormality or error when it occurs . This anomaly recovery feature is essential for embedded systems, but Father Gutara did not use Recovery testing as a system testing category. The reason is that we decided to carry out as a quasi-normal system test and an abnormal system test in the facility test that confirms each function.
The Facility function is a test that identifies and comprehensively confirms the functions of the device. In the test design of each function, confirm that the function operates normally as a normal system test. Furthermore, the operation for errors and abnormal conditions assumed at the time of design is confirmed as a quasi-normal system test , and the operation when an abnormality that was not expected at the time of design occurs is confirmed by the abnormal system test . .. I didn’t use the test category called Recovery test because the quasi-normal and abnormal tests in the Facility test are testing the same thing as the Recovery test.
Serviceability test
Embedded systems are equipped with some maintenance function to investigate the cause of the problem that occurred during operation and to deal with the problem . The serviceability test confirms this maintenance function . The maintenance function includes a maintenance function that operates independently to record an error log, and a remote maintenance function that remotely checks the status from the maintenance center of the service provider and writes the countermeasure version software . .. What kind of maintenance functions are installed should be specified in the requirement specifications and requirement definition documents of the device, so check that those functions operate normally. This Serviceability test is also essential as a system test category for embedded software.
The most important maintenance function is the software version upgrade and restart function that uses the remote maintenance function . Even if there is a bug in the embedded system software, if you can upgrade to the software that fixes the bug, you can eliminate the bug in the equipment running on the market. Also, even if the software that fixes the bug is not ready yet, if the device can be restarted remotely, it is often possible to restore the service provision to the end user.
When an end user has some problem , the remote maintenance function that remotely restarts or writes a bug-fixed version of the software as an emergency workaround can somehow improve the situation as long as this function is working. Therefore, it is the last fort of quality assurance for embedded software. That is why the remote maintenance function requires a higher quality than various other functions . Therefore, when testing the Serviceability feature, pay special attention to its stability when creating a test plan.
Documentation test
Are shipped with the product ensure the quality of the document to the Documentation test it is. If it is a product for general households, the instruction manual will be packed together with the product and shipped, and if it is a device used in an operation center, the operation manual will be delivered together with the product. In any case, customers who use the product will use the product by looking at the documentation that came with them , and if the documentation is incorrect, they will not know how to use it. The Documentation test ensures that the documentation is written correctly to prevent this from happening.
In fact, at the end of software development, product display improvements often occur. When all the functions and performances are improved, it will be possible to improve the small expressions that were low priority. As a result, the device GUI and CUI guidance and error messages are changed. It is very good that the message is easier to understand, but if the document is created in a department other than the department where the software engineer is, the change of the message in the software will be reflected in the document. It may happen that you can not make it in time. It is also the role of the Documentation test to spot such document reflection delays.
System testing of embedded software tends to focus on testing to confirm the quality of the product itself because the software is installed in the product, but the quality of the documents shipped with the product is also important . Therefore, Documentation testing is also an important item in the system testing category.
Procedure test
In the case of an embedded system, it may be required to operate according to a predetermined series of procedures in the entire embedded system . For example, when updating the software by communicating with the server installed in the center, it is required to operate according to the following procedure.
(1) Notify the server of the current software version and confirm the necessity of version upgrade. (2) If it is determined that the version needs to be upgraded, a confirmation screen of “Do you want to upgrade?" Is displayed on the GUI screen that the end user can see. ③ If the end user selects “Yes", download the latest version of the software from the server. ④ After the download is complete, restart the device to enable the new software.
If the update procedure of such software has been decided, that to make sure that the work as described in the following procedure that the Procedure test is. When a procedure that involves interaction with other devices or end users is required, various things can occur, such as waiting for a long time between individual procedures or the wrong procedure being implemented. You need to make sure that the embedded system works properly, including. Since various procedures are created by combining the individual functions of the equipment, it is often a comprehensive test, but it is also an important item in the system test category.