Test example・Add semi-normal and abnormal test to the startup

14/09/202121/10/2021Test qualityy

For embedded systems, it is also important to perform a startup test to check the stability at startup.

For embedded software , the stability of the startup process is very important as well as the stability under normal operating conditions . Embedded devices often continue to operate in places that are not visible to humans without being operated by the operator. Therefore, the startup process also works without being seen by human eyes .

When the power of the device is turned on, when the firmware is downloaded and switched to a new version, or when the restart process is executed by the instruction from the remote monitoring center to deal with the problem, the startup process is performed in various situations. increase. For embedded products that do not have an operator nearby, the stability of the startup process is important because if the startup process fails, the only way to recover is to replace the device and repair it . A boot test to confirm its stability is very important.

It is important to test the semi-normal test and the abnormal test  in the startup process.

In the startup process, it is important to check the normal test that starts up without any problems , but more than that, the semi-normal test (startup process when an expected error occurs) or abnormal test (when an unexpected error occurs). It is important to confirm the startup process). After the startup process is completed normally and the normal operation mode is entered, even if there is a problem with the device, you can use the remote maintenance function to check the device status and perform recovery operations from the monitoring center. However, if the startup process stops in the middle, such remote maintenance cannot be performed. This is why stability is important for the startup process.

In many cases , communication processing with the server and access to the built- in flash memory are centrally executed in the startup processing . As communication processing, device settings are often downloaded from the server and the latest firmware is checked , and in flash memory access, the program required for operation is read from the flash memory and expanded to DRAM. It is performed at the beginning of the process, and the process of writing the settings downloaded from the server to the flash memory is often included in the startup process.

Communication processing with this server and flash memory access are processing with a high probability of error occurring even in embedded devices, such as communication errors and flash memory access errors . Therefore, error handling that assumes various errors is built into these processes. It is a semi-normal test to confirm that these error processing is operating correctly, and it is abnormal to confirm that the processing does not hang even in a situation that is not expected by those error processing. This is a system test . Therefore, let’s make a test plan paying attention to the smi-normal test and the abnormal tet in the test of the startup process.

Please note that it is different from the normal operation mode during the group startup process.

There is one more thing to keep in mind when considering testing the startup process . That is, it is different from the normal operation mode during the startup process . It is not limited to embedded software, but it is necessary to set and initialize various hardware immediately after the power of the device is turned on, and it is normal to provide normal functions only after all of them are completed normally. The operating mode of is started. The period until this normal operation mode starts is called the startup mode . (Although the name varies depending on the development product and development organization)

Since various functions provided by the embedded software operate in the normal operation mode, all functional tests are premised on this normal operation mode . In other words, tests in boot mode, which is not normal operation mode, are not performed in other test items . Therefore, it is necessary to pay particular attention to this boot mode when considering the test in the boot process. One thing to keep in mind when considering testing in boot mode is the boot sequence . From the moment the power is turned on, the boot process , hardware initialization process , terminal certification process, etc. are processed in sequence according to the startup sequence in the embedded software . In the boot test, it is also important to confirm that each of these boot sequences is working properly.

Let’s check the movement of flash memory and server communication in the startup process

Access to flash memory is a part where potential bugs are likely to be hidden due to various factors, although Flashrom also wrote in the article on Bug’s Nest . Then, in the boot process, access to the flash memory is concentrated . In the first boot process, the executable program stored in the flash memory is read and written to the DRAM. In addition, various setting values ​​of the device are written to the flash memory . At the same time, if the latest version of the firmware is found on the server, it will be downloaded and written to flash memory .

Since these processes are executed in the boot process, more flash memory access will occur compared to the normal operation mode. In the startup test, pay attention to the access to these flash memories, and perform a quasi-normal test when the access to the flash memory fails .

In addition, communication with the server frequently occurs during the startup process . Communication with the server for various confirmations and data exchange, such as downloading terminal settings, checking the contracted service contents, checking for the existence of a new version of software, etc., is part of the startup procedure. It will be carried out. Communication processing is also processing that can cause many errors, such as packet loss on the communication path, timeout due to server response delay, and communication error . In the communication process, a large amount of quasi-normal system processing is incorporated to deal with these errors, so in order to confirm its operation, a quasi-normal system test of server communication is indispensable for the startup test. is.

Let’s also check the operation when an error occurs during the startup process

Don’t forget to check the behavior after the error occurs in the startup test . There are several ways to deal with errors found in the startup process, but they can be roughly divided into the following four types.

1. Reboot with the expectation that the error will not occur (It is a policy to expect that the error will disappear by restarting)
2. Start in degraded mode with some functions limited due to errors (It is a policy to provide only functions without errors)
3. Record the error and start in maintenance mode (the policy is to wait for investigation and recovery processing from the remote maintenance center)
4. Give up on starting, record the error log, only display the error and stop the starting process (It is a policy to give up gracefully)

By these alone or in combination, the policy of dealing with errors in the startup process should be decided, so when an error occurs, it is confirmed by the startup test that it is operating according to those policies. It will be necessary to confirm.