Test example・Simultaneous execution test of multiple functions
- 1. In embedded systems, it is also important to detect bugs hidden in the simultaneous execution of multiple processes.
- 2. Simultaneous execution of multiple functions occurs at any time
- 3. Why bugs are likely to be latent in simultaneous execution of multiple processes
- 4. It is impossible to cover the test of simultaneous execution of multiple processes
- 5. How to choose a combination of multiple processes to test
- 6. Multiple timings are also required for simultaneous execution tests of multiple functions
- 7. Next to the simultaneous execution of multiple functions is a test of abnormal / quasi-normal system at startup.
In embedded systems, it is also important to detect bugs hidden in the simultaneous execution of multiple processes.
In order for embedded software to continue to operate stably, it is important to identify and fix as many potential bugs as possible before release . Therefore, after the dynamic memory and resource leak test, timer and counter rollover test, let’s perform simultaneous execution test of multiple functions .
Simultaneous execution of multiple functions occurs at any time
There are various functions implemented in embedded software. And for each function, the test named normal system / quasi-normal system / abnormal system has confirmed the operation and stability of the single function . Built-in software only to run one by one function if, I may only test this single function, in the actual embedded software multiple functions are performed concurrently there that may be Ru.
For example, many embedded software devices have an interface, such as a GUI, that starts processing in response to instructions from the operator . In the case of such a device, the normal internal function as embedded software often operates behind the function execution in response to the instruction from the GUI . Then, when the timing of the operator’s screen operation by the GUI and the timing of the execution of the internal function match , the processing to execute the two functions is executed in parallel in the embedded software.
In this way, when the execution timing of the automatically executed internal function and the function executed by the operator’s instruction match, there are many patterns in which the processes that execute the two functions are executed at the same time in the embedded software. In addition, some embedded software has a function (multi-user / multitasking function) that executes processing from multiple users in parallel at the same time . In this case, if two users instruct the execution of some function at the same timing, the processing to execute the two functions will be executed at the same time.
Why bugs are likely to be latent in simultaneous execution of multiple processes
Even if two functions are executed at the same time, there is no problem if each function finishes operating normally . However, it is often the case that latent bugs move when two functions are executed at the same time . The most common is writing down data by accessing the same data at the same time , and reading inconsistent data in the middle of change . Both occur when there is a bug in exclusive control when multiple functions or processes access the same data .
For example, suppose you have information about sales to Mr. A, who has two data structures , product name and quantity . Suppose function A writes the information of 3 apples entered from the GUI now , and function B receives the information of 1 orange entered by Mr. A yesterday on another page from the communication line and writes it. If exclusive control is properly performed, two types of information, one orange and three apples, will be recorded in the sales contents for Mr. A.
In not been able to exclusive control , the bad timing I think what happens when the function A and function B is executed. Immediately after function A writes apple, function B receives communication and activates interrupt processing, writes one mandarin orange, and then function A returns and writes three. .. Then, as a result, Mr. A’s sales will be strange with 3 oranges.
The bug hidden in such exclusive control becomes apparent only when two functions are executed at the timing of writing data to the data at the same time . No matter how much you test on its own, you won’t find any potential bugs that depend on the timing of simultaneous execution of multiple features like this. This is just an example, but finding such potential bugs requires concurrent testing of multiple features .
It is impossible to cover the test of simultaneous execution of multiple processes
It’s true that simultaneous execution tests of multiple processes are effective in identifying potential bugs, but when you actually design a test , you run into a big wall . It 's an explosion of test items, as it is commonly called, assuming that there are 10 types of functions, there are 100 types of simultaneous execution of two functions in 10 x 10. This 100 types is when only the typical function execution of the normal system is considered.
But when you try to plan the test thinking the scene of the execution of the actual function, the normal system in addition to the one semi-normal system three and the abnormal system degree one would like added. On top of that, considering the possibility that the latent bugs that appear in the simultaneous execution of multiple functions depend on the execution timing of each function, it is tempting to assume two types with different execution timings . This alone increases the execution conditions of one function to 10 types, so when it is multiplied, it becomes (10 X 10) X (10 X 10), which is already 10,000 test items .
This number of tests is a bit impractical, as you don’t just have to do multiple function concurrency tests. In the simultaneous execution test of multiple functions, we have to give up ensuring completeness .
How to choose a combination of multiple processes to test
If it becomes impossible to ensure 100% completeness, it is necessary to make a test plan limited to combinations that can effectively identify potential bugs as much as possible . In a multi-function simultaneous operation test, which function and which function simultaneous operation should be selected as test candidates? There is no answer that this is correct, but latent bugs found by simultaneous execution of multiple functions are often found at the timing of collision of some processing, so look for a combination of functions that is likely to cause processing conflict inside the software . I think that is good. Here are some of the ideas used by Father Gutara.
(1) Select a function that is likely to use the same lower-level processing for internal processing
If the same subordinate process is used, a conflict will occur in the execution of the subordinate process called from the two processes. Then you will often find potential bugs related to the required exclusive control as introduced in the previous example .
(2) Select a function that is accessing the same data
Although the lower level processing used is different, latent bugs may be found even with functions that are accessing the same data. Like the data search function and the data update function, different types of processing that handle the same data can be used as test items for multiple functions because they identify potential bugs in exclusive control .
(3) Select a function that uses a large amount of common hardware resources such as CPU, memory, and communication buffer .
Most embedded software uses dynamic resources such as dynamic memory, communication sockets, and communication buffers. The CPU processing power itself is also a dynamic resource. The idea is that if there is a function that consumes a large amount of any of these dynamic resources, it will be executed at the same time. In this case, it is possible to identify potential bugs in processing such as waiting for resources to become available due to temporary depletion of dynamic resources .
(4) Combination of simultaneous operations that are likely to occur in actual operation
It is also a necessary selection method to select a function that is likely to be discussed at the same time when actually using the device.
Multiple timings are also required for simultaneous execution tests of multiple functions
Since it is difficult to ensure completeness in the simultaneous execution test of multiple functions, we narrow down the combinations to be tested, but on the other hand, it is also important to execute at the timing when the two internal processes collide . It is very difficult to accurately control the timing of processing, including the internal processing of software, so it is necessary to devise a test that executes multiple functions at multiple timings that are slightly shifted . Since the number of test items will increase, it is difficult to balance it with the completeness of the test, but there is no point in worrying about it too finely, so I think that there is no choice but to decide with a certain degree.
Next to the simultaneous execution of multiple functions is a test of abnormal / quasi-normal system at startup.
The dynamic memory and dynamic resource leak test, timer and counter rollover test, and multi-function concurrency test are mainly tests to check the stability during normal operation . In the embedded software, a test to confirm the stability at startup is also required in addition to the normal operation, so in the next article, let’s introduce the abnormal system / quasi-normal system test at startup.