Soft audit Checklist・No16: Test technology
At the end of the test checklist is the test content
In this article, we will introduce each item of the audit checklist used for software development audit . The audit checklist is divided into ( 1) development process, (2) requirements management, (3) testing, ( 4) design and implementation, and in this article, ( 3 ) individual check items for checking the test contents in the test are introduced. (The checklist itself can be found in the article on Software Audit Practice / Checklist 14: Development Technology / Testing (Test Management / First Half) , so please refer to that.)
The test content check confirms the test design ability
Be careful when using the checklist for test content. What I want to check using this checklist is not the quality of the test, which is whether the test content is sufficient , but the good or bad of the test design ability, which is the design ability to design the test .
The purpose of the software development audit is to judge whether the software development ability is good or bad, and as part of that, we also check the tests using the checklist. Therefore, what you want to check in this checklist is the test design ability of the partner organization. However, there is not a good way to measure test design ability, so bring out specific test items, how the other organization handles such test items, and whether they have already been implemented. We will judge the test design ability based on whether or not we feel the need for implementation.
The contents of each checklist are more specific than the items of other checklists, so if you are not aware of it, you will need to confirm whether you are implementing the test contents described in the checklist. It may end up. Instead, be careful to focus on what your organization thinks about the test items as described here. Let’s introduce the items in the checklist in order.
[Item number: TC-01]
Unit tests’s the very first test that code creator do but, Te method of purpose and test the strike definition of unit testing, etc., by the development organizations and software engineers you will be quite variation . Whether or not you are aware of the problem of variation in the definition of such unit tests, whether the definition of the unit test is clarified, whether there is a mechanism for all engineers to understand the definition correctly, etc. You can see it by checking. Paying attention to such a viewpoint, we will listen to the contents of the unit test.
[Item number: TC-02]
If you go into more specific methods of unit testing, you will need to select the necessity as a general test technique, such as a method for identifying test data and a method for evaluating the completeness of pass tests. .. Which one to choose is not the correct answer, but knowing such a specific test technique and incorporating it into the actual test is one guideline when judging the design ability of the test. Will also be. We will pay attention to such a viewpoint and confirm it.
[Item number: TC-03]
Memory leaks are not identified as test items in test designs aimed at confirming the original functions and performance. Only when a test design was made for the purpose of confirming stability, memory leaks were identified as a cause that could hinder stable operation for a long time, and test items for confirming the presence or absence of memory leaks were identified. Will be designed. Therefore, if there is a memory leak test item and the leak test method is specifically described there, it is judged that there is a test design skill not only in function and performance but also in stability. You can do things. We will pay attention to such a viewpoint and confirm it.
[Item number: TC-04]
Since memory leaks are well-known, there are many cases where this test item is included in the test. However, it is not just memory that leaks, but what is commonly referred to as dynamic resources . A certain amount is prepared by the system, and when it is needed, it is used as much as needed and returned when it is used up. If this return process is not executed due to a software bug, the resource will leak .
Such dynamic resources include sockets and message buffers as resources provided by the OS . Also, if the application has a table inside and has a mechanism to dynamically register and delete data in that table , there is a risk of leakage because these tables are also dynamic resources. In this way, we estimate that leaks will occur even in dynamic resources other than memory, and check whether we have designed test items to check for those leaks.
[Item number: TC-05]
Like leaks, timer and counter roundup issues are also well known as software bugs. Timers and counters are used in various parts of the software, but if the processing at Roundup is not properly created, the operation will be strange at the timing of Landup. This bug does not occur until the timer or counter actually lands up, so you can only find the bug with a test item aimed at checking the roundup processing of the timer or counter . I am aware of the need for such test items, and I will pay attention to whether I can actually create test items.
[Item number: TC-06]
Among the timer problems, there are problems called 49-day problems and 497-day problems with unique names. The system clock Roundup problem that the OS has has come to be called by such a name because it occurs in 49 days and 497 days. So, it overlaps with the confirmation items of TC-05, but even if it does not correspond to the general timer roundup problem of TC-05, I know the famous 49 days problem and 497 days problem. Some organizations support it. As with the timer and counter roundup problems, check whether you are aware of the keywords 49 days and 497 days, and whether you are taking measures against them.
[Item number: TC-07]
Leaks and timer / counter Landup issues are potential bugs that are difficult to find in tests because it takes a long time for a bug to occur as a bug. Another hard-to-find problem in testing is the memory corruption bug . This is reproducible because it often occurs when very rare conditions are met due to processing timing or duplication, regardless of the time the software is running , and the bugs that actually occur are not constant. It is a low bug that is difficult to reproduce by testing and to investigate the cause .
For problems that are difficult to see in actual machine tests, it is often more efficient to find and fix bugs by means of desk examination such as source code review . We will check carefully whether we are considering finding potential bugs that are difficult to find in such tests on actual machines by using a desk review method such as code review .
[Item number: TC-08]
For other resources such as memory sockets, dynamic resource leaks and double release are two sides of the same coin . If you forget to release it, a leak will occur, and if you release it once and then release it again, double release will occur. Leak, double and double release if the process of releasing resources is forgotten. This means that if you find the leak bug = forgetting the resource release process and add the resource release process, you may create another bug that causes double release under certain conditions.
To avoid this, when fixing a leak or double release bug, it is important to carefully check that the opposite symptom , the double release or leak potential bug, is not embedded . We will check whether such things are shared by engineers as a point of caution in the review.
Next, I will introduce the design and implementation of development technology.
That’s it for the test checklist. So far, we have briefly introduced the contents and precautions to be checked for the required specifications at the entrance of software development and the checklist for testing the exit. Next to the entrance and exit is the design and implementation that connects the two . In the next article, we will introduce a checklist focusing on the design and implementation that corresponds to the phase of actually creating software in the development technology.