Soft Audit overview・Purpose 3: Audit to confirm the quality of bug-fixed software　

25/12/202021/10/2021Audit for soft-Develop..

The three purposes of software development audit are to judge development ability, improve development ability, and ensure software quality.

The three purposes of software development audits conducted on software development contractors: (1) Judging development competence (　2) Improving development competence (　3) Securing the quality of software for defect countermeasures　Among the above, (1) and (2) Since the outline was introduced in the article, this article will introduce the software development audit to ensure the quality of the defect countermeasure version software in (3). For a concrete introduction to the actual software development audit, please refer to the article titled “Software Development Audit / Practice" if you are interested in the details.

3.  I want to check the quality of the troubleshooting version software

The third purpose of conducting a software development audit is to check the quality of the defect countermeasure version of the software . When a bug occurs in the market, it is often the case that the bug countermeasure version software is released and permanent countermeasures are taken, but it is difficult if the bug countermeasure version software has a new bug. Therefore, before releasing the defect countermeasure version of the software, perform a software development audit to check whether sufficient quality is ensured.

Since the purpose is to confirm the quality of the defect countermeasure version software, the points to be emphasized in the audit are slightly different from the other two purposes, and the content of the defect correction and its influence range are confirmed and the test for them is performed. The main task is to check the situation. Specifically, we will check the following items.

• Is the mechanism by which the problem occurs clarified?
• Is the bug fix method valid?
• Has the effect of the bug fix been confirmed in the test?
• Is the range of influence of bug fixes properly determined?
• Has the test confirmed that there are no secondary defects due to bug fixes?
• Are the causes of defects mixed / leaked and measures to prevent recurrence?

Confirmation of these items is often carried out while checking the report on the defect created by the development contractor, the design document on the defect part, the source code of the part corresponding to the defect correction, the test result, etc. is. Therefore, in this software development audit, there are many situations where the software development audit checklist is not used . If there is a problem in the development process when checking the cause and recurrence prevention measures at the end of the check items, the checklist items related to the development process may be used, but the ratio of using the checklist is It will be quite low.

Let’s take a look at each item in turn.

Is the mechanism by which the problem occurs clarified?

When a bug occurs in the market, the first important thing is whether the mechanism by which the bug occurs is clearly understood and whether the correct bug fix is ​​implemented as a countermeasure.　

Some kind of bug A is latent in the source code, and when a certain condition B is satisfied, the source code including the bug A is executed, and as a result, the situation C occurs, and as a result, an error E occurs in the processing D. Therefore, a defect F occurs in the market. From bug A that causes a defect to defect F that occurred in the market, a logical explanation is given , and it is carefully confirmed by some kind of test or investigation that the mechanism of occurrence is not wrong. To do.

In the confirmation work, while looking at the report on the cause of the defect prepared by the development contractor, the design document of the relevant part, and the actual source code, make sure that there are no mistakes, misunderstandings, or logical jumps in the mechanism of occurrence. confirm.

However, in some cases, the mechanism of occurrence may not be understood . With limited time and personnel, the development department cannot reproduce the defects in the market, and the investigation does not proceed. Because it is a survey of, there may be a limit. In such a case, a problem has occurred when such conditions are met, so we may adopt workaround measures to prevent the conditions from being met . It is better to understand the mechanism of occurrence clearly, but in reality, there are times when you have to take workarounds. However, even at that time, it is necessary to confirm what is known and what is not known based on the facts.

Is the bug fix method valid?

Next to checking the mechanism that causes the problem is checking how to fix it. Here, pay attention to whether the buggy source code mistake is just a coding mistake or a design problem, and check whether the fix is ​​correct.　

If it’s just a coding mistake, you can rest assured that you’ll just correct the mistake. However, if a source code error is caused by a design problem, it is necessary to go back to the design and correct it. In this case, the act of fixing the bug is a design change , so it is necessary to check the range of influence and whether or not the correction will affect others.

While paying attention to the cause of the source code error found in the bug generation mechanism, we will check whether the correction method can surely solve the problem occurring in the market.

Has the effect of the bug fix been confirmed in the test?

There are two ways to confirm that there is no problem with the bug fixing method, a desk examination and an actual machine test, but the first thing to do is to examine it on the desk . It is a work equivalent to a design review that checks the design document and source code to see if there are any mistakes in the bug fixing method . After confirming that there is no problem in the desk examination, next, check the result of the test to verify that the bug fix actually works.　

Software engineers prepare test conditions and test environments that can reliably reproduce defects that occur in the market, and execute the defect-fixed version of the software in that test environment, and it is true that defects in the market do not occur. You should have confirmed that with a test. It should have been confirmed by the test, but the software development audit reconfirms the status of the test. The point to note at this time is whether you can confirm the effect that the defect has disappeared by using an environment that can reproduce the defect.

It sounds like a Zen question, but it’s quite important. If you are an inexperienced engineer, you may misunderstand that you have completed the correction confirmation test just by confirming that the source code that fixed the bug works as expected. In short, it’s a misconception that you can do unit tests to make sure that what you’ve modified is working as you think it is, and that’s okay .

If the mechanism of defect occurrence is correctly clarified, it is okay to check the unit test level, but unfortunately, it is the “person" who is the software engineer who thought about the content of the defect investigation and countermeasures.　People make mistakes and misunderstandings, so be careful here and test in an environment that reproduces market defects, and make sure that the result of the bug fix is ​​correct or not based on the test result. Let’s check.

Is the extent of the impact of the bug fix properly determined?

Bug fixes require some changes or additions to the source code. The modified source code is a part of the processing and functions to realize the existing functions, so other parts of the source code that use those processing and functions are the scope of the impact of this modification .

After identifying all the affected areas, does the change in the source code due to this bug fix have any effect on the processing contents of the affected areas that have been identified, and if so, is there any problem with the contents? If countermeasure processing is required, check whether the countermeasure processing can be implemented correctly.

The most important thing to pay attention to when fixing a bug is to check the range of influence . If you overlook it here, you will end up in a situation where you have released a version of the software that fixes the problem, but you have added a new problem to it and caused another problem in the market.

Has the test confirmed that there are no secondary defects due to bug fixes?

At this point, the goal is near. Finally, it is confirmed whether the result of the desk examination of the influence range is reconfirmed by the test. Has a confirmation test been conducted to confirm that the functions and processes that may be affected by this modification operate without problems as before the modification ? Are functions and processes outside the scope of this fix tested to work as they did before the fix? Check the test status for these two points.

The former is often carried out together with the confirmation of this correction, but the operation confirmation outside the influence range of the latter may be forgotten. In the desk examination, there should be no problem because it is out of the range of influence, but at least a light test that the basic functions work without problems is sufficient, so I would like to test and confirm. Some organizations call such tests sanity tests and position them as a series of basic functional tests that are always performed after bug fixes.

Are the causes of defects mixed / leaked and measures to prevent recurrence?

Once the confirmation of the secondary defect is completed, the quality confirmation of this defect-fixed version software is complete. You can end the software development audit here .

However, another aspect of the software development audit, want to improve the development competence of development contractors If the rattan purpose also incorporate a little, root causes and that failure had mixed into the source code, it has been conducting a problem before the release It may be a good idea to go one step further and analyze the root cause of the outflow that was not found in advance in the in-house test, and have the development contractor consider measures to prevent its recurrence.　

It is a little out of the purpose of checking the quality of the defect countermeasure version of the software, but if the development consignment continues to the software development consignee in the future , it will take a little more time and effort to perform the software development audit. Taking measures for the purpose of the eyes will be beneficial in the future.

Next, I will introduce it from the perspective of being subject to software development audits.

In this article, we have classified the purposes of software development audits into three categories and introduced the outline of audits for each purpose. How was it? Could you give us a little overview of Gutara’s father’s software development audit? After all, it’s a method that Gutara’s father made on his own, so I think there are still many parts that are difficult to understand. In the next article, I will continue to introduce the outline of software development auditing from the perspective of the audit target area .

Next : Soft Audit overview・Target 1: Audit the development process