Release judgment criteria・Check the code quality to check the product quality

05/02/2018Release Judgment

Source code quality is paramount in determining product quality

By the way, at the time of release judgment, the first item for judging the quality of the product is the design quality , and the second item is the quality of the source code . In this article, I will show you how to judge the quality of source code (sometimes abbreviated as code), and how Gutara’s father came.

By the way, it’s a little off topic, but the most important thing for software development is the quality of the source code . Design review, code review, and testing are all about improving the quality of your source code. Review and eliminate mistakes to make a good design and good source code. Bugs that can’t be removed in the review and get into the source code are tested, detected and removed by debugging, all of which is to get good source code .　

All software development work done by us software engineers and software quality engineers is ultimately integrated into the source code . It can be said that software quality = source code quality .

Strictly speaking, the source code is compiled and becomes the executable code before it is executed on the CPU, so if there is a bug in the compiler or linker, the bug may be mixed into the executable code even if the source code is of good quality. This compiler and linker bug is very difficult to find the cause and is annoying. However, unless it is a special development such as adopting a CPU with a new architecture that has just been developed for the first time , fortunately recent compilers and linkers are getting better, so don’t worry too much about such a situation. I think it’s okay.

Source code quality is estimated by the number of bugs

As with design quality , it is easy to judge whether the quality of the source code is good or bad by judging whether the source code has many or few mistakes . It’s a bit rough, but it’s safe to assume that source code mistakes = bugs, and that the quality of the source code is good if the number of bugs found in tests and code reviews is small .

Have you found any bugs in your in-house testing?

Many bugs are found in various in- house tests conducted before the release . By the way, does your company find all the bugs in the software you release in your in-house tests ? Actually, the number of bugs that have not been found (= latent bugs) cannot be counted, so no one knows whether all the bugs have been found or if there are still bugs left. So, the reality is that we release the software, presuming that we’ve already found most of the bugs , because we’ve done so much testing and can’t find any bugs anymore .

So, if you have found a sufficient number of bugs in your internal tests, how many bugs should you find? This is also difficult. However, unless we determine that we have found most of the bugs in our internal tests, we cannot determine whether we can release the software. Here, let’s introduce the method of determining the number of bugs that Father Gutara did at the time of release determination.

Bugs found in tests and reviews

By the way, when are bugs found? If you think in the order of creating the source code, you will often find bugs in the following three situations.

  • Source code unit test
  • Source code review
  • Integration and system testing

A brief definition of the test used here

In fact, unit testing Toka, which came out on bond test Toka system test of Toka, defined as the name of the test is, there is quite a difference by the organization to develop the software. Here, the names of these three tests are used with the following meanings. Please refer to the article by replacing it with the name you use every day.

Unit test: A test in which the engineer who created the source code uses the white box method to check whether each line of the source code is working as expected or whether the created function is working as expected. .. For example, check the operation by step execution using a debugger.

Integration test: A test in which the engineer who created the source code uses a black box method to check whether the implemented function provides the function and performance that he or she expected . For example, a test that prepares a stub function and a file of pseudo input data and confirms the operation of the function.

System test: Software such as function / performance / stability / abnormal recovery / robustness, which is carried out by a test engineer who is different from the one who created the source code, by designing the test based on the software specifications. A test that confirms various aspects of.

Which test should you count for bugs?

Bugs can be found in unit tests, source code reviews, integration tests and system tests. Then, is it better to use the number of bugs found in these three tasks as information when judging the release? In fact, Gutara didn’t use all three of the bugs he found in internal testing.

Don’t count the number of bugs found in unit tests

Finding Bugs You can find quite a lot of bugs in the unit test, which is the first of the three situations . Then, is it better to use the number of bugs found in the unit test at the time of release judgment as information to judge the quality of the product?

Father Gutara didn’t use unit test bugs at the time of release decision . Unit tests are often run alongside the coding work that creates the source code, and can be considered part of the coding work. Some engineers who create source code take the style of writing the source code roughly first and then using the error detection function of the compiler to correct coding mistakes .

With this development style, there is less correlation between the number of bugs found in unit tests and the quality of the source code . This is often the case, so Dad Gutara didn’t use the number of bugs found in unit tests when making a release decision.

I don’t even count the number of bugs I found in code reviews

Regarding the number of bugs I found in the second code review , Father Gutara did not use the number of bugs at the time of release judgment . However, the reason is that unlike the bugs I found in unit tests, it was a hassle to count.

Depending on how the code review is conducted, if you try to strictly operate the rule that all newly developed and modified / deleted source code will be code reviewed, the meeting format and materials will be used as the code review method. If you do not use both circulation formats and perform code reviews efficiently, development will not proceed in a timely manner.

Also, how to record and trace the points pointed out in the code review and the countermeasures for the pointed out points is quite difficult without a software development environment with good work efficiency .　In the case of embedded systems , the development environment is often not unified, such as Windows applications and server applications, and the software development environment that can be used varies depending on the hardware used. Therefore, it may not be possible to create a system to efficiently record / aggregate the number of indications in the source code review.

Against this background, Father Gutara did not use the number of code reviews pointed out as information when making a release decision . However, if the development environment can be unified and the number of code reviews pointed out can be easily collected, I think it is better to use it as judgment information at the time of release judgment. In that case, as in the case of design quality in the first part, it is better to collect information on the quantity (code review implementation rate) and quality (code review participants) of the code review and see it together. If you do not review the code review, there will be no indications, and if there are no people with the skills to point out even if you do the reviews, the necessary indications will not come out.

But, well, if you look at the bugs found in integration tests and system tests , you can judge whether the product quality is good or bad to some extent, so even if you do not know the number of bugs found in the code review, it is not so difficult to judge the release. is not.

Release judgment based on bugs found in integration test and system test

So, in the software release judgment, Gutara’s father judged the quality of the source code based on the information about the bug (= source code error) found in the integration test and the system test .　

It may seem a little overlooked, but it is important to consider the cost performance of the judgment work even for release judgment, so it is necessary to take the plunge and discard data with a small amount of information for a lot of time and effort. increase.　

In addition to the number of bugs found in the test, we will revise the quality of the source code in other articles, but the quantity of the test, the quality of the test, the status of the bugs found, the estimated number of potential bugs, etc. I also use it. Since the quality of the source code is judged comprehensively from various points of view, it is safe to look at the number of integration tests and system tests as the number of bugs found in the in-house test . It was a closeout.

Annoying relationship between testing and bugs

By the way, is it better to find a lot of bugs in internal tests ? Or is it better not to find many bugs ?　　

Of course, if your testing objective is to find and debug bugs and you’re testing in the early stages of development, it’s better to find a lot of bugs. What I want to think about here is a pre-release test whose purpose is to make sure that there are no bugs left .

Since the purpose is to confirm that there are no bugs left, it is naturally better to find a small number of bugs. So, if you find zero bugs, can you unconditionally determine that the software is of good quality and can be released?　

Would you like to get caught in something? Yes,　there are no bugs without testing　.

Regarding the relationship between test bugs, it is a normal story that if you roughly judge the quality by the number of bugs, you can not judge correctly unless you also consider the quantity and quality of the test . The quality of the source code is not good just because the number of bugs is small , the number of bugs that can be found is small if the number of tests is small , and how many tests are done if the test method is a colander . But you won’t find many bugs. This is a little long to write, so I’ve organized it in a separate article , so please have a look there.

Next to product quality is process quality confirmation

Process quality is a companion idea to product quality. Ultimately, it is the quality of the product, that is, the quality of the product, that must be guaranteed. However, in order to improve the quality of a product, the quality of the process of making the product must be improved. With that said, it is necessary to check the status of the quality of the development process when determining the release of software, so let’s introduce it in the next article.

Next : Release judgment criteria・Confirmation of process quality is based on development practice and development management
Back : Release judgment criteria ・To check the product quality, first check the design quality
Return : Release judgment is the most important matter to guarantee the quality of software

Release Judgment

Posted by グータラ親父