Release judgment criteria・Estimate how many potential bugs remain

20/11/202020/10/2021Release Judgment

Software quality is determined by the number of potential bugs

So far, I’ve written about existing bugs I found in my tests . By the way, what is the most important thing when judging the quality of the source code that needs to be checked at the time of release judgment based on the number and contents of bugs ? I feel like I’ve forgotten something. .. .. Yes, how many potential bugs remain that have not yet been found in the test, rather than the bugs found in the test, to determine the quality of the software ? But, in fact, it is the most important thing.　

If there are no potential bugs left in the software, the quality is 100 points . But, even if how much design reviews and test Unfortunately, the potential to have bugs zero the fact that it can not be proven . Still or potential bugs that have not been discovered there is review what, determine to anyone not found because. No one can count the things you haven’t found as one or two.

Potential bugs have still been found, so there is no choice but to estimate

So what are you thinking about so far? I feel like that. In fact, the activities that have been considered so far all, still find and not latent is standing bug estimate how many stars there 's the collection of information for. No one really knows how many more potential bugs we haven’t found yet.

What we can do is collect as much information as possible about how many bugs we have already found, what tests we found them, what the quantity and quality of those tests are, and the information we have collected in various pasts. In comparison with experience, it is estimated that there are probably few potential bugs that have not been found yet because we have found enough bugs .　

Only then can you believe that there are few potential bugs left before you can decide that you can release the software .

I want a more convincing explanation for potential bugs

But with this alone, it’s just that there shouldn’t be any bugs left because I’ve done a lot of testing, so it’s not convincing . Well, it doesn’t matter because you don’t need that much persuasiveness when deciding on a software release .　

However, externally the the quality situation when describing Toka, in-house management layer quality situation to when describing the Toka is, say a little more brackets, something seems theoretically , when the potential bug I do not anymore almost left I want a way to say that.

And a convenient way to use it in such cases is the old bug curve . Bug growth curve Toka reliability growth curve Toka, Gonberutsu curve Toka logistic curve Toka, because you may be referred to by various names, I think that there is that everyone is also once heard.

It is often introduced as a tool used to judge the end of software testing ( judgment of whether it is okay to finish the test because it has already been tested sufficiently). This is an S-shaped curve, I think you’ve seen it. In this article, we will use the name Bug Growth Curve.

Gombelz curve / logistic curve

What is the bug growth curve?

In the graph above, the blue line is the Gombelz curve and the red line is the logistic curve. But with this alone , I don’t know how to use it to determine how many potential bugs remain . Before that, let’s sort out the relationship between words.

Gombelz curve or logistic curve is a name given to mean a curve drawn by Gombelz function or logistic function, which is a mathematical function that draws an S-shaped curve when drawn on a graph .

On the other hand, the biological populations and bug discovery number when you try to Te software such as, first, to slow the increase, and how, such as last also increase becomes less increased sharply increase in the middle. If you graph the increase status with time on the horizontal axis and cumulative amount on the vertical axis, you will get an S-shaped graph . This is called the growth curve .

More specifically, if you draw a graph with the horizontal axis as the amount of tests (number of tests, man-hours for test work, etc.) and the cumulative number of bugs found on the vertical axis , this is a bug. It is called a growth curve or reliability growth curve . The bug growth curve is sometimes called the bug curve for short. The word “reliability" means that the fewer bugs there are, the higher the reliability of the software, so think of the number of bugs as reliability and the general term .

And, experience shows that the bug growth curve of soft air often overlaps with the Gombelz curve and the logistic curve , according to the observation results of many software development projects. I don’t know the theory as to why this happens, but it is the wisdom of our ancestors that it would be safe to make such a judgment based on a lot of experience .

Therefore, it is common to draw a Gombelz curve or a logistic curve by superimposing it on the bug growth curve drawn based on the actual value . I’ll explain why you do that later.

To summarize, both the bug curve and the reliability growth curve are one type of growth curve . This growth curve often overlaps with the Gomberts and logistic curves. So, when you draw a bug curves and reliability growth curve based on the actual value,併to the actual value wrote there be Gonberutsu curve is drawn, and Gonberutsu curve as a whole at that time called there also. If the curve to be written together is a logistic curve, it may be called a logistic curve as a whole .　

I don’t understand …

Yes, I’m saying something like that, if you were an old man, you would be deceived by a fox . Let’s take a look at a more concrete example. This is the output screen of the software tool that draws the reliability growth curve used by my father Gutara .

Confidence growth curve generation tool

If you enter the daily test man-hours (man-days) and the number of detected bugs , the growth curve with the test man-hours on the horizontal axis and the cumulative number of bugs on the vertical axis will be displayed as a graph.

The red point of the graph is the actual value actually entered , and the purple curve is the graph in which the growth degree curve is applied to the red point of the actual result . In other words, such as Gonberutsu function Toka logistic function mathematical function , and the value of the actual adjusting the various parameters of the function as along with, it relies on the curve was fit is a curve of purple.