Release judgment criteria・The amount of test is measured by test density
- 1. In the test quality in the release judgment, first look at the sufficiency of the amount of test
- 2. The basis for determining the amount of test is the test density for the source code
- 3. In remodeling development, some code other than new and modified is reflected in the evaluation
- 4. Consider the amount of test separately for new code and existing code
- 5. When checking the amount of tests, also check the amount of tests actually performed.
In the test quality in the release judgment, first look at the sufficiency of the amount of test
Of the four criteria used when determining software release ( test quality , residual bugs , product quality , process quality ), this article will consider how to determine whether the test quality is good or bad. Gutara father is the quality of the test amount of testing and individual types of tests and test the implementation of the bug situation was watching from four perspectives of. In the articles that follow this article, let’s take a look at the idea of determining test quality. Let’s start with the amount of tests that are relatively easy to think about.
When looking at the quality of the test in terms of the quantity of the test, the simple answer is that the larger the quantity of the test, the better . If you don’t test, there are no bugs, and it’s clear to everyone that a lot of tests will find more bugs than a few tests and a lot of tests. The test is sufficient to implement the how much amount do you that? It’s quite annoying. In this article, I will introduce the idea that Father Gutara has used, whether the amount of testing is sufficient or insufficient, what kind of information should be used when making a release decision, and what kind of perspective should be used. ..
The basis for determining the amount of test is the test density for the source code
When considering the amount of tests, it is necessary to consider the test density (XX / KLOC), which is normalized by the scale of the source code, rather than the absolute number of test items (XX) . For example, a 100-item test is sufficient for 100 lines of source code, but not enough for 100,000 lines of source code. Only the test density normalized by the scale of the source code can determine whether the test amount is sufficient.
If all the source code is new development , simply set a test density target of XX / KLOC and determine the scale of the required tests. How to determine the target value is also described in the quality creation guide for embedded software development by IPA , so it is good to refer to this. However, the value of the test density changes depending on the particle size of the test item, which is what is counted as one test, so it is necessary to read it for your company.
In remodeling development, some code other than new and modified is reflected in the evaluation
By the way, developing all source code from scratch is actually a rare situation in the recent software development field where software has become large-scale. The most common is remodeling development (also called derivative development) that modifies existing software. Remodeling development in the source code to be modified source ( baseline code there also be referred to as a) Bring, there or afterthought the necessary functions to, or remove unnecessary functions, that the development of addition subtraction to the Make software.
In such a case, it is not enough to just do the tests related to the newly added source code or the modified source code. In interdependent software, the effects of adding / removing features often reveal potential bugs in undeleted / unfixed source code . However, if you try to run tests on all baseline code with exactly the same test density as the newly developed part, it will cost a lot of testing, so it is quite annoying.
Consider the amount of test separately for new code and existing code
As a result, we have thoroughly tested the newly created source code and the functions related to the modified source code, and the test density for the functions implemented in the source code that is not touched in this development. It is necessary to adjust the necessary test density by lowering the test a little, or to simplify the test . In this case, the total amount of tests required (= number of test items) is roughly determined by the following calculation.
(1) Total amount of tests = (2) Test amount of new or modified modules + (3) Test amount of unmodified modules
② Test amount of new / modified module =
Number of source codes for new / modified modules (KLOC) X (α) Test density of new / modified parts (case / KLOC)
③ Test amount of unmodified module =
Number of source codes of unmodified modules (KLOC) X (β) Test density of unmodified modules (case / KLOC)
First, (α) determine the test density (case / KLOC) of the new / corrected part by referring to the quality creation guide for embedded software development . It is a common way to determine this value by multiplying this value by the (γ) adjustment value from 0 to 1 that was decided by the eye to some extent, and (β) the test density of the unmodified part (case / KLOC). .. (Β = α X γ)
(Β) How much the (γ) adjustment value for calculating the test density of the uncorrected part should be between 0 and 1 depends on the new / modified source code and the other source code. It is difficult to judge unless you know the overall structure of development because it is related to the range of influence of the correction determined by the relationship .
It is not a value based on some achievements, but a mere rule of thumb, but in the case of software with little dependency (high independence of individual modules), Gutara’s father has a mutual adjustment value of about 0.1. I used about 0.3 as a reference value when the dependency of is strong (the modules are dependent on each other).
But, well, no one really knows the correct answer as to how much to test, so the person who has the best understanding of the overall development structure decides with a slap , and some people review the contents. Is this value all right? It would be realistic to decide. If possible, at that time, I think it is best for people who have experience in test design for various products to participate and present appropriate values based on their actual experience with products so far. The world of software is still the world of craftsmen where the judgment of skilled experts is of utmost importance.
When checking the amount of tests, also check the amount of tests actually performed.
When judging whether the test quality is good or bad, we first check the amount of tests, but what we have introduced so far is the perspective of the planned amount of tests. When looking at the amount of tests, looking at the amount of tests actually performed in addition to the planned amount of tests provides information that determines test quality from a different perspective.
In the next article, I will introduce how to see the test implementation status that Father Gutara of the test was watching, if you are interested, please have a look.