Test report・Test evaluation begins with an evaluation of the test content
- 1. The test evaluation written in the software test report starts with the quality of the test content.
- 2. Test content assesses bug density and validity of test density
- 3. Was the planned amount of testing appropriate?
- 4. Was the test plan appropriate for the priority areas of the test?
- 5. Was the test plan for detecting complex bugs reasonable?
- 6. Evaluation of test schedule and test efficiency is also required
The test evaluation written in the software test report starts with the quality of the test content.
Write the test result and test evaluation in the test report of the software . Simply put, the status of the bug detected in the test is the test result, and the good or bad of the test plan and implementation is the test evaluation . Since the test results were introduced in the previous two articles, let’s introduce the test evaluation a little more concretely from this article.
In the test evaluation, it will be easier to understand if you evaluate and write the test plan and implementation from the following three viewpoints.
- Evaluation of test contents : Was the quantity and quality of the test optimal for guaranteeing the quality of this software?
- Test Itinerary Assessment : Was the test itinerary appropriate?
- Evaluation of test efficiency : Was the cost performance plan for the test work appropriate?
First, let’s take a closer look at the evaluation of test content.
Test content assesses bug density and validity of test density
It is easy to confuse the test result and the evaluation of the test content because they analyze and evaluate the bugs found in the test, but if you are aware of the purpose of analysis and evaluation, you can see the difference. .. The test result analyzes the bug to judge the quality of the tested software, while the evaluation of the test content analyzes the bug to confirm the validity of the test item .
Checking the validity of the test items means whether the amount of tests was sufficient, whether the priority area of the test was appropriate for testing this software, and whether the test found a bug with the difficulty expected. From this perspective, it means confirming whether or not the results (= bug detection) that were planned at the time of planning were achieved. Let’s take a closer look in turn.
Was the planned amount of testing appropriate?
If the amount of tests is small, few bugs will be found, and if a large amount of tests are performed, many bugs will be found. Of course, how many bugs you can find affects the quality of your tests, but you can’t find many bugs without first completing the number of tests . “If you don’t test, there are no bugs."
From that point of view , first evaluate whether the amount of tests this time was sufficient , insufficient, or excessive in terms of the number of bugs detected, and whether the amount of tests was good or bad. It is easier to understand when evaluating the amount of test by dividing it into the total test amount and the test amount for each function.
(1) Evaluation of the total test amount
First , consider whether the amount of testing as a whole was appropriate. However, in addition to the amount of tests, the amount of bugs detected is related to various factors such as the quality of the original software, the quality of the tests, the test period, and the test environment. Therefore, it is very difficult to judge whether the amount of tests in this test plan is appropriate .
It is certainly difficult, but if it is one company or one development organization, the software development ability and testing ability will not change suddenly. In other words, we made the design and implementation team number of bugs that are mixed in soft as well, the test team the ability to detect the bug also should not change significantly is. If so, the comparison between the bug density and the test density of the bugs detected in the test should be within a certain range.
Conversely, if the comparison between the bug density detected in this test and the test density is far apart from the other tests so far, it seems that something is happening. It may seem like a very vague decision, but if you accumulate the bug density and test density of the same company or organization for about 5 years , if this organization tests with this level of test density, it will be about this level of bugs. You can see a reasonable level that the density will detect bugs. By looking at the bug density of this test using the bug density of the past test as one criterion, you can judge whether the total amount of tests was appropriate.
(2) Evaluation of test amount for each function
The evaluation of the test amount for each function cannot be compared unless there is a function similar to the software targeted for comparison with the software targeted for comparison, so devices and devices with similar functions but and if there are multiple, different version software development of the same equipment and for equipment it is the only available method in the case where there is more than one.
The evaluation of the test amount for each function is the same as the evaluation of the overall test amount. By calculating the test density and bug density for each software function and comparing them with the results of other tests, you can examine the validity of the test amount for each function of this test. If the relationship between test density and bug density for a particular feature is significantly different from other features, it may be a good idea to look back at the amount of test, suspecting that something went wrong.
Was the test plan appropriate for the priority areas of the test?
There are various ways to decide the priority area of the test depending on the software to be tested, but there are roughly two ways to decide. Whether to classify by software function to determine the priority area, or to classify by the potential bug and determine the priority area. With either method, when planning the test, plan to spend more time and effort on the priority areas than on the other areas. Judging from the test results, whether the test plan for the priority area of this test was appropriate is written in the test report.
Judging the validity of a test plan for a priority area is difficult, and ultimately it is a judgment based on the feeling of the test leader or tester , but even so , write down such on- site judgment in the report. Is important.
(1) Classify by software function to determine priority areas
In new software development , the original test procedure is to comprehensively test all functions if sufficient time and cost are available. However, the reality is that you often have to perform the most efficient tests for a limited period of time before release at a limited cost . In such a case, of course , a test plan is made to focus on the important functions, considering the importance of quality for each function of the software to be released .
In such a case, the evaluation of the test plan for the priority area in the test report will be judged from whether the function decided to focus at the time of planning could detect the bug with sufficient density . Whether the test plan was appropriate by calculating the bug density for each function in this test and analyzing the test results from the viewpoint of whether the functions decided to focus on detected bugs with sufficient bug density. To judge.
(2) Set priority areas by classifying by potential bugs
The most important thing to pay attention to when developing modifications such as adding functions or fixing defects to existing software is the inclusion of secondary defects . If code changes due to function additions or bug fixes have an unexpected effect on existing functions and embed new bugs, end users will have problems with the functions that have been used so far, so it looks like degreasing . It will end up.
In order to avoid such a thing, it is often the case to make a test plan to identify the range of influence of the correction and focus on the range of influence when adding functions or fixing defects . Even in this case, but to perform an exhaustive testing for all of the range of influence that began washing is ideal, still this is in a limited period of time and costs high is the degree of impact of modifications likely , change the words When the secondary failure is a possibility is high likely that intrudes with an emphasis in the area there is also taking a policy of planning a test.
Even in this case, analyze the test results in terms of whether the bug density in the area you decided to focus on was higher than in other areas, in other words, whether you found enough bugs in the area you decided to focus on. , Determine the validity of the test plan.
Was the test plan for detecting complex bugs reasonable?
When you make a test plan, you also think about the quality of the test . A quality test is one that has few bugs that cannot be found and missed . However, in order to reduce the number of bugs that are overlooked, it is necessary to increase the ratio of tests to detect compound condition bugs that occur when multiple conditions overlap and probability-occurring bugs that occur only with a low frequency of occurrence , and the cost of testing And the period will swell greatly .
Limited cost and duration because in must be carried out the test, out of the cost and duration that can be applied as a whole test or how much proportion of the allocated to the detection of complex conditions bugs and probability generating bug , the test This is one of the major policies to consider in the plan . The validity of the cost and period allocation in the test plan from the viewpoint of whether the bugs could be found in sufficient quantity and density with respect to the cost and period allocated for detecting the compound condition bug and the probability occurrence bug . To judge. It is difficult to judge how densely the bug should be found , but it is also possible to judge the validity of this test by comparing it with similar test results . It is good to make a judgment.
Evaluation of test schedule and test efficiency is also required
The quality of the test plan written in the test report includes the evaluation of the test schedule as well as the evaluation of the test content. Once you have written the good and bad of the test plan, the final step is to evaluate the test efficiency. In the next article, I will introduce these two in detail.