Release judgment criteria・Test implementation status based on progress confirmation

16/11/202020/10/2021Release Judgment

How to determine the test implementation status

Of the four criteria used when determining software release ( test quality , residual bugs , product quality , process quality ), this article will consider how to determine whether the test quality is good or bad. Gutara father is the quality of the test amount of testing and individual types of tests and test the implementation of the bug situation was watching from four perspectives of.

The third point of view for judging whether the test quality is good or bad is the test implementation status . There are many types of tests. The most important thing to pay attention to when judging the release is the system test, so when the term “test" is used in this article, think of it as the system test. Now, for the test, make a plan, design the test, and execute the test for each test item. The situation when running a test is the test execution status. What can you tell about the test quality from this test execution status ?

Father Gutara mainly confirmed the following three things about the test implementation status and used it as information for judging the test quality.

1. Test progress
2. Bug detection status
3. Bug fix status

Start watching the test status earlier than the release decision time

Actually, I didn’t see the situation regarding these three tests at the stage of making a release decision . Of course, it is possible to check these situations at the time of release judgment and estimate the quality of the software product from the test status. However, I’m not so happy if I find out that the product quality is poor at the time of release judgment.

At the time of release judgment, if the product quality is found to be poor, all you can do is stop the release. However, before the release decision, for example, when the test is about half advanced, if it can be estimated from the test situation that the product quality is likely to be insufficient, some measures can be taken.

If you know before the release decision, you may be able to take measures …

If the content of the test is insufficient , review the test design, if the test man-hours are insufficient , increase the number of test personnel to make up for the shortage of man-hours, if the test schedule seems to be insufficient, consult with the customer or related departments in advance and release re-adjust the schedule to, various and striking hand that might have will be Kangaerae.

Of course, there are things you can and cannot do, but it’s not a situation where you can’t do anything. However, if the product quality is improved as a result of taking some measures and the quality is improved at the time of release judgment, it will be a great pleasure. For that reason, Father Gutara has been stereotactically monitoring the status of the test since the system test process was about halfway through .

There are various backgrounds to the project, some of which can be viewed with peace of mind, and some of which are quite dangerous. The more dangerous the project, the more important it is to take proactive improvement measures so that you do not end up making a release decision in a situation where testing is insufficient .　

The story is a little off, so let’s take a look at what we have confirmed in each of the three situations.

1. Test progress

First is the progress of the test . It seems to be difficult to put it in words as progress, but in short, how many tests are you planning to do and how many tests have you completed now? 　It’s just that.　

To put it a little nicely, it 's the monitoring of the planned and actual test progress values shown in the test items . You can look at the numbers, but if you draw a line graph of the planned value and the actual value with the horizontal axis as the number of test days and the vertical axis as the cumulative number of test items, the situation will be clear at a glance. For example, this is the graph.

 

The planned value can be rough

Isn’t it so easy to estimate a test plan? Don’t worry. If the total number of tests is 600 and the test period is 20 days, you can just draw one straight line that will be 600 on the 20th day . You can also set the planned value with the momentum of the moment.

If you want to match the actual situation a little more, you can estimate the number of test workers on a daily or weekly basis and make a graph with the work man-hours (man-days) on the horizontal axis.　The plan is a guide , and if you get used to it, you will be able to draw the plan line accurately, but just because the test plan matches the actual results does not mean that the bugs will disappear, so even if it is a little appropriate, there is no problem at all.

Let’s total the actual value properly

However, let’s count the actual value, which is the number of test items actually performed . It is good to record the number of tests performed every day for each tester. If you put it together for a week, you will usually make a mistake in counting. People are creatures that are forgotten, so it can’t be helped.

If you put the achievements in the graph above, it will often be like this.

What can you tell from the graph?

 In comparison with such a rough plan values, exactly what is seen? 　You may be wondering. However, it turns out to be unexpectedly useful. For example, the most common situation is that when the test process progresses half of the planned value of the straight line as shown above, only about half of the planned value has been tested.

So what’s happening at the test site ? The test plan was originally an unreasonable plan , the test workers could only arrange half the number of people planned, and there was a labor shortage, etc., and each test item was twice as much as expected at the time of planning . There may be various reasons , such as the time and effort required, or the quality of the software being poor and the test not proceeding.

But one thing is certain: even if the rightmost day comes to the graph, which is the end date of the planned test, there will be a lot of test residue that has not been executed yet . That is, when the test period is still half left, it turns out that the actual result = the fact. Half of the test period is still left as time to take some measures after understanding the facts .

The ability of the test team can be seen based on the achievements

Whatever the reason, the current test team’s ability to carry out tests is half of the plan, so if you finish the 600-item test that you originally planned by the planning date, the other half of the schedule will be three times as much as before. First of all, it is necessary to take either measures, either by putting in the test personnel of the above, or by extending the planned end date of the test to three times the remaining schedule.

However, if the quality of the software is not good and the test is not progressing, this will not improve the situation no matter how many test personnel are added. In that case, it is necessary to freeze the test implementation and wait until the software development department can produce software of sufficient quality to withstand the test .

That said, the software release date is fixed, and the cost of testing is limited, so it can’t be helped! 　I think you can hear the voice.

Software is honest

But unfortunately the software is honest . Things that can’t be helped really can’t be helped, and even if you continue testing with this system, there is only software that is full of bugs that have not been tested even when the scheduled end date of the test comes . ..

If you forcibly release the software, a potential bug will be discovered as soon as it starts operating in the market, causing a great deal of turmoil in both the sales department and the development department. As a result, customers request us to provide bug-fixed software as soon as possible and perform a large number of retests to ensure complete quality, and we end up spending a lot of money on testing . ..　

Software will always cause problems in the market if bugs remain. Unfortunately, we can’t expect it to keep secretly hiding the latent bugs it holds. The software is very honest.

There is nothing you can do, so let’s escalate

Therefore, when such a graph of the test progress is drawn , it is unlikely that we will be able to release it to the upper management and management of the organization because the test will not be completed as it is, please take measures. increase. It is best to make a whistleblower. In other words, it 's an escalation of an important issue .

Originally, it is good for the development department to do such a move, but many development departments or software engineers struggle to the very end , even if it is impossible . is. As a result, there are many cases where it is turned over at the end, which is a problem.　

Therefore, when such a sign is seen, a third party who is involved in the test situation outside the development department rescues the development department from the death march by escalating the problem to the management. Things will also be possible.

Next to the progress of the test is the status of bug detection and fixing.

In the test implementation status, we will look at the status of bug detection and the status of fixing the detected bug, which are the results of the test as well as the progress of the test. I will introduce from what perspective in the next article, so please have a look if you are interested.

Next : Release judgment criteria・See the status of bug detection for evaluate of testing