Test plan・Decide how to classify defects and bugs and the items to be recordp

28/05/202121/10/2021Test qualityy

In the test plan, specifically describe how to classify defects and bugs and the items to be recorded.

You can find bugs and bugs by testing the software. Problems and bugs is the easy to be confused, but in this article a problem when using the devices and equipment is the problem refers to the bug becomes a cause of the defect mistake of software you want it to refer to. To be more specific, the problem is the operation of the device or device caused by a software bug or a mistake in the test procedure, and the software mistake caused by a software design mistake or a coding mistake is a bug . Hey. A bug is a specific item in a bug.

How to classify this defect and how to manage the bug must be decided concretely at the time of test planning. In the development project with the participation of multiple software engineers and test engineers, bug was found in the test and information sharing management for bug tracking system and test support tool is there are many things to use. These systems and tools are just tools, so make specific decisions at the time of test planning to make good use of them.

At the time of test design, the items to be specifically decided for managing defects and bugs are mainly as follows.

1. Classification of defects (how to classify defects such as bugs, test setting mistakes, test implementation mistakes, and environmental deficiencies)
2. Defect record (In defect record, write the defect reproduction method and management information without omission)
3. How to record a defect (whether to enter it in some system or write it in an EXCEL table)
4. State definition (what state each defect or bug transitions to and finally ends in the final state)
5. Termination judgment (who decides by what criteria the defect has been dealt with)
6. Prioritization (who and how to prioritize bug and bug investigations and countermeasures)
7. Tracking (who monitors the state transitions of individual defects and how to take measures against delays)

In this article, I will first introduce the first 1 to 3 items. Let’s take a closer look at each item.

1. 1. Defect classification (how to classify and record defects and what to do)

As a result of conducting the test , if a result different from the correct result that should be obtained originally written in the test procedure manual etc. is obtained, the test item is recorded as a defect . Each recorded bug must be categorized by cause , and if the bug is a software bug, it must be handed over to the software design / implementation team for debugging . One of the major purposes of testing is to run this debug cycle to eliminate software bugs and improve software quality.

There are several categories of bugs other than software bugs , as shown below (A) to (E) . The soft bug of (A) needs to be taken over by the design / implementation team for debugging, and (B) to (D) need to be retested by solving the problem in the test team. In addition, if a problem such as (E) is found, it will be necessary to have the planning department, etc., which determines the required specifications of the software, take action. Since the department to deal with afterwards changes depending on the classification of the defect, let’s clarify the classification of the problem to be treated as a defect in the plan.

(A) Software bug

The operation of the software is different from the required specifications due to software design mistakes and implementation mistakes. For example, it is a software bug that the calculation result is different due to the design error or implementation error of the software C = A + A against the required specification of the calculation formula C = A + B.

(B) Test design error

There is a mistake in the test design and the wrong test result is judged. For example, as a test specification of the calculation formula C = A + B, it is confirmed that the result of C = 5 is obtained for the input of A = 1 and B = 2, and the method of judging the test result is wrong. It is a mistake in the test design.

(C) Mistake in test implementation

The test design is correct, but I made a mistake when performing the test. For example, for the test specification of the calculation formula C = A + B, to confirm that the result of C = 3 is obtained for the input of A = 1 and B = 2, the input is mistaken as A = 0. If you record it as a defect because the result is C = 2, this is a mistake in the test execution.

(D) Inadequate test environment

There is a problem with the environment used for the test and the test result is incorrect. For example, the connector of the communication cable used when testing the communication has a bad contact, and as a result, the communication performed in the test results in an error, so record it as a defect, which is an inadequate test environment.

(E) There is a problem with the validity of the required specifications

The test result is defective because the required specifications of the software have a problem in the light of common sense. For example, as a test specification of the calculation formula C = A + B, it takes one minute or more for the result of C = 3 to be obtained for the input of A = 1 and B = 2. According to the required specifications, the responsiveness is within 2 minutes, so it meets the specifications, but in general, if you wait for 1 minute or more, there is a high possibility that it will be mistaken for a failure. In such cases, if the purpose of the test is to validate, the test result is recorded as a defect and the reason is the validity of the required specifications.

2. Defect record ( whether to write the defect management information and how to reproduce it without omission)

The most important information for recording a defect is how to reproduce the defect . In particular, if the bug is a software bug, the bug will not be resolved unless the person in charge of the software design / implementation team reproduces the bug and proceeds with investigation / countermeasures. If the defect cannot be reproduced, neither investigation nor countermeasures can be taken, so it is very important that the information that can be reliably reproduced is recorded concretely .

If a bug or defect can be reproduced, investigation and countermeasures will be carried out, so management information for managing the defect is also important. Then, what kind of items are there in the defect management information and information on how to reproduce it? Let’s take a look at the defect record that Father Gutara often saw as a sample example in order.

First of all, it is an item about defect management information .

(1) Bug ID number : An ID number to uniquely identify the bug is required, and it is automatically unnecessary if you use a bug tracking system.

(2) Defect title : Since the ID number does not tell what the defect is, a title that makes it easy to understand the content of the defect is also required.

(3) Occurrence software: Information such as the detailed version number that can uniquely identify the software to be tested when a defect is detected is essential.

(4) Defect event : What kind of unexpected result was obtained as a result of the test, so it is necessary to have a detailed and specific event as much as possible.

(5) Defect status : Items such as whether the defect is waiting to be reproduced, under investigation, or being corrected, and what state it is in are essential as management information.

(6) Person in charge : The name of the engineer who is currently in charge of the defect is necessary for determining and managing the concentration of work on a specific engineer.

(7) Scope of influence: If the bug is a bug, the version of the software in which the bug is latent is required to determine the scope of application of the countermeasure (write after the cause is found).

(8) Scope of countermeasures : If the bug is a bug, it is also necessary to record the version of the software that fixes the bug (in some cases, planning and actual results).

Next is the item about how to reproduce the defect .

(7) Test ID number : Which ID number of which test design document the defect was found in, this is the basic information for reproducing the defect.

(8) Test environment : Record the SER. Number of the target hardware and the individual identification number of the server, peripheral device, and communication cable used as much as possible.

(9) Test data : The test data used for the test when a defect is found is recorded as specifically as possible, such as device settings and communication data.

(10) Tester : When the defect cannot be reproduced only from the record, record who should ask the reproduction situation.

(11) Reproduction status : Record the results of the reproduction trials so far, whether the reproduction is successful or unreproduced, and whether it can be reproduced once every few times.

In addition to these items, various items are required to record defects, but basically, if there are items like the above, I think that the investigation and countermeasures for defects can be carried out smoothly.

3. 3. Defect recording method (whether to input to the system or write to EXCEL)

Failure is found in the test , et al., To the team and the person in charge to carry out the subsequent correspondence take over the problems in order, record the problems of content you. The record of this defect is the most important deliverable for the test, and after that, many people in charge proceed with various work based on the recorded contents. Therefore, it is necessary to decide specifically at the time of test planning how to record the content of the defect and share it with the related persons.

The most commonly used method of recording bugs is a management system such as a bug tracking system . Bugzilla , Mantis , Redmine, etc. are famous in the open source system, and other paid versions are also available from each company. Whether it’s an open source system or a paid version, various definitions and settings are required to master the bug tracking system, so the barrier is a little high when introducing it for the first time. However, once introduced, the merit is great, so especially when developing and testing software with a team of 10 or more people, or when developing and testing software in multiple bases, introduce a bug tracking system Is recommended.

If you don’t use the bug tracking system, you often use a list called a bug list created by EXCEL . By recording in one line of one defect EXCEL, necessary items such as defect ID number, defect title, occurrence test item number, defect content, current status (status), etc. are recorded in the horizontal direction of the table. It is common to use a table in which. This bug list is shared with related parties on a server that can be accessed by designers and testers , and bug information is shared. In this case, make sure that all the persons in charge follow the operation method that makes good use of the sharing function of EXCEL so as not to cause writing corruption due to simultaneous writing .