Soft Audit overview・Target 2: Audit the test technology　

06/01/202121/10/2021Audit for soft-Develop..

2. Check whether the development technology and test technology are sufficient from both the technical and system perspectives.

In software development audit, we will confirm the quality of skill from the four viewpoints of development process, test technology ability, requirement grasping ability, and design / mounting ability. In this article, I will introduce from what perspective we will confirm the second test technology.

Software development requires various processes such as planning, design, implementation, testing, and maintenance, and in order to carry out each process, technical skills related to that process area are required. Technology in every area is just as important, and software that can be lacking cannot be made. However, when confirming the software development skills of a contractor in a software development audit, Gutara’s father places the highest priority on test technology , especially in terms of quality .　

As a result of the increase in software scale, the proportion of software installed in products that was developed in-house has been steadily decreasing, and the proportion of purchased products, free software provided by chip vendors, and OSS (open source software) has increased. I did. In order to guarantee the quality of the software in such a state, it is essential to enhance the test . Therefore, even when estimating the development skills of a software development contractor, it is very important to determine whether the contractor has sufficient technical capabilities for testing.　

Then, what kind of things can be confirmed to know whether the test technology is good or bad? Although it is expressed in one word as test technology, in fact, test technology also has various aspects, and it is not possible to correctly judge the quality of technology unless it is judged from multiple perspectives. However, since it is difficult to find a general-purpose test technology evaluation method, Gutara’s father is outsourced to the following test technology and test system and management areas in the software development audit. It was judged whether the test technology was good or bad.

• Test team structure and test plan
• Adopted test category and department in charge
• Adopted test process
• Test progress management and plan review
• Manage detected bugs

Now, let’s take a look at what we are checking from what perspective for each audit target.

Is the test team structure and test plan clear?

The first thing to check is the structure of the test team, which team will perform the test . Depending on the size of the company and the composition of the organization, there are the following teams to test.

1. In the design and implementation team, the software engineer who designed and implemented is also in charge of testing
2. A test engineer who specializes in testing is in charge of testing in the design and implementation team
3. There is a test team in the development department that is separate from the design and implementation team, and the test engineer there is in charge of testing.
4. There is a test team in the quality assurance department separate from the design department, and the test engineer there is in charge of testing

In some cases, it applies to any of 1 to 4, and in other cases, multiple teams have a test system with division of roles. In any case, the structure of the test team in the company or organization to be audited, who is the team leader, how many team members are there, and where the test work is being performed , as specifically as possible the status of the test team confirm.

At the same time, it doesn’t matter whether it is a test team unit or software development as a whole, but we will check if there is a test plan . In organizations where test technology is firmly established in software development work, test plans are often created independently of development plans related to design and implementation. Of course, it is okay if the test plan is included in the development plan for development / implementation, but with large-scale software, the number of man-hours required for testing is larger than that for design / implementation. In many cases, it is more reassuring as a test technology to have an independent test plan from such a viewpoint.

Is the test category adopted just right and the department in charge clear?

Next, check what test category you are using. The test category is a test classification in Japanese, but it is easy to understand if you think of the tests as being classified according to the purpose of the test . For example, normal test and abnormal test are one test category. In software development audits, we ask the companies to be audited to tell us what the software test categories are, the names of the categories and their specific contents, and which test team conducts the tests for each category. I was checking if it was.

It’s pretty cool, but even if you actually ask to show the test category at the software development audit site, there are few places that will show that this is our test category. What is the test category , especially for young companies abroad ? 　Was often asked. In such a case, here is an example of a test category, and which test team is conducting the test in such a category? I was taking the method of confirming.

There are various ways to divide the test categories, but Gutara’s father found the following 15 test categories in the book The Art of Software Testing, Second Edition by Glenford J. Myers easy to understand. I was referring to.　

• Facility Testing
• Volume Testing
• Stress Testing
• Usability Testing
• Security Testing
• Performance Testing
• Storage Testing
• Configuration Testing
• Compatibility / conversion Testing
• Installability Testing
• Reliability Testing
• Recovery Testing
• Serviceability Testing
• Documentation Testing
• Procedure Testing

Even if it is called a software test, there are tests for various purposes such as function confirmation and performance confirmation. In testing, it is good to carry out tests for as many purposes as possible in a well-balanced manner, so checking the test categories adopted and the implementation status of each category is a good way to judge the quality of the test technology.

Clarify the test process adopted

After confirming the purpose of the test in the test category, the next step is to confirm what kind of test process is adopted. Software testing, like design and implementation, consists of multiple processes. Test planning, test environment construction, test outline design, test detailed design, test design review, test execution, test result recording, test result (detected bug) tracing, test result review, test There are various testing processes, such as replanning tests based on results. For software testing, it is also possible to check how specific the process has been decided, whether each test process is clearly written so that anyone can carry it out in the same way, etc. You can judge whether it is good or bad.

What is happening with test progress management and plan review?

After confirming the specific test contents in the test category and the test process, the next step is to confirm the test management . As with software development, it is necessary to manage test progress and test quality for testing, so the status of such management is also subject to software development audits.

Managing the progress of the test is relatively easy to understand.If you have a test plan that tests an average of 50 items a day, check the actual number of items to be tested every day and progress by 50 items. Is as planned. To see the progress of this test, I need work that compared to the plan on a regular basis summary action items of the test, such whether the work of comparison and aggregation is performed is planned properly Toi Is the area to be confirmed by software development audit.

Also, as we proceed with the test, we cannot create this test condition! Or maybe this test item is duplicated with another test I did earlier? Such as that the mistake of the test design itself might also be coming into view. If there are too many such things, the quality of the test design itself will be poor and the efficiency of test execution will decrease, which is a problem. It’s the management and monitoring of test quality activities, but such whether there is a monitoring and management activities is also a confirmation area for determining the test technology.

Also, if you are actually monitoring the progress of the test, you may gradually fall behind the plan. There are various causes when the delay of the test plan is large, but if there is a situation where the quality of the software to be tested is poor and the test does not proceed easily, some stage In some cases, it may be necessary to review the test plan so that the test you are currently doing is interrupted and the test is performed again after the quality of the software has improved . Another point to keep in mind when proceeding with an audit is whether it is clear who will use what information to determine the necessity of reviewing the test plan .

How do you manage the bugs you detect?

The purpose of the test is as follows: (1) Identify potential bugs in the software. (2) Guarantee that there are no bugs in the software. Among these, in the test conducted for the purpose of detecting latent bugs in (1), software bugs are detected as the result. The accepted bugs are investigated by the design and implementation department, a debug cycle is carried out to correct the source code, and the fixed software is tested again.

At this time, there is a mechanism to trace how the detected bug is recorded, whether the recorded content contains all the information necessary to reproduce the bug, and the detected bug until the correction is completed. the one, the bug has been detected, such as to ensure that that is addressed until the last bug management mechanism of the well educated, make sure in the software development audit.

In the case of small-scale development, bug tracking may be done with a list like EXCEL, the development department is distributed to multiple bases, the design / implementation team and the test team are separate bases. If you’re working in a country, bug tracking may also be running on a system on your network. Any tool can be used, but until the final release of the software, there is a mechanism to reliably trace the bugs found, and stable operation ensures the quality of the software. It is an important item in.

I think there are various ways of thinking to judge the quality of the test technology of the development contractor. The example introduced here is just an example that Father Gutara was looking at such a part from such a viewpoint, but I hope it will be helpful to everyone. Now, let’s introduce the confirmation of technical capabilities for grasping the requirements that are the next most important after test technology.