Test example・Memory leak test　

01/09/202121/10/2021Test qualityy

Memory leak test is essential for embedded software

It is an important requirement for embedded software to continue to operate stably. So what kind of test should I do to make sure it keeps working steadily? Here, using the control system and communication system embedded software that Father Gutara has experienced so far as an example, the content of the bug and the bug are detected for specific tests that may be useful for ensuring the stability of the embedded software. I will introduce how to do it. The first is the memory leak test, which is essential for testing embedded software. I hope it will be helpful when you think about testing.

Find a time bomb bug in a memory leak test

As mentioned here and there in this blog, the time bomb bug is a great enemy for embedded software that requires stable operation 24 hours a day, 365 days a year. Memory leaks are the most common cause of time bomb bugs, so be sure to perform a memory leak test before shipping your embedded software.

A memory leak is a bug that requires attention when using dynamic memory , which means that the memory used by a program is “acquired from the OS when needed and returned to the OS when it is used up" . The famous dynamic memory acquisition / return function for unix-like OS is malloc / free. There is a lot of information about memory leaks on the net, so it’s easy to see for details, but in simple terms, you forget to return the acquired memory, and the dynamic that the OS can provide. The decrease in the remaining amount of memory is called a memory leak.

Let me give you a brief example of a memory leak.

The OS reserves a certain amount of memory in advance at startup to provide the function of dynamic memory, which is called a memory pool . Then, when the application software requests the acquisition of dynamic memory using the malloc function , the OS cuts out the required amount of memory from the memory pool and rents it to the application software. The remaining amount of the memory pool will be reduced by the amount of the rented memory, but the remaining amount of the memory pool will be restored if the dynamic memory is returned by using the free memory that the application software has finished using .

However, what happens if there is a bug in the application software and you forget to return it with the free function after using the memory acquired by the malloc function ? From the OS’s point of view, the rented memory is not returned, so the remaining memory pool remains low. If the free function is repeatedly forgotten to return each time a buggy application software is called, the remaining amount of the OS memory pool will gradually decrease. This state where the remaining amount of the memory pool is decreasing is compared to the state where a hole is opened in the bottom of the bucket and the stored water is leaking, and it is expressed that the memory is leaking, that is, a memory leak is occurring. doing. (Leak in English is a leak (LEAK)) If the memory leak continues, the remaining memory pool will eventually become zero and the memory will be exhausted, so you need the application software that was trying to acquire memory with the mallco function. Dynamic memory cannot be acquired and it cannot operate normally.

Why Memory Leaks Are Annoying Time Bomb Bugs

Memory leaks are annoying time bomb bugs because they are hard to find in pre-release in- house tests . Software bugs cause problems when some conditions are met. For example, when the conditions such as inputting a specific value on the input screen or receiving specific data by communication are met, the buggy part in the software is executed and causes a problem. Therefore, in the in-house test, we make a test plan assuming various conditions and try to identify potential bugs in the software. Memory leak bugs are difficult to find in internal testing because it is difficult to create this condition .

The symptom of a memory leak, which is a decrease in the memory pool, will occur if some conditions are met , just like any other bug . However, even if a memory leak occurs , the software will operate normally without any problems as long as the memory pool is full. Software malfunctions occur only when memory leaks are repeated many times and the remaining amount of memory pool becomes zero . For example, if the amount of memory pool is 300MByte and a memory leak occurs by 1Mbyte in one day’s operation, 300 days have passed since the device was turned on and the software started to operate, and the remaining memory became zero. There will be a problem.

To find the memory leak, ① conditions that cause a memory leak in addition to, that either may be repeated many times the condition ② number of iterations you need to meet as well conditions. In order to cause a problem due to a memory leak within a limited test time, it is necessary to set both the test acceleration method to accelerate the condition where the conditions of (1) are met and the test period to satisfy the number of repetitions of (2). Therefore, it becomes difficult to test.

What should I do with a memory leak test?

The test conditions are difficult, but when you release the software, you have to make sure that there are no latent memory leak bugs, what kind of memory leak test should you do? Actually, the memory leak test is quite good if you change the way of thinking of the test a little. If it is difficult to wait for the remaining amount of the memory pool to become zero and a software malfunction occurs, you should check that the remaining amount of the memory pool is not low, and a little test judgment method Change it.

Specifically, plan a memory leak test by following the steps below.

1. The beginning of the test measuring the remaining amount of memory pool to
2. In a variety of conditions operate the software makes
3. The end of the test re-measuring the remaining amount of memory pool to
4. If the remaining amount of the memory pool does not change between the beginning and the end, it is judged that there is no memory leak.

To measure the remaining amount of the first memory pool, it is sufficient if the method of measuring the remaining amount of dynamic memory is provided by the OS interface, but if there is no method, search for a library with the same function. Or you have to develop it yourself . When developing, it is good to create a function that simply acquires dynamic memory and determines the amount of memory acquired by the time it cannot be acquired as the amount of remaining memory at that time . In detail, there are some items to consider, such as fragmentation of the memory area (fragmentation) and the necessity of writing the actual data after allocating the memory, so the process will be a little more complicated, but the rough idea of ​​the remaining memory It will be attached.

The second operation of the software under various conditions is the power of trying various operations anyway because the conditions under which a memory leak occurs are unknown . If you have an automated test system for various functions, it is best to use it to operate various functions for a long time. In addition to that, in order to find a memory leak in some error handling (which is actually the most difficult to find and easy to get to the market), I created a test environment where errors occur and overloaded it. Performing time aging is also a good idea.

To measure the remaining amount of the third memory pool, just do the same method as the first after finishing the second test. Then, the fourth is to compare the remaining amount of the first and second memory pools and judge that the remaining amount of the memory pool has not decreased if there is no change . Depending on how you measure the remaining amount of the memory pool, the remaining amount of the memory pool changes considerably dynamically depending on the actual operating conditions . Therefore, it is safe to judge that a memory leak has occurred if you can see the remaining memory phenomenon in each test by looking at the results of performing the memory leak test not only once but multiple times .

It’s not just the OS dynamic memory that leaks

In the explanation so far, the dynamic memory provided by the OS has been assumed as the dynamic memory, but it is not only the dynamic memory provided by the OS that leaks . Of course, dynamic memory leaks can also occur if you implement dynamic memory at the application level and use it from another application. Therefore, when planning a memory leak test, it is important to identify all the dynamic memory used by the software and make a memory leak test plan for each dynamic memory.