Search Posts

I can not ask anymore Linux and memory basics & detailed usage of vmstat

Sakura Internet Advent Calendar On the last day, I would like to take a fair look at the basic knowledge of Linux memory.

Recently, I became able to program without consciousness of the server and less consciousness about the free space of the memory, but when it became a situation where it is necessary to touch the low layer, the OS is stunned in front of me It seems that there are many people to do.

Basically, I think that it is better to know about performance of Linux, such as whether to fill in a lot of memory or not to swap. However, I do not want to see results such as the ps command and the free command somehow, I would like to dig into the memory a bit more.

Memory and cache

The state of memory in Linux is roughly divided into “memory in use” “cache” “free memory” “swap” is divided into four.

And swap exists in HDD, otherwise it resides in real memory, HDD is slower than real memory.


Memory in use is the memory used by processes such as kernel and applications. Cache is a memory used as a cache for speeding up I / O such as HDD access.

  • When memory in use increases, free memory decreases.
  • As long as there is free memory, the cache is basically increasing more and more.
  • If there is no free memory, the cache decreases according to the increase of the memory being used.
  • If there is no more memory allocated to the cache, I / O performance will be degraded.
  • When the memory in use becomes likely to become larger than the actual memory, it writes the overflowed part to the swap.

As you can see, as more memory is needed, the I / O performance will be degraded as the available memory for the cache will be reduced, and if the memory overflows, swap will be used and I / O Performance will decrease more and more.

For this reason, it is ideal that the memory being used is smaller than the actual memory, the cache is reserved, and the swap is not used. In the situation where the swap is constantly used, increase of memory etc. should not be considered There is no doubt.

Check the memory status

Next, let’s check the actual memory usage situation.

The memory usage status can be confirmed with the free command, the vmstat command, or the like.

In this situation, you can see that 1.7 GB (1717524 KB) of free memory is available and 184 KB and 8 MB (8280 KB) are used in the cache.

There are two kinds of cache, there are a page cache indicated by cache and a buffer cache indicated by buff. The page cache is a cache for the file system and is a cache used when accessing by file.

For example, when data is written to a file, data is left in the page cache, so you can use the data without accessing HDD at the next reading. Another buffer cache is the cache used when accessing block devices directly.

Consider cache

Let’s look at the behavior of page cache using dd and vmstat.

I think that you probably have not used swap yet, just to make sure swap is turned on / on, and make swap not used.

Looking at the display of the free command, you can see that the swap out is 0.

Then clear the cache.

Next, execute dd and write to the HDD.

Looking at the vmstat display, you can see that writing to the file (io bo) occurs and the IO wait (cpu wa) is rising. Also, the page cache usage has increased by over 500 MB, and you can see that the page cache was used at the timing of writing the file.

By the way, although writing to the file is completed in 1.76 seconds, in an environment where delayed write (write back) is effective, writing to the HDD is performed asynchronously by the kernel thread called flush. Even in the above example, it is understood that only 393,216 blocks (384 MB) are written immediately after writing, and 524 and 288 blocks (512 MB) are written together after 30 seconds.

The b of procs is temporarily set to 2 because flush is running separately from dd. It is because dq has already ended, b in procs is 1 when writing after 30 seconds because flush worked and asynchronously wrote.

Next, I will read in the written data.

Since it is stored in the page cache, it is completed in a moment as 0.28 seconds, and the result of vmstat does not generate a disk IO (bi is 0) and there is no IO weight.

This indicates that the written data is cached in the page cache and used for reading.

Confirm delayed write

I mentioned earlier that asynchronously written to HDD by kernel thread called flush. You can check the status of IO in real time by using the iotop command, so let’s try writing while executing iotop.

Open a terminal running vmstat or a new terminal and execute the iotop command.

First, attach the direct option, specify it without using the page cache, and execute it.

Then iotop knows that only the dd process is using IO.

By the way, vmstat at this time is as follows, you can see that the page cache of 500 MB or more allocated to the file called test was released. Also, b of procs is 1, and you can see that only one process is using IO.

Then do it without the direct option.

Then iotop knows that the process [flush – 253: 0] is using IO with the dd process.

Looking at vmstat, the page cache was reserved again and increased by about 500 MB, and b of procs became two processes.

In other words, in the case of direct, writing to the HDD is performed in synchronism with the process, and in normal cases, it is understood that asynchronously writes to the HDD via the page cache.

Note that it may be faster if you do not use the page cache when writing files with a low reading frequency and large capacity. For example, data such as VM image or backup data that does not use for a while keeping written is applicable.

Also, in a server with a large amount of memory, unless memory is used by processes or the like, as much memory is allocated to the page cache, even if the write seems to be fast at first glance, IO becomes a bottleneck during flush There are times when so-called “breathing” may occur.

Therefore, depending on the application, you should consider whether to use delayed writing, using the direct option, etc.

Try swapping out

So far we have seen the relationship between cache behavior and IO.







Leave a Reply