mmap Introduction

on mmap, optimization, IO, memory, and performance

mmap is essential tool for performance optimization, this article simply lists related information.

What is mmap

Simple popular imprecise saying, mmap can directly establish mapping between memory and file, process’s modification to memory can directly sync to file content modification, only one copy process from disk to memory.

This is Wikipedia’s explanation:

In computing, mmap(2) is a POSIX-compliant Unix system call that maps files or devices into memory. It is a method of memory-mapped file I/O. It implements demand paging, because file contents are not read from disk initially and do not use physical RAM at all. The actual reads from disk are performed in a “lazy” manner, after a specific location is accessed.

This is a Chinese explanation (not translation of above):

mmap is a method of memory-mapped files, that is mapping a file or other object to process’s address space, implementing one-to-one correspondence between file disk address and a virtual address segment in process’s virtual address space. After implementing such mapping relationship, process can use pointer method to read and write this memory segment, and system will automatically write back dirty pages to corresponding file disk, that is completed file operation without needing to call read, write and other system call functions. Conversely, kernel space’s modification to this region also directly reflects user space, thus can implement file sharing between different processes.

Principle

Recommend reading this article “Carefully Analyze mmap: What Why How” https://www.cnblogs.com/huxiao-tee/p/4660352.html

Difference from read/write

  1. read/write and other conventional file operations, need page cache as intermediary between kernel and application layer, so one file operation needs two data copies (kernel to page cache, page cache to application layer), while mmap saves page cache intermediary, one less data copy.
  2. mmap when memory insufficient, process Crash will automatically write back to disk. (Because of this, WeChat xlog can ensure log won’t be lost)

More information reference “File Read Write Process from Kernel File System Perspective” http://www.cnblogs.com/huxiao-tee/p/4657851.html

Usage

Specific method won’t elaborate, still reference article in “Principle” above.

pagesize

Always thought page size is 4K, but starting from arm64 is already 16K.

https://forums.developer.apple.com/thread/47532

Implementation in xnu

Example Code

mmap usage method can directly reference AppleTrace’s code https://github.com/everettjf/AppleTrace/blob/master/appletrace/appletrace/src/appletrace.mm

pagesize test code https://github.com/everettjf/Yolo/tree/master/BukuzaoArchive/sample/MmapSample/MmapSample/FastRead.m

References

  1. https://en.wikipedia.org/wiki/Mmap
  2. https://en.wikipedia.org/wiki/Fork_(system_call)
  3. https://www.safaribooksonline.com/library/view/linux-system-programming/0596009585/ch04s03.html
  4. http://www.tutorialspoint.com/unix_system_calls/mmap.htm
  5. https://en.wikipedia.org/wiki/Virtual_memory
  6. https://en.wikipedia.org/wiki/Virtual_address_space
  7. https://satanwoo.github.io/2017/07/30/xlog/
  8. http://blog.desmondyao.com/mars-xlog/

Summary

mmap is a powerful tool. Deeper principles need to analyze kernel implementation, Emmmmmm… Too inexperienced, still need to learn.

Welcome to follow subscription account “Client Technology Review”: happyhackingstudio