Exmap - Documentation

Exmap does not build on current systems and should be considered obsolete. The code is still available on github.


Introduction

Exmap is a utility which takes a snapshot of how the physical memory and swap space are currently used by all the processes on your system. It examines which page of memory are shared between which processes, so that it can share the cost of the pages fairly when calculating usage totals.

Some software systems (e.g. the GNOME and KDE desktop environments) consist of dozens of processes, each loading dozens of shared libraries.

Modern operating systems, such as Linux, try to reduce physical memory usage with various techniques. Firstly, binaries (including libraries) are generally "demand paged". This means that only the parts of the library which are actually used by a process get loaded in from disk. Also, two different processes which both load a shared library will share any read-only pages. Lastly, for writable pages, a scheme called "copy on write" is used. This means that potentially-writable pages are initially shared between processes, but when one process writes to a page it is given its own private copy of that page with that change.

Together, these facilities make it very difficult to work out how memory is actually used. Particularly because the Linux kernel doesn't export enough information to user-space to work out which processes share which pages.

Hence Exmap includes a loadable kernel module, exmap.ko, which exports information on the page usage of a process, via a simple interface using an entry in /proc. This information is collated by a userspace process and displayed in a GUI (written with the C++ binding to the GTK+ toolkit, gtkmm).

Compilation

After unpacking the tarball you should run the commands:

make

sudo insmod ./exmap.ko

make test

If all goes well, this you'll see some test output (including some warning messages) and a final line: "All tests passed successfully".

If problems occur, check that your system meet the build and kernel requirements mentioned in the FAQ.

NOTE: kernel module code effectively runs under root privileges. You should apply the same concerns and cautions as you would with running code from any other source as root. In particular, the exmap.ko module allows any process to read the per-page mapping information of any other process. Whilst I can't envisage a way this might be used as a privilege escalation attack, it is probably not wise to run exmap on a multi-user system where unfriendly users are a possibility.

Additionally, buggy code in kernel mode can completely lock your machine, so please don't try this with unsaved work in any applications. You may also want to run the command 'sync' before running exmap. Previous versions of Exmap have caused completely locked machines. Use at your own risk!

Running

From the top-level dir, you can run exmap with the command:

src/gexmap

or

sudo src/gexmap

to run as root.

Running as root may be necessary for accurate numbers on some distributions and/or kernel versions, depending on the file permissions on the /proc/X/maps files on your system.

User Interface

There are two tabs: 'Processes' and 'Files', allowing you to examine the data from different perspectives.

Each of these tabs has four lists, becoming more specific as you move downwards. The processes tab has: a list of all processes, a list of files mapped by the selected process, a list of ELF sections within the selected file and process and finally a list of ELF symbols within the selected section. The files tab is similar, but reverses the roles of file and process in the first two links.

If you select a process and a file which is an ELF file (any shared library or executable) you will see a list of ELF sections in the third list. Selecting an ELF section will cause information on the symbols within that section to be displayed in the bottom list. Note that most system binaries and libraries do not contain any symbols (they have been "stripped"), so the bottom list will generally remain blank unless you are examining binaries you have built yourself. Note again, that you don't have to build binaries with the '-g' option to get this per-symbol information, you just need to avoid running the 'strip' command on the binaries you produce.

Here's a crib sheet to some of the important ELF sections (corrections and additions gratefully received):

Interpreting the numbers

Terminology: although "swapping" originally referred to writing an entire process to the swap file, in this documentation I'll tend to say "paging in" if I mean reading in from a non-swap file (demand paging) and "swapping in" or "swapping out" for reading and writing pages to or from the swap area. Apologies if thats confusing and particularly so if I'm inconsistent.

In general, all the measured items (processes, files, ELF sections, symbols) are given the following sizes:

As an example, a processes which calls malloc(1010241024) will increase its VM size by 10Mbytes. If it doesn't read or write that memory, it probably won't take up any additional swap space or physical RAM.

Note that if processes A and B both load library L and process A has paged in some of that library but B has not accessed those pages, the 'resident' size for those pages will be solely accounted to process A. The resident size for B is only dependent on pages currently mapped and resident by B.

Kernel info: this corresponds to pte_present().

This figure won't vary as a page is swapped in or out (since it remains mapped) but under memory pressure some pages which were mapped (e.g. .text sections containing executable code) may be discarded (since they can just be loaded back in from the backing file). Such discarding of pages will reduce the mapped figure.

There is currently no way to discriminate between unmapped chunks of file-backed address space which have never been touched and those which have been discarded due to memory pressure.

Kernel info: this corresponds to a non-zero pfn or swp_entry.

Kernel info: this page is referenced by only one pte or swp_entry.

So for some items in the UI (e.g. an ELF file) the 'writable' value is a rough approximation to the amount of COW'd memory associated with it. Another way of thinking of it is the amount of memory which has been written to.

Kernel info: this corresponds to pte_write().

Rather than count a full page to each process, the page size is divided amongst all processes which make use of the page.

e.g. if a given (4096 byte) page is in use by 6 processes, each process will get 4096/6 == 682 bytes added to its effective size, rather than the 4096 which is added to the mapped size.

Kernel info: two pages with the same pfn or swp_entry are considered the same.

Feedback

I'm very interested in hearing from anyone who finds exmap useful. If you have any ideas on how to improve it, or find an aspect of it annoying or useful, please let me know. If you have any particular successes with optimising your system, again please let me know.