Tuning the AIX file caches
Introduction
By default, AIX is tuned for a mixed workload, and will grow its VMM file cache up to 80% of physical RAM. While this may be great for an NFS server, SMTP relay or web server, it is very poor for running any application which does its own cache management. This includes most databases (Oracle, DB2, Sybase, PostgreSQL, MySQL using InnoDB tables, TSM) and some other software (eg. the Squid web cache).
Common symptoms include high paging (high pgspin and pgspout in topas), high system CPU time, the lrud kernel thread using CPU, slow overall system throughput, slow backups and slow process startup.
For most database systems, the ideal solution is to use raw logical volumes. If this is not acceptable, then direct I/O and concurrent I/O should be used. If for some reason this is not possible, then the last solution is to tune the AIX file caches to be less aggressive.
Parameters
The four main parameters that should be tuned are the three controlling the size of the persistent file cache (minperm% and maxperm%) used for JFS filesystems, and the client file cache (maxclient%) used by NFS, CDRFS and JFS2 filesystems, and also the lru_file_repage parameter, which influences what pages the VMM page stealing algorithm will steal (present in AIX 5.2 ML4+ and AIX 5.3 ML1+).
- numperm%
- Defines the current size of the persistent file cache.
- minperm%
- Defines the minimum amount of RAM the persistent file cache may occupy. If numperm% is less than or equal to minperm%, file pages will not be stolen when RAM is required.
- maxperm%
- Defines the maximum amount of RAM the persistent file cache may occupy before it is used as the sole source of new pages by the page stealing algorithm. By default, numperm% may exceed maxperm% if there is free memory available. The setting strict_maxperm may be set to one to change maxperm% into a hard limit, guaranteeing numperm% will never exceed maxperm%.
- strict_maxperm
- As above, if set to 1, changes maxperm% into a hard limit.
- numclient%
- Defines the current size of the client file cache.
- maxclient%
- Defines the hard maximum size of the client file cache.
- strict_maxclient
- Introduced in 5.2 ML4, allows the changing of maxclient% into a soft limit, similar to strict_maxperm.
- lru_file_repage
- Introduced in AIX 5.2 ML4 and AIX 5.3 ML1, this influences the VMM page stealing algorithm. If set to 0, the algorithm will strongly prefer stealing file pages to satisfy memory requests.
Note that maxclient% may never exceed maxperm%. In later versions of vmtune, this is enforced by changing both parameters if necessary.
Tuning for AIX 5.1 and Earlier
The tool to use is /usr/samples/kernel/vmtune, installed as part of the bos.adt.samples fileset. If run without options, it will display the currently configured tuneable values, and some of the current runtime values.
Note: vmtume may be used to set the current runtime parameters only. To have changes take effect on reboot, vmtune must be initiated as part of the system startups.
An example of a tuning command used on a system running Oracle may be:
# /usr/samples/kernel/vmtune -p 3 -P 5 -h 1 -t 5
Which sets minperm% to 3%, maxperm% and maxclient% to 5%, and enables strict_maxperm.
Tuning for AIX 5.2 and Later
Note: AIX 5.2 includes a compatibility version of vmtune. It is probably most wise to become familiar with the new tools, instead of relying on the backwards compatibility commands.
The main tool to use is /usr/sbin/vmo, installed as part of the bos.perf.tune fileset. To display current cache sizes (numperm% and numclient%) use vmstat -v.
vmo can change both persistent (reboot) values as well as runtime values, and so does not need to be present in the startups. It stores the persistent values in the /etc/tunables/nextboot file.
Current values and characteristics may be displayed using:
# vmo -L
NAME CUR DEF BOOT MIN MAX UNIT TYPE
DEPENDENCIES
--------------------------------------------------------------------------------
memory_frames 512K 512K 4KB pages S
--------------------------------------------------------------------------------
pinnable_frames 427718 427718 4KB pages S
--------------------------------------------------------------------------------
maxfree 128 128 128 16 200K 4KB pages D
minfree
memory_frames
...
A similar example to the vmtune example above using vmo may be:
# vmo -p -o minperm%=3 -o maxperm%=5 -o strict_maxperm=1 -o maxclient%=5
And if making use of lru_file_repage:
# vmo -p -o minperm%=3 -o maxperm%=90 -o strict_maxperm=1 -o maxclient%=90 -o lru_file_repage=0
To check the current size of the persistent file cache and the client file cache, see the numperm and numclient values reported by vmstat -v:
$ vmstat -v
524288 memory pages
474939 lruable pages
...
10.0 minperm percentage
20.0 maxperm percentage
44.5 numperm percentage
211365 file pages
...
19.7 numclient percentage
20.0 maxclient percentage
94027 client pages
