2020-11-17 virtual memory limits

Just spent too much time trying to figure out why my mediawiki thumbnail generation using ImageMagick convert(1) was failing. After enabling mediawiki debugging via:

$wgDebugLogFile = "/tmp/debug-{$wgDBname}.log";
$wgShowExceptionDetails = true;

The error was singularly uninformative, with an apparently different, random library appearing on each attempt:

[thumbnail] thumbnail failed on www.stix.id.au: error 1 "/usr/lib/libgssapi.so.11: Shared object "libkrb5.so.27" not found" from "'/usr/pkg/bin/convert' '-quality' '95' …

The clue was uncovered by ktrace -dip <pid>, on the parent apache process. This uncovered the real error:

   581      1 convert  CALL  close(3)                                                                                                  
   581      1 convert  RET   close 0                                                                                                   
   581      1 convert  CALL  open(0x7f7fff9ed3f8,0,0x63)                                                                               
   581      1 convert  NAMI  "/lib/libcrypt.so.1"                                                                                      
   581      1 convert  RET   open 3                                                                                                    
   581      1 convert  CALL  __fstat50(3,0x7f7fff9ed2f8)                                                                               
   581      1 convert  RET   __fstat50 0                                                                                               
   581      1 convert  CALL  mmap(0,0x1000,PROT_READ,0x1<SHARED,FILE,ALIGN=NONE>,3,0,0)                                                
   581      1 convert  RET   mmap 135607257300992/0x7b5586668000                                                                       
   581      1 convert  CALL  munmap(0x7b5586668000,0x1000)                                                                             
   581      1 convert  RET   munmap 0                                                                                                  
   581      1 convert  CALL  mmap(0,0x20a000,PROT_READ|PROT_EXEC,0x15000002<PRIVATE,FILE,ALIGN=2MB>,3,0,0)                             
   581      1 convert  RET   mmap -1 errno 12 Cannot allocate memory                                                                   
   581      1 convert  CALL  close(3)                                                                                                  
   581      1 convert  RET   close 0

So it turns out that mediawiki limits the virtual memory of sub-processes by default, using a simple wrapper around ulimit (or cgroup magic where supported). Sure enough, on my NetBSD system, the default 300MiB is insufficient.

ksh$ ulimit -v $((300*1024))
ksh$ ulimit -a
time(cpu-seconds)    unlimited
file(blocks)         unlimited
coredump(blocks)     unlimited
data(kbytes)         1048576
stack(kbytes)        4096
lockedmem(kbytes)    5401636
memory(kbytes)       16204908
nofiles(descriptors) 1024
processes            512
threads              1024
vmemory(kbytes)      307200
sbsize(bytes)        unlimited
ksh$ /usr/pkg/bin/convert -?
/usr/lib/libgssapi.so.11: Shared object "libroken.so.20" not found

Bumping the default in LocalSettings.php fixed the issue for me.

# $wgMaxShellMemory = 307200;
$wgMaxShellMemory = 524288;