/proc/locks debug helper (cyrus deadlock tracing)

Thomas Jarosch thomas.jarosch at intra2net.com
Fri Jul 24 03:15:03 EDT 2015


Hello,

attached is a small helper script that parses
the output of /proc/locks. The file format is described here:
https://www.centos.org/docs/5/html/5.2/Deployment_Guide/s2-proc-locks.html

It resolves the program name from the pid number and also
resolves the filename from the inode number.

That way we can see which program is currently holding an exclusive
lock on a file and which program is waiting for the lock to come free.
This would probably also show an ABBA deadlock,
though I didn't encounter one in the wild yet.

You may need to adapt the paths in the "config" section of the program
to your local setup. 

The layout of /proc/locks might change depending on the kernel version,
I've tested it with 3.14.x. The script works with python 3.x and 2.7.

When you call it with "--shown-non-waiting", it will show you
which mailboxes currently hold a READ lock. The program does a bit
of filtering to keep the noise ratio down, so be sure to check
out the other options described in "--help".

Hopefully this helps others if they should run into a deadlock situation.

Cheers,
Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parse_proc_locks.py
Type: text/x-python
Size: 4461 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20150724/803308c7/attachment-0001.py 


More information about the Cyrus-devel mailing list