From dilyan.palauzov at aegee.org Sun Sep 8 09:40:11 2019 From: dilyan.palauzov at aegee.org (=?UTF-8?Q?=D0=94=D0=B8=D0=BB=D1=8F=D0=BD_?= =?UTF-8?Q?=D0=9F=D0=B0=D0=BB=D0=B0=D1=83=D0=B7=D0=BE=D0=B2?=) Date: Sun, 08 Sep 2019 13:40:11 +0000 Subject: Ephemeral files and temporary filesystems Message-ID: Hello, the file docsrc/imap/concepts/deployment/performance_recommendations.rst says: In-memory filesystems are faster then disk filesystems, but are limited in space and volatile as well. The following list includes configuration settings that could make the corresponding databases, files and directory trees be created and maintained on a temporary filesystem. * ``proc_path``: /dev/shm/cyrus-imapd/proc * ``mboxname_lockpath``: /dev/shm/cyrus-imapd/mboxname_lock * ``duplicate_db_path``: /dev/shm/cyrus-imapd/duplicate_db * ``statuscache_db_path``: /dev/shm/cyrus-imapd/statuscache_db ---- lib/imapoptions says: { "prometheus_stats_dir", NULL, STRING } /* Directory to use for gathering prometheus statistics. If specified, must be an absolute path. If not specified, the default path $configdirectory/stats/ will be used. It may be advantageous to locate this directory on ephemeral storage. */ ---- doc/examples/imapd_conf/murder-backend.conf, doc/examples/imapd_conf/murder-frontend.conf, doc/examples/imapd_conf/murder-mupdate.conf, doc/examples/imapd_conf/normal-master.conf, doc/examples/imapd_conf/normal- replica1.conf and doc/examples/imapd_conf/normal.conf say: # Directories for proc and lock files proc_path: /run/cyrus/proc mboxname_lockpath: /run/cyrus/lock # Locations for DB files # The following DB are recreated upon initialization, so should live in # ephemeral storage for best performance. duplicate_db_path: /run/cyrus/deliver.db ptscache_db_path: /run/cyrus/ptscache.db statuscache_db_path: /run/cyrus/statuscache.db tls_sessions_db_path: /run/cyrus/tls_sessions.db ---- docsrc/imap/download/packagers.rst says: Ephemeral files and temporary filesystems ######################################### In addition to Unix domain sockets and lock files, several databases used by Cyrus programs may be located in temporary filesystems, such as those backed by RAM (i.e. tmpfs, md, etc.). Here's a list of such files. In this example, the filesystem ``/run`` is on tmpfs:: proc_path: /run/cyrus/proc mboxname_lockpath: /run/cyrus/lock duplicate_db_path: /run/cyrus/deliver.db statuscache_db_path: /run/cyrus/statuscache.db ptscache_db_path: /run/cyrus/ptscache.db tls_sessions_db_path: /run/cyrus/tls_sessions.db lmtpsocket: /run/cyrus/socket/lmtp idlesocket: /run/cyrus/socket/idle notifysocket: /run/cyrus/socket/notify ---- docsrc/imap/download/upgrade.rst says: You don't need to copy the following databases as Cyrus 3.0 will recreate these for you automatically: * duplicate delivery (deliver.db), * TLS cache (tls_sessions.db), * PTS cache (ptscache.db), * STATUS cache (statuscache.db). .. note:: You may wish to consider relocating these four databases to ephemeral storage, such as ``/run/cyrus`` (Debian/Ubuntu) or ``/var/run/cyrus`` or whatever suitable tmpfs is provided on your distro. It will place less IO load on your disks and run faster. ---- docsrc/imap/reference/admin/locations/configuration-state.rst says: Ephemeral Databases ^^^^^^^^^^^^^^^^^^^ Several of the state databases may be located in RAM-backed locations, such as ``/run`` or ``/var/run`` or where ever your OS roots such filesystems (i.e. ``tmpfs`` on Linux or ``mfs`` on FreeBSD). Cyrus 3.0 will recreate the following databases for you automatically: * duplicate delivery (deliver.db), * TLS cache (tls_sessions.db), * PTS cache (ptscache.db), * STATUS cache (statuscache.db). Relocating these DBs to ephemeral storage will place less IO load on your disks and tend to run faster. ---- Where is the truth? What is the complete list of things, that will benefit on being on a tmpfs? On one place or another, STATUS cache, TLS cache, PTS cache, duplicate delivery, proc_path, mboxname_path, prometheus_stats_dir are mentioned as candidates. Do sockets (lmtpsocket) benefit from being on tmpfs, as only download/packagers.rst suggests ? Putting on a single place all candidates for tmpfs will be good, and using consistently /dev/shm or /run or /var/run to mean tmpfs in all documentation will also be good. Greetings ????? From matt at mattkeenan.net Sun Sep 8 10:43:56 2019 From: matt at mattkeenan.net (Matt) Date: Sun, 08 Sep 2019 15:43:56 +0100 Subject: Ephemeral files and temporary filesystems In-Reply-To: References: Message-ID: <9795BFE3-B827-4490-8406-4B4B5D089227@mattkeenan.net> Hi Dilyan, Although it is very common that /run is mounted as a tmpfs filesystem it is not universal; even more so on Linux systems that don't run systemd. And there are systems that don't run Linux. Also the semantics of SYSV are generally better defined on different platforms. Matt On 8 September 2019 14:40:11 GMT+01:00, "????? ????????" wrote: >Hello, > >the file >docsrc/imap/concepts/deployment/performance_recommendations.rst says: > >In-memory filesystems are faster then disk filesystems, but are limited >in space and volatile as well. The following >list includes configuration settings that could make the corresponding >databases, files and directory trees be created >and maintained on a temporary filesystem. > >* ``proc_path``: /dev/shm/cyrus-imapd/proc >* ``mboxname_lockpath``: /dev/shm/cyrus-imapd/mboxname_lock >* ``duplicate_db_path``: /dev/shm/cyrus-imapd/duplicate_db >* ``statuscache_db_path``: /dev/shm/cyrus-imapd/statuscache_db > >---- >lib/imapoptions says: >{ "prometheus_stats_dir", NULL, STRING } >/* Directory to use for gathering prometheus statistics. If specified, > must be an absolute path. If not specified, the default path >$configdirectory/stats/ will be used. It may be advantageous to locate >this > directory on ephemeral storage. */ > >---- >doc/examples/imapd_conf/murder-backend.conf, >doc/examples/imapd_conf/murder-frontend.conf, >doc/examples/imapd_conf/murder-mupdate.conf, >doc/examples/imapd_conf/normal-master.conf, >doc/examples/imapd_conf/normal- >replica1.conf and doc/examples/imapd_conf/normal.conf say: > ># Directories for proc and lock files >proc_path: /run/cyrus/proc >mboxname_lockpath: /run/cyrus/lock > ># Locations for DB files ># The following DB are recreated upon initialization, so should live in ># ephemeral storage for best performance. >duplicate_db_path: /run/cyrus/deliver.db >ptscache_db_path: /run/cyrus/ptscache.db >statuscache_db_path: /run/cyrus/statuscache.db >tls_sessions_db_path: /run/cyrus/tls_sessions.db > >---- >docsrc/imap/download/packagers.rst says: >Ephemeral files and temporary filesystems >######################################### > >In addition to Unix domain sockets and lock files, several databases >used by Cyrus programs may be located in temporary filesystems, such as >those backed by RAM (i.e. tmpfs, md, etc.). Here's a list of such >files. In this example, the filesystem ``/run`` is on tmpfs:: > > proc_path: /run/cyrus/proc > mboxname_lockpath: /run/cyrus/lock > duplicate_db_path: /run/cyrus/deliver.db > statuscache_db_path: /run/cyrus/statuscache.db > ptscache_db_path: /run/cyrus/ptscache.db > tls_sessions_db_path: /run/cyrus/tls_sessions.db > lmtpsocket: /run/cyrus/socket/lmtp > idlesocket: /run/cyrus/socket/idle > notifysocket: /run/cyrus/socket/notify > >---- >docsrc/imap/download/upgrade.rst says: >You don't need to copy the following databases as Cyrus 3.0 will >recreate these for you automatically: > >* duplicate delivery (deliver.db), >* TLS cache (tls_sessions.db), >* PTS cache (ptscache.db), >* STATUS cache (statuscache.db). > >.. note:: > You may wish to consider relocating these four databases to ephemeral > storage, such as ``/run/cyrus`` (Debian/Ubuntu) or ``/var/run/cyrus`` > or whatever suitable tmpfs is provided on your distro. It will place > less IO load on your disks and run faster. > >---- >docsrc/imap/reference/admin/locations/configuration-state.rst says: > >Ephemeral Databases >^^^^^^^^^^^^^^^^^^^ > >Several of the state databases may be located in RAM-backed locations, >such as ``/run`` or ``/var/run`` or where ever your OS roots such >filesystems (i.e. ``tmpfs`` on Linux or ``mfs`` on FreeBSD). Cyrus 3.0 >will >recreate the following databases for you automatically: > >* duplicate delivery (deliver.db), >* TLS cache (tls_sessions.db), >* PTS cache (ptscache.db), >* STATUS cache (statuscache.db). > >Relocating these DBs to ephemeral storage will place less IO load on >your disks and tend to run faster. > >---- >Where is the truth? What is the complete list of things, that will >benefit on being on a tmpfs? On one place or >another, STATUS cache, TLS cache, PTS cache, duplicate delivery, >proc_path, mboxname_path, prometheus_stats_dir are >mentioned as candidates. > >Do sockets (lmtpsocket) benefit from being on tmpfs, as only >download/packagers.rst suggests ? > >Putting on a single place all candidates for tmpfs will be good, and >using consistently /dev/shm or /run or /var/run to >mean tmpfs in all documentation will also be good. > >Greetings > ????? -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: