From clarra at cc.upv.es Mon Jun 3 08:01:27 2019 From: clarra at cc.upv.es (=?UTF-8?Q?Carlos_Larra=c3=b1aga?=) Date: Mon, 3 Jun 2019 14:01:27 +0200 Subject: backupd IOERROR reading backup files larger than 2GB Message-ID: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> Hi, We're testing backup feature un cyrus-imapd 3.0.10. There's no problem when backup is created first time, but when the backup already exists and is larger than 2GB, we get the following error from the backupd: cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or directory As said, it happens only when then backup file already exists and is larger than 2GB. The backupd process keeps reading de compressed backup file til sync_client exit with "Error from sync_do_user(xxx): bailing out!". We use xfs with latest CentOs. Seems like backupd is unable to read the compressed backup file. Anyone else with this problem? Any idea how to fix it? Thanks in advance for your help. Best regards, Carlos. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3627 bytes Desc: Firma criptogr??fica S/MIME URL: From web at 3dresearch.com Tue Jun 4 13:16:38 2019 From: web at 3dresearch.com (Janos Dohanics) Date: Tue, 4 Jun 2019 13:16:38 -0400 Subject: Open idle sockets Message-ID: <20190604131638.d4d45dba672bac3955c41d53@3dresearch.com> Hello, This is a FreeBSD 12.0-STABLE r345381 system running cyrus-imapd30-3.0.9_1 built with these options: ===> The following configuration options are available for cyrus-imapd30-3.0.9_1: AUTOCREATE=off: Enable autocreate support BACKUP=off: Enable backup support (experimental) CLAMAV=off: Use ClamAV DOCS=off: Build and/or install documentation HTTP=off: Enable HTTP support IDLED=on: Enable IMAP idled support LDAP=off: Enable LDAP support (experimental) LMDB=off: Use LMDB backend MURDER=off: Enable IMAP Murder support MYSQL=off: MySQL database support NNTP=off: Enable NNTP support PGSQL=off: PostgreSQL database support REPLICATION=off: Enable replication (experimental) SNMP=off: SNMP network protocol support SQLITE=on: SQLite database support SQUAT=on: Enable Squat support XAPIAN=off: Enable Xapian support ====> GSSAPI Security API support: you can only select none or one of them GSSAPI_HEIMDAL=off: GSSAPI support via security/heimdal GSSAPI_MIT=off: GSSAPI support via security/krb5 GSSAPI_BASE=on: GSSAPI support via base system (needs Kerberos) ====> Command line editing via libreadline: you can only select none or one of them READLINE_GNU=off: Use Term::Readline::GNU for cyradm READLINE_PERL=off: Use Term::Readline::Perl for cyradm ===> Use 'make config' to modify these settings I have disabled idled in cyrus.conf, but sockstat(1) reports many open sockets: # sockstat | grep idle cyrus pop3d 38399 6 dgram /zroot/data/cyrus/imap/socket/idle.38399 cyrus imapd 38359 6 dgram /zroot/data/cyrus/imap/socket/idle.38359 cyrus imapd 38351 6 dgram /zroot/data/cyrus/imap/socket/idle.38351 cyrus imapd 38350 6 dgram /zroot/data/cyrus/imap/socket/idle.38350 cyrus imapd 38349 6 dgram /zroot/data/cyrus/imap/socket/idle.38349 cyrus imapd 38328 6 dgram /zroot/data/cyrus/imap/socket/idle.38328 cyrus imapd 38325 6 dgram /zroot/data/cyrus/imap/socket/idle.38325 cyrus imapd 38065 6 dgram /zroot/data/cyrus/imap/socket/idle.38065 cyrus imapd 38062 6 dgram /zroot/data/cyrus/imap/socket/idle.38062 cyrus imapd 37980 6 dgram /zroot/data/cyrus/imap/socket/idle.37980 cyrus imapd 37870 6 dgram /zroot/data/cyrus/imap/socket/idle.37870 cyrus imapd 37868 6 dgram /zroot/data/cyrus/imap/socket/idle.37868 cyrus imapd 37785 6 dgram /zroot/data/cyrus/imap/socket/idle.37785 cyrus imapd 37780 6 dgram /zroot/data/cyrus/imap/socket/idle.37780 cyrus imapd 37769 6 dgram /zroot/data/cyrus/imap/socket/idle.37769 cyrus imapd 8538 6 dgram /zroot/data/cyrus/imap/socket/idle.8538 cyrus imapd 1636 6 dgram /zroot/data/cyrus/imap/socket/idle.1636 cyrus imapd 1632 6 dgram /zroot/data/cyrus/imap/socket/idle.1632 Should there be any open idle sockets if idled is disabled and not running? -- Janos Dohanics From ellie at fastmail.com Wed Jun 5 22:04:51 2019 From: ellie at fastmail.com (ellie timoney) Date: Thu, 06 Jun 2019 12:04:51 +1000 Subject: backupd IOERROR reading backup files larger than 2GB In-Reply-To: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> References: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> Message-ID: It kinda sounds like your platform might be 32bit? Or your zlib is compiled to use 32bit integer sizes? On Mon, Jun 3, 2019, at 10:07 PM, Carlos Larra?aga wrote: > Hi, > > We're testing backup feature un cyrus-imapd 3.0.10. There's no problem when backup is created first time, but when the backup already exists and is larger than 2GB, we get the following error from the backupd: >> cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or directory > As said, it happens only when then backup file already exists and is larger than 2GB. The backupd process keeps reading de compressed backup file til sync_client exit with "Error from sync_do_user(xxx): bailing out!". > > We use xfs with latest CentOs. Seems like backupd is unable to read the compressed backup file. > > Anyone else with this problem? Any idea how to fix it? > Thanks in advance for your help. > > Best regards, > Carlos. > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > *Attachments:* > * smime.p7s -------------- next part -------------- An HTML attachment was scrubbed... URL: From ismael.tanguy at univ-brest.fr Thu Jun 6 06:21:13 2019 From: ismael.tanguy at univ-brest.fr (=?UTF-8?Q?Isma=c3=abl_Tanguy?=) Date: Thu, 6 Jun 2019 12:21:13 +0200 Subject: Deleting seen state file keeps unstable seen state for mailbox Message-ID: <25209a1f-5846-2bb6-8dec-623a4c5ef408@univ-brest.fr> Hello cyrus list, due to a careless mistake, our global NFS pool became full last Monday. The mail data storage (/var/spool/imap) is a pool of this global pool. /var/lib/imap is stored somewhere else During this outage of five minutes, thousand of mailboxes undergo IOERROR. This mailboxes has been reconstruted immediately after. But a problem remains, for some users, the seen state of their mailboxes is inconstant. So, we tried to repair this with skiplist.py: https://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-July/029375.html http://oss.netfarm.it/python-cyrus.php That did'nt work for some users, (but works for some, weird!) We then delete the seen state file, but the problem still remains. Is there anywhere else that the seen state of a mailbox is kept? Thank four your help. Isma?l TANGUY -- ------------------ From jayesh.shinde at netcore.co.in Thu Jun 6 10:20:35 2019 From: jayesh.shinde at netcore.co.in (Jayesh Shinde) Date: Thu, 6 Jun 2019 19:50:35 +0530 Subject: How to exclude "Archived" email from Quota usage ? Message-ID: Hi , I am testing Archiving feature with Cyrus-imapd-3.0.9 . Is it possible to exclude the mailbox quota for "Archived emails" ( which is in different partition ) and only consider mailbox quota of default partition ? OR Is it possible to know how much quota used for end user's mailbox in Archive partition ? By listquota command I am able to view overall Mailbox Quota. Thank You Regards Jayesh Shinde https://ps1.saymails.com/client_customize/raman/reg_lp.php?utm_source=email-disclaimer&utm_medium=email&utm_campaign=beatraman -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.jpg Type: image/jpeg Size: 33120 bytes Desc: not available URL: From clarra at cc.upv.es Thu Jun 6 11:57:52 2019 From: clarra at cc.upv.es (=?UTF-8?Q?Carlos_Larra=c3=b1aga?=) Date: Thu, 6 Jun 2019 17:57:52 +0200 Subject: backupd IOERROR reading backup files larger than 2GB In-Reply-To: References: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> Message-ID: Hi Ellie, Thanks for answering. We use latest 64bit Oracle Linux (not CentOs like I said before, sorry) and zlib is also 64bit version: # uname -a Linux xxx 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 17:35:45 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux # yum list installed zlib Loaded plugins: langpacks, ulninfo Installed Packages zlib.x86_64 1.2.7-18.el7????????????????????? @ol7_latest # lsof -p 17005 |grep lib |grep -i z backupd 17005 cyrus? mem??? REG????????????? 253,0????? 90248 134400930 /usr/lib64/libz.so.1.2.7 # file /usr/lib64/libz.so.1.2.7 /usr/lib64/libz.so.1.2.7: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=b9d5f73428bd6ad68c96986b57bea3b7cedb9745, stripped # rpm -qf /usr/lib64/libz.so.1.2.7 zlib-1.2.7-18.el7.x86_64 I have summarized here some information about the backupd error reading the file descriptor 15, which is a backup larger than 2GB. The error is logged 1755 times. Instead, there is no error for fd 12, which is a backup of less than 2 GB: *#-- CLIENT SIDE ****-----------------------------------------------*** # time sync_client -v -o -n backup -A Thu Jun? 6 17:08:02 CEST 2019 USER aaa QUOTA user.aaa USER ccc Error from do_user(ccc): bailing out! real??? 30m16.223s user??? 0m0.006s sys???? 0m0.006s * * **#-- BACKUP SERVER SIDE ---------------------------------------- *# LOGFILE *# tail -f /var/log/imapd.log Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory *# lsof of the backupd PROCESS *# lsof -P -p 2584 COMMAND? PID? USER?? FD?? TYPE???????????? DEVICE SIZE/OFF??????? NODE NAME backupd 2584 cyrus? cwd??? DIR????????????? 253,0 4096???????? 128 / backupd 2584 cyrus? rtd??? DIR????????????? 253,0 4096???????? 128 / backupd 2584 cyrus? txt??? REG????????????? 253,0 768632?? 939542214 /usr/local/cyrus/libexec/backupd backupd 2584 cyrus? mem??? REG????????????? 253,0 37208?? 135746334 /usr/lib64/libnss_sss.so.2 backupd 2584 cyrus? mem??? REG????????????? 253,0 31416?? 134406707 /usr/lib64/libnss_dns-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 61632?? 134406709 /usr/lib64/libnss_files-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 43336??? 68814977 /usr/lib64/sasl2/webmail.so.0.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 57960??? 68856627 /usr/lib64/sasl2/libdigestmd5.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 24232??? 68797220 /usr/lib64/sasl2/libcrammd5.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 20088??? 68797044 /usr/lib64/sasl2/libplain.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 20056??? 68797036 /usr/lib64/sasl2/liblogin.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 1845816?? 135873512 /usr/lib64/libdb-5.3.so backupd 2584 cyrus? mem??? REG????????????? 253,0 28272??? 67113426 /usr/lib64/sasl2/libsasldb.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 20064??? 67113423 /usr/lib64/sasl2/libanonymous.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 11448?? 135872733 /usr/lib64/libfreebl3.so backupd 2584 cyrus? mem??? REG????????????? 253,0 155784?? 134400716 /usr/lib64/libselinux.so.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 15464?? 134401447 /usr/lib64/libkeyutils.so.1.5 backupd 2584 cyrus? mem??? REG????????????? 253,0 40672?? 134399432 /usr/lib64/libcrypt-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 105824?? 134781819 /usr/lib64/libresolv-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 53944?? 134406785 /usr/lib64/libjansson.so.4.10.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 88776?? 135352942 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 1137032?? 134400337 /usr/lib64/libm-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 991616?? 135352396 /usr/lib64/libstdc++.so.6.0.19 backupd 2584 cyrus? mem??? REG????????????? 253,0 19296?? 134399434 /usr/lib64/libdl-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 142008?? 134722750 /usr/lib64/libpthread-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 2151704?? 134365659 /usr/lib64/libc-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 753232?? 134401161 /usr/lib64/libsqlite3.so.0.8.6 *backupd 2584 cyrus? mem??? REG????????????? 253,0 90248?? 134400930 /usr/lib64/libz.so.1.2.7* backupd 2584 cyrus? mem??? REG????????????? 253,0 11128?? 134400947 /usr/lib64/libpcreposix.so.0.0.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 402384?? 134400940 /usr/lib64/libpcre.so.1.2.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 67104?? 135269427 /usr/lib64/libkrb5support.so.0.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 15920?? 135873508 /usr/lib64/libcom_err.so.2.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 210832?? 134781960 /usr/lib64/libk5crypto.so.3.1 backupd 2584 cyrus? mem??? REG????????????? 253,0 967864?? 134781967 /usr/lib64/libkrb5.so.3.3 backupd 2584 cyrus? mem??? REG????????????? 253,0 320400?? 134781952 /usr/lib64/libgssapi_krb5.so.2.2 backupd 2584 cyrus? mem??? REG????????????? 253,0 115856?? 134400341 /usr/lib64/libnsl-2.17.so backupd 2584 cyrus? mem??? REG????????????? 253,0 42168?? 134401419 /usr/lib64/libwrap.so.0.7.6 backupd 2584 cyrus? mem??? REG????????????? 253,0 2516640?? 134400708 /usr/lib64/libcrypto.so.1.0.2k backupd 2584 cyrus? mem??? REG????????????? 253,0 470360?? 134400711 /usr/lib64/libssl.so.1.0.2k backupd 2584 cyrus? mem??? REG????????????? 253,0 121208?? 134401258 /usr/lib64/libsasl2.so.3.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 2898664?? 838888996 /usr/local/cyrus/lib/libcyrus_imap.so.0.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 599528?? 838888999 /usr/local/cyrus/lib/libcyrus_sieve.so.0.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 20112?? 134406704 /usr/lib64/libuuid.so.1.3.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 20781704?? 136182553 /usr/lib64/libicudata.so.50.1.2 backupd 2584 cyrus? mem??? REG????????????? 253,0 1539544?? 136197152 /usr/lib64/libicuuc.so.50.1.2 backupd 2584 cyrus? mem??? REG????????????? 253,0 553264?? 838890012 /usr/local/cyrus/lib/libcyrus_min.so.0.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 1961968?? 838888994 /usr/local/cyrus/lib/libcyrus.so.0.0.0 backupd 2584 cyrus? mem??? REG????????????? 253,0 163408?? 134355369 /usr/lib64/ld-2.17.so backupd 2584 cyrus??? 0u? IPv4??????????? 2092952 0t0???????? TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) backupd 2584 cyrus??? 1u? IPv4??????????? 2092952 0t0???????? TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) backupd 2584 cyrus??? 2u? IPv4??????????? 2092952 0t0???????? TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) backupd 2584 cyrus??? 3w? FIFO??????????????? 0,9 0t0???? 2141764 pipe backupd 2584 cyrus??? 4u? IPv4??????????? 2141763 0t0???????? TCP hostb1.upv.es:2005 (LISTEN) backupd 2584 cyrus??? 5u?? REG 0,20????????? 0?????? 86644 /run/cyrus-b1/socket/backupd-0.lock backupd 2584 cyrus??? 6r? FIFO??????????????? 0,9 0t0???? 2141754 pipe backupd 2584 cyrus??? 7w? FIFO??????????????? 0,9 0t0???? 2141754 pipe backupd 2584 cyrus??? 8r? FIFO??????????????? 0,9 0t0???? 2141755 pipe backupd 2584 cyrus??? 9w? FIFO??????????????? 0,9 0t0???? 2141755 pipe backupd 2584 cyrus?? 10u? unix 0xffff9d9df53d4400 0t0???? 2093417 socket backupd 2584 cyrus?? 11w?? REG????????????? 253,5 680 18253611600 /b1/lib/log/admin/backupd-2584 backupd 2584 cyrus?? 12uW? REG????????????? 253,5 976281587 13958644353 /b1/bck/b1/a/aaa_ouuUx9 backupd 2584 cyrus?? 13ur? REG????????????? 253,5 3573760 13958644354 /b1/bck/b1/a/aaa_ouuUx9.index backupd 2584 cyrus?? 14u?? REG????????????? 253,5 2576 13958644355 /b1/bck/b1/a/aaa_ouuUx9.index-journal *backupd 2584 cyrus?? 15uW? REG????????????? 253,5 3104341527 12884902595 /b1/bck/b1/c/ccc_Mqmymx** **backupd 2584 cyrus?? 16u?? REG 253,5?? 47927296 12884902596 /b1/bck/b1/c/ccc_Mqmymx.index* *# BACKUP SIZES* # ls -lh /b1/bck/b1/a/* /b1/bck/b1/c/* *-rw------- 1 cyrus mail 932M Jun? 6 17:08 **/b1/bck/b1/a/**aaa_ouuUx9* -rw------- 1 cyrus mail 3.5M Jun? 6 16:46 /b1/bck/b1/a/aaa_ouuUx9.index -rw------- 1 cyrus mail 2.6K Jun? 6 17:08 /b1/bck/b1/a/aaa_ouuUx9.index-journal *-rw------- 1 cyrus mail 2.9G Jun? 6 17:01 /b1/bck/b1/c/ccc_Mqmymx* -rw------- 1 cyrus mail? 46M Jun? 6 17:01 /b1/bck/b1/c/ccc_Mqmymx.index *# IMAP SESSION DEBUG* # cat /b1/lib/log/admin/backupd-2584 ---------- admin Thu Jun? 6 17:08:02 2019 <15598336821559833682>OK DEFLATE active <15598336821559833698>* MAILBOX %(UNIQUEID acd014aa-7b1d-43c9-b1d3-f97b06540739 MBOXNAME user.aaa MBOXTYPE NIL LAST_UID 0 HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 UIDVALIDITY 1520512152 PARTITION default ACL "aaa lrswipkxtecdan? group:super???? lrswipkxtecdan? " OPTIONS P SYNC_CRC 0 SYNC_CRC_ANNOT 0 QUOTAROOT NIL XCONVMODSEQ 0) OK Success <15598336981559833698>OK Success <1559833698 It kinda sounds like your platform might be 32bit? Or your zlib is compiled to > use 32bit integer sizes? > > On Mon, Jun 3, 2019, at 10:07 PM, Carlos Larra?aga wrote: >> Hi, >> >> We're testing backup feature un cyrus-imapd 3.0.10. There's no problem when >> backup is created first time, but when the backup already exists and is >> larger than 2GB, we get the following error from the backupd: >> >> cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or directory >> >> As said, it happens only when then backup file already exists and is larger >> than 2GB. The backupd process keeps reading de compressed backup file til >> sync_client exit with "Error from sync_do_user(xxx): bailing out!". >> >> We use xfs with latest CentOs. Seems like backupd is unable to read the >> compressed backup file. >> >> Anyone else with this problem? Any idea how to fix it? >> Thanks in advance for your help. >> >> Best regards, >> Carlos. >> ---- >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> >> *Attachments:* >> >> * smime.p7s >> > > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3627 bytes Desc: Firma criptogr??fica S/MIME URL: From ismael.tanguy at univ-brest.fr Tue Jun 11 10:21:00 2019 From: ismael.tanguy at univ-brest.fr (=?UTF-8?Q?Isma=c3=abl_Tanguy?=) Date: Tue, 11 Jun 2019 16:21:00 +0200 Subject: Deleting seen state file keeps unstable seen state for mailbox In-Reply-To: <25209a1f-5846-2bb6-8dec-623a4c5ef408@univ-brest.fr> References: <25209a1f-5846-2bb6-8dec-623a4c5ef408@univ-brest.fr> Message-ID: <5618ecc6-8c50-5894-338f-ae52282a3671@univ-brest.fr> Hello we're still stuck with inconstant state file. This problem happen with differents clients (Thunderbid, Apple Mail, Iphone, webmail,..). We have just make a tcpdump between a client and the server. At the beginning of the capture, the client marks a mail with Seen Flag (line 1&2). Then, he moves the mail to the Trash (line 7 & 8). Thenseveral of his Seen mails is marked back as Unseen. I'm not sure that this appears in this capture, doesn't it? Also, I'm not an expert of the IMAP protocol, are the lines 10, 13, 19 OK? The numeric tag (2472, 2477) doesn't answer to client's tag and all come from the server. Is this the standard behavior? ---------------------------------------------- N??? Source? Dest ?? Length 1??? client??? server??? 90 Request: 24 uid store 158136 +Flags (\Seen) 2??? server??? client??? 71??? ??? ?? Response: 24 OK Completed 3??? client??? server??? 63??? ??? ?? Request: 25 IDLE 4??? server??? client??? 64??? ??? ?? Response: + idling 5??? client??? server??? 60??? ??? ?? Request: DONE 6??? server??? client??? 71??? ??? ?? Response: 25 OK Completed 7??? client??? server??? 88??? ?????? Request: 27 uid copy 158136 "INBOX.Trash" 8??? server??? client??? 106??? ???? Response: 27 OK [COPYUID 1125493985 158136 157935] C 9??? client??? server??? 99??? ?????? Request: 28 uid store 158136 +FLAGS (\Deleted \Seen 10??? server??? client??? 2974??? Response: * 2472 FETCH (FLAGS (\Recent \Deleted \See 11??? server??? client??? 148????? Response: d NonJunk) UID 158169) 12??? client??? server??? 77??? ??? Request: 29 uid expunge 158136 13??? server??? client??? 115????? Response: * 2472 EXPUNGE 14??? client??? server??? 63??? ??? Request: 30 noop 15??? server??? client??? 71??? ??? Response: 30 OK Completed 16??? client??? server??? 79??? ??? Request: 31 getquotaroot "INBOX" 17??? server??? client??? 147??? ? Response: * QUOTAROOT INBOX user.bob 18??? client??? server??? 85??? ??? Request: 32 UID fetch 158174:* (FLAGS) 19??? server??? client??? 136??? ? Response: * 2477 FETCH (FLAGS (\Recent \Answered) UI 20??? client??? server??? 63??? ??? Request: 33 IDLE 21??? server??? client??? 64??? ??? Response: + idling ------------------------------------- Thank for your help, Isma?l Tanguy ------------------------------------- Le 06/06/2019 ? 12:21, Isma?l Tanguy a ?crit?: > Hello cyrus list, > > due to a careless mistake, our global NFS pool became full last Monday. > The mail data storage (/var/spool/imap) is a pool of this global pool. > /var/lib/imap is stored somewhere else > > During this outage of five minutes, thousand of mailboxes undergo > IOERROR. > This mailboxes has been reconstruted immediately after. > > But a problem remains, for some users, the seen state of their > mailboxes is inconstant. > So, we tried to repair this with skiplist.py: > > https://lists.andrew.cmu.edu/pipermail/info-cyrus/2008-July/029375.html > http://oss.netfarm.it/python-cyrus.php > > That did'nt work for some users, (but works for some, weird!) > We then delete the seen state file, but the problem still remains. > > Is there anywhere else that the seen state of a mailbox is kept? > > Thank four your help. > > Isma?l TANGUY > > -- ------------------ > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus From sven.schwedas at tao.at Wed Jun 12 10:20:39 2019 From: sven.schwedas at tao.at (Sven Schwedas) Date: Wed, 12 Jun 2019 16:20:39 +0200 Subject: LDAP auth and ptloader In-Reply-To: <0c0acdee-a654-4b67-a23a-f58aa58d6672@www.fastmail.com> References: <190cb420-4836-1f98-36f9-70e6bde97629@tao.at> <0c0acdee-a654-4b67-a23a-f58aa58d6672@www.fastmail.com> Message-ID: <4b541906-b785-7397-6f6c-96e4b0f77dd4@tao.at> Sorry for the delay, I was busy with other projects. :/ On 26.04.19 10:03, ellie timoney wrote: > Hi Sven, > > I don't know much about running it in a production capacity, but our > test suite sets up the following for LDAP pts: > > imapd.conf: > ? ?... > ? ?ptloader_sock: /path/to/some/socket > ? ?auth_mech: pts > ? ?pts_module: ldap > ? ?... > > cyrus.conf: > ? ?SERVICES { > ? ? ? ... > ? ? ? ptloader cmd="ptloader" listen="/path/to/some/socket" > ? ? ? ... ?? > ? ?} > > Does this get you going? It starts now, and according to the log, ptloader is initialized, but it doesn't find any LDAP groups, and I can't really figure out why ? it just silently fails to find any groups (so users can't access shared folders), with no indication in the logs as to why, even with debug/chatty both enabled. Groups *do* work with pts disabled and libpam-winbind resolving them as native groups, so they *should* be set up correctly, I think. Relevant settings: > # These make no difference > #debug: 1 > #chatty: 1 > > # Same as in sample, path correct > #auth_mech: pts > pts_module: ldap > ptloader_sock: /var/run/cyrus/socket/pts > > # Work, verified with s_client > ldap_uri: ldaps://graz-dc-sem.ad.tao.at/ > ldap_ca_file: /usr/local/share/ca-certificates/tao-ad-ca.crt > ldap_verify_peer: yes > > ldap_version: 3 > ldap_sasl: 0 > ldap_bind_dn: CN=some_user,CN=Users,DC=ad,DC=tao,DC=at > ldap_password: some_password > # Seems to work up to here, wrong password results in a ptloader error > # message. Correct password results in no output? > > ldap_base: CN=Users,DC=ad,DC=tao,DC=at > ldap_group_base: CN=Users,DC=ad,DC=tao,DC=at > ldap_member_base: CN=Users,DC=ad,DC=tao,DC=at > > # These SHOULD work, and do work with ldapsearch, but silently fail? > ldap_group_filter: (&(|(cn=%u)(sAMAccountName=%u))(objectClass=group)) > ldap_member_attribute: memberUid > ldap_user_attribute: uid > ldap_filter: (uid=%u) Is there another way to get ptloader to spit out debug information and pinpoint what's not set up correctly? -- Mit freundlichen Gr??en, / Best Regards, Sven Schwedas, Systemadministrator ? sven.schwedas at tao.at | ? +43 680 301 7167 TAO Digital | Teil der TAO Beratungs- & Management GmbH Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach A8020 Graz | https://www.tao-digital.at -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: From ellie at fastmail.com Thu Jun 13 23:38:52 2019 From: ellie at fastmail.com (ellie timoney) Date: Fri, 14 Jun 2019 13:38:52 +1000 Subject: LDAP auth and ptloader In-Reply-To: <4b541906-b785-7397-6f6c-96e4b0f77dd4@tao.at> References: <190cb420-4836-1f98-36f9-70e6bde97629@tao.at> <0c0acdee-a654-4b67-a23a-f58aa58d6672@www.fastmail.com> <4b541906-b785-7397-6f6c-96e4b0f77dd4@tao.at> Message-ID: <3a4ca4ad-e56e-4fcc-8e29-5e98a82d0d19@www.fastmail.com> Hi Sven, On Thu, Jun 13, 2019, at 12:27 AM, Sven Schwedas wrote: > Is there another way to get ptloader to spit out debug information and > pinpoint what's not set up correctly? > I remember this thing as being very noisy, let me see... Okay, in your cyrus.conf SERVICES entry, if you add "-d1" to the ptloader line like this, ptloader cmd="ptloader -d1" listen="/path/to/some/socket" then ptloader will syslog every user that it's asked about... You need "debug: 1" in imapd.conf, which will tell Cyrus to not swallow LOG_DEBUG level log lines, but ALSO: your syslog itself must be configured to log these lines (the default is often to not). We have some makeshift instructions here but ymmv: https://www.cyrusimap.org/imap/installing.html#setting-up-syslog If you turn on the "ptloader -d1" switch and set debug:1 and *don't* start seeing entries in your logs like "ptloader[pid]: user [user]", then you need to fiddle with syslog to enable the LOG_DEBUG log level :) Here's an example of some ptloader log output from running our test suite, for example: > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: executed > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: starting: ptloader.c 3.1.6-696-gf38559858 > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: accepted connection > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: user admin > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: collecting all domains from ou=domains,o=cyrus > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Domain filter: (&(objectclass=domainrelatedobject)(associateddomain=*)) > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: we have a domain internal. > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: ptsmodule_standard_root_dn called for domain internal. > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Root DN now dc= > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Root DN now dc=internal > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Root DN now dc=internal > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Root DN now dc=internal > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: Found admin in dc=internal > Jun 14 12:08:03 debian 02032301C4/ptloader[29481]: we have found admin in dc=internal And part of another run, showing it resolving a group membership (sorry these lines got truncated during copy&paste, but you get the idea): > Jun 14 13:11:47 debian 0311460101/ptloader[16345]: accepted connection > Jun 14 13:11:47 debian 0311460101/ptloader[16345]: user group:group co > Jun 14 13:11:47 debian 0311460101/ptloader[16345]: (groups) about to search ou=groups,o=cy > Jun 14 13:11:47 debian 0311460101/imap[16344]: timeout_select exiting. r = 1; errno = 0 > Jun 14 13:11:47 debian 0311460101/imap[16344]: timeout_select: sock = 15, rp = 0x7ffca95e3 > Jun 14 13:11:47 debian 0311460101/imap[16344]: timeout_select exiting. r = 1; errno = 0 > Jun 14 13:11:47 debian 0311460101/imap[16344]: ptload read data back > Jun 14 13:11:47 debian 0311460101/imap[16344]: ptload returning data > Jun 14 13:11:47 debian 0311460101/imap[16344]: canonified group:group co -> group:group co Not sure if this is helpful, but this is the directory structure our tests are working with: https://github.com/cyrusimap/cassandane/blob/master/data/directory.ldif ... ohhhhhhh, > ldap_member_attribute: memberUid This kinda sounds like your groups are what I think of as "normal": a group in LDAP is an entry that contains a multi-valued attribute listing all the group members. Is that a good description of your schema? As far as I've been able to figure out while building tests, Cyrus seems to expect each *user* entry to contain a multi-valued attribute listing the groups it is a member of (e.g. see that directory.ldif linked above). This feels backwards to me, but maybe it's normal somewhere?? I don't understand the rationale for this choice, or whether Cyrus can support a "normal" setup... maybe using the "ldap_member_method: filter" configuration (vs the default setting of "attribute") somehow?? Hopefully this is enough for you to get some useful logging out of the thing anyway, Cheers, ellie -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellie at fastmail.com Fri Jun 14 01:42:22 2019 From: ellie at fastmail.com (ellie timoney) Date: Fri, 14 Jun 2019 15:42:22 +1000 Subject: backupd IOERROR reading backup files larger than 2GB In-Reply-To: References: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> Message-ID: <8b4ea322-c9d2-4d0b-8153-9180e3bea12b@www.fastmail.com> Hi Carlos, This is quite weird, I'm not sure why a 64bit platform would have any trouble around the 2GB mark?? What does the Cyrus ./configure report for your system's integer sizes? e.g. mine shows: > checking size of int... 4 > checking size of long... 8 > checking size of size_t... 8 > checking size of off_t... 8 > checking size of time_t... 8 > checking size of long long int... 8 > checking size of unsigned long long int... 8 > checking whether byte ordering is bigendian... no What's your level of comfort with C debugging? It'd be very helpful to see a core file+binary from the time that lseek error occurs? Cheers, ellie On Fri, Jun 7, 2019, at 1:58 AM, Carlos Larra?aga wrote: > Hi Ellie, > > Thanks for answering. We use latest 64bit Oracle Linux (not CentOs like I said before, sorry) and zlib is also 64bit version: >> # uname -a >> Linux xxx 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 17:35:45 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux >> >> # yum list installed zlib >> Loaded plugins: langpacks, ulninfo >> Installed Packages >> zlib.x86_64 1.2.7-18.el7 @ol7_latest >> >> # lsof -p 17005 |grep lib |grep -i z >> backupd 17005 cyrus mem REG 253,0 90248 134400930 /usr/lib64/libz.so.1.2.7 >> >> # file /usr/lib64/libz.so.1.2.7 >> /usr/lib64/libz.so.1.2.7: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=b9d5f73428bd6ad68c96986b57bea3b7cedb9745, stripped >> >> # rpm -qf /usr/lib64/libz.so.1.2.7 >> zlib-1.2.7-18.el7.x86_64 > I have summarized here some information about the backupd error reading the file descriptor 15, which is a backup larger than 2GB. The error is logged 1755 times. Instead, there is no error for fd 12, which is a backup of less than 2 GB: >> *#-- CLIENT SIDE* >> *-----------------------------------------------* >> >> >> # time sync_client -v -o -n backup -A >> Thu Jun 6 17:08:02 CEST 2019 >> USER aaa >> QUOTA user.aaa >> USER ccc >> Error from do_user(ccc): bailing out! >> >> real 30m16.223s >> user 0m0.006s >> sys 0m0.006s >> ** >> >> *#-- BACKUP SERVER SIDE ---------------------------------------- * >> # LOGFILE >> >> # tail -f /var/log/imapd.log >> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >> >> >> *# lsof of the backupd PROCESS *# lsof -P -p 2584 >> >> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >> backupd 2584 cyrus cwd DIR 253,0 4096 128 / >> backupd 2584 cyrus rtd DIR 253,0 4096 128 / >> backupd 2584 cyrus txt REG 253,0 768632 939542214 /usr/local/cyrus/libexec/backupd >> backupd 2584 cyrus mem REG 253,0 37208 135746334 /usr/lib64/libnss_sss.so.2 >> backupd 2584 cyrus mem REG 253,0 31416 134406707 /usr/lib64/libnss_dns-2.17.so >> backupd 2584 cyrus mem REG 253,0 61632 134406709 /usr/lib64/libnss_files-2.17.so >> backupd 2584 cyrus mem REG 253,0 43336 68814977 /usr/lib64/sasl2/webmail.so.0.0.0 >> backupd 2584 cyrus mem REG 253,0 57960 68856627 /usr/lib64/sasl2/libdigestmd5.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 24232 68797220 /usr/lib64/sasl2/libcrammd5.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 20088 68797044 /usr/lib64/sasl2/libplain.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 20056 68797036 /usr/lib64/sasl2/liblogin.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 1845816 135873512 /usr/lib64/libdb-5.3.so >> backupd 2584 cyrus mem REG 253,0 28272 67113426 /usr/lib64/sasl2/libsasldb.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 20064 67113423 /usr/lib64/sasl2/libanonymous.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 11448 135872733 /usr/lib64/libfreebl3.so >> backupd 2584 cyrus mem REG 253,0 155784 134400716 /usr/lib64/libselinux.so.1 >> backupd 2584 cyrus mem REG 253,0 15464 134401447 /usr/lib64/libkeyutils.so.1.5 >> backupd 2584 cyrus mem REG 253,0 40672 134399432 /usr/lib64/libcrypt-2.17.so >> backupd 2584 cyrus mem REG 253,0 105824 134781819 /usr/lib64/libresolv-2.17.so >> backupd 2584 cyrus mem REG 253,0 53944 134406785 /usr/lib64/libjansson.so.4.10.0 >> backupd 2584 cyrus mem REG 253,0 88776 135352942 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 >> backupd 2584 cyrus mem REG 253,0 1137032 134400337 /usr/lib64/libm-2.17.so >> backupd 2584 cyrus mem REG 253,0 991616 135352396 /usr/lib64/libstdc++.so.6.0.19 >> backupd 2584 cyrus mem REG 253,0 19296 134399434 /usr/lib64/libdl-2.17.so >> backupd 2584 cyrus mem REG 253,0 142008 134722750 /usr/lib64/libpthread-2.17.so >> backupd 2584 cyrus mem REG 253,0 2151704 134365659 /usr/lib64/libc-2.17.so >> backupd 2584 cyrus mem REG 253,0 753232 134401161 /usr/lib64/libsqlite3.so.0.8.6 >> backupd 2584 cyrus mem REG 253,0 90248 134400930 /usr/lib64/libz.so.1.2.7 >> backupd 2584 cyrus mem REG 253,0 11128 134400947 /usr/lib64/libpcreposix.so.0.0.1 >> backupd 2584 cyrus mem REG 253,0 402384 134400940 /usr/lib64/libpcre.so.1.2.0 >> backupd 2584 cyrus mem REG 253,0 67104 135269427 /usr/lib64/libkrb5support.so.0.1 >> backupd 2584 cyrus mem REG 253,0 15920 135873508 /usr/lib64/libcom_err.so.2.1 >> backupd 2584 cyrus mem REG 253,0 210832 134781960 /usr/lib64/libk5crypto.so.3.1 >> backupd 2584 cyrus mem REG 253,0 967864 134781967 /usr/lib64/libkrb5.so.3.3 >> backupd 2584 cyrus mem REG 253,0 320400 134781952 /usr/lib64/libgssapi_krb5.so.2.2 >> backupd 2584 cyrus mem REG 253,0 115856 134400341 /usr/lib64/libnsl-2.17.so >> backupd 2584 cyrus mem REG 253,0 42168 134401419 /usr/lib64/libwrap.so.0.7.6 >> backupd 2584 cyrus mem REG 253,0 2516640 134400708 /usr/lib64/libcrypto.so.1.0.2k >> backupd 2584 cyrus mem REG 253,0 470360 134400711 /usr/lib64/libssl.so.1.0.2k >> backupd 2584 cyrus mem REG 253,0 121208 134401258 /usr/lib64/libsasl2.so.3.0.0 >> backupd 2584 cyrus mem REG 253,0 2898664 838888996 /usr/local/cyrus/lib/libcyrus_imap.so.0.0.0 >> backupd 2584 cyrus mem REG 253,0 599528 838888999 /usr/local/cyrus/lib/libcyrus_sieve.so.0.0.0 >> backupd 2584 cyrus mem REG 253,0 20112 134406704 /usr/lib64/libuuid.so.1.3.0 >> backupd 2584 cyrus mem REG 253,0 20781704 136182553 /usr/lib64/libicudata.so.50.1.2 >> backupd 2584 cyrus mem REG 253,0 1539544 136197152 /usr/lib64/libicuuc.so.50.1.2 >> backupd 2584 cyrus mem REG 253,0 553264 838890012 /usr/local/cyrus/lib/libcyrus_min.so.0.0.0 >> backupd 2584 cyrus mem REG 253,0 1961968 838888994 /usr/local/cyrus/lib/libcyrus.so.0.0.0 >> backupd 2584 cyrus mem REG 253,0 163408 134355369 /usr/lib64/ld-2.17.so >> backupd 2584 cyrus 0u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >> backupd 2584 cyrus 1u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >> backupd 2584 cyrus 2u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >> backupd 2584 cyrus 3w FIFO 0,9 0t0 2141764 pipe >> backupd 2584 cyrus 4u IPv4 2141763 0t0 TCP hostb1.upv.es:2005 (LISTEN) >> backupd 2584 cyrus 5u REG 0,20 0 86644 /run/cyrus-b1/socket/backupd-0.lock >> backupd 2584 cyrus 6r FIFO 0,9 0t0 2141754 pipe >> backupd 2584 cyrus 7w FIFO 0,9 0t0 2141754 pipe >> backupd 2584 cyrus 8r FIFO 0,9 0t0 2141755 pipe >> backupd 2584 cyrus 9w FIFO 0,9 0t0 2141755 pipe >> backupd 2584 cyrus 10u unix 0xffff9d9df53d4400 0t0 2093417 socket >> backupd 2584 cyrus 11w REG 253,5 680 18253611600 /b1/lib/log/admin/backupd-2584 >> backupd 2584 cyrus 12uW REG 253,5 976281587 13958644353 /b1/bck/b1/a/aaa_ouuUx9 >> backupd 2584 cyrus 13ur REG 253,5 3573760 13958644354 /b1/bck/b1/a/aaa_ouuUx9.index >> backupd 2584 cyrus 14u REG 253,5 2576 13958644355 /b1/bck/b1/a/aaa_ouuUx9.index-journal >> backupd 2584 cyrus 15uW REG 253,5 3104341527 12884902595 /b1/bck/b1/c/ccc_Mqmymx >> backupd 2584 cyrus 16u REG 253,5 47927296 12884902596 /b1/bck/b1/c/ccc_Mqmymx.index >> >> *# BACKUP SIZES* >> # ls -lh /b1/bck/b1/a/* >> /b1/bck/b1/c/* >> *-rw------- 1 cyrus mail 932M Jun 6 17:08* >> /b1/bck/b1/a/*aaa_ouuUx9* >> -rw------- 1 cyrus mail 3.5M Jun 6 16:46 >> /b1/bck/b1/a/aaa_ouuUx9.index >> -rw------- 1 cyrus mail 2.6K Jun 6 17:08 >> /b1/bck/b1/a/aaa_ouuUx9.index-journal >> >> -rw------- 1 cyrus mail 2.9G Jun 6 17:01 /b1/bck/b1/c/ccc_Mqmymx >> -rw------- 1 cyrus mail 46M Jun 6 17:01 /b1/bck/b1/c/ccc_Mqmymx.index >> >> *# IMAP SESSION DEBUG* >> # cat /b1/lib/log/admin/backupd-2584 >> ---------- admin Thu Jun 6 17:08:02 2019 >> >> <1559833682> >1559833682>OK DEFLATE active >> <1559833682> >1559833698>* MAILBOX %(UNIQUEID acd014aa-7b1d-43c9-b1d3-f97b06540739 MBOXNAME user.aaa MBOXTYPE NIL LAST_UID 0 HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 UIDVALIDITY 1520512152 PARTITION default ACL "aaa lrswipkxtecdan group:super lrswipkxtecdan " OPTIONS P SYNC_CRC 0 SYNC_CRC_ANNOT 0 QUOTAROOT NIL XCONVMODSEQ 0) >> OK Success >> <1559833698> >1559833698>OK Success >> <1559833698 > Best regards, > Carlos > > > > El 06/06/2019 a las 4:04, ellie timoney escribi?: >> It kinda sounds like your platform might be 32bit? Or your zlib is compiled to use 32bit integer sizes? >> >> On Mon, Jun 3, 2019, at 10:07 PM, Carlos Larra?aga wrote: >>> Hi, >>> >>> We're testing backup feature un cyrus-imapd 3.0.10. There's no problem when backup is created first time, but when the backup already exists and is larger than 2GB, we get the following error from the backupd: >>>> cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or directory >>> As said, it happens only when then backup file already exists and is larger than 2GB. The backupd process keeps reading de compressed backup file til sync_client exit with "Error from sync_do_user(xxx): bailing out!". >>> >>> We use xfs with latest CentOs. Seems like backupd is unable to read the compressed backup file. >>> >>> Anyone else with this problem? Any idea how to fix it? >>> Thanks in advance for your help. >>> >>> Best regards, >>> Carlos. >>> ---- >>> Cyrus Home Page: http://www.cyrusimap.org/ >>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>> To Unsubscribe: >>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>> >>> *Attachments:* >>> * smime.p7s >> >> >> ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > *Attachments:* > * smime.p7s -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellie at fastmail.com Sun Jun 16 20:44:16 2019 From: ellie at fastmail.com (ellie timoney) Date: Mon, 17 Jun 2019 10:44:16 +1000 Subject: backupd IOERROR reading backup files larger than 2GB In-Reply-To: <8b4ea322-c9d2-4d0b-8153-9180e3bea12b@www.fastmail.com> References: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> <8b4ea322-c9d2-4d0b-8153-9180e3bea12b@www.fastmail.com> Message-ID: Hi Carlos, Sudden overnight intuition, and I think this will fix the issue -- the problem isn't that the lseek is failing at >2GB; the problem is that its off_t return value is being truncated to an int before being checked to see if it's negative for the error case (so any "I succeeded, the offset is > 2GB" response looks like "negative, it's an error"). Doh! I think this will fix the immediate issue: https://github.com/cyrusimap/cyrus-imapd/commit/63f8c09fa2076c0e2fa55436e071e3e341fe48b6 But then I found and fixed some similar ones: https://github.com/cyrusimap/cyrus-imapd/commit/0930d3af4ed9bd44d328db8cfa4d9f2e2be7bada Remains to be seen whether more similar issues pop up -- since no-one's tripped over this previously, I guess you're the first person to try this with a >2GB backup file :( Cheers, ellie On Fri, Jun 14, 2019, at 3:50 PM, ellie timoney wrote: > Hi Carlos, > > This is quite weird, I'm not sure why a 64bit platform would have any trouble around the 2GB mark?? > > What does the Cyrus ./configure report for your system's integer sizes? e.g. mine shows: > >> checking size of int... 4 >> checking size of long... 8 >> checking size of size_t... 8 >> checking size of off_t... 8 >> checking size of time_t... 8 >> checking size of long long int... 8 >> checking size of unsigned long long int... 8 >> checking whether byte ordering is bigendian... no > > What's your level of comfort with C debugging? It'd be very helpful to see a core file+binary from the time that lseek error occurs? > > Cheers, > > ellie > > On Fri, Jun 7, 2019, at 1:58 AM, Carlos Larra?aga wrote: >> Hi Ellie, >> >> Thanks for answering. We use latest 64bit Oracle Linux (not CentOs like I said before, sorry) and zlib is also 64bit version: >>> # uname -a >>> Linux xxx 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 17:35:45 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux >>> >>> # yum list installed zlib >>> Loaded plugins: langpacks, ulninfo >>> Installed Packages >>> zlib.x86_64 1.2.7-18.el7 @ol7_latest >>> >>> # lsof -p 17005 |grep lib |grep -i z >>> backupd 17005 cyrus mem REG 253,0 90248 134400930 /usr/lib64/libz.so.1.2.7 >>> >>> # file /usr/lib64/libz.so.1.2.7 >>> /usr/lib64/libz.so.1.2.7: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=b9d5f73428bd6ad68c96986b57bea3b7cedb9745, stripped >>> >>> # rpm -qf /usr/lib64/libz.so.1.2.7 >>> zlib-1.2.7-18.el7.x86_64 >> I have summarized here some information about the backupd error reading the file descriptor 15, which is a backup larger than 2GB. The error is logged 1755 times. Instead, there is no error for fd 12, which is a backup of less than 2 GB: >>> *#-- CLIENT SIDE* >>> *-----------------------------------------------* >>> >>> >>> # time sync_client -v -o -n backup -A >>> Thu Jun 6 17:08:02 CEST 2019 >>> USER aaa >>> QUOTA user.aaa >>> USER ccc >>> Error from do_user(ccc): bailing out! >>> >>> real 30m16.223s >>> user 0m0.006s >>> sys 0m0.006s >>> ** >>> >>> *#-- BACKUP SERVER SIDE ----------------------------------------* >>> # LOGFILE >>> >>> # tail -f /var/log/imapd.log >>> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >>> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >>> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >>> Jun 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: No such file or directory >>> >>> >>> *# lsof of the backupd PROCESS *# lsof -P -p 2584 >>> >>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >>> backupd 2584 cyrus cwd DIR 253,0 4096 128 / >>> backupd 2584 cyrus rtd DIR 253,0 4096 128 / >>> backupd 2584 cyrus txt REG 253,0 768632 939542214 /usr/local/cyrus/libexec/backupd >>> backupd 2584 cyrus mem REG 253,0 37208 135746334 /usr/lib64/libnss_sss.so.2 >>> backupd 2584 cyrus mem REG 253,0 31416 134406707 /usr/lib64/libnss_dns-2.17.so >>> backupd 2584 cyrus mem REG 253,0 61632 134406709 /usr/lib64/libnss_files-2.17.so >>> backupd 2584 cyrus mem REG 253,0 43336 68814977 /usr/lib64/sasl2/webmail.so.0.0.0 >>> backupd 2584 cyrus mem REG 253,0 57960 68856627 /usr/lib64/sasl2/libdigestmd5.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 24232 68797220 /usr/lib64/sasl2/libcrammd5.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 20088 68797044 /usr/lib64/sasl2/libplain.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 20056 68797036 /usr/lib64/sasl2/liblogin.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 1845816 135873512 /usr/lib64/libdb-5.3.so >>> backupd 2584 cyrus mem REG 253,0 28272 67113426 /usr/lib64/sasl2/libsasldb.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 20064 67113423 /usr/lib64/sasl2/libanonymous.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 11448 135872733 /usr/lib64/libfreebl3.so >>> backupd 2584 cyrus mem REG 253,0 155784 134400716 /usr/lib64/libselinux.so.1 >>> backupd 2584 cyrus mem REG 253,0 15464 134401447 /usr/lib64/libkeyutils.so.1.5 >>> backupd 2584 cyrus mem REG 253,0 40672 134399432 /usr/lib64/libcrypt-2.17.so >>> backupd 2584 cyrus mem REG 253,0 105824 134781819 /usr/lib64/libresolv-2.17.so >>> backupd 2584 cyrus mem REG 253,0 53944 134406785 /usr/lib64/libjansson.so.4.10.0 >>> backupd 2584 cyrus mem REG 253,0 88776 135352942 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 >>> backupd 2584 cyrus mem REG 253,0 1137032 134400337 /usr/lib64/libm-2.17.so >>> backupd 2584 cyrus mem REG 253,0 991616 135352396 /usr/lib64/libstdc++.so.6.0.19 >>> backupd 2584 cyrus mem REG 253,0 19296 134399434 /usr/lib64/libdl-2.17.so >>> backupd 2584 cyrus mem REG 253,0 142008 134722750 /usr/lib64/libpthread-2.17.so >>> backupd 2584 cyrus mem REG 253,0 2151704 134365659 /usr/lib64/libc-2.17.so >>> backupd 2584 cyrus mem REG 253,0 753232 134401161 /usr/lib64/libsqlite3.so.0.8.6 >>> backupd 2584 cyrus mem REG 253,0 90248 134400930 /usr/lib64/libz.so.1.2.7 >>> backupd 2584 cyrus mem REG 253,0 11128 134400947 /usr/lib64/libpcreposix.so.0.0.1 >>> backupd 2584 cyrus mem REG 253,0 402384 134400940 /usr/lib64/libpcre.so.1.2.0 >>> backupd 2584 cyrus mem REG 253,0 67104 135269427 /usr/lib64/libkrb5support.so.0.1 >>> backupd 2584 cyrus mem REG 253,0 15920 135873508 /usr/lib64/libcom_err.so.2.1 >>> backupd 2584 cyrus mem REG 253,0 210832 134781960 /usr/lib64/libk5crypto.so.3.1 >>> backupd 2584 cyrus mem REG 253,0 967864 134781967 /usr/lib64/libkrb5.so.3.3 >>> backupd 2584 cyrus mem REG 253,0 320400 134781952 /usr/lib64/libgssapi_krb5.so.2.2 >>> backupd 2584 cyrus mem REG 253,0 115856 134400341 /usr/lib64/libnsl-2.17.so >>> backupd 2584 cyrus mem REG 253,0 42168 134401419 /usr/lib64/libwrap.so.0.7.6 >>> backupd 2584 cyrus mem REG 253,0 2516640 134400708 /usr/lib64/libcrypto.so.1.0.2k >>> backupd 2584 cyrus mem REG 253,0 470360 134400711 /usr/lib64/libssl.so.1.0.2k >>> backupd 2584 cyrus mem REG 253,0 121208 134401258 /usr/lib64/libsasl2.so.3.0.0 >>> backupd 2584 cyrus mem REG 253,0 2898664 838888996 /usr/local/cyrus/lib/libcyrus_imap.so.0.0.0 >>> backupd 2584 cyrus mem REG 253,0 599528 838888999 /usr/local/cyrus/lib/libcyrus_sieve.so.0.0.0 >>> backupd 2584 cyrus mem REG 253,0 20112 134406704 /usr/lib64/libuuid.so.1.3.0 >>> backupd 2584 cyrus mem REG 253,0 20781704 136182553 /usr/lib64/libicudata.so.50.1.2 >>> backupd 2584 cyrus mem REG 253,0 1539544 136197152 /usr/lib64/libicuuc.so.50.1.2 >>> backupd 2584 cyrus mem REG 253,0 553264 838890012 /usr/local/cyrus/lib/libcyrus_min.so.0.0.0 >>> backupd 2584 cyrus mem REG 253,0 1961968 838888994 /usr/local/cyrus/lib/libcyrus.so.0.0.0 >>> backupd 2584 cyrus mem REG 253,0 163408 134355369 /usr/lib64/ld-2.17.so >>> backupd 2584 cyrus 0u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus 1u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus 2u IPv4 2092952 0t0 TCP hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus 3w FIFO 0,9 0t0 2141764 pipe >>> backupd 2584 cyrus 4u IPv4 2141763 0t0 TCP hostb1.upv.es:2005 (LISTEN) >>> backupd 2584 cyrus 5u REG 0,20 0 86644 /run/cyrus-b1/socket/backupd-0.lock >>> backupd 2584 cyrus 6r FIFO 0,9 0t0 2141754 pipe >>> backupd 2584 cyrus 7w FIFO 0,9 0t0 2141754 pipe >>> backupd 2584 cyrus 8r FIFO 0,9 0t0 2141755 pipe >>> backupd 2584 cyrus 9w FIFO 0,9 0t0 2141755 pipe >>> backupd 2584 cyrus 10u unix 0xffff9d9df53d4400 0t0 2093417 socket >>> backupd 2584 cyrus 11w REG 253,5 680 18253611600 /b1/lib/log/admin/backupd-2584 >>> backupd 2584 cyrus 12uW REG 253,5 976281587 13958644353 /b1/bck/b1/a/aaa_ouuUx9 >>> backupd 2584 cyrus 13ur REG 253,5 3573760 13958644354 /b1/bck/b1/a/aaa_ouuUx9.index >>> backupd 2584 cyrus 14u REG 253,5 2576 13958644355 /b1/bck/b1/a/aaa_ouuUx9.index-journal >>> backupd 2584 cyrus 15uW REG 253,5 3104341527 12884902595 /b1/bck/b1/c/ccc_Mqmymx >>> backupd 2584 cyrus 16u REG 253,5 47927296 12884902596 /b1/bck/b1/c/ccc_Mqmymx.index >>> >>> *# BACKUP SIZES* >>> # ls -lh /b1/bck/b1/a/* >>> /b1/bck/b1/c/* >>> *-rw------- 1 cyrus mail 932M Jun 6 17:08* >>> /b1/bck/b1/a/*aaa_ouuUx9* >>> -rw------- 1 cyrus mail 3.5M Jun 6 16:46 >>> /b1/bck/b1/a/aaa_ouuUx9.index >>> -rw------- 1 cyrus mail 2.6K Jun 6 17:08 >>> /b1/bck/b1/a/aaa_ouuUx9.index-journal >>> >>> -rw------- 1 cyrus mail 2.9G Jun 6 17:01 /b1/bck/b1/c/ccc_Mqmymx >>> -rw------- 1 cyrus mail 46M Jun 6 17:01 /b1/bck/b1/c/ccc_Mqmymx.index >>> >>> *# IMAP SESSION DEBUG* >>> # cat /b1/lib/log/admin/backupd-2584 >>> ---------- admin Thu Jun 6 17:08:02 2019 >>> >>> <1559833682>> >1559833682>OK DEFLATE active >>> <1559833682>> >1559833698>* MAILBOX %(UNIQUEID acd014aa-7b1d-43c9-b1d3-f97b06540739 MBOXNAME user.aaa MBOXTYPE NIL LAST_UID 0 HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 UIDVALIDITY 1520512152 PARTITION default ACL "aaa lrswipkxtecdan group:super lrswipkxtecdan " OPTIONS P SYNC_CRC 0 SYNC_CRC_ANNOT 0 QUOTAROOT NIL XCONVMODSEQ 0) >>> OK Success >>> <1559833698>> >1559833698>OK Success >>> <1559833698> >> Best regards, >> Carlos >> >> >> >> El 06/06/2019 a las 4:04, ellie timoney escribi?: >>> It kinda sounds like your platform might be 32bit? Or your zlib is compiled to use 32bit integer sizes? >>> >>> On Mon, Jun 3, 2019, at 10:07 PM, Carlos Larra?aga wrote: >>>> Hi, >>>> >>>> We're testing backup feature un cyrus-imapd 3.0.10. There's no problem when backup is created first time, but when the backup already exists and is larger than 2GB, we get the following error from the backupd: >>>>> cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or directory >>>> As said, it happens only when then backup file already exists and is larger than 2GB. The backupd process keeps reading de compressed backup file til sync_client exit with "Error from sync_do_user(xxx): bailing out!". >>>> >>>> We use xfs with latest CentOs. Seems like backupd is unable to read the compressed backup file. >>>> >>>> Anyone else with this problem? Any idea how to fix it? >>>> Thanks in advance for your help. >>>> >>>> Best regards, >>>> Carlos. >>>> ---- >>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>> To Unsubscribe: >>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>> >>>> *Attachments:* >>>> * smime.p7s >>> >>> >>> ---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: >>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >> >> >> *Attachments:* >> * smime.p7s > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -------------- next part -------------- An HTML attachment was scrubbed... URL: From clarra at cc.upv.es Mon Jun 17 06:50:06 2019 From: clarra at cc.upv.es (=?UTF-8?Q?Carlos_Larra=c3=b1aga?=) Date: Mon, 17 Jun 2019 12:50:06 +0200 Subject: backupd IOERROR reading backup files larger than 2GB In-Reply-To: References: <6eac4b65-a73d-eba4-b9e3-464699959777@cc.upv.es> <8b4ea322-c9d2-4d0b-8153-9180e3bea12b@www.fastmail.com> Message-ID: Ufffff ... Thank you so much Ellie, now it works fine. Right now I was fighting against my rusty gdb knowledge, although I had almost defeated him ;) Cheers, Carlos El 17/06/2019 a las 2:44, ellie timoney escribi?: > Hi Carlos, > > Sudden overnight intuition, and I think this will fix the issue -- the problem > isn't that the lseek is failing at >2GB; the problem is that its off_t return > value is being truncated to an int before being checked to see if it's > negative for the error case (so any "I succeeded, the offset is > 2GB" > response looks like "negative, it's an error"). Doh! > > I think this will fix the immediate issue: > https://github.com/cyrusimap/cyrus-imapd/commit/63f8c09fa2076c0e2fa55436e071e3e341fe48b6 > > But then I found and fixed some similar ones: > https://github.com/cyrusimap/cyrus-imapd/commit/0930d3af4ed9bd44d328db8cfa4d9f2e2be7bada > > Remains to be seen whether more similar issues pop up -- since no-one's > tripped over this previously, I guess you're the first person to try this with > a >2GB backup file :( > > Cheers, > > ellie > > On Fri, Jun 14, 2019, at 3:50 PM, ellie timoney wrote: >> Hi Carlos, >> >> This is quite weird, I'm not sure why a 64bit platform would have any trouble >> around the 2GB mark?? >> >> What does the Cyrus ./configure report for your system's integer sizes? ?e.g. >> mine shows: >> >>> checking size of int... 4 >>> checking size of long... 8 >>> checking size of size_t... 8 >>> checking size of off_t... 8 >>> checking size of time_t... 8 >>> checking size of long long int... 8 >>> checking size of unsigned long long int... 8 >>> checking whether byte ordering is bigendian... no >> >> What's your level of comfort with C debugging? ?It'd be very helpful to see a >> core file+binary from the time that lseek error occurs? >> >> Cheers, >> >> ellie >> >> On Fri, Jun 7, 2019, at 1:58 AM, Carlos Larra?aga wrote: >>> Hi Ellie, >>> >>> Thanks for answering. We use latest 64bit Oracle Linux (not CentOs like I >>> said before, sorry) and zlib is also 64bit version: >>> >>> # uname -a >>> Linux xxx 3.10.0-957.12.2.el7.x86_64 #1 SMP Tue May 14 17:35:45 PDT 2019 >>> x86_64 x86_64 x86_64 GNU/Linux >>> >>> # yum list installed zlib >>> Loaded plugins: langpacks, ulninfo >>> Installed Packages >>> zlib.x86_64 1.2.7-18.el7????????????????????? @ol7_latest >>> >>> # lsof -p 17005 |grep lib |grep -i z >>> backupd 17005 cyrus? mem??? REG????????????? 253,0 90248?? 134400930 >>> /usr/lib64/libz.so.1.2.7 >>> >>> # file /usr/lib64/libz.so.1.2.7 >>> /usr/lib64/libz.so.1.2.7: ELF 64-bit LSB shared object, x86-64, version >>> 1 (SYSV), dynamically linked, >>> BuildID[sha1]=b9d5f73428bd6ad68c96986b57bea3b7cedb9745, stripped >>> >>> # rpm -qf /usr/lib64/libz.so.1.2.7 >>> zlib-1.2.7-18.el7.x86_64 >>> >>> I have summarized here some information about the backupd error reading the >>> file descriptor 15, which is a backup larger than 2GB. The error is logged >>> 1755 times. Instead, there is no error for fd 12, which is a backup of less >>> than 2 GB: >>> >>> *#-- CLIENT SIDE* >>> *-----------------------------------------------* >>> >>> >>> # time sync_client -v -o -n backup -A >>> Thu Jun? 6 17:08:02 CEST 2019 >>> USER aaa >>> QUOTA user.aaa >>> USER ccc >>> Error from do_user(ccc): bailing out! >>> >>> real??? 30m16.223s >>> user??? 0m0.006s >>> sys???? 0m0.006s >>> >>> >>> *#-- BACKUP SERVER SIDE ----------------------------------------* >>> # LOGFILE >>> >>> # tail -f /var/log/imapd.log >>> Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: >>> No such file or directory >>> Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: >>> No such file or directory >>> Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: >>> No such file or directory >>> Jun? 6 17:09:11 bcrux cyrus/backupd[2584]: IOERROR: gzuc_read: lseek 15: >>> No such file or directory >>> >>> >>> *# lsof of the backupd PROCESS >>> *# lsof -P -p 2584 >>> >>> COMMAND PID? USER?? FD?? TYPE???????????? DEVICE SIZE/OFF??????? NODE NAME >>> backupd 2584 cyrus? cwd??? DIR????????????? 253,0 4096???????? 128 / >>> backupd 2584 cyrus? rtd??? DIR????????????? 253,0 4096???????? 128 / >>> backupd 2584 cyrus? txt??? REG????????????? 253,0???? 768632 939542214 >>> /usr/local/cyrus/libexec/backupd >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 37208 135746334 >>> /usr/lib64/libnss_sss.so.2 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 31416 134406707 >>> /usr/lib64/libnss_dns-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 61632 134406709 >>> /usr/lib64/libnss_files-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 43336 68814977 >>> /usr/lib64/sasl2/webmail.so.0.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 57960 68856627 >>> /usr/lib64/sasl2/libdigestmd5.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 24232 68797220 >>> /usr/lib64/sasl2/libcrammd5.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 20088 68797044 >>> /usr/lib64/sasl2/libplain.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 20056 68797036 >>> /usr/lib64/sasl2/liblogin.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 1845816 135873512 >>> /usr/lib64/libdb-5.3.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 28272 67113426 >>> /usr/lib64/sasl2/libsasldb.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 20064 67113423 >>> /usr/lib64/sasl2/libanonymous.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 11448 135872733 >>> /usr/lib64/libfreebl3.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 155784 134400716 >>> /usr/lib64/libselinux.so.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 15464 134401447 >>> /usr/lib64/libkeyutils.so.1.5 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 40672 134399432 >>> /usr/lib64/libcrypt-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 105824 134781819 >>> /usr/lib64/libresolv-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 53944 134406785 >>> /usr/lib64/libjansson.so.4.10.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 88776 135352942 >>> /usr/lib64/libgcc_s-4.8.5-20150702.so.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 1137032 134400337 >>> /usr/lib64/libm-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 991616 135352396 >>> /usr/lib64/libstdc++.so.6.0.19 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 19296 134399434 >>> /usr/lib64/libdl-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 142008 134722750 >>> /usr/lib64/libpthread-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 2151704 134365659 >>> /usr/lib64/libc-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 753232 134401161 >>> /usr/lib64/libsqlite3.so.0.8.6 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 90248 134400930 >>> /usr/lib64/libz.so.1.2.7 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 11128 134400947 >>> /usr/lib64/libpcreposix.so.0.0.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 402384 134400940 >>> /usr/lib64/libpcre.so.1.2.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 67104 135269427 >>> /usr/lib64/libkrb5support.so.0.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 15920 135873508 >>> /usr/lib64/libcom_err.so.2.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 210832 134781960 >>> /usr/lib64/libk5crypto.so.3.1 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 967864 134781967 >>> /usr/lib64/libkrb5.so.3.3 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 320400 134781952 >>> /usr/lib64/libgssapi_krb5.so.2.2 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 115856 134400341 >>> /usr/lib64/libnsl-2.17.so >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 42168 134401419 >>> /usr/lib64/libwrap.so.0.7.6 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 2516640 134400708 >>> /usr/lib64/libcrypto.so.1.0.2k >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 470360 134400711 >>> /usr/lib64/libssl.so.1.0.2k >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 121208 134401258 >>> /usr/lib64/libsasl2.so.3.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 2898664 838888996 >>> /usr/local/cyrus/lib/libcyrus_imap.so.0.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 599528 838888999 >>> /usr/local/cyrus/lib/libcyrus_sieve.so.0.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0????? 20112 134406704 >>> /usr/lib64/libuuid.so.1.3.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0?? 20781704 136182553 >>> /usr/lib64/libicudata.so.50.1.2 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 1539544 136197152 >>> /usr/lib64/libicuuc.so.50.1.2 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 553264 838890012 >>> /usr/local/cyrus/lib/libcyrus_min.so.0.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0??? 1961968 838888994 >>> /usr/local/cyrus/lib/libcyrus.so.0.0.0 >>> backupd 2584 cyrus? mem??? REG????????????? 253,0???? 163408 134355369 >>> /usr/lib64/ld-2.17.so >>> backupd 2584 cyrus??? 0u? IPv4??????????? 2092952 0t0???????? TCP >>> hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus??? 1u? IPv4??????????? 2092952 0t0???????? TCP >>> hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus??? 2u? IPv4??????????? 2092952 0t0???????? TCP >>> hostb1.upv.es:2005->host1.upv.es:27107 (ESTABLISHED) >>> backupd 2584 cyrus??? 3w? FIFO??????????????? 0,9??????? 0t0 2141764 pipe >>> backupd 2584 cyrus??? 4u? IPv4??????????? 2141763 0t0???????? TCP >>> hostb1.upv.es:2005 (LISTEN) >>> backupd 2584 cyrus??? 5u?? REG?????????????? 0,20 0?????? 86644 >>> /run/cyrus-b1/socket/backupd-0.lock >>> backupd 2584 cyrus??? 6r? FIFO??????????????? 0,9??????? 0t0 2141754 pipe >>> backupd 2584 cyrus??? 7w? FIFO??????????????? 0,9??????? 0t0 2141754 pipe >>> backupd 2584 cyrus??? 8r? FIFO??????????????? 0,9??????? 0t0 2141755 pipe >>> backupd 2584 cyrus??? 9w? FIFO??????????????? 0,9??????? 0t0 2141755 pipe >>> backupd 2584 cyrus?? 10u? unix 0xffff9d9df53d4400??????? 0t0 2093417 socket >>> backupd 2584 cyrus?? 11w?? REG????????????? 253,5??????? 680 18253611600 >>> /b1/lib/log/admin/backupd-2584 >>> backupd 2584 cyrus?? 12uW? REG????????????? 253,5? 976281587 13958644353 >>> /b1/bck/b1/a/aaa_ouuUx9 >>> backupd 2584 cyrus?? 13ur? REG????????????? 253,5??? 3573760 13958644354 >>> /b1/bck/b1/a/aaa_ouuUx9.index >>> backupd 2584 cyrus?? 14u?? REG????????????? 253,5?????? 2576 13958644355 >>> /b1/bck/b1/a/aaa_ouuUx9.index-journal >>> backupd 2584 cyrus?? 15uW? REG????????????? 253,5 3104341527 12884902595 >>> /b1/bck/b1/c/ccc_Mqmymx >>> backupd 2584 cyrus?? 16u?? REG????????????? 253,5?? 47927296 12884902596 >>> /b1/bck/b1/c/ccc_Mqmymx.index >>> >>> *# BACKUP SIZES* >>> # ls -lh /b1/bck/b1/a/* >>> /b1/bck/b1/c/* >>> *-rw------- 1 cyrus mail 932M Jun? 6 17:08* >>> /b1/bck/b1/a/*aaa_ouuUx9* >>> -rw------- 1 cyrus mail 3.5M Jun? 6 16:46 >>> /b1/bck/b1/a/aaa_ouuUx9.index >>> -rw------- 1 cyrus mail 2.6K Jun? 6 17:08 >>> /b1/bck/b1/a/aaa_ouuUx9.index-journal >>> >>> -rw------- 1 cyrus mail 2.9G Jun? 6 17:01 /b1/bck/b1/c/ccc_Mqmymx >>> -rw------- 1 cyrus mail? 46M Jun? 6 17:01 /b1/bck/b1/c/ccc_Mqmymx.index >>> >>> *# IMAP SESSION DEBUG* >>> # cat /b1/lib/log/admin/backupd-2584 >>> ---------- admin Thu Jun? 6 17:08:02 2019 >>> >>> <1559833682>> >1559833682>OK DEFLATE active >>> <1559833682>> >1559833698>* MAILBOX %(UNIQUEID acd014aa-7b1d-43c9-b1d3-f97b06540739 MBOXNAME >>> user.aaa MBOXTYPE NIL LAST_UID 0 HIGHESTMODSEQ 2 RECENTUID 0 RECENTTIME >>> 0 LAST_APPENDDATE 0 POP3_LAST_LOGIN 0 POP3_SHOW_AFTER 0 UIDVALIDITY >>> 1520512152 PARTITION default ACL "aaa??? lrswipkxtecdan group:super???? >>> lrswipkxtecdan? " OPTIONS P SYNC_CRC 0 SYNC_CRC_ANNOT 0 QUOTAROOT NIL >>> XCONVMODSEQ 0) >>> OK Success >>> <1559833698>> >1559833698>OK Success >>> <1559833698>> >>> >>> Best regards, >>> Carlos >>> >>> >>> >>> El 06/06/2019 a las 4:04, ellie timoney escribi?: >>>> It kinda sounds like your platform might be 32bit? Or your zlib is compiled >>>> to use 32bit integer sizes? >>>> >>>> On Mon, Jun 3, 2019, at 10:07 PM, Carlos Larra?aga wrote: >>>>> Hi, >>>>> >>>>> We're testing backup feature un cyrus-imapd 3.0.10. There's no problem >>>>> when backup is created first time, but when the backup already exists and >>>>> is larger than 2GB, we get the following error from the backupd: >>>>> >>>>> cyrus/backupd[xxxx]: IOERROR: gzuc_read: lseek 12: No such file or >>>>> directory >>>>> >>>>> As said, it happens only when then backup file already exists and is >>>>> larger than 2GB. The backupd process keeps reading de compressed backup >>>>> file til sync_client exit with "Error from sync_do_user(xxx): bailing out!". >>>>> >>>>> We use xfs with latest CentOs. Seems like backupd is unable to read the >>>>> compressed backup file. >>>>> >>>>> Anyone else with this problem? Any idea how to fix it? >>>>> Thanks in advance for your help. >>>>> >>>>> Best regards, >>>>> Carlos. >>>>> ---- >>>>> Cyrus Home Page: http://www.cyrusimap.org/ >>>>> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>>> To Unsubscribe: >>>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>>>> >>>>> *Attachments:* >>>>> >>>>> * smime.p7s >>>>> >>>> >>>> >>>> ---- >>>> Cyrus Home Page:http://www.cyrusimap.org/ >>>> List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >>>> To Unsubscribe: >>>> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus >>> >>> >>> *Attachments:* >>> >>> * smime.p7s >>> >> >> ---- >> Cyrus Home Page: http://www.cyrusimap.org/ >> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ >> To Unsubscribe: >> https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus > > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3627 bytes Desc: Firma criptogr??fica S/MIME URL: