problem with one user after a crash

[David Lang]

On Thu, 10 Jan 2013, Andrew Morgan wrote:

> On Thu, 10 Jan 2013, David Lang wrote:
>
>> On Thu, 10 Jan 2013, Andrew Morgan wrote:
>> 
>>> A corrupted seen file is the only thing that makes sense to me.  If other 
>>> users can open the same folder, then the cyrus.header and cyrus.index 
>>> files must be sane.
>>> 
>>> As an experiment, you could move your seen file from lang.seen (or 
>>> whatever it's called) to lang.seen.bak.  Then connect to IMAP as yourself 
>>> and try to open the folder.  If it works, then it must have been a 
>>> corrupted seen file, and you can use skiplist.py to recover as much of it 
>>> as possible.
>> 
>> Ok, the good news is that this seems to be the problem.
>> 
>> unfortunantly the skiplist recovery tool is not working.
>> 
>> # ./skiplist.py dlang.seen.bak >dlang.seen.txt
>> Traceback (most recent call last):
>>  File "./skiplist.py", line 172, in <module>
>>    values, keys = getkeys(fp)
>>  File "./skiplist.py", line 152, in getkeys
>>    spointer = unpack('>I', str_p)[0]
>> struct.error: unpack requires a string argument of length 4
>> 
>> # file dlang.seen.bak
>> dlang.seen.bak: Cyrus skiplist DB
>> 
>> I tried enabling debug mode in skiplist.py and I'm not seeing anything 
>> different. This confuses me. I'm not that familiar with python, but as I 
>> read the code, get_header() should be writing a bunch of stuff before it 
>> gets to the getkeys() section that failing.
>
> Hmmm, I haven't looked at the code in skiplist.py much.  I have an older 
> version of skiplist.py, which I have attached to this email.  Honestly, I 
> haven't used this since I upgraded to Cyrus v2.3.something.  I think there 
> were some bugs in skiplist on the older versions.  :)
>
> Give the attached skiplist.py a shot!  Worst case, you'll have to start over 
> with no Seen history.  :(

It dies as well. It turns out that it's sending the debug messages to stdout not 
stderr. It looks like it processes 96 sections before it dies.

with debug off I just get the error, with debug on I get a bunch of stuff that 
looks fairly similar, ending with:

[*] 
------------------------------------------------------------------------------
[INORDER] Record type INORDER
[INORDER] Key size 16 (16)
[INORDER] Key String 7536cca646f98c5d
[INORDER] Data size 259 (260)
[INORDER] Data String 1 1351316330 4755 1351316487 
1,285,528,596,807,811,984:1003,1005:1010,1012,1014,1016,1018:1019,1075,1239,1619,1821,1920,2409,2511:2513,2646,2713,2787,3076,3080:3081,3084:3085,3090,3124,3126,3148,3160,3385,4056,4108,4186,4221,4256,4320,4390,4401,4585,4618,4627
[INORDER] Skip pointer 16264
[INORDER] Total Skip pointers: 1
[*] 
------------------------------------------------------------------------------
[INORDER] Record type INORDER
[INORDER] Key size 16 (16)
[INORDER] Key String 782cc11f4fb291b5
[INORDER] Data size 127 (128)
[INORDER] Data String 1 1351316303 46217 1351316313 
1:13755,22847:22986,26559:26563,26565:26749,29906:29910,30316,
Traceback (most recent call last):
   File "/home/dlang/skiplist.py", line 167, in <module>
     values, keys = getkeys(fp)
   File "/home/dlang/skiplist.py", line 147, in getkeys
     spointer = unpack('>I', str_p)[0]
struct.error: unpack requires a string argument of length 4

with >24k messages in my inbox, I really hope I don't loose too much of my seen 
data. anyone else have suggestions?

David Lang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: skiplist.py
Type: text/x-python
Size: 4593 bytes
Desc: 
Url : http://lists.andrew.cmu.edu/pipermail/info-cyrus/attachments/20130110/ce350c9a/attachment.py