cyrus-murder problems with database corruption in the frontend/master

[João Assad]

Derrick J Brashear wrote:

>
> So, prior to this presumably you've mmap2()'d some memory, have there 
> been any munmaps

for these mmap2s

00:25:58.583761 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68e8000
00:25:58.585461 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68e3000
00:25:58.586673 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68df000
00:25:58.588172 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68db000
00:25:58.589402 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68d7000
00:25:58.592491 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000
00:25:58.593194 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000
00:25:58.594149 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000
00:25:58.594661 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000
00:25:58.595493 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000
00:25:58.596004 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000


I have the corresponding munmaps.

00:25:58.584008 munmap(0xb68e8000, 4096) = 0
00:25:58.585683 munmap(0xb68e3000, 4096) = 0
00:25:58.586896 munmap(0xb68df000, 4096) = 0
00:25:58.588394 munmap(0xb68db000, 4096) = 0
00:25:58.589621 munmap(0xb68d7000, 4096) = 0
00:25:58.592712 munmap(0xb68cc000, 4096) = 0
00:25:58.593409 munmap(0xb68cc000, 4096) = 0
00:25:58.594363 munmap(0xb68cc000, 4096) = 0
00:25:58.594898 munmap(0xb68cc000, 4096) = 0
00:25:58.595706 munmap(0xb68cc000, 4096) = 0
00:25:58.596225 munmap(0xb68cc000, 4096) = 0

Im generating full traces now separated by threads, with each thread 
logging in a different file. So I belive we will be able to nail down 
exactly what the hell is going on.

I've also changed mupdate_workers options to the following values:

mupdate_workers_start: 50
mupdate_workers_minspare: 50
mupdate_workers_maxspare: 50
mupdate_workers_max: 50

to make sure I dont end up with 100000s strace files to look into.

---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html