Francesco,<br><br>I'm glad my previous procedures were helpful to you. That's why I wrote them =)<br><br>I understand what you are saying about changing the subsystem. For example, I am simulating only 1 SSD. The original Financial traces are simulating multiple hard disks. I have modified the trace so that for devicenum, it is always 0 (hard disk 0). I believe this is a valid modification to the trace since it doesn't not change the request arrival times, size of each request, etc. It is only changing the physical disks I have available to run the simulation through. However, I'm not sure that changing the arrival times is a legitimate change. For example, the Financial traces are very write intensive - the time between writes is very small, i.e. the write request rate is very high. If I changed these arrival times to be further spaced apart, I cannot legitimately say that I ran my GC algorithm or FTL through the Financial trace as now it is not as write intensive as it should be. However, changing things about my subsystem: how many disks I have, what kind of disks they are, etc. are completely valid changes. Hopefully that clears up what I was trying to say.<br>
<br>With that said, I'm not sure if I understand exactly what you meant by this:<br><font face="Arial" size="2"><<<br>Regarding this I am still in trouble since I read
somewhere that it is possible to configure simulator in order to make it behave
as a closed subsystem model also on external I/O traces making it ignore
the 'arrival times' have you an idea on how this can be done?<br>>><br><br>I'm not sure how that can be done.<br><br>I read the link you sent me. I don't think I am implementing any magnetic disks. Do you know where I can look in the code to check exactly which disks are implemented? (I'm not that familiar with DiskSim yet).<br>
<br>Thanks,<br>Jonathan<br></font><br><div class="gmail_quote">On Wed, Mar 23, 2011 at 12:44 AM, Francesco Falanga <span dir="ltr"><<a href="mailto:ffalanga@fastwebnet.it">ffalanga@fastwebnet.it</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div bgcolor="#ffffff">
<div><font face="Arial" size="2">Hi Jonatan,</font></div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2">I just find out that I am following your same steps
with FlashSim encouneing problem that you've already solved :)...</font></div>
<div><font face="Arial" size="2">Regarding your statement:</font></div><div class="im">
<div><font face="Arial" size="2"></font> </div>
<div>"So if I slowed down the request-ratio, this would mean modifying the
arrival times, which would invalidate the trace as it would no longer be real
world"</div>
<div> </div>
</div><div><font face="Arial" size="2">I have to respectfully disagree since a
"trace" gives you the performance of a very specific system configuration.
Changing one or more subsystem may have still sense in order to see how the new
configuration will perform given the same request sequence.</font></div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2">Regarding this I am still in trouble since I read
somewhere that it is possible to configure simulator in order to make it behave
as a closed subsystem model also on external I/O traces making it ignore
the 'arrival times' have you an idea on how this can be done?</font></div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2">Regarding segmentation fault I found this, but I
guess you have already given a glance:</font></div>
<div><a href="https://sos.ece.cmu.edu/pipermail/disksim-users/2010-December/000556.html" target="_blank">https://sos.ece.cmu.edu/pipermail/disksim-users/2010-December/000556.html</a></div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2"></font> </div>
<div><font face="Arial" size="2">Ciao!</font></div>
<div><font face="Arial" size="2"></font> </div><font color="#888888">
<div><font face="Arial" size="2">Francesco.</font></div>
<div><br><br></div>
</font><blockquote style="padding-right: 0px; padding-left: 5px; margin-left: 5px; border-left: 2px solid rgb(0, 0, 0); margin-right: 0px;"><div><div></div><div class="h5">
<div style="font: 10pt arial;">----- Original Message ----- </div>
<div style="background: none repeat scroll 0% 0% rgb(228, 228, 228); font: 10pt arial;"><b>From:</b>
<a title="jontjioe@gmail.com" href="mailto:jontjioe@gmail.com" target="_blank">Jonathan
Tjioe</a> </div>
<div style="font: 10pt arial;"><b>To:</b> <a title="alexander.lochmann@tu-dortmund.de" href="mailto:alexander.lochmann@tu-dortmund.de" target="_blank">Alexander Lochmann</a> </div>
<div style="font: 10pt arial;"><b>Cc:</b> <a title="axg354@cse.psu.edu" href="mailto:axg354@cse.psu.edu" target="_blank">Aayush Gupta</a> ; <a title="andresblanco_89@yahoo.com" href="mailto:andresblanco_89@yahoo.com" target="_blank">Andres
Blanco</a> ; <a title="disksim-users@ece.cmu.edu" href="mailto:disksim-users@ece.cmu.edu" target="_blank">disksim-users@ece.cmu.edu</a> ; <a title="youkim@cse.psu.edu" href="mailto:youkim@cse.psu.edu" target="_blank">Youngjae Kim</a>
</div>
<div style="font: 10pt arial;"><b>Sent:</b> Wednesday, March 23, 2011 3:47
AM</div>
<div style="font: 10pt arial;"><b>Subject:</b> Re: [Disksim-users] DiskSim
3.0/FlashSim Simulation problems</div>
<div><br></div>Alex,<br><br>I've moved your responses up to the top so the
rest of the forum won't get confused trying to distinguish the your responses
from my questions.<br><br>I've also CCed Aayush Gupta and Youngjae Kim as they
were the authors of the DFTL paper that my environment is based on. Hopefully,
they can shed some light on what I'm doing wrong.<br><br>On your response to
Error #1 (Segmentation fault)...<br><br>When I run it for the Financial1 trace
file using all 3 FTLs, I get the following output to screen:<br><br><span style="font-family: courier new,monospace;">Running Pagemap FTL...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">start_blk_no: 1107318608,
block_cnt: 4, total_util_sect_num: 2097152</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Running DFTL...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">./runtest: line 11: 5905
Segmentation fault ../src/disksim dftl.parv
dftl.outv ascii ./trace/test.file 0</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Running FAST...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">start_blk_no: 1107318608,
block_cnt: 4, total_util_sect_num: 2097152</span><br><br>As stated earlier, if
I check the .outv files that were generated for each of these FTLs, none of
the simulations ever completed.<br><br>I have not run it using gdb. The code
is compiled already. I suppose I could try to do so, but I would think that I
should be able to run it as is.<br><br>I have also attached the exact script
that I run when I simulate the 3 FTLs. Again, this is the same exact
environment that the DFTL authors used.<br><br>On your response to Error #2
(simulation stopped due to saturation), when running runtest script against
the Financial2 trace file, I get this output to the screen.<br><br><span style="font-family: courier new,monospace;">Running Pagemap FTL...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Stopping simulation because of
saturation: simtime 108.435867, totalreqs 10394</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
average: 17.003924</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
std.dev.: 15.487314</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Running DFTL...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Stopping simulation because of
saturation: simtime 107.151359, totalreqs 10255</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
average: 25.488591</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
std.dev.: 19.068197</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Running FAST...</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Stopping simulation because of
saturation: simtime 106.554985, totalreqs 10188</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
average: 10.128997</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">IOdriver Response time
std.dev.:
8.028486</span><br><br><br>As far as slowing down the request-ratio...well, I
could be misunderstanding what you are saying, but I thought that the way it
works is that the real world requests come in exactly as the arrival times
state in the trace file. Then if the hard drive(s) is(are) busy, then they
just go in the request queue waiting to be serviced. I could understand that
for a synthetically generated trace, it might be worth slowing down the
request ratio just to see what happens, but this is a real world trace. So if
I slowed down the request-ratio, this would mean modifying the arrival times,
which would invalidate the trace as it would no longer be real
world.<br><br>Thanks,<br>Jonathan<br><br><br><br>
<div class="gmail_quote">On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann
<span dir="ltr"><<a href="mailto:alexander.lochmann@tu-dortmund.de" target="_blank">alexander.lochmann@tu-dortmund.de</a>></span> wrote:
<div bgcolor="#ffffff" text="#000000">Hi!<br></div>
<div><br>On Error #1 (Segmentation fault), could give us some more
information?<br>Have you tried to run it with gdb und typed
"bt"?<br> <br>On Error #2 (Simulation stopped due to saturation), maybe
your harddisk model is too slow to serve the requests within a appropriate
amount of time so the requestqueue doesn't get saturated. In a real system
you've got a operating system which keeps track of this issue. Linux for
example, slows down every readahead and writeback activity to reduce the
number of requests if it detects congestion conditions. It has a requestqueue
which is on top of the driver holding every request.<br>Have you tried to slow
down the request-ratio?<br><br>Greetings<br>Alex<br><br></div>
<blockquote class="gmail_quote" style="padding-left: 1ex; margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204);">
<div bgcolor="#ffffff" text="#000000">Am 22.03.2011 03:05, schrieb Jonathan
Tjioe:
<div>
<blockquote type="cite">DiskSim users,<br><br>First, let me apologize for
making this email so long. I wanted it to be thorough so my problem is
explained clearly.<br><br>I'm running DiskSim 3.0 with FlashSim (same as
the FlashSim in the DFTL paper). It's basically DiskSim 3.0 with support
for SSD.<br><br>I've been running the "runtest" script which basically
runs a small sample test file through all 3 FTLs: Page mapped, DFTL, and
FAST. The test trace that was included is very small (8-9MB) and I get the
intended results as the authors of the DFTL paper did. <br><br>All the
runtest script does is run the same trace file on these 3 FTLs:<br><span style="font-family: courier new,monospace;">../src/disksim pagemap.parv
pagemap.outv ascii ./trace/test.file 0</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">../src/disksim dftl.parv
dftl.outv ascii ./trace/test.file 0</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">../src/disksim fast.parv
fast.outv ascii ./trace/test.file 0</span><br><br><br>Upon successful
completion of the simulation, I noticed that in the .outv files, I will
see:<br><span style="font-family: courier new,monospace;"><<</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">...</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">loadparams complete</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Initialization
complete</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Simulation complete</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">...</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">>></span><br><br>After
this, I will see many performance statistics along with every request. I
get the appropriate results when simulating DFTL, FAST, and pure page
mapped FTL.<br><br>However, once I tried putting any other real world
trace (which is much longer) in the simulation, it does not complete
successfully. It should be noted that I have not made any modifications to
any of the 3 FTLs during these tests. I have verified that my format of
the trace files is correct:<br><span style="font-family: courier new,monospace;"><arrival time>
<devicenum> <LBA> <size in # of sectors> <1 or 0 for
read or write></span><br><br>There are 3 different results that I get,
all of which are unsuccessful simulations:<br>1) Segmentation
fault</blockquote></div>
<div>
<blockquote type="cite">2) Simulation is stopped because of saturation
</blockquote></div>
<blockquote type="cite">
<div>3) Simulation seems like it finished, but when you check the .outv
files, the last line says "initialization complete" (in otherwords it does
not ever say "simulation complete"<br><br>One example (Error #1) is when I
run the entire finanical trace in, I get a segmentation fault simulating
DFTL. Although I didn't get segmentation faults for FAST or pure page
mapped, their corresponding .outv files do not look correct. They seem
like they never finished simulating. In the .outv files for all 3 FTLs, it
shows:<br><br><span style="font-family: courier new,monospace;"><<</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">...</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">loadparams complete</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Initialization
complete</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">>></span><br><br>But it
never shows "simulation complete" and thus the results are not shown in
the .outv file. The weird thing is that if I just include maybe several
hundred lines of the Financial1 trace instead of the entire thing, it
completes successfully with no problem, so I know my format of the file is
correct.<br><br>Another example (Error #2) I noticed that I the Financial2
trace did not have any segmentation faults but instead I had a different
message:<br><span style="font-family: courier new,monospace;"><<</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">Stopping simulation because of
saturation: simtime 108.435867, totalreqs 10394</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">>></span><br><br>I did
some research and found that there is a define statement in
disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of
the instances when the simulation stopped due to saturation, the totalreqs
number was just over 10000. I imagine the queue length is getting very
large due to the fact that I only have a device num of 0 since my GC
algorithm will be a local GC algorithm, not a inter-disk
algorithm.<br><br>And lastly, if (Error #1) or (Error #2) occur, the
simulation will never complete and no simulation summary will be in the
.outv file (which is Error #3).<br><br>My questions are as
follows:<br>Regarding (Error #1), I have no idea why I am getting
segmentation faults. Do you think it is some type of buffer overflow issue
b/c there is so much data with the real world traces? Remember, that if I
just take a smaller subset of the real world data, it simulates to
completion without any problem.<br><br>Regarding (Error #2), I can try
just increasing the MAX_QUEUE_LENGTH to maybe 100000 and see what happens,
but is that the right solution or is there something else that I need to
be aware of.<br><br>Regarding (Error #3), I would assume that this will be
solved once I find the solutions to (Error #1) and (Error #2).<br><br>I
really appreciate any help or hints you can offer. I will also post this
to the disksims mailing list.<br><br>Thanks for your valuable
time,<br>Jonathan Tjioe<br></div><pre><fieldset></fieldset>
_______________________________________________
Disksim-users mailing list
<a href="mailto:Disksim-users@ece.cmu.edu" target="_blank">Disksim-users@ece.cmu.edu</a>
<a href="https://sos.ece.cmu.edu/mailman/listinfo/disksim-users" target="_blank">https://sos.ece.cmu.edu/mailman/listinfo/disksim-users</a>
</pre></blockquote><br></div></blockquote></div><br>
</div></div><p>
</p><hr><div class="im">
<p></p>_______________________________________________<br>Disksim-users
mailing
list<br><a href="mailto:Disksim-users@ece.cmu.edu" target="_blank">Disksim-users@ece.cmu.edu</a><br><a href="https://sos.ece.cmu.edu/mailman/listinfo/disksim-users" target="_blank">https://sos.ece.cmu.edu/mailman/listinfo/disksim-users</a><br>
</div></blockquote></div>
</blockquote></div><br>