[Disksim-users] DiskSim 3.0/FlashSim Simulation problems

Wed Mar 23 20:32:34 EDT 2011

Francesco,

I'm glad my previous procedures were helpful to you. That's why I wrote them
=)

I understand what you are saying about changing the subsystem. For example,
I am simulating only 1 SSD. The original Financial traces are simulating
multiple hard disks. I have modified the trace so that for devicenum, it is
always 0 (hard disk 0). I believe this is a valid modification to the trace
since it doesn't not change the request arrival times, size of each request,
etc. It is only changing the physical disks I have available to run the
simulation through. However, I'm not sure that changing the arrival times is
a legitimate change. For example, the Financial traces are very write
intensive - the time between writes is very small, i.e. the write request
rate is very high. If I changed these arrival times to be further spaced
apart, I cannot legitimately say that I ran my GC algorithm or FTL through
the Financial trace as now it is not as write intensive as it should be.
However, changing things about my subsystem: how many disks I have, what
kind of disks they are, etc. are completely valid changes. Hopefully that
clears up what I was trying to say.

With that said, I'm not sure if I understand exactly what you meant by this:
<<
Regarding this I am still in trouble since I read somewhere that it is
possible to configure simulator in order to make it behave as a closed
subsystem model also on external I/O traces making it ignore the 'arrival
times' have you an idea on how this can be done?
>>

I'm not sure how that can be done.

I read the link you sent me. I don't think I am implementing any magnetic
disks. Do you know where I can look in the code to check exactly which disks
are implemented? (I'm not that familiar with DiskSim yet).

Thanks,
Jonathan

On Wed, Mar 23, 2011 at 12:44 AM, Francesco Falanga
<ffalanga at fastwebnet.it>wrote:

>  Hi Jonatan,
>
> I just find out that I am following your same steps with FlashSim
> encouneing problem that you've already solved :)...
> Regarding your statement:
>
> "So if I slowed down the request-ratio, this would mean modifying the
> arrival times, which would invalidate the trace as it would no longer be
> real world"
>
> I have to respectfully disagree since a "trace" gives you the performance
> of a very specific system configuration. Changing one or more subsystem may
> have still sense in order to see how the new configuration will perform
> given the same request sequence.
>
> Regarding this I am still in trouble since I read somewhere that it is
> possible to configure simulator in order to make it behave as a closed
> subsystem model also on external I/O traces making it ignore the 'arrival
> times' have you an idea on how this can be done?
>
>
> Regarding segmentation fault I found this, but I guess you have already
> given a glance:
> https://sos.ece.cmu.edu/pipermail/disksim-users/2010-December/000556.html
>
>
> Ciao!
>
> Francesco.
>
>
>  ----- Original Message -----
> *From:* Jonathan Tjioe <jontjioe at gmail.com>
> *To:* Alexander Lochmann <alexander.lochmann at tu-dortmund.de>
> *Cc:* Aayush Gupta <axg354 at cse.psu.edu> ; Andres Blanco<andresblanco_89 at yahoo.com>;
> disksim-users at ece.cmu.edu ; Youngjae Kim <youkim at cse.psu.edu>
> *Sent:* Wednesday, March 23, 2011 3:47 AM
> *Subject:* Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems
>
> Alex,
>
> I've moved your responses up to the top so the rest of the forum won't get
> confused trying to distinguish the your responses from my questions.
>
> I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of
> the DFTL paper that my environment is based on. Hopefully, they can shed
> some light on what I'm doing wrong.
>
> On your response to Error #1 (Segmentation fault)...
>
> When I run it for the Financial1 trace file using all 3 FTLs, I get the
> following output to screen:
>
> Running Pagemap FTL...
> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152
> Running DFTL...
> ./runtest: line 11:  5905 Segmentation fault      ../src/disksim dftl.parv
> dftl.outv ascii ./trace/test.file 0
> Running FAST...
> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152
>
> As stated earlier, if I check the .outv files that were generated for each
> of these FTLs, none of the simulations ever completed.
>
> I have not run it using gdb. The code is compiled already. I suppose I
> could try to do so, but I would think that I should be able to run it as is.
>
> I have also attached the exact script that I run when I simulate the 3
> FTLs. Again, this is the same exact environment that the DFTL authors used.
>
> On your response to Error #2 (simulation stopped due to saturation), when
> running runtest script against the Financial2 trace file, I get this output
> to the screen.
>
> Running Pagemap FTL...
> Stopping simulation because of saturation: simtime 108.435867, totalreqs
> 10394
> IOdriver Response time average:         17.003924
> IOdriver Response time std.dev.:        15.487314
> Running DFTL...
> Stopping simulation because of saturation: simtime 107.151359, totalreqs
> 10255
> IOdriver Response time average:         25.488591
> IOdriver Response time std.dev.:        19.068197
> Running FAST...
> Stopping simulation because of saturation: simtime 106.554985, totalreqs
> 10188
> IOdriver Response time average:         10.128997
> IOdriver Response time std.dev.:        8.028486
>
>
> As far as slowing down the request-ratio...well, I could be
> misunderstanding what you are saying, but I thought that the way it works is
> that the real world requests come in exactly as the arrival times state in
> the trace file. Then if the hard drive(s) is(are) busy, then they just go in
> the request queue waiting to be serviced. I could understand that for a
> synthetically generated trace, it might be worth slowing down the request
> ratio just to see what happens, but this is a real world trace. So if I
> slowed down the request-ratio, this would mean modifying the arrival times,
> which would invalidate the trace as it would no longer be real world.
>
> Thanks,
> Jonathan
>
>
>
> On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann <
> alexander.lochmann at tu-dortmund.de> wrote:
> Hi!
>
> On Error #1 (Segmentation fault), could give us some more information?
> Have you tried to run it with gdb und typed "bt"?
>
> On Error #2 (Simulation stopped due to saturation), maybe your harddisk
> model is too slow to serve the requests within a appropriate amount of time
> so the requestqueue doesn't get saturated. In a real system you've got a
> operating system which keeps track of this issue. Linux for example, slows
> down every readahead and writeback activity to reduce the number of requests
> if it detects congestion conditions. It has a requestqueue which is on top
> of the driver holding every request.
> Have you tried to slow down the request-ratio?
>
> Greetings
> Alex
>
>  Am 22.03.2011 03:05, schrieb Jonathan Tjioe:
>>
>> DiskSim users,
>>
>> First, let me apologize for making this email so long. I wanted it to be
>> thorough so my problem is explained clearly.
>>
>> I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL
>> paper). It's basically DiskSim 3.0 with support for SSD.
>>
>> I've been running the "runtest" script which basically runs a small sample
>> test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace
>> that was included is very small (8-9MB) and I get the intended results as
>> the authors of the DFTL paper did.
>>
>> All the runtest script does is run the same trace file on these 3 FTLs:
>> ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0
>> ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0
>> ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0
>>
>>
>> Upon successful completion of the simulation, I noticed that in the .outv
>> files, I will see:
>> <<
>> ...
>> loadparams complete
>> Initialization complete
>> Simulation complete
>> ...
>> >>
>>
>> After this, I will see many performance statistics along with every
>> request. I get the appropriate results when simulating DFTL, FAST, and pure
>> page mapped FTL.
>>
>> However, once I tried putting any other real world trace (which is much
>> longer) in the simulation, it does not complete successfully. It should be
>> noted that I have not made any modifications to any of the 3 FTLs during
>> these tests. I have verified that my format of the trace files is correct:
>> <arrival time> <devicenum> <LBA> <size in # of sectors> <1 or 0 for read
>> or write>
>>
>> There are 3 different results that I get, all of which are unsuccessful
>> simulations:
>> 1) Segmentation fault
>>
>>  2) Simulation is stopped because of saturation
>>
>>  3) Simulation seems like it finished, but when you check the .outv
>> files, the last line says "initialization complete" (in otherwords it does
>> not ever say "simulation complete"
>>
>> One example (Error #1) is when I run the entire finanical trace in, I get
>> a segmentation fault simulating DFTL. Although I didn't get segmentation
>> faults for FAST or pure page mapped, their corresponding .outv files do not
>> look correct. They seem like they never finished simulating. In the .outv
>> files for all 3 FTLs, it shows:
>>
>> <<
>> ...
>> loadparams complete
>> Initialization complete
>> >>
>>
>> But it never shows "simulation complete" and thus the results are not
>> shown in the .outv file. The weird thing is that if I just include maybe
>> several hundred lines of the Financial1 trace instead of the entire thing,
>> it completes successfully with no problem, so I know my format of the file
>> is correct.
>>
>> Another example (Error #2) I noticed that I the Financial2 trace did not
>> have any segmentation faults but instead I had a different message:
>> <<
>> Stopping simulation because of saturation: simtime 108.435867, totalreqs
>> 10394
>> >>
>>
>> I did some research and found that there is a define statement in
>> disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the
>> instances when the simulation stopped due to saturation, the totalreqs
>> number was just over 10000. I imagine the queue length is getting very large
>> due to the fact that I only have a device num of 0 since my GC algorithm
>> will be a local GC algorithm, not a inter-disk algorithm.
>>
>> And lastly, if (Error #1) or (Error #2) occur, the simulation will never
>> complete and no simulation summary will be in the .outv file (which is Error
>> #3).
>>
>> My questions are as follows:
>> Regarding (Error #1), I have no idea why I am getting segmentation faults.
>> Do you think it is some type of buffer overflow issue b/c there is so much
>> data with the real world traces? Remember, that if I just take a smaller
>> subset of the real world data, it simulates to completion without any
>> problem.
>>
>> Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to
>> maybe 100000 and see what happens, but is that the right solution or is
>> there something else that I need to be aware of.
>>
>> Regarding (Error #3), I would assume that this will be solved once I find
>> the solutions to (Error #1) and (Error #2).
>>
>> I really appreciate any help or hints you can offer. I will also post this
>> to the disksims mailing list.
>>
>> Thanks for your valuable time,
>> Jonathan Tjioe
>>
>>
>> _______________________________________________
>> Disksim-users mailing listDisksim-users at ece.cmu.eduhttps://sos.ece.cmu.edu/mailman/listinfo/disksim-users
>>
>>
>>
>  ------------------------------
>
> _______________________________________________
> Disksim-users mailing list
> Disksim-users at ece.cmu.edu
> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/disksim-users/attachments/20110323/5e3307aa/attachment.html>