[Disksim-users] 'Make - clean' issue

[Jonathan Tjioe]

No problem. Glad to help. The word file was in .docx format. I think you
need Office 2007 or newer for it. Or you would have had to download/install
the free office compatibility pack. But attached is the pdf.

Here you go.

Jonathan

On Sat, Apr 2, 2011 at 11:22 AM, Francesco Falanga
<ffalanga at fastwebnet.it>wrote:

>  Hi Jonathan,
>
> your continuos support is very appreciated.
> Would you kindly try to send again Compiling_FlashSim doc again but in pdf
> format? I never got it before.
> I am having some issue while trying to open with word ( some conversion
> error).
>
> Thank you!
>
> Francesco.
>
> ----- Original Message -----
>
> *From:* Jonathan Tjioe <jontjioe at gmail.com>
> *To:* Francesco Falanga <ffalanga at fastwebnet.it>
> *Cc:* disksim-users at ece.cmu.edu
> *Sent:* Saturday, April 02, 2011 6:26 PM
> *Subject:* Re: 'Make - clean' issue
>
> Francesco,
>
> I had this same issue whenever I would do a 'make clean'. I am fortunate
> b/c the only files I really need to modify are dftl.c, fast.c, and
> pagemap.c. My workaround isn't the most elegant solution, but it works. In
> the Compiling FlashSim guide that I sent you previously, right before I do
> step 7 (which is the 'make' step), I create a snapshot of my Ubuntu VM.
> After the snapshot is complete, I can easily revert back to the snapshot,
> make any changes to my c code and then run 'make' again. Another way you can
> do it is to just create a copy of your 'src' directory prior to running a
> 'make' on it ('cp -p src src.orig'). And whenever you want to modify the
> code and recompile,  you need to just delete the src directory and copy the
> original one back in place ('rm -r src; cp -p src.orig src').
>
> One of the problems for the 'make clean' is that it deletes the files. When
> it tries to regenerate them again with the 'make', it puts several of the
> fprintf statements across multiple lines, which for some reason do not
> compile in the gcc version that we are running (I'm assuming it is the gcc
> version that is the culprit...remember, this version of disksim 3.0/flashsim
> is tightly coupled to gcc 3.3.6). So when I originally got disksim working,
> I had to manually grep for fprintf in the src directory (`grep fprintf
> ~/src`) and find all of the files that had fprintf statements that spanned
> multiple lines and then go modify all of these to be on a single line. I
> probably had to do this for 15-20 fprintf statements in various files
> throughout the code. So if you look at my steps in the Compiling Flash Sim
> guide, I have already modified all of these fprintf statements for you in
> the attached .gz file. Once you extract this src directory according to the
> instructions, this is what your src.orig should be based off of.
>
> Again, this isn't the most graceful solution but this worked for me.
>
> Also, please include the disksim users email list in your replies to me so
> that everyone can benefit from our discussion as I know many others are
> having the same issues. Thanks!
>
> To the other DiskSim users out there, if anyone knows the webadmin to this
> page: http://csl.cse.psu.edu/?q=node/322 could you please have them email
> me? I would like to post my FlashSim instructions and the updated tar file
> (both attached) on there for others to use so they can easily get FlashSim
> running.
>
> Thanks!
> Jonathan
>
> On Sat, Apr 2, 2011 at 8:25 AM, Francesco Falanga <ffalanga at fastwebnet.it>wrote:
>
>>  Hi Jonathan,
>>
>> As I have already mentioned everything is ok when I follow your
>> instruction for getting disksim 3.0 working. But when I issue a 'make clean'
>> to rebuild all the project ( I am trying to debug with code block) then I
>> get some src file modified again so that I get compilation errors.
>>
>> Can you please help me on this?
>>
>> Best Regards,
>>
>> Francesco.
>>
>> ----- Original Message -----
>> *From:* Jonathan Tjioe <jontjioe at gmail.com>
>> *To:* changyue <cy_3108079008 at hotmail.com>
>> *Cc:* disksim-users at ece.cmu.edu
>> *Sent:* Thursday, March 24, 2011 2:40 AM
>> *Subject:* Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems
>>
>> Can you be more specific? Which parameters in particular do I need to
>> modify?
>>
>> Basically, the only thing that I did was the following:
>> 1) Divided the request size (bytes) by 512 because my sectors are
>> 512Bytes. So now the size is in sectors. To my understanding, this is what
>> DiskSim requires.
>>
>> 2) I changed the device number to always be 0 (since I am only simulating
>> one SSD)
>>
>> 3) I moved the parameters around from SPC format to disksim format as
>> follows:
>> <arrival time> <devicenum> <LBA> <size in # of sectors> <1 or 0 for read
>> or write>
>>
>> Are there other modifications that I need to do besides that?
>>
>> Thanks,
>> Jonathan
>>
>> 2011/3/23 changyue <cy_3108079008 at hotmail.com>
>>
>>> Hi,
>>>      If you want to use the IO trace file directly that download from the
>>> website (for example,Financial 1), you must filter the trace file to fit
>>> your disk parameter. It is a hard question that I have tried do it nearly ,
>>> but not successful.
>>>      Thanks!
>>> ------------------------------
>>> Date: Tue, 22 Mar 2011 19:47:59 -0700
>>> From: jontjioe at gmail.com
>>> To: alexander.lochmann at tu-dortmund.de
>>> CC: axg354 at cse.psu.edu; andresblanco_89 at yahoo.com;
>>> disksim-users at ece.cmu.edu; youkim at cse.psu.edu
>>>
>>> Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems
>>>
>>>  Alex,
>>>
>>> I've moved your responses up to the top so the rest of the forum won't
>>> get confused trying to distinguish the your responses from my questions.
>>>
>>> I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of
>>> the DFTL paper that my environment is based on. Hopefully, they can shed
>>> some light on what I'm doing wrong.
>>>
>>> On your response to Error #1 (Segmentation fault)...
>>>
>>> When I run it for the Financial1 trace file using all 3 FTLs, I get the
>>> following output to screen:
>>>
>>> Running Pagemap FTL...
>>> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152
>>> Running DFTL...
>>> ./runtest: line 11:  5905 Segmentation fault      ../src/disksim
>>> dftl.parv dftl.outv ascii ./trace/test.file 0
>>> Running FAST...
>>> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152
>>>
>>> As stated earlier, if I check the .outv files that were generated for
>>> each of these FTLs, none of the simulations ever completed.
>>>
>>> I have not run it using gdb. The code is compiled already. I suppose I
>>> could try to do so, but I would think that I should be able to run it as is.
>>>
>>> I have also attached the exact script that I run when I simulate the 3
>>> FTLs. Again, this is the same exact environment that the DFTL authors used.
>>>
>>> On your response to Error #2 (simulation stopped due to saturation), when
>>> running runtest script against the Financial2 trace file, I get this output
>>> to the screen.
>>>
>>> Running Pagemap FTL...
>>> Stopping simulation because of saturation: simtime 108.435867, totalreqs
>>> 10394
>>> IOdriver Response time average:         17.003924
>>> IOdriver Response time std.dev.:        15.487314
>>> Running DFTL...
>>> Stopping simulation because of saturation: simtime 107.151359, totalreqs
>>> 10255
>>> IOdriver Response time average:         25.488591
>>> IOdriver Response time std.dev.:        19.068197
>>> Running FAST...
>>> Stopping simulation because of saturation: simtime 106.554985, totalreqs
>>> 10188
>>> IOdriver Response time average:         10.128997
>>> IOdriver Response time std.dev.:        8.028486
>>>
>>>
>>> As far as slowing down the request-ratio...well, I could be
>>> misunderstanding what you are saying, but I thought that the way it works is
>>> that the real world requests come in exactly as the arrival times state in
>>> the trace file. Then if the hard drive(s) is(are) busy, then they just go in
>>> the request queue waiting to be serviced. I could understand that for a
>>> synthetically generated trace, it might be worth slowing down the request
>>> ratio just to see what happens, but this is a real world trace. So if I
>>> slowed down the request-ratio, this would mean modifying the arrival times,
>>> which would invalidate the trace as it would no longer be real world.
>>>
>>> Thanks,
>>> Jonathan
>>>
>>>
>>>
>>> On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann <
>>> alexander.lochmann at tu-dortmund.de> wrote:
>>> Hi!
>>>
>>> On Error #1 (Segmentation fault), could give us some more information?
>>> Have you tried to run it with gdb und typed "bt"?
>>>
>>> On Error #2 (Simulation stopped due to saturation), maybe your harddisk
>>> model is too slow to serve the requests within a appropriate amount of time
>>> so the requestqueue doesn't get saturated. In a real system you've got a
>>> operating system which keeps track of this issue. Linux for example, slows
>>> down every readahead and writeback activity to reduce the number of requests
>>> if it detects congestion conditions. It has a requestqueue which is on top
>>> of the driver holding every request.
>>> Have you tried to slow down the request-ratio?
>>>
>>> Greetings
>>> Alex
>>>
>>>  Am 22.03.2011 03:05, schrieb Jonathan Tjioe:
>>>
>>> DiskSim users,
>>>
>>> First, let me apologize for making this email so long. I wanted it to be
>>> thorough so my problem is explained clearly.
>>>
>>> I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL
>>> paper). It's basically DiskSim 3.0 with support for SSD.
>>>
>>> I've been running the "runtest" script which basically runs a small
>>> sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test
>>> trace that was included is very small (8-9MB) and I get the intended results
>>> as the authors of the DFTL paper did.
>>>
>>> All the runtest script does is run the same trace file on these 3 FTLs:
>>> ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0
>>> ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0
>>> ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0
>>>
>>>
>>> Upon successful completion of the simulation, I noticed that in the .outv
>>> files, I will see:
>>> <<
>>> ...
>>> loadparams complete
>>> Initialization complete
>>> Simulation complete
>>> ...
>>> >>
>>>
>>> After this, I will see many performance statistics along with every
>>> request. I get the appropriate results when simulating DFTL, FAST, and pure
>>> page mapped FTL.
>>>
>>> However, once I tried putting any other real world trace (which is much
>>> longer) in the simulation, it does not complete successfully. It should be
>>> noted that I have not made any modifications to any of the 3 FTLs during
>>> these tests. I have verified that my format of the trace files is correct:
>>> <arrival time> <devicenum> <LBA> <size in # of sectors> <1 or 0 for read
>>> or write>
>>>
>>> There are 3 different results that I get, all of which are unsuccessful
>>> simulations:
>>> 1) Segmentation fault
>>>
>>>  2) Simulation is stopped because of saturation
>>>
>>>  3) Simulation seems like it finished, but when you check the .outv
>>> files, the last line says "initialization complete" (in otherwords it does
>>> not ever say "simulation complete"
>>>
>>> One example (Error #1) is when I run the entire finanical trace in, I get
>>> a segmentation fault simulating DFTL. Although I didn't get segmentation
>>> faults for FAST or pure page mapped, their corresponding .outv files do not
>>> look correct. They seem like they never finished simulating. In the .outv
>>> files for all 3 FTLs, it shows:
>>>
>>> <<
>>> ...
>>> loadparams complete
>>> Initialization complete
>>> >>
>>>
>>> But it never shows "simulation complete" and thus the results are not
>>> shown in the .outv file. The weird thing is that if I just include maybe
>>> several hundred lines of the Financial1 trace instead of the entire thing,
>>> it completes successfully with no problem, so I know my format of the file
>>> is correct.
>>>
>>> Another example (Error #2) I noticed that I the Financial2 trace did not
>>> have any segmentation faults but instead I had a different message:
>>> <<
>>> Stopping simulation because of saturation: simtime 108.435867, totalreqs
>>> 10394
>>> >>
>>>
>>> I did some research and found that there is a define statement in
>>> disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the
>>> instances when the simulation stopped due to saturation, the totalreqs
>>> number was just over 10000. I imagine the queue length is getting very large
>>> due to the fact that I only have a device num of 0 since my GC algorithm
>>> will be a local GC algorithm, not a inter-disk algorithm.
>>>
>>> And lastly, if (Error #1) or (Error #2) occur, the simulation will never
>>> complete and no simulation summary will be in the .outv file (which is Error
>>> #3).
>>>
>>> My questions are as follows:
>>> Regarding (Error #1), I have no idea why I am getting segmentation
>>> faults. Do you think it is some type of buffer overflow issue b/c there is
>>> so much data with the real world traces? Remember, that if I just take a
>>> smaller subset of the real world data, it simulates to completion without
>>> any problem.
>>>
>>> Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to
>>> maybe 100000 and see what happens, but is that the right solution or is
>>> there something else that I need to be aware of.
>>>
>>> Regarding (Error #3), I would assume that this will be solved once I find
>>> the solutions to (Error #1) and (Error #2).
>>>
>>> I really appreciate any help or hints you can offer. I will also post
>>> this to the disksims mailing list.
>>>
>>> Thanks for your valuable time,
>>> Jonathan Tjioe
>>>
>>>
>>> _______________________________________________
>>> Disksim-users mailing list
>>> Disksim-users at ece.cmu.edu
>>> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users
>>>
>>>
>>>
>>>
>>> _______________________________________________ Disksim-users mailing
>>> list Disksim-users at ece.cmu.edu
>>> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users
>>>
>>
>>  ------------------------------
>>
>> _______________________________________________
>> Disksim-users mailing list
>> Disksim-users at ece.cmu.edu
>> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.andrew.cmu.edu/pipermail/disksim-users/attachments/20110402/b920646d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Compiling_FlashSim_v1.0.pdf
Type: application/pdf
Size: 72194 bytes
Desc: not available
URL: <http://lists.andrew.cmu.edu/pipermail/disksim-users/attachments/20110402/b920646d/attachment.pdf>