From krish097 at umn.edu Fri Apr 1 02:48:49 2011 From: krish097 at umn.edu (Chaithra Krishnarajanagar Nataraj) Date: Fri, 1 Apr 2011 01:48:49 -0500 Subject: [Disksim-users] IODriver Response time - With FlashSim integration Message-ID: Hi All, I have integrated the Object-Oriented FlashSim simulator from PSU with DiskSim along with FTL and GC modules (http://csl.cse.psu.edu/?q=node/321) and I am able to run the Financial 2 trace as well. However, I am not able to find out where to plug-in the read/write delays etc. for the SSD. The values that I want to plug-in can be seen in the attached file. The SSD is created by creating an object of the Ssd class and all I/O requests go to this class. The event_arrive() function returns the time taken to process the event. How do I integrate this time with the IODriver response time of the DiskSim simulator? Should the return value of event_arrive() be integrated with the IODriver response time or should I plug-in these delays to the DiskSim at a different place? It would be great if any of you can give me some information on these things. Thanks, Chaithra -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ssd.conf Type: application/octet-stream Size: 1458 bytes Desc: not available URL: From ffalanga at fastwebnet.it Sat Apr 2 14:22:49 2011 From: ffalanga at fastwebnet.it (Francesco Falanga) Date: Sat, 2 Apr 2011 20:22:49 +0200 Subject: [Disksim-users] 'Make - clean' issue References: <4D88594D.2050007@tu-dortmund.de> Message-ID: Hi Jonathan, your continuos support is very appreciated. Would you kindly try to send again Compiling_FlashSim doc again but in pdf format? I never got it before. I am having some issue while trying to open with word ( some conversion error). Thank you! Francesco. ----- Original Message ----- From: Jonathan Tjioe To: Francesco Falanga Cc: disksim-users at ece.cmu.edu Sent: Saturday, April 02, 2011 6:26 PM Subject: Re: 'Make - clean' issue Francesco, I had this same issue whenever I would do a 'make clean'. I am fortunate b/c the only files I really need to modify are dftl.c, fast.c, and pagemap.c. My workaround isn't the most elegant solution, but it works. In the Compiling FlashSim guide that I sent you previously, right before I do step 7 (which is the 'make' step), I create a snapshot of my Ubuntu VM. After the snapshot is complete, I can easily revert back to the snapshot, make any changes to my c code and then run 'make' again. Another way you can do it is to just create a copy of your 'src' directory prior to running a 'make' on it ('cp -p src src.orig'). And whenever you want to modify the code and recompile, you need to just delete the src directory and copy the original one back in place ('rm -r src; cp -p src.orig src'). One of the problems for the 'make clean' is that it deletes the files. When it tries to regenerate them again with the 'make', it puts several of the fprintf statements across multiple lines, which for some reason do not compile in the gcc version that we are running (I'm assuming it is the gcc version that is the culprit...remember, this version of disksim 3.0/flashsim is tightly coupled to gcc 3.3.6). So when I originally got disksim working, I had to manually grep for fprintf in the src directory (`grep fprintf ~/src`) and find all of the files that had fprintf statements that spanned multiple lines and then go modify all of these to be on a single line. I probably had to do this for 15-20 fprintf statements in various files throughout the code. So if you look at my steps in the Compiling Flash Sim guide, I have already modified all of these fprintf statements for you in the attached .gz file. Once you extract this src directory according to the instructions, this is what your src.orig should be based off of. Again, this isn't the most graceful solution but this worked for me. Also, please include the disksim users email list in your replies to me so that everyone can benefit from our discussion as I know many others are having the same issues. Thanks! To the other DiskSim users out there, if anyone knows the webadmin to this page: http://csl.cse.psu.edu/?q=node/322 could you please have them email me? I would like to post my FlashSim instructions and the updated tar file (both attached) on there for others to use so they can easily get FlashSim running. Thanks! Jonathan On Sat, Apr 2, 2011 at 8:25 AM, Francesco Falanga wrote: Hi Jonathan, As I have already mentioned everything is ok when I follow your instruction for getting disksim 3.0 working. But when I issue a 'make clean' to rebuild all the project ( I am trying to debug with code block) then I get some src file modified again so that I get compilation errors. Can you please help me on this? Best Regards, Francesco. ----- Original Message ----- From: Jonathan Tjioe To: changyue Cc: disksim-users at ece.cmu.edu Sent: Thursday, March 24, 2011 2:40 AM Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems Can you be more specific? Which parameters in particular do I need to modify? Basically, the only thing that I did was the following: 1) Divided the request size (bytes) by 512 because my sectors are 512Bytes. So now the size is in sectors. To my understanding, this is what DiskSim requires. 2) I changed the device number to always be 0 (since I am only simulating one SSD) 3) I moved the parameters around from SPC format to disksim format as follows: <1 or 0 for read or write> Are there other modifications that I need to do besides that? Thanks, Jonathan 2011/3/23 changyue Hi, If you want to use the IO trace file directly that download from the website (for example,Financial 1), you must filter the trace file to fit your disk parameter. It is a hard question that I have tried do it nearly , but not successful. Thanks! ------------------------------------------------------------------------ Date: Tue, 22 Mar 2011 19:47:59 -0700 From: jontjioe at gmail.com To: alexander.lochmann at tu-dortmund.de CC: axg354 at cse.psu.edu; andresblanco_89 at yahoo.com; disksim-users at ece.cmu.edu; youkim at cse.psu.edu Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems Alex, I've moved your responses up to the top so the rest of the forum won't get confused trying to distinguish the your responses from my questions. I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of the DFTL paper that my environment is based on. Hopefully, they can shed some light on what I'm doing wrong. On your response to Error #1 (Segmentation fault)... When I run it for the Financial1 trace file using all 3 FTLs, I get the following output to screen: Running Pagemap FTL... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 Running DFTL... ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 Running FAST... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 As stated earlier, if I check the .outv files that were generated for each of these FTLs, none of the simulations ever completed. I have not run it using gdb. The code is compiled already. I suppose I could try to do so, but I would think that I should be able to run it as is. I have also attached the exact script that I run when I simulate the 3 FTLs. Again, this is the same exact environment that the DFTL authors used. On your response to Error #2 (simulation stopped due to saturation), when running runtest script against the Financial2 trace file, I get this output to the screen. Running Pagemap FTL... Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 IOdriver Response time average: 17.003924 IOdriver Response time std.dev.: 15.487314 Running DFTL... Stopping simulation because of saturation: simtime 107.151359, totalreqs 10255 IOdriver Response time average: 25.488591 IOdriver Response time std.dev.: 19.068197 Running FAST... Stopping simulation because of saturation: simtime 106.554985, totalreqs 10188 IOdriver Response time average: 10.128997 IOdriver Response time std.dev.: 8.028486 As far as slowing down the request-ratio...well, I could be misunderstanding what you are saying, but I thought that the way it works is that the real world requests come in exactly as the arrival times state in the trace file. Then if the hard drive(s) is(are) busy, then they just go in the request queue waiting to be serviced. I could understand that for a synthetically generated trace, it might be worth slowing down the request ratio just to see what happens, but this is a real world trace. So if I slowed down the request-ratio, this would mean modifying the arrival times, which would invalidate the trace as it would no longer be real world. Thanks, Jonathan On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann wrote: Hi! On Error #1 (Segmentation fault), could give us some more information? Have you tried to run it with gdb und typed "bt"? On Error #2 (Simulation stopped due to saturation), maybe your harddisk model is too slow to serve the requests within a appropriate amount of time so the requestqueue doesn't get saturated. In a real system you've got a operating system which keeps track of this issue. Linux for example, slows down every readahead and writeback activity to reduce the number of requests if it detects congestion conditions. It has a requestqueue which is on top of the driver holding every request. Have you tried to slow down the request-ratio? Greetings Alex Am 22.03.2011 03:05, schrieb Jonathan Tjioe: DiskSim users, First, let me apologize for making this email so long. I wanted it to be thorough so my problem is explained clearly. I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL paper). It's basically DiskSim 3.0 with support for SSD. I've been running the "runtest" script which basically runs a small sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace that was included is very small (8-9MB) and I get the intended results as the authors of the DFTL paper did. All the runtest script does is run the same trace file on these 3 FTLs: ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 Upon successful completion of the simulation, I noticed that in the .outv files, I will see: << ... loadparams complete Initialization complete Simulation complete ... >> After this, I will see many performance statistics along with every request. I get the appropriate results when simulating DFTL, FAST, and pure page mapped FTL. However, once I tried putting any other real world trace (which is much longer) in the simulation, it does not complete successfully. It should be noted that I have not made any modifications to any of the 3 FTLs during these tests. I have verified that my format of the trace files is correct: <1 or 0 for read or write> There are 3 different results that I get, all of which are unsuccessful simulations: 1) Segmentation fault 2) Simulation is stopped because of saturation 3) Simulation seems like it finished, but when you check the .outv files, the last line says "initialization complete" (in otherwords it does not ever say "simulation complete" One example (Error #1) is when I run the entire finanical trace in, I get a segmentation fault simulating DFTL. Although I didn't get segmentation faults for FAST or pure page mapped, their corresponding .outv files do not look correct. They seem like they never finished simulating. In the .outv files for all 3 FTLs, it shows: << ... loadparams complete Initialization complete >> But it never shows "simulation complete" and thus the results are not shown in the .outv file. The weird thing is that if I just include maybe several hundred lines of the Financial1 trace instead of the entire thing, it completes successfully with no problem, so I know my format of the file is correct. Another example (Error #2) I noticed that I the Financial2 trace did not have any segmentation faults but instead I had a different message: << Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 >> I did some research and found that there is a define statement in disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the instances when the simulation stopped due to saturation, the totalreqs number was just over 10000. I imagine the queue length is getting very large due to the fact that I only have a device num of 0 since my GC algorithm will be a local GC algorithm, not a inter-disk algorithm. And lastly, if (Error #1) or (Error #2) occur, the simulation will never complete and no simulation summary will be in the .outv file (which is Error #3). My questions are as follows: Regarding (Error #1), I have no idea why I am getting segmentation faults. Do you think it is some type of buffer overflow issue b/c there is so much data with the real world traces? Remember, that if I just take a smaller subset of the real world data, it simulates to completion without any problem. Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to maybe 100000 and see what happens, but is that the right solution or is there something else that I need to be aware of. Regarding (Error #3), I would assume that this will be solved once I find the solutions to (Error #1) and (Error #2). I really appreciate any help or hints you can offer. I will also post this to the disksims mailing list. Thanks for your valuable time, Jonathan Tjioe _______________________________________________Disksim-users mailing listDisksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users _______________________________________________ Disksim-users mailing list Disksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------------------------------------------------------------------- _______________________________________________ Disksim-users mailing list Disksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From jontjioe at gmail.com Sat Apr 2 14:40:01 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Sat, 2 Apr 2011 11:40:01 -0700 Subject: [Disksim-users] 'Make - clean' issue In-Reply-To: References: <4D88594D.2050007@tu-dortmund.de> Message-ID: No problem. Glad to help. The word file was in .docx format. I think you need Office 2007 or newer for it. Or you would have had to download/install the free office compatibility pack. But attached is the pdf. Here you go. Jonathan On Sat, Apr 2, 2011 at 11:22 AM, Francesco Falanga wrote: > Hi Jonathan, > > your continuos support is very appreciated. > Would you kindly try to send again Compiling_FlashSim doc again but in pdf > format? I never got it before. > I am having some issue while trying to open with word ( some conversion > error). > > Thank you! > > Francesco. > > ----- Original Message ----- > > *From:* Jonathan Tjioe > *To:* Francesco Falanga > *Cc:* disksim-users at ece.cmu.edu > *Sent:* Saturday, April 02, 2011 6:26 PM > *Subject:* Re: 'Make - clean' issue > > Francesco, > > I had this same issue whenever I would do a 'make clean'. I am fortunate > b/c the only files I really need to modify are dftl.c, fast.c, and > pagemap.c. My workaround isn't the most elegant solution, but it works. In > the Compiling FlashSim guide that I sent you previously, right before I do > step 7 (which is the 'make' step), I create a snapshot of my Ubuntu VM. > After the snapshot is complete, I can easily revert back to the snapshot, > make any changes to my c code and then run 'make' again. Another way you can > do it is to just create a copy of your 'src' directory prior to running a > 'make' on it ('cp -p src src.orig'). And whenever you want to modify the > code and recompile, you need to just delete the src directory and copy the > original one back in place ('rm -r src; cp -p src.orig src'). > > One of the problems for the 'make clean' is that it deletes the files. When > it tries to regenerate them again with the 'make', it puts several of the > fprintf statements across multiple lines, which for some reason do not > compile in the gcc version that we are running (I'm assuming it is the gcc > version that is the culprit...remember, this version of disksim 3.0/flashsim > is tightly coupled to gcc 3.3.6). So when I originally got disksim working, > I had to manually grep for fprintf in the src directory (`grep fprintf > ~/src`) and find all of the files that had fprintf statements that spanned > multiple lines and then go modify all of these to be on a single line. I > probably had to do this for 15-20 fprintf statements in various files > throughout the code. So if you look at my steps in the Compiling Flash Sim > guide, I have already modified all of these fprintf statements for you in > the attached .gz file. Once you extract this src directory according to the > instructions, this is what your src.orig should be based off of. > > Again, this isn't the most graceful solution but this worked for me. > > Also, please include the disksim users email list in your replies to me so > that everyone can benefit from our discussion as I know many others are > having the same issues. Thanks! > > To the other DiskSim users out there, if anyone knows the webadmin to this > page: http://csl.cse.psu.edu/?q=node/322 could you please have them email > me? I would like to post my FlashSim instructions and the updated tar file > (both attached) on there for others to use so they can easily get FlashSim > running. > > Thanks! > Jonathan > > On Sat, Apr 2, 2011 at 8:25 AM, Francesco Falanga wrote: > >> Hi Jonathan, >> >> As I have already mentioned everything is ok when I follow your >> instruction for getting disksim 3.0 working. But when I issue a 'make clean' >> to rebuild all the project ( I am trying to debug with code block) then I >> get some src file modified again so that I get compilation errors. >> >> Can you please help me on this? >> >> Best Regards, >> >> Francesco. >> >> ----- Original Message ----- >> *From:* Jonathan Tjioe >> *To:* changyue >> *Cc:* disksim-users at ece.cmu.edu >> *Sent:* Thursday, March 24, 2011 2:40 AM >> *Subject:* Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems >> >> Can you be more specific? Which parameters in particular do I need to >> modify? >> >> Basically, the only thing that I did was the following: >> 1) Divided the request size (bytes) by 512 because my sectors are >> 512Bytes. So now the size is in sectors. To my understanding, this is what >> DiskSim requires. >> >> 2) I changed the device number to always be 0 (since I am only simulating >> one SSD) >> >> 3) I moved the parameters around from SPC format to disksim format as >> follows: >> <1 or 0 for read >> or write> >> >> Are there other modifications that I need to do besides that? >> >> Thanks, >> Jonathan >> >> 2011/3/23 changyue >> >>> Hi, >>> If you want to use the IO trace file directly that download from the >>> website (for example,Financial 1), you must filter the trace file to fit >>> your disk parameter. It is a hard question that I have tried do it nearly , >>> but not successful. >>> Thanks! >>> ------------------------------ >>> Date: Tue, 22 Mar 2011 19:47:59 -0700 >>> From: jontjioe at gmail.com >>> To: alexander.lochmann at tu-dortmund.de >>> CC: axg354 at cse.psu.edu; andresblanco_89 at yahoo.com; >>> disksim-users at ece.cmu.edu; youkim at cse.psu.edu >>> >>> Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems >>> >>> Alex, >>> >>> I've moved your responses up to the top so the rest of the forum won't >>> get confused trying to distinguish the your responses from my questions. >>> >>> I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of >>> the DFTL paper that my environment is based on. Hopefully, they can shed >>> some light on what I'm doing wrong. >>> >>> On your response to Error #1 (Segmentation fault)... >>> >>> When I run it for the Financial1 trace file using all 3 FTLs, I get the >>> following output to screen: >>> >>> Running Pagemap FTL... >>> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 >>> Running DFTL... >>> ./runtest: line 11: 5905 Segmentation fault ../src/disksim >>> dftl.parv dftl.outv ascii ./trace/test.file 0 >>> Running FAST... >>> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 >>> >>> As stated earlier, if I check the .outv files that were generated for >>> each of these FTLs, none of the simulations ever completed. >>> >>> I have not run it using gdb. The code is compiled already. I suppose I >>> could try to do so, but I would think that I should be able to run it as is. >>> >>> I have also attached the exact script that I run when I simulate the 3 >>> FTLs. Again, this is the same exact environment that the DFTL authors used. >>> >>> On your response to Error #2 (simulation stopped due to saturation), when >>> running runtest script against the Financial2 trace file, I get this output >>> to the screen. >>> >>> Running Pagemap FTL... >>> Stopping simulation because of saturation: simtime 108.435867, totalreqs >>> 10394 >>> IOdriver Response time average: 17.003924 >>> IOdriver Response time std.dev.: 15.487314 >>> Running DFTL... >>> Stopping simulation because of saturation: simtime 107.151359, totalreqs >>> 10255 >>> IOdriver Response time average: 25.488591 >>> IOdriver Response time std.dev.: 19.068197 >>> Running FAST... >>> Stopping simulation because of saturation: simtime 106.554985, totalreqs >>> 10188 >>> IOdriver Response time average: 10.128997 >>> IOdriver Response time std.dev.: 8.028486 >>> >>> >>> As far as slowing down the request-ratio...well, I could be >>> misunderstanding what you are saying, but I thought that the way it works is >>> that the real world requests come in exactly as the arrival times state in >>> the trace file. Then if the hard drive(s) is(are) busy, then they just go in >>> the request queue waiting to be serviced. I could understand that for a >>> synthetically generated trace, it might be worth slowing down the request >>> ratio just to see what happens, but this is a real world trace. So if I >>> slowed down the request-ratio, this would mean modifying the arrival times, >>> which would invalidate the trace as it would no longer be real world. >>> >>> Thanks, >>> Jonathan >>> >>> >>> >>> On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann < >>> alexander.lochmann at tu-dortmund.de> wrote: >>> Hi! >>> >>> On Error #1 (Segmentation fault), could give us some more information? >>> Have you tried to run it with gdb und typed "bt"? >>> >>> On Error #2 (Simulation stopped due to saturation), maybe your harddisk >>> model is too slow to serve the requests within a appropriate amount of time >>> so the requestqueue doesn't get saturated. In a real system you've got a >>> operating system which keeps track of this issue. Linux for example, slows >>> down every readahead and writeback activity to reduce the number of requests >>> if it detects congestion conditions. It has a requestqueue which is on top >>> of the driver holding every request. >>> Have you tried to slow down the request-ratio? >>> >>> Greetings >>> Alex >>> >>> Am 22.03.2011 03:05, schrieb Jonathan Tjioe: >>> >>> DiskSim users, >>> >>> First, let me apologize for making this email so long. I wanted it to be >>> thorough so my problem is explained clearly. >>> >>> I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL >>> paper). It's basically DiskSim 3.0 with support for SSD. >>> >>> I've been running the "runtest" script which basically runs a small >>> sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test >>> trace that was included is very small (8-9MB) and I get the intended results >>> as the authors of the DFTL paper did. >>> >>> All the runtest script does is run the same trace file on these 3 FTLs: >>> ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 >>> ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 >>> ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 >>> >>> >>> Upon successful completion of the simulation, I noticed that in the .outv >>> files, I will see: >>> << >>> ... >>> loadparams complete >>> Initialization complete >>> Simulation complete >>> ... >>> >> >>> >>> After this, I will see many performance statistics along with every >>> request. I get the appropriate results when simulating DFTL, FAST, and pure >>> page mapped FTL. >>> >>> However, once I tried putting any other real world trace (which is much >>> longer) in the simulation, it does not complete successfully. It should be >>> noted that I have not made any modifications to any of the 3 FTLs during >>> these tests. I have verified that my format of the trace files is correct: >>> <1 or 0 for read >>> or write> >>> >>> There are 3 different results that I get, all of which are unsuccessful >>> simulations: >>> 1) Segmentation fault >>> >>> 2) Simulation is stopped because of saturation >>> >>> 3) Simulation seems like it finished, but when you check the .outv >>> files, the last line says "initialization complete" (in otherwords it does >>> not ever say "simulation complete" >>> >>> One example (Error #1) is when I run the entire finanical trace in, I get >>> a segmentation fault simulating DFTL. Although I didn't get segmentation >>> faults for FAST or pure page mapped, their corresponding .outv files do not >>> look correct. They seem like they never finished simulating. In the .outv >>> files for all 3 FTLs, it shows: >>> >>> << >>> ... >>> loadparams complete >>> Initialization complete >>> >> >>> >>> But it never shows "simulation complete" and thus the results are not >>> shown in the .outv file. The weird thing is that if I just include maybe >>> several hundred lines of the Financial1 trace instead of the entire thing, >>> it completes successfully with no problem, so I know my format of the file >>> is correct. >>> >>> Another example (Error #2) I noticed that I the Financial2 trace did not >>> have any segmentation faults but instead I had a different message: >>> << >>> Stopping simulation because of saturation: simtime 108.435867, totalreqs >>> 10394 >>> >> >>> >>> I did some research and found that there is a define statement in >>> disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the >>> instances when the simulation stopped due to saturation, the totalreqs >>> number was just over 10000. I imagine the queue length is getting very large >>> due to the fact that I only have a device num of 0 since my GC algorithm >>> will be a local GC algorithm, not a inter-disk algorithm. >>> >>> And lastly, if (Error #1) or (Error #2) occur, the simulation will never >>> complete and no simulation summary will be in the .outv file (which is Error >>> #3). >>> >>> My questions are as follows: >>> Regarding (Error #1), I have no idea why I am getting segmentation >>> faults. Do you think it is some type of buffer overflow issue b/c there is >>> so much data with the real world traces? Remember, that if I just take a >>> smaller subset of the real world data, it simulates to completion without >>> any problem. >>> >>> Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to >>> maybe 100000 and see what happens, but is that the right solution or is >>> there something else that I need to be aware of. >>> >>> Regarding (Error #3), I would assume that this will be solved once I find >>> the solutions to (Error #1) and (Error #2). >>> >>> I really appreciate any help or hints you can offer. I will also post >>> this to the disksims mailing list. >>> >>> Thanks for your valuable time, >>> Jonathan Tjioe >>> >>> >>> _______________________________________________ >>> Disksim-users mailing list >>> Disksim-users at ece.cmu.edu >>> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users >>> >>> >>> >>> >>> _______________________________________________ Disksim-users mailing >>> list Disksim-users at ece.cmu.edu >>> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users >>> >> >> ------------------------------ >> >> _______________________________________________ >> Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Compiling_FlashSim_v1.0.pdf Type: application/pdf Size: 72194 bytes Desc: not available URL: From jontjioe at gmail.com Sat Apr 2 12:26:39 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Sat, 2 Apr 2011 09:26:39 -0700 Subject: [Disksim-users] 'Make - clean' issue In-Reply-To: References: <4D88594D.2050007@tu-dortmund.de> Message-ID: Francesco, I had this same issue whenever I would do a 'make clean'. I am fortunate b/c the only files I really need to modify are dftl.c, fast.c, and pagemap.c. My workaround isn't the most elegant solution, but it works. In the Compiling FlashSim guide that I sent you previously, right before I do step 7 (which is the 'make' step), I create a snapshot of my Ubuntu VM. After the snapshot is complete, I can easily revert back to the snapshot, make any changes to my c code and then run 'make' again. Another way you can do it is to just create a copy of your 'src' directory prior to running a 'make' on it ('cp -p src src.orig'). And whenever you want to modify the code and recompile, you need to just delete the src directory and copy the original one back in place ('rm -r src; cp -p src.orig src'). One of the problems for the 'make clean' is that it deletes the files. When it tries to regenerate them again with the 'make', it puts several of the fprintf statements across multiple lines, which for some reason do not compile in the gcc version that we are running (I'm assuming it is the gcc version that is the culprit...remember, this version of disksim 3.0/flashsim is tightly coupled to gcc 3.3.6). So when I originally got disksim working, I had to manually grep for fprintf in the src directory (`grep fprintf ~/src`) and find all of the files that had fprintf statements that spanned multiple lines and then go modify all of these to be on a single line. I probably had to do this for 15-20 fprintf statements in various files throughout the code. So if you look at my steps in the Compiling Flash Sim guide, I have already modified all of these fprintf statements for you in the attached .gz file. Once you extract this src directory according to the instructions, this is what your src.orig should be based off of. Again, this isn't the most graceful solution but this worked for me. Also, please include the disksim users email list in your replies to me so that everyone can benefit from our discussion as I know many others are having the same issues. Thanks! To the other DiskSim users out there, if anyone knows the webadmin to this page: http://csl.cse.psu.edu/?q=node/322 could you please have them email me? I would like to post my FlashSim instructions and the updated tar file (both attached) on there for others to use so they can easily get FlashSim running. Thanks! Jonathan On Sat, Apr 2, 2011 at 8:25 AM, Francesco Falanga wrote: > Hi Jonathan, > > As I have already mentioned everything is ok when I follow your instruction > for getting disksim 3.0 working. But when I issue a 'make clean' to rebuild > all the project ( I am trying to debug with code block) then I get some src > file modified again so that I get compilation errors. > > Can you please help me on this? > > Best Regards, > > Francesco. > > ----- Original Message ----- > *From:* Jonathan Tjioe > *To:* changyue > *Cc:* disksim-users at ece.cmu.edu > *Sent:* Thursday, March 24, 2011 2:40 AM > *Subject:* Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems > > Can you be more specific? Which parameters in particular do I need to > modify? > > Basically, the only thing that I did was the following: > 1) Divided the request size (bytes) by 512 because my sectors are 512Bytes. > So now the size is in sectors. To my understanding, this is what DiskSim > requires. > > 2) I changed the device number to always be 0 (since I am only simulating > one SSD) > > 3) I moved the parameters around from SPC format to disksim format as > follows: > <1 or 0 for read or > write> > > Are there other modifications that I need to do besides that? > > Thanks, > Jonathan > > 2011/3/23 changyue > >> Hi, >> If you want to use the IO trace file directly that download from the >> website (for example,Financial 1), you must filter the trace file to fit >> your disk parameter. It is a hard question that I have tried do it nearly , >> but not successful. >> Thanks! >> ------------------------------ >> Date: Tue, 22 Mar 2011 19:47:59 -0700 >> From: jontjioe at gmail.com >> To: alexander.lochmann at tu-dortmund.de >> CC: axg354 at cse.psu.edu; andresblanco_89 at yahoo.com; >> disksim-users at ece.cmu.edu; youkim at cse.psu.edu >> >> Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems >> >> Alex, >> >> I've moved your responses up to the top so the rest of the forum won't get >> confused trying to distinguish the your responses from my questions. >> >> I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of >> the DFTL paper that my environment is based on. Hopefully, they can shed >> some light on what I'm doing wrong. >> >> On your response to Error #1 (Segmentation fault)... >> >> When I run it for the Financial1 trace file using all 3 FTLs, I get the >> following output to screen: >> >> Running Pagemap FTL... >> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 >> Running DFTL... >> ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv >> dftl.outv ascii ./trace/test.file 0 >> Running FAST... >> start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 >> >> As stated earlier, if I check the .outv files that were generated for each >> of these FTLs, none of the simulations ever completed. >> >> I have not run it using gdb. The code is compiled already. I suppose I >> could try to do so, but I would think that I should be able to run it as is. >> >> I have also attached the exact script that I run when I simulate the 3 >> FTLs. Again, this is the same exact environment that the DFTL authors used. >> >> On your response to Error #2 (simulation stopped due to saturation), when >> running runtest script against the Financial2 trace file, I get this output >> to the screen. >> >> Running Pagemap FTL... >> Stopping simulation because of saturation: simtime 108.435867, totalreqs >> 10394 >> IOdriver Response time average: 17.003924 >> IOdriver Response time std.dev.: 15.487314 >> Running DFTL... >> Stopping simulation because of saturation: simtime 107.151359, totalreqs >> 10255 >> IOdriver Response time average: 25.488591 >> IOdriver Response time std.dev.: 19.068197 >> Running FAST... >> Stopping simulation because of saturation: simtime 106.554985, totalreqs >> 10188 >> IOdriver Response time average: 10.128997 >> IOdriver Response time std.dev.: 8.028486 >> >> >> As far as slowing down the request-ratio...well, I could be >> misunderstanding what you are saying, but I thought that the way it works is >> that the real world requests come in exactly as the arrival times state in >> the trace file. Then if the hard drive(s) is(are) busy, then they just go in >> the request queue waiting to be serviced. I could understand that for a >> synthetically generated trace, it might be worth slowing down the request >> ratio just to see what happens, but this is a real world trace. So if I >> slowed down the request-ratio, this would mean modifying the arrival times, >> which would invalidate the trace as it would no longer be real world. >> >> Thanks, >> Jonathan >> >> >> >> On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann < >> alexander.lochmann at tu-dortmund.de> wrote: >> Hi! >> >> On Error #1 (Segmentation fault), could give us some more information? >> Have you tried to run it with gdb und typed "bt"? >> >> On Error #2 (Simulation stopped due to saturation), maybe your harddisk >> model is too slow to serve the requests within a appropriate amount of time >> so the requestqueue doesn't get saturated. In a real system you've got a >> operating system which keeps track of this issue. Linux for example, slows >> down every readahead and writeback activity to reduce the number of requests >> if it detects congestion conditions. It has a requestqueue which is on top >> of the driver holding every request. >> Have you tried to slow down the request-ratio? >> >> Greetings >> Alex >> >> Am 22.03.2011 03:05, schrieb Jonathan Tjioe: >> >> DiskSim users, >> >> First, let me apologize for making this email so long. I wanted it to be >> thorough so my problem is explained clearly. >> >> I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL >> paper). It's basically DiskSim 3.0 with support for SSD. >> >> I've been running the "runtest" script which basically runs a small sample >> test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace >> that was included is very small (8-9MB) and I get the intended results as >> the authors of the DFTL paper did. >> >> All the runtest script does is run the same trace file on these 3 FTLs: >> ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 >> ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 >> ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 >> >> >> Upon successful completion of the simulation, I noticed that in the .outv >> files, I will see: >> << >> ... >> loadparams complete >> Initialization complete >> Simulation complete >> ... >> >> >> >> After this, I will see many performance statistics along with every >> request. I get the appropriate results when simulating DFTL, FAST, and pure >> page mapped FTL. >> >> However, once I tried putting any other real world trace (which is much >> longer) in the simulation, it does not complete successfully. It should be >> noted that I have not made any modifications to any of the 3 FTLs during >> these tests. I have verified that my format of the trace files is correct: >> <1 or 0 for read >> or write> >> >> There are 3 different results that I get, all of which are unsuccessful >> simulations: >> 1) Segmentation fault >> >> 2) Simulation is stopped because of saturation >> >> 3) Simulation seems like it finished, but when you check the .outv >> files, the last line says "initialization complete" (in otherwords it does >> not ever say "simulation complete" >> >> One example (Error #1) is when I run the entire finanical trace in, I get >> a segmentation fault simulating DFTL. Although I didn't get segmentation >> faults for FAST or pure page mapped, their corresponding .outv files do not >> look correct. They seem like they never finished simulating. In the .outv >> files for all 3 FTLs, it shows: >> >> << >> ... >> loadparams complete >> Initialization complete >> >> >> >> But it never shows "simulation complete" and thus the results are not >> shown in the .outv file. The weird thing is that if I just include maybe >> several hundred lines of the Financial1 trace instead of the entire thing, >> it completes successfully with no problem, so I know my format of the file >> is correct. >> >> Another example (Error #2) I noticed that I the Financial2 trace did not >> have any segmentation faults but instead I had a different message: >> << >> Stopping simulation because of saturation: simtime 108.435867, totalreqs >> 10394 >> >> >> >> I did some research and found that there is a define statement in >> disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the >> instances when the simulation stopped due to saturation, the totalreqs >> number was just over 10000. I imagine the queue length is getting very large >> due to the fact that I only have a device num of 0 since my GC algorithm >> will be a local GC algorithm, not a inter-disk algorithm. >> >> And lastly, if (Error #1) or (Error #2) occur, the simulation will never >> complete and no simulation summary will be in the .outv file (which is Error >> #3). >> >> My questions are as follows: >> Regarding (Error #1), I have no idea why I am getting segmentation faults. >> Do you think it is some type of buffer overflow issue b/c there is so much >> data with the real world traces? Remember, that if I just take a smaller >> subset of the real world data, it simulates to completion without any >> problem. >> >> Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to >> maybe 100000 and see what happens, but is that the right solution or is >> there something else that I need to be aware of. >> >> Regarding (Error #3), I would assume that this will be solved once I find >> the solutions to (Error #1) and (Error #2). >> >> I really appreciate any help or hints you can offer. I will also post this >> to the disksims mailing list. >> >> Thanks for your valuable time, >> Jonathan Tjioe >> >> >> _______________________________________________ >> Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users >> >> >> >> >> _______________________________________________ Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users >> > > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Compiling_FlashSim_v1.0.doc Type: application/msword Size: 120314 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: src.release.20101205.tar.gz Type: application/x-gzip Size: 651570 bytes Desc: not available URL: From myidpt at gmail.com Mon Apr 4 15:26:36 2011 From: myidpt at gmail.com (Yonggang Liu) Date: Mon, 4 Apr 2011 19:26:36 +0000 (UTC) Subject: [Disksim-users] Yonggang Liu wants to stay in touch on LinkedIn Message-ID: <726913805.3912972.1301945196235.JavaMail.app@ela4-bed35.prod> LinkedIn ------------ I'd like to add you to my professional network on LinkedIn. - Yonggang Liu Yonggang Liu Research Assistant at University of Florida Gainesville, Florida Area Confirm that you know Yonggang Liu https://www.linkedin.com/e/fn51j5-gm3sjyrt-26/isd/2633020046/VDh2mWAv/ -- (c) 2011, LinkedIn Corporation -------------- next part -------------- An HTML attachment was scrubbed... URL: From cy_3108079008 at hotmail.com Tue Apr 5 10:45:48 2011 From: cy_3108079008 at hotmail.com (changyue) Date: Tue, 5 Apr 2011 14:45:48 +0000 Subject: [Disksim-users] RAM TIME Message-ID: Hi,all Did someone know how the latency of RAM infulence the disk's transfer rate? I add the cache function into the disksim, but find that the running time is so huge because of the reading of the RAM information is too slow. But by the increasing the running time, the response time of disk is not increase. That means the reponse time of disk is not conrresponding to the simulation time. Can anyone give an explanation? Thank you! Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From frajaei55 at gmail.com Sat Apr 9 07:27:37 2011 From: frajaei55 at gmail.com (f.rajaei) Date: Sat, 9 Apr 2011 15:57:37 +0430 Subject: [Disksim-users] parallel flash chips Message-ID: Dear All, Does anybody know if it is possible to write or read on an SSD emelents (flash chips) in parallel fasion (striping) since simulation results show different number of writes or reads on different elements of an SSD. Regards, Farzaneh -------------- next part -------------- An HTML attachment was scrubbed... URL: From frajaei55 at gmail.com Sun Apr 10 23:40:26 2011 From: frajaei55 at gmail.com (f.rajaei) Date: Sun, 10 Apr 2011 20:40:26 -0700 Subject: [Disksim-users] number of parallel units Message-ID: Hi All, Does anybody know that number of parallel unit in a parameter file for SSD extension is related to planes or elements? Regards, Farzaneh -------------- next part -------------- An HTML attachment was scrubbed... URL: From adamcrume at gmail.com Fri Apr 15 14:45:52 2011 From: adamcrume at gmail.com (Adam Crume) Date: Fri, 15 Apr 2011 11:45:52 -0700 Subject: [Disksim-users] Compile error - duplicate case value Message-ID: I ran into the same problem. The code won't compile, and the problem seems to be this assertion: StaticAssert (sizeof(ioreq_event) <= DISKSIM_EVENT_SIZE); Any fixes/workarounds? Here's my setup: Linux birgitte 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:46 UTC 2011 x86_64 GNU/Linux gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3 -------------- next part -------------- An HTML attachment was scrubbed... URL: From adamcrume at gmail.com Fri Apr 15 14:49:08 2011 From: adamcrume at gmail.com (Adam Crume) Date: Fri, 15 Apr 2011 11:49:08 -0700 Subject: [Disksim-users] Compile error - duplicate case value In-Reply-To: References: Message-ID: Sorry, it looks like the in-reply-to got swallowed. Here's the original message: https://sos.ece.cmu.edu/pipermail/disksim-users/2009-January/000359.html On Fri, Apr 15, 2011 at 11:45 AM, Adam Crume wrote: > I ran into the same problem. The code won't compile, and the problem seems > to be this assertion: > > StaticAssert (sizeof(ioreq_event) <= DISKSIM_EVENT_SIZE); > > Any fixes/workarounds? > > Here's my setup: > Linux birgitte 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:46 UTC 2011 > x86_64 GNU/Linux > gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stroucki at ece.cmu.edu Fri Apr 15 15:35:30 2011 From: stroucki at ece.cmu.edu (Michael Stroucken) Date: Fri, 15 Apr 2011 15:35:30 -0400 Subject: [Disksim-users] Compile error - duplicate case value In-Reply-To: References: Message-ID: <4DA89E02.9080600@ece.cmu.edu> Adam Crume wrote: > Sorry, it looks like the in-reply-to got swallowed. Here's the > original message: > https://sos.ece.cmu.edu/pipermail/disksim-users/2009-January/000359.html > > On Fri, Apr 15, 2011 at 11:45 AM, Adam Crume > wrote: > > I ran into the same problem. The code won't compile, and the > problem seems to be this assertion: > > StaticAssert (sizeof(ioreq_event) <= DISKSIM_EVENT_SIZE); > > Any fixes/workarounds? > > Here's my setup: > Linux birgitte 2.6.32-30-generic #59-Ubuntu SMP Tue Mar 1 21:30:46 > UTC 2011 x86_64 GNU/Linux > gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3 > > Here's what I would say:- StaticAssert is a macro defined in disksim_global.h. #define StaticAssert(c) switch (c) case 0: case (c): The point is to detect a problem before the code is run, although the error message that is generated is confusing. Since sizeof(ioreq_event) and DISKSIM_EVENT_SIZE are known at compile time, I would assume the preprocessor does the evaluation and inserts the result in the place of the comparison. Let's say the result is 0, ie the condition asserted to be true is in fact false. Then this code will turn into:- switch (0) case 0: case (0): And gcc will complain. The source reason for this failure is you are trying to compile disksim on a 64 bit system. disksim is not 64-bit clean, so compile this in 32-bit mode by changing the gcc invocation to have the -m32 flag. Hope this helps, Michael. From dahlawi at IEEE.ORG Fri Apr 29 08:12:36 2011 From: dahlawi at IEEE.ORG (Al-Dahlawi) Date: Fri, 29 Apr 2011 08:12:36 -0400 Subject: [Disksim-users] Trace File Message-ID: Greeting all I am trying to figure out the block size in terms of KB for each line in the trace files that is part of disksim package. For the first line R : means Read 265552 : means start at logical block number 265552 2 : means read 2 block but what is the block size in this case ??? what are the other two number 14901.00000 & 4757.708232 ?? R Hit 265552 2 14901.000000 4757.708232 R Hit 751567 44 21725.000000 7050.375042 R Hit 310927 37 24368.000000 15410.132732 W Hit 311664 30 12628.000000 17035.849408 W Hit 493734 12 18952.000000 11015.968446 Thanks -- Abdullah Al-Dahlawi Department. Of Electrical & Computer Engineering George Washington University ---- Check The Fastest 500 Super Computers Worldwide http://www.top500.org/list/2010/06/100 -------------- next part -------------- An HTML attachment was scrubbed... URL: From haghdoost at gmail.com Fri Apr 29 08:42:38 2011 From: haghdoost at gmail.com (Alireza Haghdoost) Date: Fri, 29 Apr 2011 17:12:38 +0430 Subject: [Disksim-users] Trace File In-Reply-To: References: Message-ID: > > 2 : means read 2 block > but what is the block size in this case ??? > 2 means reading 2 sector , each sector typically is 512 byte. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dahlawi at IEEE.ORG Fri Apr 29 08:44:57 2011 From: dahlawi at IEEE.ORG (Al-Dahlawi) Date: Fri, 29 Apr 2011 08:44:57 -0400 Subject: [Disksim-users] Trace File In-Reply-To: References: Message-ID: Thanks Alireza Can I modify the Parv file to change the sector size ? On Fri, Apr 29, 2011 at 8:42 AM, Alireza Haghdoost wrote: > 2 : means read 2 block >> but what is the block size in this case ??? >> > > > 2 means reading 2 sector , each sector typically is 512 byte. > > > -- Abdullah Al-Dahlawi PhD Candidate High Performance Computing LAB Department. Of Electrical & Computer Engineering George Washington University ---- Check The Fastest 500 Super Computers Worldwide http://www.top500.org/list/2010/06/100 -------------- next part -------------- An HTML attachment was scrubbed... URL: