From cy_3108079008 at hotmail.com Mon Mar 7 00:18:16 2011 From: cy_3108079008 at hotmail.com (changyue) Date: Mon, 7 Mar 2011 05:18:16 +0000 Subject: [Disksim-users] access time modify Message-ID: Hello, Everyone, I want to modify the read and write time of the simpledisk module. Does anybody know what can I do if I do change the access time of simpledisk module's behavior. So I can set the read time and write time of the simpledisk! Waitting for the reply! Thanks! Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From cy_3108079008 at hotmail.com Thu Mar 10 03:30:01 2011 From: cy_3108079008 at hotmail.com (changyue) Date: Thu, 10 Mar 2011 08:30:01 +0000 Subject: [Disksim-users] HDD physical address Message-ID: Hello, Everyone who use the Disksim simulation tool. May I ask a simple question about disksim? If you have the logical sector address from the IO request, then how you can get the physical sector address from the disksim? Does disksim have the function that show the physical sector address of the Hard Disk. Waiting for the reply! Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.lochmann at tu-dortmund.de Thu Mar 10 03:36:39 2011 From: alexander.lochmann at tu-dortmund.de (Alexander Lochmann) Date: Thu, 10 Mar 2011 09:36:39 +0100 Subject: [Disksim-users] HDD physical address In-Reply-To: References: Message-ID: <4D788D97.7080301@tu-dortmund.de> Hi! Am 10.03.2011 09:30, schrieb changyue: > > Hello, Everyone who use the Disksim simulation tool. May I ask a > simple question about disksim? > If you have the logical sector address from the IO request, then how > you can get the physical sector address from the disksim? I think you're wrong. The disksim receives a physical sector address and a request size. With these information he computes a duration for this request. What you mean is to translate a logical address to a physical address. A filesystem has to cover this action. A filesystems task is to map a logical address belonging to a file into a physical sektor address. Best regards Alex > Does disksim have the function that show the physical sector address > of the Hard Disk. > Waiting for the reply! > Best regards! > > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From chensinhome at gmail.com Fri Mar 11 10:58:25 2011 From: chensinhome at gmail.com (=?UTF-8?B?6Zmz5L+h5a6P?=) Date: Fri, 11 Mar 2011 23:58:25 +0800 Subject: [Disksim-users] Disksim-users Digest, Vol 65, Issue 2 In-Reply-To: References: Message-ID: <8615322550621205486@unknownmsgid> I don't think filesystem have to take responsible to logic-physical translation Sent from SH "disksim-users-request at ece.cmu.edu" ? 2011/3/11 ??1:00 ??? > Send Disksim-users mailing list submissions to > disksim-users at ece.cmu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > or, via email, send a message with subject or body 'help' to > disksim-users-request at ece.cmu.edu > > You can reach the person managing the list at > disksim-users-owner at ece.cmu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Disksim-users digest..." > > > Today's Topics: > > 1. HDD physical address (changyue) > 2. Re: HDD physical address (Alexander Lochmann) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 10 Mar 2011 08:30:01 +0000 > From: changyue > Subject: [Disksim-users] HDD physical address > To: > Message-ID: > Content-Type: text/plain; charset="gb2312" > > > > Hello, Everyone who use the Disksim simulation tool. May I ask a simple question about disksim? > If you have the logical sector address from the IO request, then how you can get the physical sector address from the disksim? Does disksim have the function that show the physical sector address of the Hard Disk. > Waiting for the reply! > Best regards! > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Message: 2 > Date: Thu, 10 Mar 2011 09:36:39 +0100 > From: Alexander Lochmann > Subject: Re: [Disksim-users] HDD physical address > To: changyue > Cc: disksim-users at ece.cmu.edu > Message-ID: <4D788D97.7080301 at tu-dortmund.de> > Content-Type: text/plain; charset="gb2312" > > Hi! > > Am 10.03.2011 09:30, schrieb changyue: >> >> Hello, Everyone who use the Disksim simulation tool. May I ask a >> simple question about disksim? >> If you have the logical sector address from the IO request, then how >> you can get the physical sector address from the disksim? > > I think you're wrong. The disksim receives a physical sector address and > a request size. With these information he computes a duration for this > request. > What you mean is to translate a logical address to a physical address. A > filesystem has to cover this action. > A filesystems task is to map a logical address belonging to a file into > a physical sektor address. > > Best regards > Alex >> Does disksim have the function that show the physical sector address >> of the Hard Disk. >> Waiting for the reply! >> Best regards! >> >> >> _______________________________________________ >> Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > End of Disksim-users Digest, Vol 65, Issue 2 > ******************************************** From ganger at ece.cmu.edu Fri Mar 11 11:41:23 2011 From: ganger at ece.cmu.edu (Greg Ganger) Date: Fri, 11 Mar 2011 11:41:23 -0500 (EST) Subject: [Disksim-users] Disksim-users Digest, Vol 65, Issue 2 In-Reply-To: <8615322550621205486@unknownmsgid> References: <8615322550621205486@unknownmsgid> Message-ID: Correct. The simulator includes support for doing the same sort of mappings that disk drives do (or "did", since some of the newer stuff isn't in there). Greg On Fri, 11 Mar 2011, ?~Y??????~O wrote: > I don't think filesystem have to take responsible to logic-physical translation Sent from SH "disksim-users-request at ece.cmu.edu" ??? 2011/3/11 ??????1:00 ????????? > Send Disksim-users mailing list submissions to > disksim-users at ece.cmu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > or, via email, send a message with subject or body 'help' to > disksim-users-request at ece.cmu.edu > > You can reach the person managing the list at > disksim-users-owner at ece.cmu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Disksim-users digest..." > > > Today's Topics: > > 1. HDD physical address (changyue) > 2. Re: HDD physical address (Alexander Lochmann) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 10 Mar 2011 08:30:01 +0000 > From: changyue > Subject: [Disksim-users] HDD physical address > To: > Message-ID: > Content-Type: text/plain; charset="gb2312" > > > > Hello, Everyone who use the Disksim simulation tool. May I ask a simple question about disksim? > If you have the logical sector address from the IO request, then how you can get the physical sector address from the disksim? Does disksim have the function that show the physical sector address of the Hard Disk. > Waiting for the reply! > Best regards! > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Message: 2 > Date: Thu, 10 Mar 2011 09:36:39 +0100 > From: Alexander Lochmann > Subject: Re: [Disksim-users] HDD physical address > To: changyue > Cc: disksim-users at ece.cmu.edu > Message-ID: <4D788D97.7080301 at tu-dortmund.de> > Content-Type: text/plain; charset="gb2312" > > Hi! > > Am 10.03.2011 09:30, schrieb changyue: >> >> Hello, Everyone who use the Disksim simulation tool. May I ask a >> simple question about disksim? >> If you have the logical sector address from the IO request, then how >> you can get the physical sector address from the disksim? > > I think you're wrong. The disksim receives a physical sector address and > a request size. With these information he computes a duration for this > request. > What you mean is to translate a logical address to a physical address. A > filesystem has to cover this action. > A filesystems task is to map a logical address belonging to a file into > a physical sektor address. > > Best regards > Alex >> Does disksim have the function that show the physical sector address >> of the Hard Disk. >> Waiting for the reply! >> Best regards! >> >> >> _______________________________________________ >> Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > End of Disksim-users Digest, Vol 65, Issue 2 > ******************************************** _______________________________________________ Disksim-users mailing list Disksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users From k.u.sanoj at gmail.com Sat Mar 12 01:17:17 2011 From: k.u.sanoj at gmail.com (sanoj) Date: Sat, 12 Mar 2011 11:47:17 +0530 Subject: [Disksim-users] how to add address translation at the controller Message-ID: Hello all I would like to study a storage system comprising of SSDs and HDD's in different RAID combinations. ( similar to autoraid, DHIS ) I would like to use an address translation layer at the controller, which decides the location of blocks during reads / writes based on factors such as the queue length , throughput, current disk head position. The problem is that when using disksim , the location of blocks ( for read / write ) should be known before running the simulation. Is there anyway using which i can provide this information online. Thanks, sanoj -------------- next part -------------- An HTML attachment was scrubbed... URL: From frajaei55 at gmail.com Wed Mar 16 09:03:52 2011 From: frajaei55 at gmail.com (f.rajaei) Date: Wed, 16 Mar 2011 06:03:52 -0700 Subject: [Disksim-users] Response time Message-ID: Hi All, Does anybody know what is the unit of response time (s or ms) which is reported in outv file in Disksim? Regards, Farzaneh -------------- next part -------------- An HTML attachment was scrubbed... URL: From leitian.hust at gmail.com Wed Mar 16 09:24:10 2011 From: leitian.hust at gmail.com (Lei Tian) Date: Wed, 16 Mar 2011 08:24:10 -0500 Subject: [Disksim-users] Response time In-Reply-To: References: Message-ID: The unit of response time is ms. Lei On Mar 16, 2011, at 8:03 AM, f.rajaei wrote: > Hi All, > > Does anybody know what is the unit of response time (s or ms) which is reported in outv file in Disksim? > > Regards, > Farzaneh > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users From cy_3108079008 at hotmail.com Wed Mar 16 22:57:22 2011 From: cy_3108079008 at hotmail.com (changyue) Date: Thu, 17 Mar 2011 02:57:22 +0000 Subject: [Disksim-users] unexpected request location of logorg request Message-ID: Hello,everyone. If I want use one disk to cache another disk, but the cache disk is too small that the IO request logical address is out of the capacity of the cache disk. So the disksim assert error, because the request logorg is -1, and the logical address request is out of range. Does anyone account for this problem. If do, please reply me. Thanks! Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From jontjioe at gmail.com Mon Mar 21 22:05:06 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Mon, 21 Mar 2011 19:05:06 -0700 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems Message-ID: DiskSim users, First, let me apologize for making this email so long. I wanted it to be thorough so my problem is explained clearly. I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL paper). It's basically DiskSim 3.0 with support for SSD. I've been running the "runtest" script which basically runs a small sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace that was included is very small (8-9MB) and I get the intended results as the authors of the DFTL paper did. All the runtest script does is run the same trace file on these 3 FTLs: ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 Upon successful completion of the simulation, I noticed that in the .outv files, I will see: << ... loadparams complete Initialization complete Simulation complete ... >> After this, I will see many performance statistics along with every request. I get the appropriate results when simulating DFTL, FAST, and pure page mapped FTL. However, once I tried putting any other real world trace (which is much longer) in the simulation, it does not complete successfully. It should be noted that I have not made any modifications to any of the 3 FTLs during these tests. I have verified that my format of the trace files is correct: <1 or 0 for read or write> There are 3 different results that I get, all of which are unsuccessful simulations: 1) Segmentation fault 2) Simulation is stopped because of saturation 3) Simulation seems like it finished, but when you check the .outv files, the last line says "initialization complete" (in otherwords it does not ever say "simulation complete" One example (Error #1) is when I run the entire finanical trace in, I get a segmentation fault simulating DFTL. Although I didn't get segmentation faults for FAST or pure page mapped, their corresponding .outv files do not look correct. They seem like they never finished simulating. In the .outv files for all 3 FTLs, it shows: << ... loadparams complete Initialization complete >> But it never shows "simulation complete" and thus the results are not shown in the .outv file. The weird thing is that if I just include maybe several hundred lines of the Financial1 trace instead of the entire thing, it completes successfully with no problem, so I know my format of the file is correct. Another example (Error #2) I noticed that I the Financial2 trace did not have any segmentation faults but instead I had a different message: << Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 >> I did some research and found that there is a define statement in disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the instances when the simulation stopped due to saturation, the totalreqs number was just over 10000. I imagine the queue length is getting very large due to the fact that I only have a device num of 0 since my GC algorithm will be a local GC algorithm, not a inter-disk algorithm. And lastly, if (Error #1) or (Error #2) occur, the simulation will never complete and no simulation summary will be in the .outv file (which is Error #3). My questions are as follows: Regarding (Error #1), I have no idea why I am getting segmentation faults. Do you think it is some type of buffer overflow issue b/c there is so much data with the real world traces? Remember, that if I just take a smaller subset of the real world data, it simulates to completion without any problem. Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to maybe 100000 and see what happens, but is that the right solution or is there something else that I need to be aware of. Regarding (Error #3), I would assume that this will be solved once I find the solutions to (Error #1) and (Error #2). I really appreciate any help or hints you can offer. I will also post this to the disksims mailing list. Thanks for your valuable time, Jonathan Tjioe -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.lochmann at tu-dortmund.de Tue Mar 22 04:09:49 2011 From: alexander.lochmann at tu-dortmund.de (Alexander Lochmann) Date: Tue, 22 Mar 2011 09:09:49 +0100 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems In-Reply-To: References: Message-ID: <4D88594D.2050007@tu-dortmund.de> Hi! Am 22.03.2011 03:05, schrieb Jonathan Tjioe: > DiskSim users, > > First, let me apologize for making this email so long. I wanted it to > be thorough so my problem is explained clearly. > > I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the > DFTL paper). It's basically DiskSim 3.0 with support for SSD. > > I've been running the "runtest" script which basically runs a small > sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The > test trace that was included is very small (8-9MB) and I get the > intended results as the authors of the DFTL paper did. > > All the runtest script does is run the same trace file on these 3 FTLs: > ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 > ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 > ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 > > > Upon successful completion of the simulation, I noticed that in the > .outv files, I will see: > << > ... > loadparams complete > Initialization complete > Simulation complete > ... > >> > > After this, I will see many performance statistics along with every > request. I get the appropriate results when simulating DFTL, FAST, and > pure page mapped FTL. > > However, once I tried putting any other real world trace (which is > much longer) in the simulation, it does not complete successfully. It > should be noted that I have not made any modifications to any of the 3 > FTLs during these tests. I have verified that my format of the trace > files is correct: > <1 or 0 for > read or write> > > There are 3 different results that I get, all of which are > unsuccessful simulations: > 1) Segmentation fault Could give us some more information? Have you tried to run it with gdb und typed "bt"? > 2) Simulation is stopped because of saturation Maybe your harddisk model is too slow to serve the requests within a appropriate amount of time so the requestqueue doesn't get saturated. In a real system you've got a operating system which keeps track of this issue. Linux for example, slows down every readahead and writeback activity to reduce the number of requests if it detects congestion conditions. It has a requestqueue which is on top of the driver holding every request. Have you tried to slow down the request-ratio? Greetings Alex > 3) Simulation seems like it finished, but when you check the .outv > files, the last line says "initialization complete" (in otherwords it > does not ever say "simulation complete" > > One example (Error #1) is when I run the entire finanical trace in, I > get a segmentation fault simulating DFTL. Although I didn't get > segmentation faults for FAST or pure page mapped, their corresponding > .outv files do not look correct. They seem like they never finished > simulating. In the .outv files for all 3 FTLs, it shows: > > << > ... > loadparams complete > Initialization complete > >> > > But it never shows "simulation complete" and thus the results are not > shown in the .outv file. The weird thing is that if I just include > maybe several hundred lines of the Financial1 trace instead of the > entire thing, it completes successfully with no problem, so I know my > format of the file is correct. > > Another example (Error #2) I noticed that I the Financial2 trace did > not have any segmentation faults but instead I had a different message: > << > Stopping simulation because of saturation: simtime 108.435867, > totalreqs 10394 > >> > > I did some research and found that there is a define statement in > disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each > of the instances when the simulation stopped due to saturation, the > totalreqs number was just over 10000. I imagine the queue length is > getting very large due to the fact that I only have a device num of 0 > since my GC algorithm will be a local GC algorithm, not a inter-disk > algorithm. > > And lastly, if (Error #1) or (Error #2) occur, the simulation will > never complete and no simulation summary will be in the .outv file > (which is Error #3). > > My questions are as follows: > Regarding (Error #1), I have no idea why I am getting segmentation > faults. Do you think it is some type of buffer overflow issue b/c > there is so much data with the real world traces? Remember, that if I > just take a smaller subset of the real world data, it simulates to > completion without any problem. > > Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH > to maybe 100000 and see what happens, but is that the right solution > or is there something else that I need to be aware of. > > Regarding (Error #3), I would assume that this will be solved once I > find the solutions to (Error #1) and (Error #2). > > I really appreciate any help or hints you can offer. I will also post > this to the disksims mailing list. > > Thanks for your valuable time, > Jonathan Tjioe > > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From chensinhome at gmail.com Tue Mar 22 20:41:59 2011 From: chensinhome at gmail.com (=?Big5?B?q0inu7Ov?=) Date: Wed, 23 Mar 2011 08:41:59 +0800 Subject: [Disksim-users] Disksim-users Digest, Vol 65, Issue 7 In-Reply-To: References: Message-ID: about Error #1 (segment fault), check if logic address range of your trace is out of that your device can take. 2011/3/23 > Send Disksim-users mailing list submissions to > disksim-users at ece.cmu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > or, via email, send a message with subject or body 'help' to > disksim-users-request at ece.cmu.edu > > You can reach the person managing the list at > disksim-users-owner at ece.cmu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Disksim-users digest..." > > > Today's Topics: > > 1. DiskSim 3.0/FlashSim Simulation problems (Jonathan Tjioe) > 2. Re: DiskSim 3.0/FlashSim Simulation problems (Alexander Lochmann) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 21 Mar 2011 19:05:06 -0700 > From: Jonathan Tjioe > Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems > To: disksim-users at ece.cmu.edu > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > DiskSim users, > > First, let me apologize for making this email so long. I wanted it to be > thorough so my problem is explained clearly. > > I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL > paper). It's basically DiskSim 3.0 with support for SSD. > > I've been running the "runtest" script which basically runs a small sample > test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace > that was included is very small (8-9MB) and I get the intended results as > the authors of the DFTL paper did. > > All the runtest script does is run the same trace file on these 3 FTLs: > ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 > ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 > ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 > > > Upon successful completion of the simulation, I noticed that in the .outv > files, I will see: > << > ... > loadparams complete > Initialization complete > Simulation complete > ... > >> > > After this, I will see many performance statistics along with every > request. > I get the appropriate results when simulating DFTL, FAST, and pure page > mapped FTL. > > However, once I tried putting any other real world trace (which is much > longer) in the simulation, it does not complete successfully. It should be > noted that I have not made any modifications to any of the 3 FTLs during > these tests. I have verified that my format of the trace files is correct: > <1 or 0 for read or > write> > > There are 3 different results that I get, all of which are unsuccessful > simulations: > 1) Segmentation fault > 2) Simulation is stopped because of saturation > 3) Simulation seems like it finished, but when you check the .outv files, > the last line says "initialization complete" (in otherwords it does not > ever > say "simulation complete" > > One example (Error #1) is when I run the entire finanical trace in, I get a > segmentation fault simulating DFTL. Although I didn't get segmentation > faults for FAST or pure page mapped, their corresponding .outv files do not > look correct. They seem like they never finished simulating. In the .outv > files for all 3 FTLs, it shows: > > << > ... > loadparams complete > Initialization complete > >> > > But it never shows "simulation complete" and thus the results are not shown > in the .outv file. The weird thing is that if I just include maybe several > hundred lines of the Financial1 trace instead of the entire thing, it > completes successfully with no problem, so I know my format of the file is > correct. > > Another example (Error #2) I noticed that I the Financial2 trace did not > have any segmentation faults but instead I had a different message: > << > Stopping simulation because of saturation: simtime 108.435867, totalreqs > 10394 > >> > > I did some research and found that there is a define statement in > disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the > instances when the simulation stopped due to saturation, the totalreqs > number was just over 10000. I imagine the queue length is getting very > large > due to the fact that I only have a device num of 0 since my GC algorithm > will be a local GC algorithm, not a inter-disk algorithm. > > And lastly, if (Error #1) or (Error #2) occur, the simulation will never > complete and no simulation summary will be in the .outv file (which is > Error > #3). > > My questions are as follows: > Regarding (Error #1), I have no idea why I am getting segmentation faults. > Do you think it is some type of buffer overflow issue b/c there is so much > data with the real world traces? Remember, that if I just take a smaller > subset of the real world data, it simulates to completion without any > problem. > > Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to > maybe 100000 and see what happens, but is that the right solution or is > there something else that I need to be aware of. > > Regarding (Error #3), I would assume that this will be solved once I find > the solutions to (Error #1) and (Error #2). > > I really appreciate any help or hints you can offer. I will also post this > to the disksims mailing list. > > Thanks for your valuable time, > Jonathan Tjioe > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://sos.ece.cmu.edu/pipermail/disksim-users/attachments/20110321/e86aff31/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Tue, 22 Mar 2011 09:09:49 +0100 > From: Alexander Lochmann > Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems > To: Jonathan Tjioe > Cc: disksim-users at ece.cmu.edu > Message-ID: <4D88594D.2050007 at tu-dortmund.de> > Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" > > Hi! > > Am 22.03.2011 03:05, schrieb Jonathan Tjioe: > > DiskSim users, > > > > First, let me apologize for making this email so long. I wanted it to > > be thorough so my problem is explained clearly. > > > > I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the > > DFTL paper). It's basically DiskSim 3.0 with support for SSD. > > > > I've been running the "runtest" script which basically runs a small > > sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The > > test trace that was included is very small (8-9MB) and I get the > > intended results as the authors of the DFTL paper did. > > > > All the runtest script does is run the same trace file on these 3 FTLs: > > ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 > > ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 > > ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 > > > > > > Upon successful completion of the simulation, I noticed that in the > > .outv files, I will see: > > << > > ... > > loadparams complete > > Initialization complete > > Simulation complete > > ... > > >> > > > > After this, I will see many performance statistics along with every > > request. I get the appropriate results when simulating DFTL, FAST, and > > pure page mapped FTL. > > > > However, once I tried putting any other real world trace (which is > > much longer) in the simulation, it does not complete successfully. It > > should be noted that I have not made any modifications to any of the 3 > > FTLs during these tests. I have verified that my format of the trace > > files is correct: > > <1 or 0 for > > read or write> > > > > There are 3 different results that I get, all of which are > > unsuccessful simulations: > > 1) Segmentation fault > Could give us some more information? > Have you tried to run it with gdb und typed "bt"? > > 2) Simulation is stopped because of saturation > Maybe your harddisk model is too slow to serve the requests within a > appropriate amount of time so the requestqueue doesn't get saturated. > In a real system you've got a operating system which keeps track of this > issue. Linux for example, slows down every readahead and writeback > activity to reduce the number of requests if it detects congestion > conditions. It has a requestqueue which is on top of the driver holding > every request. > Have you tried to slow down the request-ratio? > > Greetings > Alex > > > 3) Simulation seems like it finished, but when you check the .outv > > files, the last line says "initialization complete" (in otherwords it > > does not ever say "simulation complete" > > > > One example (Error #1) is when I run the entire finanical trace in, I > > get a segmentation fault simulating DFTL. Although I didn't get > > segmentation faults for FAST or pure page mapped, their corresponding > > .outv files do not look correct. They seem like they never finished > > simulating. In the .outv files for all 3 FTLs, it shows: > > > > << > > ... > > loadparams complete > > Initialization complete > > >> > > > > But it never shows "simulation complete" and thus the results are not > > shown in the .outv file. The weird thing is that if I just include > > maybe several hundred lines of the Financial1 trace instead of the > > entire thing, it completes successfully with no problem, so I know my > > format of the file is correct. > > > > Another example (Error #2) I noticed that I the Financial2 trace did > > not have any segmentation faults but instead I had a different message: > > << > > Stopping simulation because of saturation: simtime 108.435867, > > totalreqs 10394 > > >> > > > > I did some research and found that there is a define statement in > > disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each > > of the instances when the simulation stopped due to saturation, the > > totalreqs number was just over 10000. I imagine the queue length is > > getting very large due to the fact that I only have a device num of 0 > > since my GC algorithm will be a local GC algorithm, not a inter-disk > > algorithm. > > > > And lastly, if (Error #1) or (Error #2) occur, the simulation will > > never complete and no simulation summary will be in the .outv file > > (which is Error #3). > > > > My questions are as follows: > > Regarding (Error #1), I have no idea why I am getting segmentation > > faults. Do you think it is some type of buffer overflow issue b/c > > there is so much data with the real world traces? Remember, that if I > > just take a smaller subset of the real world data, it simulates to > > completion without any problem. > > > > Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH > > to maybe 100000 and see what happens, but is that the right solution > > or is there something else that I need to be aware of. > > > > Regarding (Error #3), I would assume that this will be solved once I > > find the solutions to (Error #1) and (Error #2). > > > > I really appreciate any help or hints you can offer. I will also post > > this to the disksims mailing list. > > > > Thanks for your valuable time, > > Jonathan Tjioe > > > > > > _______________________________________________ > > Disksim-users mailing list > > Disksim-users at ece.cmu.edu > > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://sos.ece.cmu.edu/pipermail/disksim-users/attachments/20110322/0f5c6347/attachment-0001.html > > > > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > End of Disksim-users Digest, Vol 65, Issue 7 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jontjioe at gmail.com Tue Mar 22 22:47:59 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Tue, 22 Mar 2011 19:47:59 -0700 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems In-Reply-To: <4D88594D.2050007@tu-dortmund.de> References: <4D88594D.2050007@tu-dortmund.de> Message-ID: Alex, I've moved your responses up to the top so the rest of the forum won't get confused trying to distinguish the your responses from my questions. I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of the DFTL paper that my environment is based on. Hopefully, they can shed some light on what I'm doing wrong. On your response to Error #1 (Segmentation fault)... When I run it for the Financial1 trace file using all 3 FTLs, I get the following output to screen: Running Pagemap FTL... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 Running DFTL... ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 Running FAST... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 As stated earlier, if I check the .outv files that were generated for each of these FTLs, none of the simulations ever completed. I have not run it using gdb. The code is compiled already. I suppose I could try to do so, but I would think that I should be able to run it as is. I have also attached the exact script that I run when I simulate the 3 FTLs. Again, this is the same exact environment that the DFTL authors used. On your response to Error #2 (simulation stopped due to saturation), when running runtest script against the Financial2 trace file, I get this output to the screen. Running Pagemap FTL... Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 IOdriver Response time average: 17.003924 IOdriver Response time std.dev.: 15.487314 Running DFTL... Stopping simulation because of saturation: simtime 107.151359, totalreqs 10255 IOdriver Response time average: 25.488591 IOdriver Response time std.dev.: 19.068197 Running FAST... Stopping simulation because of saturation: simtime 106.554985, totalreqs 10188 IOdriver Response time average: 10.128997 IOdriver Response time std.dev.: 8.028486 As far as slowing down the request-ratio...well, I could be misunderstanding what you are saying, but I thought that the way it works is that the real world requests come in exactly as the arrival times state in the trace file. Then if the hard drive(s) is(are) busy, then they just go in the request queue waiting to be serviced. I could understand that for a synthetically generated trace, it might be worth slowing down the request ratio just to see what happens, but this is a real world trace. So if I slowed down the request-ratio, this would mean modifying the arrival times, which would invalidate the trace as it would no longer be real world. Thanks, Jonathan On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann < alexander.lochmann at tu-dortmund.de> wrote: Hi! On Error #1 (Segmentation fault), could give us some more information? Have you tried to run it with gdb und typed "bt"? On Error #2 (Simulation stopped due to saturation), maybe your harddisk model is too slow to serve the requests within a appropriate amount of time so the requestqueue doesn't get saturated. In a real system you've got a operating system which keeps track of this issue. Linux for example, slows down every readahead and writeback activity to reduce the number of requests if it detects congestion conditions. It has a requestqueue which is on top of the driver holding every request. Have you tried to slow down the request-ratio? Greetings Alex Am 22.03.2011 03:05, schrieb Jonathan Tjioe: > > DiskSim users, > > First, let me apologize for making this email so long. I wanted it to be > thorough so my problem is explained clearly. > > I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL > paper). It's basically DiskSim 3.0 with support for SSD. > > I've been running the "runtest" script which basically runs a small sample > test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace > that was included is very small (8-9MB) and I get the intended results as > the authors of the DFTL paper did. > > All the runtest script does is run the same trace file on these 3 FTLs: > ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 > ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 > ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 > > > Upon successful completion of the simulation, I noticed that in the .outv > files, I will see: > << > ... > loadparams complete > Initialization complete > Simulation complete > ... > >> > > After this, I will see many performance statistics along with every > request. I get the appropriate results when simulating DFTL, FAST, and pure > page mapped FTL. > > However, once I tried putting any other real world trace (which is much > longer) in the simulation, it does not complete successfully. It should be > noted that I have not made any modifications to any of the 3 FTLs during > these tests. I have verified that my format of the trace files is correct: > <1 or 0 for read or > write> > > There are 3 different results that I get, all of which are unsuccessful > simulations: > 1) Segmentation fault > > 2) Simulation is stopped because of saturation > > 3) Simulation seems like it finished, but when you check the .outv files, > the last line says "initialization complete" (in otherwords it does not ever > say "simulation complete" > > One example (Error #1) is when I run the entire finanical trace in, I get a > segmentation fault simulating DFTL. Although I didn't get segmentation > faults for FAST or pure page mapped, their corresponding .outv files do not > look correct. They seem like they never finished simulating. In the .outv > files for all 3 FTLs, it shows: > > << > ... > loadparams complete > Initialization complete > >> > > But it never shows "simulation complete" and thus the results are not shown > in the .outv file. The weird thing is that if I just include maybe several > hundred lines of the Financial1 trace instead of the entire thing, it > completes successfully with no problem, so I know my format of the file is > correct. > > Another example (Error #2) I noticed that I the Financial2 trace did not > have any segmentation faults but instead I had a different message: > << > Stopping simulation because of saturation: simtime 108.435867, totalreqs > 10394 > >> > > I did some research and found that there is a define statement in > disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the > instances when the simulation stopped due to saturation, the totalreqs > number was just over 10000. I imagine the queue length is getting very large > due to the fact that I only have a device num of 0 since my GC algorithm > will be a local GC algorithm, not a inter-disk algorithm. > > And lastly, if (Error #1) or (Error #2) occur, the simulation will never > complete and no simulation summary will be in the .outv file (which is Error > #3). > > My questions are as follows: > Regarding (Error #1), I have no idea why I am getting segmentation faults. > Do you think it is some type of buffer overflow issue b/c there is so much > data with the real world traces? Remember, that if I just take a smaller > subset of the real world data, it simulates to completion without any > problem. > > Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to > maybe 100000 and see what happens, but is that the right solution or is > there something else that I need to be aware of. > > Regarding (Error #3), I would assume that this will be solved once I find > the solutions to (Error #1) and (Error #2). > > I really appreciate any help or hints you can offer. I will also post this > to the disksims mailing list. > > Thanks for your valuable time, > Jonathan Tjioe > > > _______________________________________________ > Disksim-users mailing listDisksim-users at ece.cmu.eduhttps://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: runtest Type: application/octet-stream Size: 633 bytes Desc: not available URL: From jontjioe at gmail.com Wed Mar 23 20:32:34 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Wed, 23 Mar 2011 17:32:34 -0700 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems In-Reply-To: <32ABBCE83DCC4BFD8CB72B84580D1B89@numonyx.local> References: <4D88594D.2050007@tu-dortmund.de> <32ABBCE83DCC4BFD8CB72B84580D1B89@numonyx.local> Message-ID: Francesco, I'm glad my previous procedures were helpful to you. That's why I wrote them =) I understand what you are saying about changing the subsystem. For example, I am simulating only 1 SSD. The original Financial traces are simulating multiple hard disks. I have modified the trace so that for devicenum, it is always 0 (hard disk 0). I believe this is a valid modification to the trace since it doesn't not change the request arrival times, size of each request, etc. It is only changing the physical disks I have available to run the simulation through. However, I'm not sure that changing the arrival times is a legitimate change. For example, the Financial traces are very write intensive - the time between writes is very small, i.e. the write request rate is very high. If I changed these arrival times to be further spaced apart, I cannot legitimately say that I ran my GC algorithm or FTL through the Financial trace as now it is not as write intensive as it should be. However, changing things about my subsystem: how many disks I have, what kind of disks they are, etc. are completely valid changes. Hopefully that clears up what I was trying to say. With that said, I'm not sure if I understand exactly what you meant by this: << Regarding this I am still in trouble since I read somewhere that it is possible to configure simulator in order to make it behave as a closed subsystem model also on external I/O traces making it ignore the 'arrival times' have you an idea on how this can be done? >> I'm not sure how that can be done. I read the link you sent me. I don't think I am implementing any magnetic disks. Do you know where I can look in the code to check exactly which disks are implemented? (I'm not that familiar with DiskSim yet). Thanks, Jonathan On Wed, Mar 23, 2011 at 12:44 AM, Francesco Falanga wrote: > Hi Jonatan, > > I just find out that I am following your same steps with FlashSim > encouneing problem that you've already solved :)... > Regarding your statement: > > "So if I slowed down the request-ratio, this would mean modifying the > arrival times, which would invalidate the trace as it would no longer be > real world" > > I have to respectfully disagree since a "trace" gives you the performance > of a very specific system configuration. Changing one or more subsystem may > have still sense in order to see how the new configuration will perform > given the same request sequence. > > Regarding this I am still in trouble since I read somewhere that it is > possible to configure simulator in order to make it behave as a closed > subsystem model also on external I/O traces making it ignore the 'arrival > times' have you an idea on how this can be done? > > > Regarding segmentation fault I found this, but I guess you have already > given a glance: > https://sos.ece.cmu.edu/pipermail/disksim-users/2010-December/000556.html > > > Ciao! > > Francesco. > > > ----- Original Message ----- > *From:* Jonathan Tjioe > *To:* Alexander Lochmann > *Cc:* Aayush Gupta ; Andres Blanco; > disksim-users at ece.cmu.edu ; Youngjae Kim > *Sent:* Wednesday, March 23, 2011 3:47 AM > *Subject:* Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems > > Alex, > > I've moved your responses up to the top so the rest of the forum won't get > confused trying to distinguish the your responses from my questions. > > I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of > the DFTL paper that my environment is based on. Hopefully, they can shed > some light on what I'm doing wrong. > > On your response to Error #1 (Segmentation fault)... > > When I run it for the Financial1 trace file using all 3 FTLs, I get the > following output to screen: > > Running Pagemap FTL... > start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 > Running DFTL... > ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv > dftl.outv ascii ./trace/test.file 0 > Running FAST... > start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 > > As stated earlier, if I check the .outv files that were generated for each > of these FTLs, none of the simulations ever completed. > > I have not run it using gdb. The code is compiled already. I suppose I > could try to do so, but I would think that I should be able to run it as is. > > I have also attached the exact script that I run when I simulate the 3 > FTLs. Again, this is the same exact environment that the DFTL authors used. > > On your response to Error #2 (simulation stopped due to saturation), when > running runtest script against the Financial2 trace file, I get this output > to the screen. > > Running Pagemap FTL... > Stopping simulation because of saturation: simtime 108.435867, totalreqs > 10394 > IOdriver Response time average: 17.003924 > IOdriver Response time std.dev.: 15.487314 > Running DFTL... > Stopping simulation because of saturation: simtime 107.151359, totalreqs > 10255 > IOdriver Response time average: 25.488591 > IOdriver Response time std.dev.: 19.068197 > Running FAST... > Stopping simulation because of saturation: simtime 106.554985, totalreqs > 10188 > IOdriver Response time average: 10.128997 > IOdriver Response time std.dev.: 8.028486 > > > As far as slowing down the request-ratio...well, I could be > misunderstanding what you are saying, but I thought that the way it works is > that the real world requests come in exactly as the arrival times state in > the trace file. Then if the hard drive(s) is(are) busy, then they just go in > the request queue waiting to be serviced. I could understand that for a > synthetically generated trace, it might be worth slowing down the request > ratio just to see what happens, but this is a real world trace. So if I > slowed down the request-ratio, this would mean modifying the arrival times, > which would invalidate the trace as it would no longer be real world. > > Thanks, > Jonathan > > > > On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann < > alexander.lochmann at tu-dortmund.de> wrote: > Hi! > > On Error #1 (Segmentation fault), could give us some more information? > Have you tried to run it with gdb und typed "bt"? > > On Error #2 (Simulation stopped due to saturation), maybe your harddisk > model is too slow to serve the requests within a appropriate amount of time > so the requestqueue doesn't get saturated. In a real system you've got a > operating system which keeps track of this issue. Linux for example, slows > down every readahead and writeback activity to reduce the number of requests > if it detects congestion conditions. It has a requestqueue which is on top > of the driver holding every request. > Have you tried to slow down the request-ratio? > > Greetings > Alex > > Am 22.03.2011 03:05, schrieb Jonathan Tjioe: >> >> DiskSim users, >> >> First, let me apologize for making this email so long. I wanted it to be >> thorough so my problem is explained clearly. >> >> I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL >> paper). It's basically DiskSim 3.0 with support for SSD. >> >> I've been running the "runtest" script which basically runs a small sample >> test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace >> that was included is very small (8-9MB) and I get the intended results as >> the authors of the DFTL paper did. >> >> All the runtest script does is run the same trace file on these 3 FTLs: >> ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 >> ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 >> ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 >> >> >> Upon successful completion of the simulation, I noticed that in the .outv >> files, I will see: >> << >> ... >> loadparams complete >> Initialization complete >> Simulation complete >> ... >> >> >> >> After this, I will see many performance statistics along with every >> request. I get the appropriate results when simulating DFTL, FAST, and pure >> page mapped FTL. >> >> However, once I tried putting any other real world trace (which is much >> longer) in the simulation, it does not complete successfully. It should be >> noted that I have not made any modifications to any of the 3 FTLs during >> these tests. I have verified that my format of the trace files is correct: >> <1 or 0 for read >> or write> >> >> There are 3 different results that I get, all of which are unsuccessful >> simulations: >> 1) Segmentation fault >> >> 2) Simulation is stopped because of saturation >> >> 3) Simulation seems like it finished, but when you check the .outv >> files, the last line says "initialization complete" (in otherwords it does >> not ever say "simulation complete" >> >> One example (Error #1) is when I run the entire finanical trace in, I get >> a segmentation fault simulating DFTL. Although I didn't get segmentation >> faults for FAST or pure page mapped, their corresponding .outv files do not >> look correct. They seem like they never finished simulating. In the .outv >> files for all 3 FTLs, it shows: >> >> << >> ... >> loadparams complete >> Initialization complete >> >> >> >> But it never shows "simulation complete" and thus the results are not >> shown in the .outv file. The weird thing is that if I just include maybe >> several hundred lines of the Financial1 trace instead of the entire thing, >> it completes successfully with no problem, so I know my format of the file >> is correct. >> >> Another example (Error #2) I noticed that I the Financial2 trace did not >> have any segmentation faults but instead I had a different message: >> << >> Stopping simulation because of saturation: simtime 108.435867, totalreqs >> 10394 >> >> >> >> I did some research and found that there is a define statement in >> disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the >> instances when the simulation stopped due to saturation, the totalreqs >> number was just over 10000. I imagine the queue length is getting very large >> due to the fact that I only have a device num of 0 since my GC algorithm >> will be a local GC algorithm, not a inter-disk algorithm. >> >> And lastly, if (Error #1) or (Error #2) occur, the simulation will never >> complete and no simulation summary will be in the .outv file (which is Error >> #3). >> >> My questions are as follows: >> Regarding (Error #1), I have no idea why I am getting segmentation faults. >> Do you think it is some type of buffer overflow issue b/c there is so much >> data with the real world traces? Remember, that if I just take a smaller >> subset of the real world data, it simulates to completion without any >> problem. >> >> Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to >> maybe 100000 and see what happens, but is that the right solution or is >> there something else that I need to be aware of. >> >> Regarding (Error #3), I would assume that this will be solved once I find >> the solutions to (Error #1) and (Error #2). >> >> I really appreciate any help or hints you can offer. I will also post this >> to the disksims mailing list. >> >> Thanks for your valuable time, >> Jonathan Tjioe >> >> >> _______________________________________________ >> Disksim-users mailing listDisksim-users at ece.cmu.eduhttps://sos.ece.cmu.edu/mailman/listinfo/disksim-users >> >> >> > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jontjioe at gmail.com Wed Mar 23 20:40:10 2011 From: jontjioe at gmail.com (Jonathan Tjioe) Date: Wed, 23 Mar 2011 17:40:10 -0700 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems In-Reply-To: References: <4D88594D.2050007@tu-dortmund.de> Message-ID: Can you be more specific? Which parameters in particular do I need to modify? Basically, the only thing that I did was the following: 1) Divided the request size (bytes) by 512 because my sectors are 512Bytes. So now the size is in sectors. To my understanding, this is what DiskSim requires. 2) I changed the device number to always be 0 (since I am only simulating one SSD) 3) I moved the parameters around from SPC format to disksim format as follows: <1 or 0 for read or write> Are there other modifications that I need to do besides that? Thanks, Jonathan 2011/3/23 changyue > Hi, > If you want to use the IO trace file directly that download from the > website (for example,Financial 1), you must filter the trace file to fit > your disk parameter. It is a hard question that I have tried do it nearly , > but not successful. > Thanks! > ------------------------------ > Date: Tue, 22 Mar 2011 19:47:59 -0700 > From: jontjioe at gmail.com > To: alexander.lochmann at tu-dortmund.de > CC: axg354 at cse.psu.edu; andresblanco_89 at yahoo.com; > disksim-users at ece.cmu.edu; youkim at cse.psu.edu > > Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems > > Alex, > > I've moved your responses up to the top so the rest of the forum won't get > confused trying to distinguish the your responses from my questions. > > I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of > the DFTL paper that my environment is based on. Hopefully, they can shed > some light on what I'm doing wrong. > > On your response to Error #1 (Segmentation fault)... > > When I run it for the Financial1 trace file using all 3 FTLs, I get the > following output to screen: > > Running Pagemap FTL... > start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 > Running DFTL... > ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv > dftl.outv ascii ./trace/test.file 0 > Running FAST... > start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 > > As stated earlier, if I check the .outv files that were generated for each > of these FTLs, none of the simulations ever completed. > > I have not run it using gdb. The code is compiled already. I suppose I > could try to do so, but I would think that I should be able to run it as is. > > I have also attached the exact script that I run when I simulate the 3 > FTLs. Again, this is the same exact environment that the DFTL authors used. > > On your response to Error #2 (simulation stopped due to saturation), when > running runtest script against the Financial2 trace file, I get this output > to the screen. > > Running Pagemap FTL... > Stopping simulation because of saturation: simtime 108.435867, totalreqs > 10394 > IOdriver Response time average: 17.003924 > IOdriver Response time std.dev.: 15.487314 > Running DFTL... > Stopping simulation because of saturation: simtime 107.151359, totalreqs > 10255 > IOdriver Response time average: 25.488591 > IOdriver Response time std.dev.: 19.068197 > Running FAST... > Stopping simulation because of saturation: simtime 106.554985, totalreqs > 10188 > IOdriver Response time average: 10.128997 > IOdriver Response time std.dev.: 8.028486 > > > As far as slowing down the request-ratio...well, I could be > misunderstanding what you are saying, but I thought that the way it works is > that the real world requests come in exactly as the arrival times state in > the trace file. Then if the hard drive(s) is(are) busy, then they just go in > the request queue waiting to be serviced. I could understand that for a > synthetically generated trace, it might be worth slowing down the request > ratio just to see what happens, but this is a real world trace. So if I > slowed down the request-ratio, this would mean modifying the arrival times, > which would invalidate the trace as it would no longer be real world. > > Thanks, > Jonathan > > > > On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann < > alexander.lochmann at tu-dortmund.de> wrote: > Hi! > > On Error #1 (Segmentation fault), could give us some more information? > Have you tried to run it with gdb und typed "bt"? > > On Error #2 (Simulation stopped due to saturation), maybe your harddisk > model is too slow to serve the requests within a appropriate amount of time > so the requestqueue doesn't get saturated. In a real system you've got a > operating system which keeps track of this issue. Linux for example, slows > down every readahead and writeback activity to reduce the number of requests > if it detects congestion conditions. It has a requestqueue which is on top > of the driver holding every request. > Have you tried to slow down the request-ratio? > > Greetings > Alex > > Am 22.03.2011 03:05, schrieb Jonathan Tjioe: > > DiskSim users, > > First, let me apologize for making this email so long. I wanted it to be > thorough so my problem is explained clearly. > > I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL > paper). It's basically DiskSim 3.0 with support for SSD. > > I've been running the "runtest" script which basically runs a small sample > test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace > that was included is very small (8-9MB) and I get the intended results as > the authors of the DFTL paper did. > > All the runtest script does is run the same trace file on these 3 FTLs: > ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 > ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 > ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 > > > Upon successful completion of the simulation, I noticed that in the .outv > files, I will see: > << > ... > loadparams complete > Initialization complete > Simulation complete > ... > >> > > After this, I will see many performance statistics along with every > request. I get the appropriate results when simulating DFTL, FAST, and pure > page mapped FTL. > > However, once I tried putting any other real world trace (which is much > longer) in the simulation, it does not complete successfully. It should be > noted that I have not made any modifications to any of the 3 FTLs during > these tests. I have verified that my format of the trace files is correct: > <1 or 0 for read or > write> > > There are 3 different results that I get, all of which are unsuccessful > simulations: > 1) Segmentation fault > > 2) Simulation is stopped because of saturation > > 3) Simulation seems like it finished, but when you check the .outv files, > the last line says "initialization complete" (in otherwords it does not ever > say "simulation complete" > > One example (Error #1) is when I run the entire finanical trace in, I get a > segmentation fault simulating DFTL. Although I didn't get segmentation > faults for FAST or pure page mapped, their corresponding .outv files do not > look correct. They seem like they never finished simulating. In the .outv > files for all 3 FTLs, it shows: > > << > ... > loadparams complete > Initialization complete > >> > > But it never shows "simulation complete" and thus the results are not shown > in the .outv file. The weird thing is that if I just include maybe several > hundred lines of the Financial1 trace instead of the entire thing, it > completes successfully with no problem, so I know my format of the file is > correct. > > Another example (Error #2) I noticed that I the Financial2 trace did not > have any segmentation faults but instead I had a different message: > << > Stopping simulation because of saturation: simtime 108.435867, totalreqs > 10394 > >> > > I did some research and found that there is a define statement in > disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the > instances when the simulation stopped due to saturation, the totalreqs > number was just over 10000. I imagine the queue length is getting very large > due to the fact that I only have a device num of 0 since my GC algorithm > will be a local GC algorithm, not a inter-disk algorithm. > > And lastly, if (Error #1) or (Error #2) occur, the simulation will never > complete and no simulation summary will be in the .outv file (which is Error > #3). > > My questions are as follows: > Regarding (Error #1), I have no idea why I am getting segmentation faults. > Do you think it is some type of buffer overflow issue b/c there is so much > data with the real world traces? Remember, that if I just take a smaller > subset of the real world data, it simulates to completion without any > problem. > > Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to > maybe 100000 and see what happens, but is that the right solution or is > there something else that I need to be aware of. > > Regarding (Error #3), I would assume that this will be solved once I find > the solutions to (Error #1) and (Error #2). > > I really appreciate any help or hints you can offer. I will also post this > to the disksims mailing list. > > Thanks for your valuable time, > Jonathan Tjioe > > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > > > _______________________________________________ Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From challen at buffalo.edu Thu Mar 24 16:47:47 2011 From: challen at buffalo.edu (Geoffrey Challen) Date: Thu, 24 Mar 2011 16:47:47 -0400 Subject: [Disksim-users] Power Modelling Message-ID: There have been some recent questions about this but I didn't see any answers. What is the easiest way to attach a power model to DiskSim to get power estimates for a given workload? Thanks in advance. -gwa- -------------- next part -------------- An HTML attachment was scrubbed... URL: From ffalanga at fastwebnet.it Fri Mar 25 16:55:08 2011 From: ffalanga at fastwebnet.it (Francesco Falanga) Date: Fri, 25 Mar 2011 21:55:08 +0100 Subject: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems References: <4D88594D.2050007@tu-dortmund.de><32ABBCE83DCC4BFD8CB72B84580D1B89@numonyx.local> Message-ID: <821F7E20031643C1B091A52A8CFA3A58@numonyx.local> That's exactly why a trace should be submitted to your SSD so that a new request is scheduled only when the previous one has been completed (this is what I mean by closed subsystem model). In this way you are able to evaluate the performance of various FTL schemes given a same access sequence (trace) In other words your original trace is intensive just because, being the requests dispatched in parallel over more than one SSD, the average execution time is smaller causing next access arrival time to be nearer to the previous. Does anyone know hot to configure the simulator so that even with an external trace a new request is scheduled only when the previous one has been executed (making simulator ignore the 'arrival times') ? Regarding disksim I am just trying to run a debugging session with code block to get more familiar with the code but I am not so familiar with Linux, makefile. I am having trouble when giving 'make clean' and 'make'. It seems that 'clean' command changes also all the .c files causing the original compilation error :(. What IDE are you using? Francesco From: Jonathan Tjioe To: Francesco Falanga Cc: disksim-users at ece.cmu.edu Sent: Thursday, March 24, 2011 1:32 AM Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems Francesco, I'm glad my previous procedures were helpful to you. That's why I wrote them =) I understand what you are saying about changing the subsystem. For example, I am simulating only 1 SSD. The original Financial traces are simulating multiple hard disks. I have modified the trace so that for devicenum, it is always 0 (hard disk 0). I believe this is a valid modification to the trace since it doesn't not change the request arrival times, size of each request, etc. It is only changing the physical disks I have available to run the simulation through. However, I'm not sure that changing the arrival times is a legitimate change. For example, the Financial traces are very write intensive - the time between writes is very small, i.e. the write request rate is very high. If I changed these arrival times to be further spaced apart, I cannot legitimately say that I ran my GC algorithm or FTL through the Financial trace as now it is not as write intensive as it should be. However, changing things about my subsystem: how many disks I have, what kind of disks they are, etc. are completely valid changes. Hopefully that clears up what I was trying to say. With that said, I'm not sure if I understand exactly what you meant by this: << Regarding this I am still in trouble since I read somewhere that it is possible to configure simulator in order to make it behave as a closed subsystem model also on external I/O traces making it ignore the 'arrival times' have you an idea on how this can be done? >> I'm not sure how that can be done. I read the link you sent me. I don't think I am implementing any magnetic disks. Do you know where I can look in the code to check exactly which disks are implemented? (I'm not that familiar with DiskSim yet). Thanks, Jonathan On Wed, Mar 23, 2011 at 12:44 AM, Francesco Falanga wrote: Hi Jonatan, I just find out that I am following your same steps with FlashSim encouneing problem that you've already solved :)... Regarding your statement: "So if I slowed down the request-ratio, this would mean modifying the arrival times, which would invalidate the trace as it would no longer be real world" I have to respectfully disagree since a "trace" gives you the performance of a very specific system configuration. Changing one or more subsystem may have still sense in order to see how the new configuration will perform given the same request sequence. Regarding this I am still in trouble since I read somewhere that it is possible to configure simulator in order to make it behave as a closed subsystem model also on external I/O traces making it ignore the 'arrival times' have you an idea on how this can be done? Regarding segmentation fault I found this, but I guess you have already given a glance: https://sos.ece.cmu.edu/pipermail/disksim-users/2010-December/000556.html Ciao! Francesco. ----- Original Message ----- From: Jonathan Tjioe To: Alexander Lochmann Cc: Aayush Gupta ; Andres Blanco ; disksim-users at ece.cmu.edu ; Youngjae Kim Sent: Wednesday, March 23, 2011 3:47 AM Subject: Re: [Disksim-users] DiskSim 3.0/FlashSim Simulation problems Alex, I've moved your responses up to the top so the rest of the forum won't get confused trying to distinguish the your responses from my questions. I've also CCed Aayush Gupta and Youngjae Kim as they were the authors of the DFTL paper that my environment is based on. Hopefully, they can shed some light on what I'm doing wrong. On your response to Error #1 (Segmentation fault)... When I run it for the Financial1 trace file using all 3 FTLs, I get the following output to screen: Running Pagemap FTL... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 Running DFTL... ./runtest: line 11: 5905 Segmentation fault ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 Running FAST... start_blk_no: 1107318608, block_cnt: 4, total_util_sect_num: 2097152 As stated earlier, if I check the .outv files that were generated for each of these FTLs, none of the simulations ever completed. I have not run it using gdb. The code is compiled already. I suppose I could try to do so, but I would think that I should be able to run it as is. I have also attached the exact script that I run when I simulate the 3 FTLs. Again, this is the same exact environment that the DFTL authors used. On your response to Error #2 (simulation stopped due to saturation), when running runtest script against the Financial2 trace file, I get this output to the screen. Running Pagemap FTL... Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 IOdriver Response time average: 17.003924 IOdriver Response time std.dev.: 15.487314 Running DFTL... Stopping simulation because of saturation: simtime 107.151359, totalreqs 10255 IOdriver Response time average: 25.488591 IOdriver Response time std.dev.: 19.068197 Running FAST... Stopping simulation because of saturation: simtime 106.554985, totalreqs 10188 IOdriver Response time average: 10.128997 IOdriver Response time std.dev.: 8.028486 As far as slowing down the request-ratio...well, I could be misunderstanding what you are saying, but I thought that the way it works is that the real world requests come in exactly as the arrival times state in the trace file. Then if the hard drive(s) is(are) busy, then they just go in the request queue waiting to be serviced. I could understand that for a synthetically generated trace, it might be worth slowing down the request ratio just to see what happens, but this is a real world trace. So if I slowed down the request-ratio, this would mean modifying the arrival times, which would invalidate the trace as it would no longer be real world. Thanks, Jonathan On Tue, Mar 22, 2011 at 1:09 AM, Alexander Lochmann wrote: Hi! On Error #1 (Segmentation fault), could give us some more information? Have you tried to run it with gdb und typed "bt"? On Error #2 (Simulation stopped due to saturation), maybe your harddisk model is too slow to serve the requests within a appropriate amount of time so the requestqueue doesn't get saturated. In a real system you've got a operating system which keeps track of this issue. Linux for example, slows down every readahead and writeback activity to reduce the number of requests if it detects congestion conditions. It has a requestqueue which is on top of the driver holding every request. Have you tried to slow down the request-ratio? Greetings Alex Am 22.03.2011 03:05, schrieb Jonathan Tjioe: DiskSim users, First, let me apologize for making this email so long. I wanted it to be thorough so my problem is explained clearly. I'm running DiskSim 3.0 with FlashSim (same as the FlashSim in the DFTL paper). It's basically DiskSim 3.0 with support for SSD. I've been running the "runtest" script which basically runs a small sample test file through all 3 FTLs: Page mapped, DFTL, and FAST. The test trace that was included is very small (8-9MB) and I get the intended results as the authors of the DFTL paper did. All the runtest script does is run the same trace file on these 3 FTLs: ../src/disksim pagemap.parv pagemap.outv ascii ./trace/test.file 0 ../src/disksim dftl.parv dftl.outv ascii ./trace/test.file 0 ../src/disksim fast.parv fast.outv ascii ./trace/test.file 0 Upon successful completion of the simulation, I noticed that in the .outv files, I will see: << ... loadparams complete Initialization complete Simulation complete ... >> After this, I will see many performance statistics along with every request. I get the appropriate results when simulating DFTL, FAST, and pure page mapped FTL. However, once I tried putting any other real world trace (which is much longer) in the simulation, it does not complete successfully. It should be noted that I have not made any modifications to any of the 3 FTLs during these tests. I have verified that my format of the trace files is correct: <1 or 0 for read or write> There are 3 different results that I get, all of which are unsuccessful simulations: 1) Segmentation fault 2) Simulation is stopped because of saturation 3) Simulation seems like it finished, but when you check the .outv files, the last line says "initialization complete" (in otherwords it does not ever say "simulation complete" One example (Error #1) is when I run the entire finanical trace in, I get a segmentation fault simulating DFTL. Although I didn't get segmentation faults for FAST or pure page mapped, their corresponding .outv files do not look correct. They seem like they never finished simulating. In the .outv files for all 3 FTLs, it shows: << ... loadparams complete Initialization complete >> But it never shows "simulation complete" and thus the results are not shown in the .outv file. The weird thing is that if I just include maybe several hundred lines of the Financial1 trace instead of the entire thing, it completes successfully with no problem, so I know my format of the file is correct. Another example (Error #2) I noticed that I the Financial2 trace did not have any segmentation faults but instead I had a different message: << Stopping simulation because of saturation: simtime 108.435867, totalreqs 10394 >> I did some research and found that there is a define statement in disksim_logorg.c that sets the MAX_QUEUE_LENGTH to be 10000. In each of the instances when the simulation stopped due to saturation, the totalreqs number was just over 10000. I imagine the queue length is getting very large due to the fact that I only have a device num of 0 since my GC algorithm will be a local GC algorithm, not a inter-disk algorithm. And lastly, if (Error #1) or (Error #2) occur, the simulation will never complete and no simulation summary will be in the .outv file (which is Error #3). My questions are as follows: Regarding (Error #1), I have no idea why I am getting segmentation faults. Do you think it is some type of buffer overflow issue b/c there is so much data with the real world traces? Remember, that if I just take a smaller subset of the real world data, it simulates to completion without any problem. Regarding (Error #2), I can try just increasing the MAX_QUEUE_LENGTH to maybe 100000 and see what happens, but is that the right solution or is there something else that I need to be aware of. Regarding (Error #3), I would assume that this will be solved once I find the solutions to (Error #1) and (Error #2). I really appreciate any help or hints you can offer. I will also post this to the disksims mailing list. Thanks for your valuable time, Jonathan Tjioe _______________________________________________ Disksim-users mailing list Disksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------------------------------------------------------------------- _______________________________________________ Disksim-users mailing list Disksim-users at ece.cmu.edu https://sos.ece.cmu.edu/mailman/listinfo/disksim-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From ffalanga at fastwebnet.it Fri Mar 25 17:01:23 2011 From: ffalanga at fastwebnet.it (Francesco Falanga) Date: Fri, 25 Mar 2011 22:01:23 +0100 Subject: [Disksim-users] FlashSim - Block Age statistics References: Message-ID: <55F06646ED7F431087D0CE3626D90068@numonyx.local> Hi All, Has anyone already implemented on FlashSim or SSD sim a method to track the NAND block P/E cycles so that it is possible to have a blocks wear levelling figure/statistic at the end of a specific trace? Francesco -------------- next part -------------- An HTML attachment was scrubbed... URL: From sun.yashan at gmail.com Tue Mar 29 20:03:19 2011 From: sun.yashan at gmail.com (yashan sun) Date: Tue, 29 Mar 2011 17:03:19 -0700 Subject: [Disksim-users] several questions about ioqueue Message-ID: Hi, There is a long member list in ioqueue structure, and I cannot find any explanation on these members in length. Would you like help me to clarify them? What are the meanings and relationship of: iobufcnt ~ reqoutstanding listlen ~ numoutstanding numcomplete I plugged SSD module into the Disksim. I use synthio to generate the random write events to SSD module. I met a problem. 1. when the synthio found the synthion_cnt is equal to the set value, then stop the disksim 2. However, there still have some requests not be handled by the SSD module. 3. If I disable disksim_stop() when synthio_cnt == max_cnt. Howevere, there is a long queue in ssd->queue->base.numoutstanding How can I solve my problem? Thanks a lot, Yashan