From 6375625 at gmail.com Wed Jul 3 12:31:17 2013 From: 6375625 at gmail.com (q q) Date: Wed, 3 Jul 2013 10:31:17 -0600 Subject: [Disksim-users] Disksim developers' manual Message-ID: I saw the reference manual of DiskSim 4.0. However, it is designed for users but not developers (those who implement new functionalities into DiskSim). Is there a developers' manual for DiskSim? Thank you very much. From ganger at ece.cmu.edu Wed Jul 3 19:03:51 2013 From: ganger at ece.cmu.edu (Greg Ganger) Date: Wed, 3 Jul 2013 19:03:51 -0400 (EDT) Subject: [Disksim-users] Disksim developers' manual In-Reply-To: References: Message-ID: No. The closest thing would be the appendix to Ganger's dissertation, though that's more of a design description than any sort of developer's manual... and it related to the original version, rather than to 4.0. Best luck, Greg On Wed, 3 Jul 2013, q q wrote: > I saw the reference manual of DiskSim 4.0. However, it is designed for > users but not developers (those who implement new functionalities into > DiskSim). Is there a developers' manual for DiskSim? > > Thank you very much. > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > From mujtaba.tarihi at gmail.com Sat Jul 6 03:43:27 2013 From: mujtaba.tarihi at gmail.com (Mujtaba Tarihi) Date: Sat, 6 Jul 2013 12:13:27 +0430 Subject: [Disksim-users] Serious problems using dixtrac extracted parameters In-Reply-To: <51BEE149.4030009@mpi-sws.org> References: <51BEE149.4030009@mpi-sws.org> Message-ID: Hi Anjo, Thanks for getting back to me. I used the maxtor300g/maxtor146g models that were released and I did look at the pdfs from the validation stage, which do display a good correlation. However! When you feed a completely sequential load into disksim, the performance is on par with a completely random load, which absolutely does not make sense. I tracked down the issue to the disk_buffer_sector_done function inside disksim_diskctlr.c There are some useful printf() lines in there. What I explain was observed with a trace containing two sequential reads with maxtor300g * it takes a single sector for the bcount (a field which is used as a state identifier in this function!) to go to 1. As a result, it loses the opportunity to succeed this condition: if(curr->bcount == 1) { if(currblkno == firstblkno) { As a result, it misses the opportunity to finish the second request immediately and does a complete seek around the track to start the second request. Which is pretty terrible. As the code in this function is really complicated, a simple fix will not work (I tried and started tripping asserts). The only way that works for me (as I am making use of syssim) is to intentionally trim the requests (provided they are large enough) by one or two sectors, allowing the disk model to successfully start doing the next request immediately. This workaround allowed me to get a 4-5x speed up in disk model performance. I did a comparison with one of the included disks (hp_c2490a). This model appears to be much less detailed, anyways, apparently bcount remains in value 1 for both the disksim requests, so I didn't observe this bug with it! * I only watched the outputs from maxtor300g but I suspect it is the same for the rest of Dixtrac-generated models, because they have the same issue with disk performance. Regards, -Tarihi On Mon, Jun 17, 2013 at 2:43 PM, Anjo Vahldiek wrote: > Hi, > > In general it is difficult to reason about differences without knowing how > you issued the requests to disks (e.g. O_direct, by SCSI cmd, flushing > caches, ...) and which exact models you used. > > My suggest is that you look at the validation step of dixtrac. At the end of > dixtrac it runs a validation workload and produces 4 pdfs showing the > difference between simulated & real device. (called mixed(.hist).pdf and > random(.hist).pdf which are part of the tar) Especially the mixed workload > in case of 146g shows a significant difference. The 300g model shows less > differences. (not sure which of the two you used) You should be able to use > the code part to prodcude the same pdfs using your disks instead of ours. > > Hopefully this helps. > > Thanks, > Anjo > > > On Mon 17 Jun 2013 09:39:58 AM CEST, Mujtaba Tarihi wrote: >> >> Dear all, >> >> I have been attempting to make use of some parameters extracted via >> dixtrac, such as the ones graciously provided by Anjo Vahldiek. >> However, I have come across very surprising results, performance-wise. >> I generated a random trace of aligned 16KB read requests and a >> sequential trace of 16KB back-to-back read requests. >> This is how long it took to run each trace (it is in milliseconds) >> For the record, are the results obtained with the 2GB HP C2490A which >> is supplied along with Disksim: >> Random: 2232995.378781 >> Sequential: 624599.551791 >> >> To make the comparison (somewhat) fair, I had to use the same random >> trace for the larger drives: >> >> And these are the results obtained with the 300GB Maxtor drive: >> Random: 237932.073905 >> Sequential: 619900.348468 >> >> And these are the results obtained with the 146GB Maxtor drive: >> Random: 345077.024293 >> Sequential: 329766.817393 >> >> And these results are from the 146GB Seagate drive: >> Random: 209403.240946 >> Sequential: 423689.049873 >> >> Basically, with the exception of *slight* difference in case of the >> 146GB maxtor file, the sequential trace takes longer to run, nearly >> 2.5x times in the worst case! >> >> I just copied the files in the tar.gz files and used them and >> layout.mappings is not necessary. >> >> My guess is that the model generated by the tool has issues with the >> modelling firmware behavior, something like the scheduling algorithm, >> or maybe it is botching up the mappings? >> >> Any help with be more than welcome :) >> >> Regards, >> -Tarihi >> _______________________________________________ >> Disksim-users mailing list >> Disksim-users at ece.cmu.edu >> https://sos.ece.cmu.edu/mailman/listinfo/disksim-users From ghasemi.saeed at gmail.com Sat Jul 6 15:09:13 2013 From: ghasemi.saeed at gmail.com (saeed ghasemi) Date: Sat, 6 Jul 2013 12:09:13 -0700 Subject: [Disksim-users] Need help to compile disksim on ubuntu 12.04 -32bit Message-ID: Hi I am M.Sc student and want to work my thesis on cloud computing by used of disksim. i have problem in install disksim. i do this work: *sudo apt-get install flex bison*>* wget http://www.pdl.cmu.edu/PDL-FTP/DriveChar/disksim-4.0.tar.gz*>* tar xzf disksim-4.0.tar.gz*>* cd disksim-4.0*>* make after first make, it has problem on sqrt............. by using below code i solve sqrt problem *>* mems_seektest: mems_seektest.o libmems_internals.a*>* $(CC) -o $@ mems_seektest.o $(LDFLAGS) $(CFLAGS) -lmems_internals*>**>* to:*>**>* mems_seektest: mems_seektest.o libmems_internals.a*>* $(CC) -o $@ mems_seektest.o $(CFLAGS) -lmems_internals $(LDFLAGS)* but i face with new problem make -C libddbg make[1]: Entering directory `/home/saeed/Desktop/disksim-4.0/libddbg' cc -g -I../libparam -c -o ddbg_assert.o ddbg_assert.c cc -g -I../libparam -c -o ddbg_trace.o ddbg_trace.c ar cru libddbg.a ddbg_assert.o ddbg_trace.o mkdir -p include/libddbg lib cp -p libddbg.h include/libddbg cp -p libddbg.a lib make[1]: Leaving directory `/home/saeed/Desktop/disksim-4.0/libddbg' make -C libparam make[1]: Entering directory `/home/saeed/Desktop/disksim-4.0/libparam' cc -g -I../libddbg/include -Wall -c -o util.o util.c bison -plibparam --defines libparam.y cc -g -I../libddbg/include -Wall -c -o libparam.tab.o libparam.tab.c libparam.tab.c: In function ?libparamparse?: libparam.tab.c:1375:7: warning: implicit declaration of function ?libparamlex? [-Wimplicit-function-declaration] libparam.tab.c:1998:9: warning: passing argument 1 of ?libparamerror? discards ?const? qualifier from pointer target type [enabled by default] libparam.y:41:6: note: expected ?char *? but argument is of type ?const char *? flex -Plibparam libparam.lex cc -g -I../libddbg/include -Wall -c -o lex.libparam.o lex.libparam.c libparam.lex: In function ?libparamlex?: libparam.lex:124:15: warning: variable ?base? set but not used [-Wunused-but-set-variable] libparam.lex: At top level: lex.libparam.c:1460:17: warning: ?yyunput? defined but not used [-Wunused-function] lex.libparam.c:1505:16: warning: ?input? defined but not used [-Wunused-function] ar cru libparam.a util.o libparam.tab.o lex.libparam.o mkdir -p lib include/libparam cp -p libparam.a lib cp -p bitvector.h libparam.h include/libparam make[1]: Leaving directory `/home/saeed/Desktop/disksim-4.0/libparam' make -C diskmodel make[1]: Entering directory `/home/saeed/Desktop/disksim-4.0/diskmodel' make -C modules make[2]: Entering directory `/home/saeed/Desktop/disksim-4.0/diskmodel/modules' ../../libparam/mod.pl dm disk.modspec indent dm_disk_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_disk_param.o dm_disk_param.c ../../libparam/mod.pl dm mech_g1.modspec indent dm_mech_g1_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_mech_g1_param.o dm_mech_g1_param.c ../../libparam/mod.pl dm layout_g1.modspec indent dm_layout_g1_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_layout_g1_param.o dm_layout_g1_param.c ../../libparam/mod.pl dm layout_g1_zone.modspec indent dm_layout_g1_zone_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_layout_g1_zone_param.o dm_layout_g1_zone_param.c ../../libparam/mod.pl dm layout_g2.modspec indent dm_layout_g2_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_layout_g2_param.o dm_layout_g2_param.c ../../libparam/mod.pl dm layout_g2_zone.modspec indent dm_layout_g2_zone_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_layout_g2_zone_param.o dm_layout_g2_zone_param.c ../../libparam/mod.pl dm layout_g4.modspec indent dm_layout_g4_param.c 2>/dev/null >/dev/null || true cc -g -D_DM_SOURCE -I../../libddbg/include -I../../libparam/include -c -o dm_layout_g4_param.o dm_layout_g4_param.c ../../libparam/make_modules_h.pl dm *.modspec > modules.h make[2]: Leaving directory `/home/saeed/Desktop/disksim-4.0/diskmodel/modules' mkdir -p include/diskmodel/modules cp -p dm.h dm_types.h dm_config.h marshal.h include/diskmodel cp -p modules/*.h include/diskmodel/modules gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o mech_g1_seektime.o mech_g1_seektime.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o mech_g1.o mech_g1.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g1.o layout_g1.c layout_g1.c: In function ?g1_get_zone?: layout_g1.c:1941:5: warning: ?z? is used uninitialized in this function [-Wuninitialized] gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o sqrt.o sqrt.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o marshal.o marshal.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g2.o layout_g2.c layout_g2.c:480:3: warning: initialization from incompatible pointer type [enabled by default] layout_g2.c:480:3: warning: (near initialization for ?layout_g2.dm_get_track_boundaries?) [enabled by default] gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g4.o layout_g4.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o mech_g1_load.o mech_g1_load.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g1_load.o layout_g1_load.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o dm_load.o dm_load.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o convert.o convert.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g2_load.o layout_g2_load.c gcc -g -I. -I../libparam/include -I../libddbg/include -D_DM_SOURCE -Wall -Wshadow -Wno-unused -MD -c -o layout_g4_load.o layout_g4_load.c make -C modules make[2]: Entering directory `/home/saeed/Desktop/disksim-4.0/diskmodel/modules' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/home/saeed/Desktop/disksim-4.0/diskmodel/modules' Linking libdiskmodel.a modules/*.o mech_g1_load.o layout_g1_load.o dm_load.o convert.o layout_g2_load.o layout_g4_load.o ar cru libdiskmodel.a mech_g1_seektime.o mech_g1.o layout_g1.o sqrt.o marshal.o layout_g2.o layout_g4.o modules/*.o mech_g1_load.o layout_g1_load.o dm_load.o convert.o layout_g2_load.o layout_g4_load.o mkdir -p lib cp -p libdiskmodel.a lib make -C tests make[2]: Entering directory `/home/saeed/Desktop/disksim-4.0/diskmodel/tests' cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -c -o driver.o driver.c cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_simple layout_simple.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_simple_0t layout_simple_0t.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_track layout_track.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_raw layout_raw.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_bogon layout_bogon.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o layout_skew layout_skew.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg cc -g -I../../diskmodel/include -I../../libparam/include -I../../libddbg/include -o mech_seek mech_seek.c driver.o -L../../diskmodel/lib -ldiskmodel -L../../libparam/lib -lparam -L../../libddbg/lib -lddbg rm driver.o make[2]: Leaving directory `/home/saeed/Desktop/disksim-4.0/diskmodel/tests' make[1]: Leaving directory `/home/saeed/Desktop/disksim-4.0/diskmodel' make -C memsmodel make[1]: Entering directory `/home/saeed/Desktop/disksim-4.0/memsmodel' Makefile:81: *** missing separator (did you mean TAB instead of 8 spaces?). Stop. make[1]: Leaving directory `/home/saeed/Desktop/disksim-4.0/memsmodel' make: *** [all] Error 2 -- Regards, Saeed Ghasemi M.Sc. Student Department of Computer Science and Engineering and IT School of Electrical Engineering and Computer Tabriz University Tabriz; Iran -------------- next part -------------- An HTML attachment was scrubbed... URL: From ykwok2 at miners.utep.edu Wed Jul 10 22:30:23 2013 From: ykwok2 at miners.utep.edu (Yipkei Kwok) Date: Wed, 10 Jul 2013 20:30:23 -0600 Subject: [Disksim-users] Problem: No. of in-flight requests bounded by 20 Message-ID: Hi, I want to use synthetic workload generators to simulate a scenario where 50 single-thread independent processes are concurrently accessing a single-disk storage system. All I/O requests are synchronous. I expect that the number of pending request in the entire I/O hierarchy (I/O driver + controller + disk) to be 50. However, I found that the number of pending requests drops from 50, at the beginning, to 20, later in the simulation. Here is the summary of the DiskSim parv file (http://utminers.utep.edu/ykwok2/tmp/20130710-0.parv.txt) * I/O driver: FCFS * single disk: MAXTOR_ATLAS10K5_300 (disk model provided by Anjo at http://ovahldy.blogspot.com/2011/03/recent-diskmodels-for-disksim.html) * Think time from call to request = 0.0 * Think time from request to return = 0.0 * number of synthetic workload generators = 50 * System call/return with each request = 1 (i.e., each request occurs within the context of a system call) Note: I gave a try to set "System call/return with each request" to 0 but I encountered the same problem. * For each generator, Probability of time-critical request = 1.0 This is how I measure the number of pending requests in the I/O hierarchy. I keep a counter in the simulator. When a request arrives the I/O driver, iodriver_request() increments the counter. When a request is serviced, iodriver_access_complete() decrements the counter. The plot of the number of pending requests vs. time is here (http://utminers.utep.edu/ykwok2/tmp/Slide1.PNG). I repeated the experiment with 35, 20, and 15 generators (all results are shown on the same slide). The number of pending requests with 35 generators, too, drops to 20, while the numbers of pending requests with 20 and 15 generators remain constant. I want the number of pending requests to be 50 when 50 generators are used. Why is the number of pending requests dropping? Please kindly help. Thank you very much. Regards, Yipkei -- ******************************************** Yipkei Kwok Ph.D. Candidate HiPerSys Lab Department of Computer Science The University of Texas at El Paso Phone: 915 747 6433 (O) E-mail: ykwok2 at miners dot utep dot edu ******************************************** From chensinhome at gmail.com Thu Jul 11 23:56:09 2013 From: chensinhome at gmail.com (=?Big5?B?q0inu7Ov?=) Date: Fri, 12 Jul 2013 11:56:09 +0800 Subject: [Disksim-users] Disksim-users Digest, Vol 91, Issue 4 In-Reply-To: References: Message-ID: hello, dear Yipkei Kwok is that possible that your storage constantly finish your IO request (30 IOPS ?) so the pending requests does not match the request rate 2013/7/12 > Send Disksim-users mailing list submissions to > disksim-users at ece.cmu.edu > > To subscribe or unsubscribe via the World Wide Web, visit > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > or, via email, send a message with subject or body 'help' to > disksim-users-request at ece.cmu.edu > > You can reach the person managing the list at > disksim-users-owner at ece.cmu.edu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Disksim-users digest..." > > > Today's Topics: > > 1. Problem: No. of in-flight requests bounded by 20 (Yipkei Kwok) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 10 Jul 2013 20:30:23 -0600 > From: Yipkei Kwok > Subject: [Disksim-users] Problem: No. of in-flight requests bounded by > 20 > To: Disksim Users Mailing List > Message-ID: > < > CAF75uxwp0y1Obdb_Nxpsa4ubD5J2GRZBPxkSXFZ0gYc3Kv+twQ at mail.gmail.com> > Content-Type: text/plain; charset="ISO-8859-1" > > Hi, > > I want to use synthetic workload generators to simulate a scenario > where 50 single-thread independent processes are concurrently > accessing a single-disk storage system. All I/O requests are > synchronous. I expect that the number of pending request in the entire > I/O hierarchy (I/O driver + controller + disk) to be 50. However, I > found that the number of pending requests drops from 50, at the > beginning, to 20, later in the simulation. > > Here is the summary of the DiskSim parv file > (http://utminers.utep.edu/ykwok2/tmp/20130710-0.parv.txt) > * I/O driver: FCFS > * single disk: MAXTOR_ATLAS10K5_300 (disk model provided by Anjo at > http://ovahldy.blogspot.com/2011/03/recent-diskmodels-for-disksim.html) > * Think time from call to request = 0.0 > * Think time from request to return = 0.0 > * number of synthetic workload generators = 50 > * System call/return with each request = 1 (i.e., each request occurs > within the context of a system call) > Note: I gave a try to set "System call/return with each request" to 0 > but I encountered the same problem. > * For each generator, Probability of time-critical request = 1.0 > > This is how I measure the number of pending requests in the I/O > hierarchy. I keep a counter in the simulator. When a request arrives > the I/O driver, iodriver_request() increments the counter. When a > request is serviced, iodriver_access_complete() decrements the > counter. > > The plot of the number of pending requests vs. time is here > (http://utminers.utep.edu/ykwok2/tmp/Slide1.PNG). I repeated the > experiment with 35, 20, and 15 generators (all results are shown on > the same slide). The number of pending requests with 35 generators, > too, drops to 20, while the numbers of pending requests with 20 and 15 > generators remain constant. > > I want the number of pending requests to be 50 when 50 generators are > used. Why is the number of pending requests dropping? > > Please kindly help. Thank you very much. > > Regards, > Yipkei > > -- > ******************************************** > Yipkei Kwok > Ph.D. Candidate > HiPerSys Lab > Department of Computer Science > The University of Texas at El Paso > Phone: 915 747 6433 (O) > E-mail: ykwok2 at miners dot utep dot edu > ******************************************** > > > > ------------------------------ > > _______________________________________________ > Disksim-users mailing list > Disksim-users at ece.cmu.edu > https://sos.ece.cmu.edu/mailman/listinfo/disksim-users > > > End of Disksim-users Digest, Vol 91, Issue 4 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: