<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><span class="" style="font-family: Optima-Regular;"># P-RECS '18 Call for Papers</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">First International Workshop on Practical Reproducible Evaluation of </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Computer Systems.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">June 11, 2018. In conjunction with HPDC'18 <</span><a href="http://hpdc.org/2018/" class="" style="font-family: Optima-Regular;">http://hpdc.org/2018/</a><span class="" style="font-family: Optima-Regular;">>. In </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">cooperation with SIGHPC (pending).</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Independent evaluation of experimental results in the area of computer </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">and networking systems is a challenging task. Recreating the </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">environment where an experiment originally ran is commonly considered </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">impractical or even impossible. This workshop will focus heavily on </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">practical, actionable aspects of reproducibility in broad areas of </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">computational science and data exploration, with special emphasis on </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">issues in which community collaboration can be essential for adopting </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">novel methodologies, techniques and frameworks aimed at addressing </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">some of the challenges we face today. The workshop will bring together </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">researchers and experts to share experiences and advance the state of </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">the art in the reproducible evaluation of computer systems, featuring </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">contributed papers and invited talks.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Topics</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">We expect submissions from topics such as, but not limited to:</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Experiment dependency management.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Software citation and persistence.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Data versioning and preservation.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Provenance of data-intensive experiments.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Tools and techniques for incorporating provenance into publications.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Automated experiment execution and validation.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Experiment portability for code, performance, and related metrics.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Experiment discoverability for re-use.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Cost-benefit analysis frameworks for reproducibility.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Usability and adaptability of reproducibility frameworks into already-established domain-specific tools.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Long-term artifact archiving for future reproducibility.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Frameworks for sociological constructs to incentivize paradigm shifts.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Policies around publication of articles/software.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Blinding and selecting artifacts for review while maintaining history.</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Reproducibility-aware computational infrastructure.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Submission</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Submit via EasyChair </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"><</span><a href="https://easychair.org/conferences/?conf=precs18" class="" style="font-family: Optima-Regular;">https://easychair.org/conferences/?conf=precs18</a><span class="" style="font-family: Optima-Regular;">>. We look for two </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">categories of submissions:</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * **Position papers**. This category is for papers whose goal is to </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> propose solutions (or scope the work that needs to be done) to </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> address some of the issues outlined above. We hope that a research </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> agenda comes out of this and that we can create a community that </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> meets yearly to report on our status in addressing these problems.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * **Experience papers**. This category consists of papers reporting </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> on the authors' experience in automating one or more </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> experimentation pipelines. The committee will look for submissions </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> reporting on their experience: what worked? What aspects of </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> experiment automation and validation are hard in your domain? What </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> can be done to improve the tooling for your domain? As part of the </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> submission, authors need to provide a URL to the automation </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> service they use (e.g., [TravisCI](</span><a href="https://travis-ci.org" class="" style="font-family: Optima-Regular;">https://travis-ci.org</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> [GitLabCI](</span><a href="https://about.gitlab.com/gitlab-ci/" class="" style="font-family: Optima-Regular;">https://about.gitlab.com/gitlab-ci/</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> [CircleCI](</span><a href="https://circleci.com" class="" style="font-family: Optima-Regular;">https://circleci.com</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> [Jenkins](</span><a href="https://jenkins-ci.org" class="" style="font-family: Optima-Regular;">https://jenkins-ci.org</a><span class="" style="font-family: Optima-Regular;">), etc.) so reviewers can verify </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> that there is one or more automated pipelines associated to the </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> submission.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">### Format</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Authors are invited to submit manuscripts in English not exceeding 5 </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">pages of content. The 5-page limit includes figures, tables and </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">appendices, but does not include references, for which there is no </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">page limit. Submissions must use the [ACM Master </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Template](</span><a href="https://www.acm.org/publications/proceedings-template" class="" style="font-family: Optima-Regular;">https://www.acm.org/publications/proceedings-template</a><span class="" style="font-family: Optima-Regular;">) </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">(please use the `sigconf` format with default options).</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">### Proceedings</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">The proceedings will be archived in both the ACM Digital Library and </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">IEEE Xplore through SIGHPC.</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">### Tools</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">These tools can be optionally used used to automate your experiments: </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">[CWL](</span><a href="http://commonwl.org" class="" style="font-family: Optima-Regular;">http://commonwl.org</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">[Popper](</span><a href="https://github.com/systemslab/popper" class="" style="font-family: Optima-Regular;">https://github.com/systemslab/popper</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">[ReproZip](</span><a href="http://reprozip.org" class="" style="font-family: Optima-Regular;">http://reprozip.org</a><span class="" style="font-family: Optima-Regular;">), [Sciunit](</span><a href="http://sciunit.run" class="" style="font-family: Optima-Regular;">http://sciunit.run</a><span class="" style="font-family: Optima-Regular;">), </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">[Sumatra](</span><a href="https://github.com/open-research/sumatra" class="" style="font-family: Optima-Regular;">https://github.com/open-research/sumatra</a><span class="" style="font-family: Optima-Regular;">).</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Important Dates</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Submissions due: *<b class="">April 9</b>*, 2018</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Acceptance notification: April 30, 2018</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Camera-ready paper submission: May 6, 2018</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Workshop: June 11, 2018</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Organizers</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Ivo Jimenez, UC Santa Cruz</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Carlos Maltzahn, UC Santa Cruz</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Jay Lofstead, Sandia National Laboratories</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Program Committee</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Divyashri Bhat, UMass Amherst</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Michael Crusoe, Project Lead, Common Workflow Language project</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Anja Feldmann, TU Berlin</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Todd Gamblin, LLNL</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Mike Heroux, Sandia National Laboratories</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Torsten Hoefler, ETH Zürich</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Neil Chue Hong, Software Sustainability Institute / University of </span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> Edinburgh, UK</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Dan Katz, NCSA</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Kate Keahey, Argonne National Lab / ChameleonCloud</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Ignacio Laguna, LLNL</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Arnaud Legrand, Bâtiment IMAG</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Reed Milewicz, Sandia National Laboratories</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Robert Ricci, University of Utah / CloudLab</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Victoria Stodden, UIUC</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Violet R. Syrotiuk, ASU</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Michela Taufer, University of Delaware</span><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;"> * Michael Zink, UMass Amherst</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">## Contact</span><br class="" style="font-family: Optima-Regular;"><br class="" style="font-family: Optima-Regular;"><span class="" style="font-family: Optima-Regular;">Please address workshop questions to <</span><a href="mailto:ivo@cs.ucsc.edu" class="" style="font-family: Optima-Regular;">ivo@cs.ucsc.edu</a><span class="" style="font-family: Optima-Regular;">>.</span><div class=""><span class="" style="font-family: Optima-Regular;"><br class=""></span></div><div class=""><span class="" style="font-family: Optima-Regular;"><br class=""></span></div></body></html>