Chameleon Virtual Reproducibility Hackathon
Learning how to produce reproducible experiments on Chameleon
We are excited to announce our Virtual Reproducibility Hackathon taking place on December 15, 2023, hosted by the Chameleon Team! See the details below for more information and to register.
About the Event
Reproducibility is not just a sound scientific practice – it also has many potential practical applications in teaching and creating new research results. At this event, the attendees will learn about how to package their computer science research experiments in such a way that they are not only reproducible – but also practically reproducible, i.e., capable of being reproduced easily enough to be used as a mainstream method of interactive scientific exploration and exchange.
Sign up to learn about practical reproducibility on Chameleon, receive guidance on packaging your experiments, share insights and feedback on the reproducibility process, and much more!
We hope that participants can use their initial work and what they learn at the Hackathon to make future contributions to reproducibility workshops and conferences such as ACM REP.
Reproducibility Sessions
The Hackathon will consist of three sessions held throughout the day to accommodate folks outside of the Central time zone:
- Session I: Tutorial – Packaging Experiments for Practical Reproducibility – 10 CT AM to 12 CT PM
- Session II: Open Work Period – Packaging Your Experiments – 1 CT PM to 4 CT PM
- Session III: Open Discussion – Share Your Experience – 4:10 CT PM to 5 CT PM
Participants are encouraged, but not required, to attend all three sessions.
Tutorial: Packaging Experiments for Practical Reproducibility
Session I will include an overview of Chameleon services, how to run experiments on Chameleon, how to package experiments for reproducibility, how to make such artifacts searchable and shareable, and other best practices for practical reproducibility. If you are interested in learning more about Chameleon Cloud and our digital artifact hub (Trovi), this session is for you!
Concepts may include how to use the NSF-funded Chameleon platform, one of the largest academic clouds, and how to package experiments for reproducibility using this platform; tools and services Chameleon provides to share experiments, including using Jupyter to manage the full experimental workflow including creating the environment, implementing the experiment body, and data analytics; using Chameleon daypass to give access to the testbed for reproducibility; as well as Trovi, an experiment sharing portal integrated with the testbed. The tutorial will also cover the existing experiment patterns available via Trovi representing common elements of experimental configurations such as e.g., configuring storage with RAID, NFS, or RDMA.
Open Work Period: Packaging Actual Experiments
During Session II, participants will work on packaging their experiments to share on the Chameleon digital artifact hub, Trovi, following the guidelines covered in Session I. Technical support and “office hours” will be provided for individuals and teams during this session. If you have an experiment that you have been wanting to share with others, this is a great opportunity to finally get that artifact online with some extra help and support from the Chameleon team!
No experiment to share at the moment? No problem! Participants are also encouraged to use this session to reproduce existing artifacts from a curated list of recommended experiments that we will post two weeks before the event.
How to come prepared for reproducibility success:
We recommend that participants who attend this session bring one of their own experiments that they would like to package and make available for reproducibility on Trovi. Experiments that already have an established code base with some documentation on GitHub or other public repositories are typically quicker to package and will be easier to work with during the session. Some knowledge of Chameleon and general cloud computing is also helpful (such as concepts covered in Session I). However, participants are not required to use Session II to package their experiments. Participants can also use the time to run other experiments that are already on Trovi.
Open Discussion: Share Your Experience
Participants will share their experiences during the hackathon and have time to ask more questions about the platform, give feedback on the Chameleon and Trovi sites, and brainstorm ideas to facilitate experiment reproducibility on public testbeds.
We hope to see you all on December 15!