Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
Quentin Guilloteau
Quentin Guilloteau
@GuilloteauQ@social.sciences.re  ·  activity timestamp 4 weeks ago

> "Our results show that
Docker does not guarantee reproducibility under any tested definition, nor is there a “silver bullet” set of rules for
writing Dockerfiles yielding reproducible images"

Nice paper from @luj @zacchiro @Zimm_i48 on the use of Docker and reproducibility [1]

Somewhat related to what we were trying to show at ACM REP'25 [2], but at different scales (in time and amount of dockerfiles).

The scientific community cannot say that it has not been warned of Docker's reproducibility pitfalls ! :)

[1] https://arxiv.org/pdf/2601.12811
[2] https://dl.acm.org/doi/10.1145/3736731.3746146

https://arxiv.org/pdf/2601.12811
  • Copy link
  • Flag this post
  • Block
Ludovic Courtès
Ludovic Courtès
@civodul@toot.aquilenet.fr  ·  activity timestamp 4 weeks ago

@GuilloteauQ @hannes @Zimm_i48 @zacchiro @luj Amazing work.

Many of us have been repeating “Dockerfiles cannot yield reproducible images” for years, but a thorough literature review and experimental demonstration like this really hits the nail.

#ReproducibleResearch #OpenScience

  • Copy link
  • Flag this comment
  • Block
Stefano Zacchiroli
Stefano Zacchiroli
@zacchiro@mastodon.xyz  ·  activity timestamp 4 weeks ago

@civodul @GuilloteauQ @hannes @Zimm_i48 @luj the most interesting part for me personally was indeed the misalignment (for the most part) between beliefs documented in the literature and reality.

Notwithstanding that, of course, we are not the first to raise the specific alarm about Docker not being a reproducibility silver bullet. The more, the merrier!

There is also a lot to say about the impact of technology hype on science (remember the early Docker days?), but it'll be for another day...

  • Copy link
  • Flag this comment
  • Block
Konrad Hinsen
Konrad Hinsen
@khinsen@scholar.social  ·  activity timestamp 4 weeks ago

@zacchiro @GuilloteauQ @hannes @Zimm_i48 @luj My impression is that most people who believe in Docker as a reproducibility silver bullet have a very superficial understanding of reproducibility and why / when / how it matters. Pointing out to them what precisely does *not* work with Docker should help with explaining the issues. So... thanks for this work!

@civodul

  • Copy link
  • Flag this comment
  • Block
Davide Eynard (+mala)
Davide Eynard (+mala)
@mala@fosstodon.org  ·  activity timestamp 4 weeks ago

@zacchiro very interesting indeed! How deoes this lack of reproducibility translate in reproducibility of the experiments the docker images were set up for, i.e. did you find a correlation between a higher discrepancy in packages and a lower statistical significance of the difference in downstream experimental results? I have been recently inspired by https://www.sfscon.it/talks/why-you-cannot-apt-install-an-llm-yet/ and would like to find a sound, principled way to make data+code+exp available for reproducibility of ML-based packages

SFSCON

Why you cannot “apt install” an LLM (yet) - SFSCON

SFSCON Talk
  • Copy link
  • Flag this comment
  • Block

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.2-alpha.32 no JS en
Automatic federation enabled
Log in
Instance logo
  • Explore
  • About
  • Members
  • Code of Conduct