Post
Just opened an issue for a major new task for #Fedify: building an #interoperability smoke test suite.
To ensure Fedify-built servers federate correctly with the wider #fediverse, we're planning to run automated E2E tests in #CI against live instances of Mastodon, Misskey, and more. This is crucial for a framework's reliability.
You can see the full plan and discussion here:
@hongminhee funfedi.dev already runs some compatibility tests on Hollo
@hongminhee funfedi.dev already runs some compatibility tests on Hollo
@hongminhee By the way, according to this test Hollo can't process an activity if @context is not set:
https://funfedi.dev/support_tables/context/
I would appreciate if you fix that.
@hongminhee /cc @helge because of the overlap with https://pasture.funfedi.dev/
@hongminhee /cc @helge because of the overlap with https://pasture.funfedi.dev/
@hongminhee Sounds ambitious but worthwhile!
You might have some overlap with https://github.com/Letterbook/Sandcastles by @jenniferplusplus, @feditest, or https://github.com/berkmancenter/activitypub-fuzzer/ by @darius. Hopefully there are some facets that are already solved or at least lessons to incorporate.
@julian @hongminhee @jenniferplusplus @feditest I'd love for any of these projects to use the data I've collected at the Observation btw
@darius @julian This is fantastic, thank you! A dataset of “in-the-wild” objects is the perfect complement to the testing plan.
I can see this being hugely useful for creating our test cases, ensuring we handle all the variations and quirks that different servers actually produce.
Bookmarked! We really appreciate you sharing this.
@hongminhee just saw this, we do run live tests against things like Mastodon and Wordpress + plugins in various configurations.
@julian Wow, thank you so much for these links! This is exactly the kind of prior art and community effort we need to be aware of.
I'll definitely be studying Sandcastles, @feditest, and ActivityPub Fuzzer. It's super helpful to see what's already been done. We're all for incorporating existing lessons (or even tools) rather than reinventing everything.
Really appreciate you pointing these out!
@hongminhee wow, this is very cool, and inspiring.
I'll be watching and may have to borrow this idea.
You really think of everything 😎
@dansup That's very kind of you, thank you!
We're excited about it too. And “borrowing” is highly encouraged! We'll be happy if our idea can be useful to other projects.
A space for Bonfire maintainers and contributors to communicate
