Post · bonfire.cafe

@FabMusacchio@mastodon.social · 14 hours ago

… okay, I didn't expect Mastodon to speed up the playback of attached GIFs 🙈

@FabMusacchio@mastodon.social · yesterday

I actually wrote a short introduction to #WassersteinDistance and #OptimalTransport some time ago, if you’re looking for a more intuitive entry point:

🌍 https://www.fabriziomusacchio.com/blog/2023-07-23-wasserstein_distance/

#Wasserstein

3 media

Two Gaussian distributions with different standard deviations. The Wasserstein distance increases as the standard deviation of the target distribution increases (upper two panels). A shift along the
x
-axis still has just a smaller effect (lower panel). — Two Gaussian distributions with different standard deviations. The Wasserstein distance increases as the standard deviation of the target distribution increases (upper two panels). A shift along the x -axis still has just a smaller effect (lower panel).

Different sets of two 2D discrete random, but normally distributed datasets. The distributions are identical, but shifted apart from each other. The source distribution is centered at μ 1 = ( 0 , 0 ) . The target distribution is shifted from the source distribution by 0 units ( μ 2 = ( 0 , 0 ) , upper panel), + 2 units ( μ 2 = ( 2 , 2 ) , middle panel), and + 4 units ( μ 2 = ( 4 , 4 ) , lower panel) into x - and y -direction, respectively. The standard deviations for all distributions is σ 1 = σ 2 = 1 . The corresponding Wasserstein distance is 0, ∼ 0.15 and ∼ 0.31, respectively. Left panels show the two distributions, middle panels the cost matrix, and right panels the optimal transport plan.

Fabrizio Musacchio

Wasserstein distance and optimal transport

The Wasserstein distance, also known as the Earth Mover’s Distance (EMD), provides a robust and insightful approach for comparing probability distributions and finds application in various fields such as machine learning, data science, image processing, and information theory. In this post, we take a look at the optimal transport problem, required to calculate the Wasserstein distance, and how to calculate the distance metric in Python.

Fabrizio Musacchio

@FabMusacchio@mastodon.social · 14 hours ago

Following up on this, I also explored a more direct use of #WassersteinDistance in #WGANs: Instead of training a discriminator, the generator is optimized by explicitly computing the #OptimalTransport distance between real and generated samples. This turns the loss into the actual metric of interest and removes the adversarial setup, leading to a more direct and stable training signal. And we can generate cool animations, too ^_^

🌍 https://www.fabriziomusacchio.com/blog/2023-07-30-wgan_with_direct_wasserstein_distance/

#MachineLearning #Wasserstein

4 media

GIF

Fabrizio Musacchio

Eliminating the middleman: Direct Wasserstein distance computation in WGANs without discriminator

We explore an alternative approach to implementing WGANs. Contrasting from the standard implementation that requires both a generator and discriminator, the method discussed here employs the optimal transport to compute the Wasserstein distance directly between the real and generated data distributions, eliminating the need for a discriminator.

Fabrizio Musacchio

@FabMusacchio@mastodon.social · 14 hours ago

Taking this one step further, I also looked at #ConditionalGANs: Extending #GANs by conditioning both generator and discriminator on labels, so you can explicitly control what is generated. In the #MNIST case, this means generating specific digits and even smoothly interpolating between them:

🌍 https://www.fabriziomusacchio.com/blog/2023-07-30-cgan/

#MachineLearning #GenerativeAI #CGAN #GAN

(The attached GIF shows the interpolation between the digits 6 and 1)