mlx_rs::nn

Function mish

Source
pub fn mish(x: impl AsRef<Array>) -> Result<Array>
Expand description

Applies the Mish function, element-wise.

Mish: A Self Regularized Non-Monotonic Neural Activation Function.

Reference: https://arxiv.org/abs/1908.08681

This is:

x * tanh(softplus(x))