Struct Adam

pub struct Adam {
    pub lr: Array,
    pub betas: (Array, Array),
    pub eps: Array,
    pub state: State<(Array, Array)>,
}

The Adam optimizer.

Please refer to the original paper for more details:

[1]: Kingma, D.P. and Ba, J., 2015. Adam: A method for stochastic optimization. ICLR 2015.

Fields

lr: Array

The learning rate

betas: (Array, Array)

The coefficients used for computing running averages of the gradient and its square

Default to Adam::DEFAULT_BETAS

eps: Array

The epsilon added to the denominator to improve numerical stability

Default to Adam::DEFAULT_EPS

state: State<(Array, Array)>

Inner state

Implementations

impl Adam

pub fn new(lr: impl Into<f32>) -> Self

Creates a new instance of Adam.

impl Adam

pub const DEFAULT_BETAS: (f32, f32) = _

Default values for betas

pub const DEFAULT_EPS: f32 = 9.99999993E-9f32

Default value for eps

Trait Implementations

impl Buildable for Adam

type Builder = AdamBuilder

The builder type for this buildable type

impl Builder<Adam> for AdamBuilder

type Error = Infallible

Error with building

fn build(self) -> Result<Adam, Self::Error>

Build the type

impl Clone for Adam

fn clone(&self) -> Adam

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Adam

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for Adam

type State = HashMap<Rc<str>, (Array, Array)>

State of the optimizer.

fn state(&self) -> &Self::State

Get the state of the optimizer.

fn state_mut(&mut self) -> &mut Self::State

Get the mutable state of the optimizer.

fn update_single( &mut self, key: &Rc<str>, gradient: &Array, parameter: &mut Array, ) -> Result<()>

Update a single parameter with the given gradient.

fn update<M>( &mut self, model: &mut M, gradients: impl Borrow<FlattenedModuleParam>, ) -> Result<()>

where M: ModuleParameters,

Apply the gradients to the parameters of the model and update the model with the new parameters.

impl Updatable for &mut Adam

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.

impl Updatable for Adam

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.