Struct AdaGrad

pub struct AdaGrad {
    pub lr: Array,
    pub eps: Array,
    pub state: State,
}

The Adagrad optimizer.

Please refer to the original paper for more details:

[1]: Duchi, J., Hazan, E. and Singer, Y., 2011. Adaptive subgradient methods for online learning and stochastic optimization. JMLR 2011.

Fields

lr: Array

Learning rate

eps: Array

The epsilon added to the denominator to improve numerical stability. Default to AdaGrad::DEFAULT_EPS.

state: State

Inner state

Implementations

impl AdaGrad

pub fn new(lr: impl Into<f32>) -> Self

Creates a new instance of AdaGrad.

impl AdaGrad

pub const DEFAULT_EPS: f32 = 9.99999993E-9f32

Default value for eps.

Trait Implementations

impl Buildable for AdaGrad

type Builder = AdaGradBuilder

The builder type for this buildable type

impl Builder<AdaGrad> for AdaGradBuilder

type Error = Infallible

Error with building

fn build(self) -> Result<AdaGrad, Self::Error>

Build the type

impl Clone for AdaGrad

fn clone(&self) -> AdaGrad

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AdaGrad

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for AdaGrad

type State = HashMap<Rc<str>, Array>

State of the optimizer.

fn state(&self) -> &Self::State

Get the state of the optimizer.

fn state_mut(&mut self) -> &mut Self::State

Get the mutable state of the optimizer.

fn update_single( &mut self, key: &Rc<str>, gradient: &Array, parameter: &mut Array, ) -> Result<()>

Update a single parameter with the given gradient.

fn update<M>( &mut self, model: &mut M, gradients: impl Borrow<FlattenedModuleParam>, ) -> Result<()>

where M: ModuleParameters,

Apply the gradients to the parameters of the model and update the model with the new parameters.

impl Updatable for &mut AdaGrad

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.

impl Updatable for AdaGrad

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.