Struct RmsProp

pub struct RmsProp {
    pub lr: Array,
    pub alpha: Array,
    pub epsilon: Array,
    pub state: State,
}

The RMSprop optimizer [1].

[1]: Tieleman, T. and Hinton, G. 2012. Lecture 6.5-rmsprop, coursera: Neural networks for machine learning

Fields

lr: Array

Learning rate

alpha: Array

The smoothing constant. Default to RmsProp::DEFAULT_ALPHA if not specified.

epsilon: Array

The epsilon added to the denominator to improve numerical stability. Default to RmsProp::DEFAULT_EPSILON if not specified.

state: State

Inner state

Implementations

impl RmsProp

pub fn new( lr: impl Into<f32>, ) -> Result<Self, <<Self as Buildable>::Builder as Builder<Self>>::Error>

Creates a new instance of RmsProp.

impl RmsProp

pub const DEFAULT_ALPHA: f32 = 0.990000009f32

Default alpha if not specified.

pub const DEFAULT_EPSILON: f32 = 9.99999993E-9f32

Default epsilon if not specified.

Trait Implementations

impl Buildable for RmsProp

type Builder = RmsPropBuilder

The builder type for this buildable type

impl Builder<RmsProp> for RmsPropBuilder

type Error = RmsPropBuildError

Error with building

fn build(self) -> Result<RmsProp, Self::Error>

Build the type

impl Clone for RmsProp

fn clone(&self) -> RmsProp

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RmsProp

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for RmsProp

type State = HashMap<Rc<str>, Array>

State of the optimizer.

fn state(&self) -> &Self::State

Get the state of the optimizer.

fn state_mut(&mut self) -> &mut Self::State

Get the mutable state of the optimizer.

fn update_single( &mut self, key: &Rc<str>, gradient: &Array, parameter: &mut Array, ) -> Result<()>

Update a single parameter with the given gradient.

fn update<M>( &mut self, model: &mut M, gradients: impl Borrow<FlattenedModuleParam>, ) -> Result<()>

where M: ModuleParameters,

Apply the gradients to the parameters of the model and update the model with the new parameters.

impl Updatable for &mut RmsProp

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.

impl Updatable for RmsProp

fn updatable_states(&self) -> impl IntoIterator<Item = &Array>

Returns a list of references to the updatable states.

fn updatable_states_mut(&mut self) -> impl IntoIterator<Item = &mut Array>

Returns a list of mutable references to the updatable states.