Struct QuantizedLinear

pub struct QuantizedLinear {
    pub group_size: i32,
    pub bits: i32,
    pub scales: Param<Array>,
    pub biases: Param<Array>,
    pub inner: Linear,
}

Applies an affine transformation to the input using a quantized weight matrix.

It is the quantized equivalent of Linear. For now its parameters are frozen and will not be included in any gradient computation but this will probably change in the future.

QuantizedLinear also provides several useful static to convert linear layers to QuantizedLinear layers.

Fields

group_size: i32

Quantization group size. Default to QuantizedLinear::DEFAULT_GROUP_SIZE

bits: i32

Bits per parameter. Default to QuantizedLinear::DEFAULT_BITS

scales: Param<Array>

Scales

biases: Param<Array>

Biases

inner: Linear

Inner linear layer

Implementations

impl QuantizedLinear

pub fn new( input_dims: impl Into<i32>, output_dims: impl Into<i32>, ) -> Result<Self, <<Self as Buildable>::Builder as Builder<Self>>::Error>

Creates a new instance of QuantizedLinear.

impl QuantizedLinear

pub const DEFAULT_GROUP_SIZE: i32 = 64i32

Default group size

pub const DEFAULT_BITS: i32 = 4i32

Default bits

pub fn try_from_linear( linear: Linear, group_size: impl Into<Option<i32>>, bits: impl Into<Option<i32>>, ) -> Result<Self, Exception>

Convert a linear layer to a quantized linear layer.

Params

• linear: The linear layer to convert.
• group_size: The group size to use for the quantized weight. Default to QuantizedLinear::DEFAULT_GROUP_SIZE
• bits: The bit width to use for the quantized weight. Default to QuantizedLinear::DEFAULT_BITS

Trait Implementations

impl Buildable for QuantizedLinear

type Builder = QuantizedLinearBuilder

The builder type for this buildable type

impl Builder<QuantizedLinear> for QuantizedLinearBuilder

type Error = Exception

Error with building

fn build(self) -> Result<QuantizedLinear, Self::Error>

Build the type

impl Clone for QuantizedLinear

fn clone(&self) -> QuantizedLinear

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedLinear

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module<&Array> for QuantizedLinear

type Error = Exception

Error type for the module.

type Output = Array

Output type of the module.

fn forward(&mut self, x: &Array) -> Result<Array, Self::Error>

Forward pass of the module.

fn training_mode(&mut self, mode: bool)

Set whether the module is in training mode.

impl ModuleParameters for QuantizedLinear

fn freeze_parameters(&mut self, recursive: bool)

Freeze all parameters in the module.

fn unfreeze_parameters(&mut self, recursive: bool)

Unfreeze all parameters in the module.

fn parameters(&self) -> ModuleParamRef<'_>

Get references to the module parameters.

fn parameters_mut(&mut self) -> ModuleParamMut<'_>

Get mutable references to the module parameters.

fn trainable_parameters(&self) -> ModuleParamRef<'_>

Get references to the trainable parameters. A parameter is trainable if it is NOT frozen.

fn all_frozen(&self) -> Option<bool>

Check if all parameters in the module are frozen. Returns None if there are no parameters.

fn any_frozen(&self) -> Option<bool>

Check if any parameter in the module is frozen. Returns None if there are no parameters.

fn update(&mut self, parameters: ModuleParam)

Update the module parameters.

fn update_flattened(&mut self, flattened_parameters: FlattenedModuleParam)

Update the module parameters from a flattened representation.

impl TryFrom<Linear> for QuantizedLinear

type Error = Exception

The type returned in the event of a conversion error.

fn try_from(linear: Linear) -> Result<Self, Self::Error>

Performs the conversion.