Function quantize

pub fn quantize<M>(
    module: M,
    group_size: impl Into<Option<i32>>,
    bits: impl Into<Option<i32>>,
) -> Result<M::Quantized, M::QuantizationError>where
    M: Quantizable,

Expand description

Quantize a module.

§Params

module: The module to quantize.
group_size: The group size to use for the quantized weight. Default to Quantizable::DEFAULT_GROUP_SIZE
bits: The bit width to use for the quantized weight. Default to Quantizable::DEFAULT_BITS