pub trait Quantizable {
type Quantized;
type QuantizationError;
const DEFAULT_GROUP_SIZE: i32 = 64i32;
const DEFAULT_BITS: i32 = 4i32;
// Required method
fn try_into_quantized(
self,
group_size: i32,
bits: i32,
) -> Result<Self::Quantized, Self::QuantizationError>;
}Expand description
Trait for quantization of modules.
Provided Associated Constants§
Sourceconst DEFAULT_GROUP_SIZE: i32 = 64i32
const DEFAULT_GROUP_SIZE: i32 = 64i32
The default group size for quantization.
Sourceconst DEFAULT_BITS: i32 = 4i32
const DEFAULT_BITS: i32 = 4i32
The default number of bits for quantization.
Required Associated Types§
Sourcetype QuantizationError
type QuantizationError
The error type for quantization.
Required Methods§
Sourcefn try_into_quantized(
self,
group_size: i32,
bits: i32,
) -> Result<Self::Quantized, Self::QuantizationError>
fn try_into_quantized( self, group_size: i32, bits: i32, ) -> Result<Self::Quantized, Self::QuantizationError>
Quantize the module with the specified group size and number of bits.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.