Struct AttentionParams 

pub struct AttentionParams {
    pub batch: i64,
    pub seq_q: i64,
    pub seq_kv: i64,
    pub heads_q: i64,
    pub heads_kv: i64,
    pub head_dim: i64,
    pub mask: AttentionMask,
    pub scale: f64,
    pub dropout: f32,
    pub dropout_seed: u64,
}

Attention parameters.

Fields

batch: i64

seq_q: i64

seq_kv: i64

heads_q: i64

heads_kv: i64

head_dim: i64

mask: AttentionMask

scale: f64

Scale on the QK^T product. Typically 1/sqrt(head_dim).

dropout: f32

Dropout probability on attention scores. 0.0 disables.

dropout_seed: u64

Implementations

impl AttentionParams

pub fn new( batch: i64, seq_q: i64, seq_kv: i64, heads_q: i64, heads_kv: i64, head_dim: i64, ) -> Self

pub fn with_mask(self, m: AttentionMask) -> Self

pub fn with_dropout(self, p: f32, seed: u64) -> Self

pub fn is_gqa(&self) -> bool

Trait Implementations

impl Clone for AttentionParams

fn clone(&self) -> AttentionParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AttentionParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for AttentionParams

fn eq(&self, other: &AttentionParams) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for AttentionParams