How RuView's RuvSense Implements Attention-Weighted Cross-Viewpoint Embedding for Multistatic Wi-Fi Sensing

RuView's RuvSense system implements attention-weighted cross-viewpoint embedding through a two-stage hierarchical attention mechanism that first fuses raw Wi-Fi CSI data using correlation-based attention weights, then applies geometrically-biased scaled-dot-product attention to blend learned embeddings across spatially diverse ESP-32 nodes.

RuView is an open-source Wi-Fi sensing platform that leverages a mesh of ESP-32 nodes to perform device-free pose estimation and breathing detection. At the core of its perception pipeline lies attention-weighted cross-viewpoint embedding, a technique implemented in the RuvSense multistatic fusion system to aggregate signals from multiple viewpoints while suppressing multipath artifacts and emphasizing geometric diversity.

Hierarchical Attention Architecture

The RuvSense fusion pipeline consists of two tightly-coupled stages that together realize a hierarchical attention strategy:

Stage	Purpose	Core Implementation
Multistatic fusion	Aggregates raw CSI amplitude/phase across nodes using an attention-weighted mean. Implements a per-subcarrier attention gate that amplifies signals correlated across viewpoints and suppresses node-specific multipath artefacts.	[multistatic.rs](https://github.com/ruvnet/RuView/blob/main/rust-port/wifi-densepose-rs/crates/wifi-densepose-signal/src/ruvsense/multistatic.rs)
Cross-viewpoint embedding attention	Operates on higher-level learned embeddings (e.g., spectrogram or AETHER features) and injects a geometric bias that rewards spatially diverse viewpoints.	[attention.rs](https://github.com/ruvnet/RuView/blob/main/rust-port/wifi-densepose-rs/crates/wifi-densepose-ruvector/src/viewpoint/attention.rs)

Raw CSI amplitudes are first cleaned with an attention gate, then the resulting per-node feature vectors are further blended with geometry-aware attention. This yields a single fused embedding that captures both signal consensus and the physical diversity of the sensor array.

Stage 1: Multistatic Fusion with Attention-Weighted CSI

Data Flow and Algorithm

The MultistaticFuser in multistatic.rs processes Wi-Fi CSI frames through the following pipeline:

1. Collect N MultiBandCsiFrames from the TDMA mesh (one per ESP-32).
2. Extract the first channel's amplitude and phase vectors (Vec<f32>).
3. Compute a consensus amplitude (mean_amp).
4. Derive per-node attention logits from the cosine similarity of each node's amplitude to mean_amp. The logits are scaled by a temperature (config.attention_temperature).
5. Softmax the logits to produce attention weights that sum to 1.
6. Weighted average of amplitudes and phase (phase is fused via sin/cos averaging).
7. Emit FusedSensingFrame containing the fused vectors, timestamps, node geometry, and a cross-node coherence score (entropy-derived from the attention distribution).

The heart of the algorithm lives in attention_weighted_fusion (lines 40-84 of multistatic.rs). The softmax step is numerically stable (max-subtraction) and the coherence metric (compute_weight_coherence) provides a quick quality indicator for later model gating.

API and Usage

/// Configuration (guard interval, min nodes, temperature, …)
pub struct MultistaticConfig { /* … */ }

/// The fusion engine.
pub struct MultistaticFuser {
    config: MultistaticConfig,
    node_positions: Vec<[f32; 3]>,
}

Create a fuser with default or custom configuration:

let fuser = MultistaticFuser::new();               // default config
// or with a custom config:
let cfg = MultistaticConfig {
    guard_interval_us: 3000,
    min_nodes: 3,
    attention_temperature: 0.8,
    ..Default::default()
};
let fuser = MultistaticFuser::with_config(cfg);

Fuse a batch of frames:

let fused = fuser.fuse(&node_frames)?;   // `node_frames: &[MultiBandCsiFrame]`
println!("Coherence: {:.2}", fused.cross_node_coherence);

All error handling is via MultistaticError (empty view, dimension mismatch, timestamp spread, etc.) – see the enum definition at the top of multistatic.rs.

Stage 2: Cross-Viewpoint Embedding Attention with Geometric Bias

Once a per-node feature embedding (Vec<f32>) has been derived from the fused CSI (e.g., a spectrogram passed through the wifi-densepose-nn backbone), RuView applies a second attention layer that is aware of node geometry.

Geometric Bias Matrix

GeometricBias encodes a scalar bias for each viewpoint pair i, j:

G_bias[i,j] = w_angle * cos(theta_ij) + w_dist * exp(-d_ij / d_ref)

• theta_ij – azimuthal angle difference between nodes i and j.
• d_ij – Euclidean distance on the floor-plan.
• w_angle, w_dist, d_ref are learnable parameters (default 1.0, 1.0, 5.0 m).

The matrix is built by GeometricBias::build_matrix (lines 56-73 of attention.rs). It is symmetric and has a maximal self-bias (w_angle + w_dist) on the diagonal.

Scaled-Dot-Product with Bias

The classic attention score Q·Kᵀ / sqrt(d) is augmented by adding G_bias before the softmax:

let score = (dot_product + g_bias[i * n + j]) * scale; // line 70 in attend()

This makes pairs of spatially diverse nodes receive a higher raw score, encouraging the model to blend complementary viewpoints rather than redundant ones.

API and Usage

pub struct CrossViewpointAttention {
    pub weights: ProjectionWeights, // Q, K, V linear maps
    pub bias: GeometricBias,
}

Create with identity projections and default bias:

let attn = CrossViewpointAttention::new(128); // 128-dim embeddings

Fuse embeddings to a single vector:

let fused = attn.fuse(&embeddings, &viewpoint_geoms)?;

Inspect raw attention weights for debugging or visualization:

let weights = attn.attention_weights(&embeddings, &viewpoint_geoms)?;

Both fuse and attention_weights return Result<_, AttentionError> that captures empty viewpoint sets or dimension mismatches (see AttentionError at the top of attention.rs).

End-to-End Implementation Example

Below is a minimal, self-contained Rust snippet that demonstrates the full pipeline:

use wifi_densepose_signal::ruvsense::{
    MultistaticFuser, multistatic::MultistaticConfig,
};
use wifi_densepose_ruvector::viewpoint::{
    CrossViewpointAttention, ViewpointGeometry,
};

// 1️⃣  Gather raw frames from the mesh (mocked here)
let node_frames = vec![
    make_test_frame(0, 1000, 56, 1.0),
    make_test_frame(1, 1001, 56, 1.1),
    // … more nodes …
];

// 2️⃣  Fuse raw CSI (attention-weighted)
let fuser = MultistaticFuser::new();
let fused = fuser.fuse(&node_frames).expect("fusion failed");

// 3️⃣  Derive per-node embeddings (placeholder: simple mean of amplitude)
let embeddings: Vec<Vec<f32>> = fused.node_frames.iter()
    .map(|frame| {
        let amp = &frame.channel_frames[0].amplitude;
        let mean = amp.iter().sum::<f32>() / amp.len() as f32;
        vec![mean; 128]          // dummy 128-dim embedding
    })
    .collect();

// 4️⃣  Build geometry descriptors for each node
let viewpoint_geoms: Vec<ViewpointGeometry> = fused.node_positions.iter()
    .enumerate()
    .map(|(i, pos)| ViewpointGeometry {
        azimuth: (i as f32) * std::f32::consts::FRAC_PI_2, // just an example layout
        position: (pos[0], pos[1]),
    })
    .collect();

// 5️⃣  Apply cross-viewpoint attention with geometric bias
let cross_attn = CrossViewpointAttention::new(128);
let final_embedding = cross_attn.fuse(&embeddings, &viewpoint_geoms)
    .expect("cross-viewpoint attention failed");

println!("Final fused embedding length: {}", final_embedding.len());

Key takeaways from the example:

• MultistaticFuser removes node-specific artefacts before any learned representation is built.
• The geometric bias matrix ensures that the second-level attention prefers spatially diverse embeddings, which improves robustness to occlusion and multipath.
• Both stages expose diagnostics (cross_node_coherence, attention_weights) that can be fed back to system health monitors.

Key Source Files

File	Role	Direct link
multistatic.rs	Low-level, attention-weighted CSI fusion (core of Ru Sense).	multistatic.rs
attention.rs	Cross-viewpoint scaled-dot-product attention with geometric bias.	attention.rs
pose_tracker.rs	Consumes FusedSensingFrame to drive Kalman-based pose tracking (shows downstream consumption).	pose_tracker.rs
spectrogram.rs	Example of generating per-node embeddings that are later fed into CrossViewpointAttention.	spectrogram.rs
geometric_bias.rs (inline in attention.rs)	Computes the bias matrix used for geometry-aware weighting.	Same as attention.rs (see lines 85-115)

These files together embody Ru View's hierarchical attention strategy: raw CSI is first cleaned with an attention gate, then learned embeddings are fused with a geometry-aware bias, yielding robust multistatic perception across the Wi-Fi mesh.

Summary

• RuView's attention-weighted cross-viewpoint embedding combines two distinct attention mechanisms in a hierarchical pipeline.
• The multistatic fusion stage in multistatic.rs uses cosine-similarity attention weights to suppress node-specific multipath while amplifying correlated signals across the ESP-32 mesh.
• The cross-viewpoint attention stage in attention.rs applies a geometric bias matrix to scaled-dot-product attention, rewarding spatially diverse viewpoints and improving robustness to occlusion.
• Both stages expose diagnostic metrics (cross_node_coherence, attention_weights) for system health monitoring and downstream model gating.

Frequently Asked Questions

How does the attention mechanism in RuvSense differ from standard transformer attention?

Standard transformer attention computes scaled dot-product scores between query and key vectors without geometric priors. RuvSense's cross-viewpoint embedding attention augments this with a learnable geometric bias matrix G_bias that encodes azimuthal angle differences and inter-node distances, explicitly rewarding spatially diverse viewpoints rather than treating all nodes interchangeably.

What is the purpose of the attention temperature parameter in MultistaticConfig?

The attention_temperature parameter in MultistaticConfig controls the sharpness of the softmax distribution when computing fusion weights. Lower temperatures (e.g., 0.5) produce sharper, more selective attention that favors highly correlated nodes, while higher temperatures (e.g., 1.5) produce softer weights that blend contributions more uniformly across the mesh.

How does the geometric bias matrix improve robustness in the cross-viewpoint stage?

The geometric bias matrix G_bias adds a scalar offset to attention scores based on the physical layout of nodes. By adding w_angle * cos(theta_ij) and w_dist * exp(-d_ij / d_ref) to the dot product, the mechanism increases attention weights for node pairs with large angular separation and optimal distances. This ensures that the fused embedding incorporates complementary geometric perspectives, making the system robust to occlusion and directional signal fading.

Can the attention weights be inspected for debugging or visualization?

Yes. Both stages expose their internal attention distributions. The MultistaticFuser provides cross_node_coherence (an entropy-derived metric) in the FusedSensingFrame, while CrossViewpointAttention offers the attention_weights method, which returns the raw softmax distribution over viewpoint pairs. These diagnostics enable real-time monitoring of mesh health and attention pattern visualization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}