Bifrost CSL Architecture Specification
Comprehensive mathematical derivations, continuous receptive field formulations, and zero KV-cache memory proofs behind the Valkir model family.
Bifrost CSL: Sub-quadratic Linear Foundation Models
Abstract
Transformer self-attention incurs $O(N^2)$ time and $O(N)$ KV-cache spatial complexity during inference, creating massive hardware bottlenecks for long context and local edge deployment. We introduce Bifrost CSL (Convolutional Sub-quadratic Linear), which synthesizes hierarchical grouped dilated convolutions with input-dependent SiLU gating and SwiGLU feedforward networks. Evaluated on 4.74 billion pretraining tokens on a single consumer AMD Radeon RX 9070 XT GPU, our 116.8M reference model (Valkir 16L) matches or exceeds Pythia-160M and OPT-125M across OpenAI HumanEval (54.0% AST pass), WinoGrande (51.62%), and ARC-Easy (33.08%) while maintaining a strictly immutable $O(1)$ 8.8 MB state buffer.
Eliminates quadratic attention complexity by utilizing concurrent short and dilated temporal convolutions, establishing an expansive 1,233-token receptive field across 16 layers.
Dynamically modulates contextual information flow based on token importance without storing past key-value vectors in VRAM, guaranteeing sub-2ms time-to-first-token.
Calibrated scaling factors applied to residual connections completely eliminate NaN loss anomalies during marathon pretraining runs on consumer AMD hardware.
While traditional Transformers consume gigabytes of VRAM at 16K or 32K context, Bifrost CSL maintains an immutable 8.86 MB buffer, enabling infinite context streaming on edge devices.
Empirical Pretraining Curriculum (4.74 Billion Cumulative Tokens)
Cite Bifrost CSL
@article{valerois2026bifrost,
title={Bifrost CSL: Convolutional Sub-quadratic Linear Foundation Models with Zero KV-Cache},
author={Valerois AI Research Team},
journal={Valerois Technical Report},
volume={1},
number={1},
year={2026},
url={https://valerois.com/research}
}