← Library

2026-01-10

3 saved articles

  1. Deep Delta Learning

    [Submitted on 1 Jan 2026 (v1), last revised 12 May 2026 (this version, v3)] · arXiv.org

    Transformer residual streams evolve by additive accumulation: each layer appends a feature update to a shared hidden state, but has no direct mechanism for replacing content that has become obsolete or conflicting. We introduce Deep Delta Learning (DDL), a residual update rule that preserves the identity path while giving every layer the ability to selectively rewrite residual content. DDL reads the current state along a learned direction, compares it with a learned target value, and writes back