Recency Biased Causal Attention for Time-series Forecasting

A simple mechanism that introduces recency bias into Transformer attention by reweighting scores with a smooth heavy-tailed decay.

• Strengthens local temporal dependencies without sacrificing longer-range flexibility.
• Aligns Transformer attention more closely with the read–ignore–write operations of RNNs.
• Achieves competitive or superior performance on challenging time-series forecasting benchmarks.