Regulatory DNA is characterized by a severe class imbalance: most positions are uninformative, while important signals are sparse and short (e.g., TF motifs). As a result, standard language modeling struggles—its loss is dominated by reconstructing the uninformative positions 2/n