DBRX MoE-ADBRX MoE-ADBRX MoE-A

Smallest DBRX family member: 7.7B total parameters, 2.2B active. Used internally by Databricks to study MoE training efficiency. Achieves 30.5% on the Databricks Gauntlet with 3.7x fewer FLOPs than MPT-7B.