Has anyone else hit expert homogeneity collapse in small MoE models?

 (github.com)

1 points | by rfi-irfos  9 hours ago

1 comments