Tencent releases Hy3 open-source model that allegedly matches models up to five times its active size
Tencent has released an open-source language model called Hy3, which is surprisingly efficient despite its large size. Hy3 has 295 billion parameters, but only 21 billion are active at a time.
Read the breakdown