Tagged with moe
Running a 744B Model on a Laptop, Then Telling the Author Not to Fix That Bug
colibri claims you can run GLM-5.2, all 744 billion parameters, on about 25GB of RAM. I actually ran it on an M1 Max. Seven seconds per token. Three counterintuitive findings came out of it, and one of them turned my upstream report from "please fix this bug" into "please don't".