Too much! Especially with the nand shortage and high GPU prices. I don't "need" it but if you want to do *anything* with a model larger than 32B parameters, you either need to accept heavy quantization or it's going to be so slow that it's not useful. I could, of course, rent a machine in the cloud, but that precludes you from doing other interesting hardware things and getting close to the bare metal.
Let's hear more about Daddy!
Very cool. How much did it cost? Why do you "need" such a powerful machine?
Too much! Especially with the nand shortage and high GPU prices. I don't "need" it but if you want to do *anything* with a model larger than 32B parameters, you either need to accept heavy quantization or it's going to be so slow that it's not useful. I could, of course, rent a machine in the cloud, but that precludes you from doing other interesting hardware things and getting close to the bare metal.