Including questions of LLM origin. Seems like the OP might have submitted that one (47431685) although there's another copy now (beyond this SCP entry from 3 days ago)
Tl;Dr it's actually a CIX p1 + 32gb (similar to orange pi 6) and a "160TOPS" NPU accelerator with 48gb - attached via NVME. models will either have to fit in one pool or deal with shuttling data over m.2, the company has some optimizations regarding this but it's still a serious limitation.
There you go, two sentences without burying the lede.
Is it maybe competitive value anyways though? Even if you only think of the accelerators, 48gb+160TOPS seems comparable to some Strix Halo mini PCS with 64gb - lower memory bandwidth but a few hundred dollars cheaper. If they sold just the accelerator card for $800 or something that would be potentially very interesting.
Previously (23 points, 6 days ago, 6 comments) https://news.ycombinator.com/item?id=47395786
Including questions of LLM origin. Seems like the OP might have submitted that one (47431685) although there's another copy now (beyond this SCP entry from 3 days ago)
Tl;Dr it's actually a CIX p1 + 32gb (similar to orange pi 6) and a "160TOPS" NPU accelerator with 48gb - attached via NVME. models will either have to fit in one pool or deal with shuttling data over m.2, the company has some optimizations regarding this but it's still a serious limitation.
There you go, two sentences without burying the lede.
Is it maybe competitive value anyways though? Even if you only think of the accelerators, 48gb+160TOPS seems comparable to some Strix Halo mini PCS with 64gb - lower memory bandwidth but a few hundred dollars cheaper. If they sold just the accelerator card for $800 or something that would be potentially very interesting.
Great research and write-up, maybe a bit too elaborate.
Will be interesting to see if a public outcry will happen once these boxes start arriving at those who funded the kickstarter.
It’s LLM slop and very shallow, in my opinion.
whats shallow about the research? it all seems to check out?
The blogpost?
ctrl-f for "This isn't" and note how many instances of this pattern there are:
> This isn't X. It's Y.