32 points | by zyoralabs 4 hours ago
1 comments
This is so freaking awesome, I am working on a project trying run 10 models on two GPUs, loading/off loading is the only solution I have in mind.
Will try getting this deployed.
Does cold start timings advertised for a condition where there is no other model loaded on GPUs?
This is so freaking awesome, I am working on a project trying run 10 models on two GPUs, loading/off loading is the only solution I have in mind.
Will try getting this deployed.
Does cold start timings advertised for a condition where there is no other model loaded on GPUs?