Dedicated Model bugs
in progress
n
ninja
When I start an dedicated 70B model, it shows at low cost that it chooses 4x h200s and says "insufficient quota" or something like that. The fix for that is that I have to manually stop the dedicated model and start it again so that it only uses 2x of them, even though I had low cost selected.
M.R.
in progress
We have the ability to pin the hardware you request for the model, will be released to prod soon.
M.R.
under review
M.R.
Hi, This is great feedback, we are working on how our system schedules on the GPU's and our fleet for the private beta was limited to 16 H200's, we've since allocated H100's and A100's for the private beta. But this is a great note to consider.