Provider Optional Multi Bid Feature

[SGC41]

Multi Model GPU Providers have a serious handicap, due to the "random" selection of which GPU's are offered, when no GPU is specified in the SDL when creating a Deployment.

To solve this i would like to see a multi bidding option, which would allow a provider to create a number of bids...
maybe one for each GPU model, or maybe using some sort of configuration... like say one could set a number of bids and define that it would be the cheapest GPU and the most expensive GPU available on the provider at that time.

This is ofc also relevant for other features, but for this first suggestion, i'm just focusing on the GPU aspect.
but it would be relevant for CPU features and such also...

duno the exact best way to structure multiple bids, for GPU's its not that difficult, but for something like CPU features, it could quickly lead to excessive bids and thus fee costs from providers.

Thus the multi bid feature, would most likely need to have some kind of highly customizable / programable configuration method.
Without something like that, it might not be useful and adaptable down the line.

[dominikusbrian]

Maybe this can be done by allowing Providers to have a sort of "Priority List" for their GPU resources ?
So when bid come through without exact GPU specs in SDL, the bid will goes to the preferred GPU node/card designated by providers.
For instance, provided have 4 x 4090 and 2x h100, the preference for this particular provider is set to be 4090 first then h100, so if not asked specifically, and 4090 can fulfil the required task the bid will be deployed for 4090 and not h100.

Other providers may prefer different positioning for their GPUs, but i guess this is simple enough ? so when bidding the bid engine will first read through a manifest of GPU priorities, and go bid for the first one available on the priority.

Maybe this make more sense ? Multi bid may cost quite a bit, since the bid will be opened and launched for several gpus/cpus while the winner will most likely just one of them.

Though Multi Bid will be nice if the goal is ensure fair pricing. But there's also a logic in which user who doesn't specify spec usually simply trying to find the best and cheapest, so the other offers (that is not the cheapest) from the same provider, most likely doesn't win anyway, while all the extra tx fee still need to be paid for.

[instafinanzas]

Great points!

Is there a way to make the GPU specification mandatory for users when they are creating a sdl for deployments?

Q2 since each bid generated costs money to generate, is there a way to have "premade" bids based on whats more requested by users??

[SGC41]

The added bid fee's are certainly a concern, but lets break down the numbers...
also it could be possible that multi bid shouldn't cost multi fees.... should providers be punished for hosting more gear on a single provider, rather than splitting it up across multiple providers...

i think not.

checked the number of bids over the last 12 hours and it was something 40 with a few of those being close leases.
i know this might be on the low end, but its a number... we can consider it at a 10x rate also to ensure that its not really a major factor.

a create bid fee is currently 0.004 AKT, thus 80 in a day, 2400 a month, will only look at the monthly costs, because deployment costs are set based on monthly.

0.004 x 2400 = 9.6 AKT a month = 33.6$ at 3.5$ AKT
so yeah adding more fee cost is certainly not very favorable... however without multiple bids, a provider with say 4 different GPU's can will have a 25% chance to offer the right GPU for a task.
and having lower price GPU's might not even be any help, because customers might often just go for the cheapest.

But yeah.... maybe adding more fees isn't really the way to go...

I would push for multiple bids within some sort of limit of maybe 3 - 5 unique hardware bids from a single provider to be only a single create bid "bids" fee.

so it doesn't become a punishment to supply different hardware options for customers, this might also solve the issue with all the different possible cpu features when having a fair number of servers... it would be easy to imagine larger providers bidding 5 - 10 different hardware configurations.

ofc how many customers select based on cpu architecture, but some will for performance or unique cpu features such as AVX512 or https://en.wikipedia.org/wiki/Trusted_execution_environment or similar CPU features.

or maybe secondary create bid fee's should come with at discount, if they can't be rolled into a single transaction.

Is there a way to make the GPU specification mandatory for users when they are creating a sdl for deployments?

don't think this is a viable solution, forcing customers to actively select or deselect GPU's on creation will mean the majority of options on the network wouldn't get offered.

it might be possible for a customer to somehow designate what they are looking for tho.... like best possible prices or high vram or something like that.... but again this means akash will have to restrain options, which are essentially infinite, into neat little catagories.

would work for cheapest GPU's and CPU prices tho, but just can't imagine it working well on a large scale with many options.

i mean today NPU's are making their way into CPU's, are we going to add seperate catagories for that, or will it just be another detail grabbed from the CPU features, which is hardcoded it them and thus can be grabbed just like GPU models, vram amounts and other suchs details.

People might not even know what they are looking for...
Thus forcing them to limit themselves would be detrimental to their experience and Akash growth.
a consumer user might not discover SXM for AI training, or a cheap NPU that could solve their inferencing issue along with the CPU needs they got.

Well this extended rant and me pondering this more exact gave me, some interesting new ideas as how to solve this problem, maybe without needing multiple bids... so might have to morph this feature request, but will wait and see until there has been some dev feedback on the viability of this in its current form.

still think it might be a worth while feature, but this as a solution for what i consider the problem might be overkill.
i'm sure multiple bids are no easy feature to add.... or i would assume so.