Feature request: Akash storage as a separate deployment

[shimpa1]

dear all,

This is a feature that I believe will tie everything Akash has to offer together.

Basically the way it would work is storage can be leased separately and then plugged into a different lease/deployment as long as they both have the same owner.

The storage would be accessible for as long as it's being paid for and would be of persistent type.
The storage classes might also be a good idea, perhaps a good metric would be IOPS and bandwidth - not sure if it can be implemented this way. Perhaps having 4 or 5 performance classes.
It should also be accessible by multiple deployments - again as long as they have the same owner.

This would be a better substitution for the VOLUME Docker stanza that is currently disabled.

In my humble opinion this feature is what would make Akash a true IAAS marketplace.

Sort of how LEGO works - there are blocks that you can combine to create a bigger "thing".

Discuss!

Shimpa | Europlots

[piber-dev]

i like this idea! my only addition is that i would want functions for data migration, backups, and exporting to a local machine.

[shimpa1]

Yeah, all of those can be separate deployments that have access to the storage. For instance if one needs S3 object storage they can spin up a deployment that runs S3 and is using the storage deployment (that's separate but owned by the same key).
This way it's super scalable and modular.

[piber-dev]

great solution to my ideas, yeah all of those could be either separate deployments if not built out as functions. if this would only work on one provider and not through the internet, these solutions would be flawed if the user cannot deploy another deployment on the same provider though. so either this must work through the internet and not only local network / service to service by default, or there must be built-in functions by default.

[rodri-r]

I like this idea too, I see tons of storage available in many providers, which is just waiting there to be utilized somehow.

[shimpa1]

Perhaps utilize persistent storage so that it doesn't get deleted on pod restart

[SGC41]

yes i think this would be a great idea.
there are a few more things to consider but that might fall into the aforementioned multiple storage classes.

1. A Owner of the storage might not want all their deployments to be able to easily access their storage, so i think there should be atleast an option to restrict access to some deployments.

2. A more cloud like approach, again this would most likely be its own storage class.
   Akash has a partnership with Storj, and Storj are in the process of creating community run satellites, which i think can run their own pricing schemes, this would mean some providers or akash could run a Storj Satellite dedicated to Akash Cloud storage, which would then be stored on providers running a Storj Storage Node connected to the Akash Storj Satellite.
   .
   This would allow for very powerful and reliable storage across providers, ofc it will never be as efficient as the local solution inside kubernetes.
   but it would provider other benefits which is why i think it would be a good thing to include.

That being said any kind of long term storage that is independent of tenants usual deployment leases would be a big upgrade, and add a super useful feature that will give a much improved customer experience.

SGC | DCnorse

[rodri-r]

Agreed and it could be a good way for providers to earn some AKT from their unused storage.

[31trainman]

Going off the Lego idea. What if storage could be specified as either on host or cloud based. And with cloud based you can specify your storage login credentials upon container startup or sdl.

[shimpa1]

Honestly my initial thought was for the storage to be local to every provider. That would make it perform well enough so that the other deployments accessing it can actually work with it in real time.
Using other platforms to offload storage can already be achieved.

[vpavlin]

Thanks for posting this @shimpa1! This is something I am struggling with as a Kubernetes/OpenShift user - not being able to specify storage (Persistent Volume Claim) separately from deployment is a massive bummer, because it results in loss of data anytime something goes wrong with my deployment.

Just today I have updated my deployment, but had a typo there, which resulted in back-off which never recovered. I am fine redeploying from scratch, but the data stored on persistent storage is lost, so separation of deployment and storage is a must.

What can we do to prioritize this for the Akash team?

[31trainman]

If your on akash discord contact me and I'll see what I can do to help you. My discord is 31trainman.