Apicurio Registry integration with Kubernetes

[EricWittmann]

Discussion topic from @eshepelyuk on twitter:

I see a several "operator" repositories here https://github.com/Apicurio?q=operator&type=&language=&sort=

Some are written in Go, but some of them are empty

• https://github.com/Apicurio/apicurio-studio-operator - empty but mentions Quarkus
• https://github.com/Apicurio/apicurio-registry-content-sync-operator - empty

I am asking to better understand your intention and strategy for creating K8S native extensions for Apicurio, especially Registry.

I had in mind for sometime already the idea of creating the controller that can watch deployed resources ( Service, CRD) and populate Apicurio registry with API exposed by those resources, for instance poll OpenAPI and push them to Apicurio, thus creating the ultimate develoerp portal with all the documentaiton located in k8s cluster.

Maybe you have smth similar in mind and alrady working on that, If yes - please share your vision and tools you're going to use.

I'm interested in collaboratino and contribution in Java or Python.

[EricWittmann]

We do have a few operators currently. Off the top of my head we have these:

• Apicurito installer operator (helm chart I think)
• Apicurio Registry installer operator (golang)
• Apicurio Studio installer operator (java/quarkus) [in progress]
• Apicurio Registry Sync operator (java/quarkus) [in progress]

I think the one that is most relevant to your discussion here is the Sync Operator:

https://github.com/Apicurio/apicurio-registry-content-sync-operator

The repository was empty but only because I hadn't yet merged the initial PR. I've now merged that so you can have a look at what it does if you like.

@famartinrh has written that, and I think it is similar to what you are proposing. It's not quite the same, assuming I understand your described use-case. I think what you are suggesting is an operator that actually registers the OpenAPI design of actual running services deployed to Kubernetes. So e.g. any micro-service that advertises its own API description via e.g. OpenAPI could be automatically registered in Apicurio Registry.

For the record I think that's a fantastic idea. And I think we could use the sync operator to make that happen.

[eshepelyuk]

Thanks for the explanation. I should take a look at apicurio-registry-content-sync-operator.
That's just amazing that you're not blindly writing k8s stuff in golang ,
but instead are open to more mainstream languages and frameworks, known by a wider audience of potential contributors.

[eshepelyuk]

Hello

I've skimmed over the operator code and I consider that chosen strategy is more superior than my original proposal.
Having custom resources that describe schemas is extremely flexible, thus deployed microservices could provide as much as possible Artifact CRD to be stored in Apicurio Schema registry.

For instance if microservice installed with Helm, the chart could generate Avro Artifact for Kafka and OpenAPI Artifact for REST API, picking up the content from application source code.

The only good addition would be to implement fetching OpenAPI from URL, so in case OpenAPI can't be generated in build time - the operator could fetch later OpenAPI declaration from the URL of running microservice instance.

One additonal note I'd like to raise - is the presentation of this operator. Since it not only a synchronization tool, but (at least for me) - it could transforms Apicurio Schema Registry into Develoler Portal, containing all the APIs used by the project \ platform.
Compare to those products

• https://docs.solo.io/gloo-portal/latest/
• https://www.getambassador.io/docs/edge-stack/latest/topics/using/dev-portal/

But unlikely beforementioned, Apicurio is not bound to any API Gatewa, could do more formats than OpenAPI and free !

[EricWittmann]

Great feedback thanks. @famartinrh ^^^

I actually thought it had the ability to fetch the OpenAPI from a URL, but perhaps I'm mistaken.

Hey @famartinrh perhaps a blog post about this operator and its capabilities makes sense! :)

[eshepelyuk]

@EricWittmann @famartinrh I see that apicurio-registry-content-sync-operator doesn't provide Helm chart.
If you're interested I could help you with creating it.

[famarting]

how? https://github.com/Apicurio/apicurio-registry-content-sync-operator#how-to-deploy-this this way for instance? kubectl apply

[kmandal-volvo]

I pull the latest image from dockerhub docker pull apicurio/apicurio-registry-operator:latest
Create a image push to our Azure kubernet registry . Used https://raw.githubusercontent.com/apicurio/apicurio-registry-operator/master/docs/resources/install.yaml deployment descriptor but finally pod deployment failed ---- "error":"no matches for kind "ApicurioRegistry" in version "apicur.io/v1alpha1"","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/runner/go/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\nmain.main\n\tapicurio-registry-operator/cmd/manager/main.go:144\nruntime.main\n\t/opt/hostedtoolcache/go/1.14.4/x64/src/runtime/proc.go:203"}

[kmandal-volvo]

how? https://github.com/Apicurio/apicurio-registry-content-sync-operator#how-to-deploy-this this way for instance? kubectl apply

ComparisonError
failed to convert *unstructured.Unstructured to *v1beta1.Ingress: no kind "Ingress" is registered for version "networking.k8s.io/v1" in scheme "pkg/runtime/scheme.go:101"

[famarting]

those are errors with https://github.com/Apicurio/apicurio-registry-operator , if you could create an issue there

[eshepelyuk]

@kmandal-volvo
please check Apicurio/apicurio-registry-content-sync-operator#11