Running Apicurio Registry without docker

[kevinlam]

Hi there,

I understand that docker is the preferred way of having Apicurio Regisgry running. But my company hasn't migrated to use docker yet ( I know :( ) . Is there anyway to download any pre-build artifact and get the registry up and running ? like java -jar xxx.jar?

Thanks for your help

[EricWittmann]

Thanks for the question, @kevinlam

Currently we don't have a pre-built JAR that you can just download and run. However I think that's a reasonable thing for us to have. We are very much not bound to Docker in any way. We always make sure that our projects can be run in a variety of ways.

In this case, if you wanted to use the latest version (currently 2.2.3.Final) you would need to build it yourself. I will see what we can do about upgrading our release processes to attach a pre-built binary to the GitHub release whenever we do one. It might not be available for the next release (this week) but we'll see.

[EricWittmann]

• Attach built JARs and Libraries to the release #2538

[kevinlam]

Many thanks for getting back to me. I did try building it but I seem to have hit a couple of dependency/lib issues. I could probably resolve them given time. Looking forward to see the release with the prebuild jar :) Kevin Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Eric Wittmann ***@***.***> Sent: Tuesday, May 31, 2022 6:11:32 PM To: Apicurio/apicurio-registry ***@***.***> Cc: kevinlam ***@***.***>; Mention ***@***.***> Subject: Re: [Apicurio/apicurio-registry] Running Apicurio Registry without docker (Discussion #2519) Thanks for the question, @kevinlam<https://github.com/kevinlam> Currently we don't have a pre-built JAR that you can just download and run. However I think that's a reasonable thing for us to have. We are very much not bound to Docker in any way. We always make sure that our projects can be run in a variety of ways. In this case, if you wanted to use the latest version (currently 2.2.3.Final) you would need to build it yourself. I will see what we can do about upgrading our release processes to attach a pre-built binary to the GitHub release whenever we do one. It might not be available for the next release (this week) but we'll see. — Reply to this email directly, view it on GitHub<#2519 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACWCMOZ7XZ6KBZKLHZYAVLVMZB4JANCNFSM5W43RVMA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[EricWittmann]

Ask and you shall receive (sometimes):

https://github.com/Apicurio/apicurio-registry/releases/tag/2.2.4.Final

The binaries are attached to the release. I don't have any documentation right now about how to use them. But here is some ad-hoc docs:

• apicurio-registry-app-2.2.4.Final-all.tar.gz : Apicurio Registry with a non-persistent storage (in-memory only, data will be lost whenever you restart the application). Useful for testing.
• apicurio-registry-storage-kafkasql-2.2.4.Final-all.tar.gz : Apicurio Registry with Kafka as the persistent storage. You'll need to create your own Kafka cluster and then configure the app to point to it.
• apicurio-registry-storage-sql-2.2.4.Final-all.tar.gz : Apicurio Registry with PostgreSQL as the persistent storage. You'll need to provision your own PG database instance and configure the app to point to it.

You should be able to find docs on https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/index.html - all of the environment variables that are documented there (for docker) are the same here as well.

Side note: you're running Outlook on an Android device? That's blowing my mind. :)

[kevinlam]

Hi team. Really sorry to bother you again. I have been looking at your documentation but couldn’t find how to enable auto schema register? Is there such a function in your registry? What I want to do is that if the schema doesn’t exist already, the registry will auto register it for next time’s use? I am getting no artifact found at the moment when that happens. BTW, I got the schema register installed and running on the server, many thanks for the added artifacts. Regards, Kevin From: Kevin Lam ***@***.***> Sent: 03 June 2022 22:17 To: Apicurio/apicurio-registry ***@***.***>; Apicurio/apicurio-registry ***@***.***> Cc: Mention ***@***.***> Subject: Re: [Apicurio/apicurio-registry] Running Apicurio Registry without docker (Discussion #2519) That's amazing! Thank you so much for the effort. The turnaround is well exceeding my expectation. Thought the email was gonna be replied with some standard message and get backlogged for the next few years :) Don't know how to answer your side note without feeling too embarrassed 🤣😂 Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Eric Wittmann ***@***.******@***.***>> Sent: Friday, June 3, 2022 6:43:55 PM To: Apicurio/apicurio-registry ***@***.******@***.***>> Cc: kevinlam ***@***.******@***.***>>; Mention ***@***.******@***.***>> Subject: Re: [Apicurio/apicurio-registry] Running Apicurio Registry without docker (Discussion #2519) Ask and you shall receive (sometimes): https://github.com/Apicurio/apicurio-registry/releases/tag/2.2.4.Final The binaries are attached to the release. I don't have any documentation right now about how to use them. But here is some ad-hoc docs: * apicurio-registry-app-2.2.4.Final-all.tar.gz : Apicurio Registry with a non-persistent storage (in-memory only, data will be lost whenever you restart the application). Useful for testing. * apicurio-registry-storage-kafkasql-2.2.4.Final-all.tar.gz : Apicurio Registry with Kafka as the persistent storage. You'll need to create your own Kafka cluster and then configure the app to point to it. * apicurio-registry-storage-sql-2.2.4.Final-all.tar.gz : Apicurio Registry with PostgreSQL as the persistent storage. You'll need to provision your own PG database instance and configure the app to point to it. You should be able to find docs on https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/index.html - all of the environment variables that are documented there (for docker) are the same here as well. Side note: you're running Outlook on an Android device? That's blowing my mind. :) — Reply to this email directly, view it on GitHub<#2519 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACWCMI3SMIWGQJSA5I3CXTVNI75XANCNFSM5W43RVMA>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[EricWittmann]

Auto register is a feature of the client rather than the server. So you'll need to explain your use-case a bit. Is this for a Kafka application? If yes, are you using Apicurio Registry provided SerDes classes or Confluent SerDes? Or are you using the API directly to register? Or perhaps the UI?

In our REST API, there is an option called ifExists when creating a new artifact that controls what to do if the artifact already exists:

https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/assets-attachments/registry-rest-api.htm#tag/Artifacts/operation/createArtifact

If you're using our SerDes classes then there is an option to configure auto registration of schemas. For example:

https://github.com/Apicurio/apicurio-registry-examples/blob/main/simple-avro/src/main/java/io/apicurio/registry/examples/simple/avro/SimpleAvroExample.java#L150

If you're using Confluent SerDes, there is a way to do this as well, but I'm not sure offhand what it is.

There are some complicated scenarios here, so you may need to be specific about your use-case.

[EricWittmann]

Also note: the UI does not currently support the ifExists feature. It might be interesting if it did though....

[kevinlam]

I see. Thanks for getting back to me. Our stack is springboot + kafka. I tried using your serdes in our spring cloud stream app but it keeps complaining the artifact is not found on the registry. I managed to switch back to confluence compatible api (apis/ccompat/v6) with their serde and it seems to do the trick. May be I was missing the config props.putIfAbsent(SerdeConfig.AUTO_REGISTER_ARTIFACT, Boolean.TRUE); ? Is that defaulted to false by default? Btw, if I choose to have kafka backed registry, would I be able to start up multiple registries on different node pointing to the same kakfa brokers? Sorry for all the naïve questions as I am really new to this. Thanks again for your help.

[EricWittmann]

Yes the default value for SerdeConfig.AUTO_REGISTER_ARTIFACT is false - so it makes sense why you were getting that error. Our SerDes classes are a little harder to configure than the confluent ones - we have more config options essentially.

If you use the Kafka backed storage in registry, yes you can start up multiple registry nodes/instances/pods. That's no problem at all. They all need to point to the same topic on the same broker. Each instance will simply consume all the same messages on that topic and construct the same in-memory state of the registry data. It should work just fine. :)

[kevinlam]

That's amazing! Thank you so much for the effort. The turnaround is well exceeding my expectation. Thought the email was gonna be replied with some standard message and get backlogged for the next few years :) Don't know how to answer your side note without feeling too embarrassed 🤣😂 Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Eric Wittmann ***@***.***> Sent: Friday, June 3, 2022 6:43:55 PM To: Apicurio/apicurio-registry ***@***.***> Cc: kevinlam ***@***.***>; Mention ***@***.***> Subject: Re: [Apicurio/apicurio-registry] Running Apicurio Registry without docker (Discussion #2519) Ask and you shall receive (sometimes): https://github.com/Apicurio/apicurio-registry/releases/tag/2.2.4.Final The binaries are attached to the release. I don't have any documentation right now about how to use them. But here is some ad-hoc docs: * apicurio-registry-app-2.2.4.Final-all.tar.gz : Apicurio Registry with a non-persistent storage (in-memory only, data will be lost whenever you restart the application). Useful for testing. * apicurio-registry-storage-kafkasql-2.2.4.Final-all.tar.gz : Apicurio Registry with Kafka as the persistent storage. You'll need to create your own Kafka cluster and then configure the app to point to it. * apicurio-registry-storage-sql-2.2.4.Final-all.tar.gz : Apicurio Registry with PostgreSQL as the persistent storage. You'll need to provision your own PG database instance and configure the app to point to it. You should be able to find docs on https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/index.html - all of the environment variables that are documented there (for docker) are the same here as well. Side note: you're running Outlook on an Android device? That's blowing my mind. :) — Reply to this email directly, view it on GitHub<#2519 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AACWCMI3SMIWGQJSA5I3CXTVNI75XANCNFSM5W43RVMA>. You are receiving this because you were mentioned.Message ID: ***@***.***>