Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): [SCv2] Automatically create and upload a custom HF model to seldon-models in GCS on every new MLServer version #5103

Conversation

vtaskow
Copy link
Contributor

@vtaskow vtaskow commented Aug 23, 2023

What this PR does / why we need it:
The next version of MLServer will be allowing users to create custom HuggingFace models. Users will be able to specify a location in which model artefacts will be located and later used. This will allow users to customize certain model parameters, such as tokenizers, feature extractors, image processors, etc and then build a HuggingFace model and use it, not limiting themselves to existing models on the official hub.

Automating the creation and the upload of a simple custom HuggingFace model to our collection of models in our GCS bucket - seldon-models, will pave the way to use said model in upcoming demos, showcasing how users can actually load a custom model and use it in SCv2 or in our paid products.

By having this automation, we ensure that on every new MLServer version release, the custom HF model is added to our collection of models in the bucket and demo pages are kept up-to-date.

Testing

  • The resulted model was generated correctly by running make text-generation-huggingface
  • The generated model was uploaded correctly to a public GCS bucket(mine in this case) by running make upload-text-generation-huggingface
  • Used the Seldon CLI to load a Model K8s resource, pointing to a GCS location with a custom HF model and correctly predicted

Seldon Deploy Testing

  • Passed said model GCS location in Seldon Deploy Wizard and successfully created and predicted for a Deployment and for a Pipeline

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

@vtaskow vtaskow changed the title [DRAFT] V2D-1257 Create and upload a custom HF model to seldon-models in GCS feat(docs): Create and upload a custom HF model to seldon-models in GCS Aug 23, 2023
@vtaskow vtaskow marked this pull request as ready for review August 23, 2023 08:50
@vtaskow vtaskow requested review from agrski and removed request for sakoush August 23, 2023 09:56
@vtaskow vtaskow changed the title feat(docs): Create and upload a custom HF model to seldon-models in GCS feat(docs): Create and upload a custom HF model to seldon-models in GCS for SCv2 Aug 23, 2023
@vtaskow vtaskow changed the title feat(docs): Create and upload a custom HF model to seldon-models in GCS for SCv2 feat(docs): [SCv2] Automatically create and upload a custom HF model to seldon-models in GCS on every new MLServer version Aug 23, 2023
Copy link
Contributor

@adriangonz adriangonz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@vtaskow vtaskow merged commit aaf7b1d into SeldonIO:v2 Aug 24, 2023
6 checks passed
adriangonz pushed a commit that referenced this pull request Aug 31, 2023
…to seldon-models in GCS on every new MLServer version (#5103)

* V2D-1257 Create and upload a custom HF model to seldon-models in GCS

* Change GCS location to make it obvious it is a custom hf model
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants