Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): [SCv1] Automatically create and upload a custom HF model to seldon-models in GCS on every version release #5104

Conversation

vtaskow
Copy link
Contributor

@vtaskow vtaskow commented Aug 23, 2023

What this PR does / why we need it:
The next version of MLServer will be allowing users to create custom HuggingFace models. Users will be able to specify a location in which model artefacts will be located and later used. This will allow users to customize certain model parameters, such as tokenizers, feature extractors, image processors, etc and then build a HuggingFace model and use it, not limiting themselves to existing models on the official hub.

Automating the creation and the upload of a simple custom HuggingFace model to our collection of models in our GCS bucket - seldon-models, will pave the way to use said model in upcoming demos, showcasing how users can actually load a custom model and use it in SCv2 or in our paid products.

By having this automation, we ensure that on every new MLServer version release, the custom HF model is added to our collection of models in the bucket and demo pages are kept up-to-date.

By having this change separated from a similar SCv2-related change, I am ensuring that SCv1 related demo pages are using models uploaded in non-SCv2 folders in GCS. This will avoid confusion as to why a SCv1-related demo is using a model uploaded to gs://seldon-models/scv2/... folder.

Testing

  • The resulted model was generated correctly by running make env && make train
  • The generated model was uploaded correctly to a public GCS bucket(mine in this case) by running make upload
  • The uploaded model was used correctly by creating a K8s SeldonDeployment resource and was able to correctly make a prediction without any additional downloads
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: huggingface-model
spec:
  protocol: v2
  predictors:
  - graph:
      name: transformer
      implementation: HUGGINGFACE_SERVER
      modelUri: gs://viktor-models/v1.18.0-dev/huggingface/text-generation
      parameters:
      - name: task
        type: STRING
        value: text-generation
    componentSpecs:
      - spec:
          containers:
            - name: transformer
              resources:
                limits:
                  cpu: 1
                  memory: 4Gi
                requests:
                  cpu: 100m
                  memory: 3Gi
    name: default
    replicas: 1

image

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

@vtaskow vtaskow marked this pull request as ready for review August 23, 2023 11:19
@vtaskow vtaskow changed the title feat(docs): Create and upload a custom HF model to seldon-models in GCS for SCv1 feat(docs): [SCv1] Automatically create and upload a custom HF model to seldon-models in GCS on every version release Aug 23, 2023
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Contributor

@adriangonz adriangonz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one @vtaskow! Looks good to me.

@vtaskow vtaskow merged commit 7c2d4f1 into SeldonIO:master Aug 24, 2023
11 of 12 checks passed
vtaskow added a commit to vtaskow/seldon-core that referenced this pull request Aug 29, 2023
…to seldon-models in GCS on every version release (SeldonIO#5104)

* V2D-1258 Create and upload a custom HF model to seldon-models in GCS

* Remove left over link for kfserving storage initialiser

* Change GCS location to make it obvious it is a custom hf model
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants