Skip to content

Latest commit

 

History

History
57 lines (41 loc) · 1.44 KB

README.md

File metadata and controls

57 lines (41 loc) · 1.44 KB
Raw

SageMaker JumpStart Foundation Model Endpoint

Description

Deploys an endpoint for a foundation model from SageMaker JumpStart Foundation Models.

The module uses AWS Generative AI CDK Constructs.

Architecture

SageMaker JumpStart Foundation Model Endpoint Module Architecture

Inputs/Outputs

Input Parameters

Required

  • jump-start-model-name - model name from SageMaker JumpStart Foundation Models
  • instance-type - inference container instance type

Optional

  • vpc-id - VPC id
  • subnet-ids - VPC subnet ids

Module Metadata Outputs

  • EndpointArn - endpoint ARN.
  • RoleArn - IAM role ARN.

Examples

Example manifest:

name: hf-mistral-endpoint
path: modules/fmops/sagemaker-jumpstart-fm-endpoint
targetAccount: primary
parameters:
  - name: jump-start-model-name
    value: HUGGINGFACE_LLM_MISTRAL_7B_2_1_0
  - name: instance-type
    value: inf1.xlarge
  - name: vpc_id
    valueFrom:
      moduleMetadata:
        group: networking
        name: networking
        key: VpcId
  - name: subnet_ids
    valueFrom:
      moduleMetadata:
        group: networking
        name: networking
        key: PrivateSubnetIds