- Support SentenceTransformers models inference by @aleksandr-mokrov in #865
from optimum.intel.openvino import OVSentenceTransformer
model_id = "sentence-transformers/all-mpnet-base-v2"
model = OVSentenceTransformer.from_pretrained(model_id, export=True)
sentences = ["This is an example sentence", "Each sentence is converted"]
embeddings = model.encode(sentences)- Infer if the model needs to be exported or not by @echarlaix in #825
from optimum.intel import OVModelForCausalLM
- model = OVModelForCausalLM.from_pretrained("gpt2", export=True)
+ model = OVModelForCausalLM.from_pretrained("gpt2")Compatible with transformers>=4.36,<=4.44
Full Changelog: v1.18.0...v1.19.0
Contributors
aleksandr-mokrov and echarlaix