-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ort v1.15.0 ov changes customer a #314
base: ort_v1.15.0_ov_changes
Are you sure you want to change the base?
Ort v1.15.0 ov changes customer a #314
Conversation
* Add support for LUID and handle OV warnings
@@ -88,7 +88,7 @@ CreateOVModel(const ONNX_NAMESPACE::ModelProto& model_proto, const GlobalContext | |||
size_t index = results.size() - 1; | |||
|
|||
for (auto it = results.rbegin(); it != results.rend(); ++it) { | |||
if (auto const_node = std::dynamic_pointer_cast<ngraph::op::Constant>((*it)->input_value(0).get_node_shared_ptr())) { | |||
if (auto const_node = std::dynamic_pointer_cast<ov::op::v0::Constant>((*it)->input_value(0).get_node_shared_ptr())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we using OV API 1.0 or 2.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are with API 2.0 as we are supporting only the latest three release of OV.
@@ -57,7 +57,7 @@ BasicBackend::BasicBackend(const ONNX_NAMESPACE::ModelProto& model_proto, | |||
LOGS_DEFAULT(INFO) << log_tag << "Loaded model to the plugin"; | |||
} else { | |||
#if defined(OPENVINO_2023_0) | |||
if (subgraph_context.precision != InferenceEngine::Precision::FP16) { | |||
if (!subgraph_context_.has_dynamic_input_shape && dev_prec!="CPU_FP16") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Has a bug been raised with OV team for CPU FP 16 Precision
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not yet. Shall raise one.
@@ -257,7 +252,7 @@ void BackendManager::Compute(OrtKernelContext* context) { | |||
} | |||
#endif | |||
bool use_dynamic_backend = true; | |||
if (GetGlobalContext().enable_dynamic_shapes && subgraph_context_.has_dynamic_input_shape && | |||
if (subgraph_context_.has_dynamic_input_shape && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For NPU we still need dynamic backend so maybe this logic wont be suitable going forward
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This logic is only skipping whether enable_dynamic_shapes runtime option is provided. According to this logic, if a model has dynamic shaped input it will be handled with dynamic backend.
std::pair<std::string, ov::Any> device_property; | ||
device_property = std::make_pair("PLUGIN_THROTTLE", "1"); | ||
device_config.emplace(ov::device::properties("GPU_CONFIG_KEY", device_property)); | ||
// device_config[GPU_CONFIG_KEY(PLUGIN_THROTTLE)] = "1"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove commented code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please look at inline comments
a62adeb
to
9112813
Compare
Add 2023.1 support; Fix string nullptr bug
Fix for when device_type is empty
Description
Changes part of Customer A release - 1.15.0
Motivation and Context
Enabled features based on client requirements
This PR is created to review for SDLE.