Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add QNN EP option context_node_name_prefix to set EPContext node name prefix #21236

Merged
merged 8 commits into from
Jul 26, 2024

Conversation

HectorSVC
Copy link
Contributor

Description

Add QNN EP option context_node_name_prefix to set EPContext node name prefix

Motivation and Context

For the case to workaround QNN context PD memory limit, user need split the model into pieces and generate the QNN context model separately. It could happen that the generated EPContext node in separate graph has same node name. This will cause issue if glue those EPContext nodes together into a single model.
To avoid this user can set this context_node_name_prefix for each split pieces to make the node name unique.

@HectorSVC HectorSVC added the ep:QNN issues related to QNN exeution provider label Jul 3, 2024
@jywu-msft
Copy link
Member

one still has to glue separate context nodes together in one graph/model right?
Would another option be to provide that "glue" tool and have the tool update the node names there to ensure uniqueness?
it seems a bit clunky to do it in QNN EP context model generation.

@HectorSVC
Copy link
Contributor Author

one still has to glue separate context nodes together in one graph/model right? Would another option be to provide that "glue" tool and have the tool update the node names there to ensure uniqueness? it seems a bit clunky to do it in QNN EP context model generation.

It's hard to have a common script to do the glue work. It's so hard to know what the user want to do. I can imagine there's could be many trick work.

@jywu-msft
Copy link
Member

one still has to glue separate context nodes together in one graph/model right? Would another option be to provide that "glue" tool and have the tool update the node names there to ensure uniqueness? it seems a bit clunky to do it in QNN EP context model generation.

It's hard to have a common script to do the glue work. It's so hard to know what the user want to do. I can imagine there's could be many trick work.

sync'd with @HectorSVC offline. we agreed that it's probably better to hold off on this PR until the workflow for splitting/gluing back together the model has been finalized.

jywu-msft
jywu-msft previously approved these changes Jul 25, 2024
@HectorSVC HectorSVC merged commit fb61e14 into main Jul 26, 2024
97 of 99 checks passed
@HectorSVC HectorSVC deleted the qnn_ctx_node_prefix branch July 26, 2024 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:QNN issues related to QNN exeution provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants