Added Cpp Inferencing API to generate the text #156
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
asmigosw
force-pushed
the
cpp_inference
branch
2 times, most recently
from
eb7df46
to
454fa96
Compare
|
can you move the code files from examples/xyz.cpp to examples/cpp_execution/xyz.cpp? |
asmigosw
force-pushed
the
cpp_inference
branch
from
454fa96
to
7b209af
Compare
|
Can you add a readme.md to cpp_execution directory? |
asmigosw
force-pushed
the
cpp_inference
branch
3 times, most recently
from
95c053c
to
99158e7
Compare
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
asmigosw
force-pushed
the
cpp_inference
branch
from
99158e7
to
15bb5e1
Compare
quic-mamta
requested changes
Oct 16, 2024
quic-rishinr
requested changes
Oct 16, 2024
quic-mamta
reviewed
Oct 16, 2024
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
quic-mamta
requested changes
Oct 17, 2024
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
ochougul
requested changes
Oct 17, 2024
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
ochougul
requested changes
Nov 11, 2024
examples/cpp_execution/README.md
Outdated
| make -j 8 | ||
|
|
||
| # Run the python script to get the generated text | ||
| cd ../../../ |
There was a problem hiding this comment.
Is this required? Can we not run the file from any other path?
examples/cpp_execution/README.md
Outdated
|
|
||
| ## Prerequisite | ||
| 1. PyBind11 | ||
| 2. Cpp17 or above |
There was a problem hiding this comment.
is this clear enough, it's better to include what gcc version you have tested on. You can add a line saying the below readme file is validated with cpp version ** and gcc version **.
examples/cpp_execution/README.md
Outdated
| This example demonstrates how to execute a model on AI 100 using Efficient Transformers and C++ APIs. The Efficient Transformers library is utilized for transforming and compiling the model, while the QPC is executed using C++ APIs. It is tested on both x86 and ARM platform. | ||
|
|
||
| ## Prerequisite | ||
| 1. PyBind11 |
There was a problem hiding this comment.
instead add pip3 install pybind11
| try: | ||
| import InferenceSetIOBuffer # noqa: E402 | ||
| except ImportError: | ||
| logger.info("Error importing InferenceSetIOBuffer Module") |
There was a problem hiding this comment.
raise error here. We should not continue execution when we are not able to import the SO file
| except ImportError: | ||
| logger.info("Error importing InferenceSetIOBuffer Module") | ||
| else: | ||
| logger.info("so file's folder not found") |
There was a problem hiding this comment.
raise error saying FileNotFoundError("Please follow README instructions to first compile the cpp files")
| aic_enable_depth_first: bool = False, | ||
| mos: int = -1, | ||
| batch_size: int = 1, | ||
| full_batch_size: Optional[int] = None, |
Comment on lines
+141
to
+143
| enable_debug_logs: bool = False, | ||
| stream: bool = True, | ||
| full_batch_size: Optional[int] = None, |
asmigosw
force-pushed
the
cpp_inference
branch
from
340cab0
to
b26c3d9
Compare
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com>
quic-rishinr
approved these changes
Nov 12, 2024
quic-mamta
approved these changes
Nov 12, 2024
Signed-off-by: Onkar Chougule <quic_ochougul@quicinc.com>
ochougul
approved these changes
Nov 12, 2024
quic-akuruvil
pushed a commit
to quic-akuruvil/efficient_transformers
that referenced
this pull request
Nov 12, 2024
* Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * Added Cpp Inferencing API to generate the text Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> * fixed rasing errors and README Signed-off-by: Onkar Chougule <quic_ochougul@quicinc.com> --------- Signed-off-by: Asmita Goswami <quic_asmigosw@quicinc.com> Signed-off-by: Onkar Chougule <quic_ochougul@quicinc.com> Co-authored-by: Onkar Chougule <quic_ochougul@quicinc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This example demonstrates how to execute a model on AI 100 using Efficient Transformers and C++ APIs. The Efficient Transformers library is utilized for transforming and compiling the model, while the QPC is executed using C++ APIs.