This repository tests the idea of a proxy evaluation method for text to BPMN model pipeline. The proxy evaluation involves a round-trip pipeline, "text to bpmn to text" and calculating an average text to text similarity in the absence of a ground truth BPMN. To show if the proxy method is effective, we first must investigate how the existing BPMN to BPMN evaluation from model_evaluation module correlates with the proxy text to text method. This work is inspired by this publication on text to code round-tripping.
The requirements are in this pyproject.toml file. After cloning the repository, run:
poetry installTo run the pipeline, use a command similar to this:
screen -d -m python genai_gpt_pipeline.py --model-path ./data/pet/ground_json --text-path ./data/pet/process_descriptions --example pet --direction t2t The csv files are written to the results directory. The jupyter notebooks are used to visualize the results.
No known issue.
Create an issue in this repository if you find a bug or have questions about the content.
If you wish to contribute code, offer fixes or improvements, please send a pull request. Due to legal reasons, contributors will be asked to accept a DCO when they create the first pull request to this project. This happens in an automated fashion during the submission process. SAP uses the standard DCO text of the Linux Foundation.
Copyright (c) 2024 SAP SE or an SAP affiliate company. All rights reserved. This project is licensed under the Apache Software License, version 2.0.