Skip to content

RyanZong2233/Struc-Bench-gersteinlab

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Struc-Bench

[📄 Paper]

Welcome to Struc-Bench! We employ the same finetune.py and generate.py as used in alpaca-lora. Moreover, we have divided generate.py into three distinct categories: table, html, and latex. The scoring results can be found in the score directory, with scores derived from GPT housed in the gpt_score subdirectory. The remaining evaluations are segregated into table_scores.py, html_scores.py, and latex_scores.py, respectively.

How to use generate.py

You can use the command:

python generate.py read_json_path output_file_path

For example:

python generate.py "table_test.json" "table_test_output.txt"

Our data

You can access our data in the google drive: data

Citation

If you find our work useful in your research, please kindly consider cite:

@article{tang2023struc,
  title={Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?},
  author={Xiangru Tang and Yiming Zong and Jason Phang and Yilun Zhao and Wangchunshu Zhou and Arman Cohan and Mark B. Gerstein},
  journal={arXiv preprint arXiv:2309.08963},
  year={2023}
}

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 67.8%
  • Perl 32.1%
  • Shell 0.1%