Skip to content

Latest commit

 

History

History
32 lines (27 loc) · 1.38 KB

to-parquet-pr.md

File metadata and controls

32 lines (27 loc) · 1.38 KB
Raw

to-parquet-pr

  • domain(s): pairs
  • accepts: ldc.api.supervised.pairs.PairData

Writes prompt/output pairs in Parquet database format.

usage: to-parquet-pr [-h] [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
                     [-N LOGGER_NAME] -o OUTPUT [--col_instruction COL]
                     [--col_input COL] [--col_output COL] [--col_id COL]

Writes prompt/output pairs in Parquet database format.

optional arguments:
  -h, --help            show this help message and exit
  -l {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --logging_level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        The logging level to use. (default: WARN)
  -N LOGGER_NAME, --logger_name LOGGER_NAME
                        The custom name to use for the logger, uses the plugin
                        name by default (default: None)
  -o OUTPUT, --output OUTPUT
                        Path of the CSV file to write (directory when
                        processing multiple files) (default: None)
  --col_instruction COL
                        The name of the column for the instructions (default:
                        None)
  --col_input COL       The name of the column for the inputs (default: None)
  --col_output COL      The name of the column for the outputs (default: None)
  --col_id COL          The name of the column for the row IDs (uses 'id' from
                        meta-data) (default: None)