Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata correction for 2024.teachingnlp-1.12 #4164

Closed
mjpost opened this issue Dec 18, 2024 · 2 comments · Fixed by #4289
Closed

Metadata correction for 2024.teachingnlp-1.12 #4164

mjpost opened this issue Dec 18, 2024 · 2 comments · Fixed by #4289
Assignees
Labels
approved Used to note team approval of metadata requests correction for corrections submitted to the anthology metadata Correction to metadata

Comments

@mjpost
Copy link
Member

mjpost commented Dec 18, 2024

{
  "anthology_id": "2024.teachingnlp-1.12",
  "title": "A Prompting Assignment for Exploring Pretrained <fixed-case>LLM</fixed-case>s",
  "authors": [
    {
      "first": "Carolyn Jane",
      "last": "Anderson",
      "id": "carolyn-anderson",
      "affiliation": "Wellesley College"
    }
  ],
  "abstract": "As the scale of publicly-available large language models (LLMs) has increased, so has interest in few-shot prompting methods. This paper presents an assignment that asks students to explore three aspects of large language model capabilities (commonsense reasoning, factuality, and wordplay) with a prompt engineering focus. The assignment consists of three tasks designed to share a common programming framework, so that students can reuse and adapt code from earlier tasks. Two of the tasks also involve dataset construction: students are asked to construct a simple dataset for the wordplay task, and a more challenging dataset for the factuality task. In addition, the assignment includes reflection questions that ask students to think critically about what they observe."
}
@mjpost mjpost added correction for corrections submitted to the anthology metadata Correction to metadata labels Dec 18, 2024
Copy link

LLM Validation Failed

Details:

  • The proposed title 'A Prompting Assignment for Exploring Pretrained LLMs' does not match the title in the PDF, which is 'A Prompting Assignment for Exploring Pretrained Language Models'. The words 'Language Models' are missing in the proposed title.
  • The proposed list of authors only includes 'Carolyn Jane Anderson', while the PDF lists multiple authors. Therefore, the proposed list does not contain all authors.
  • The order of authors is also incorrect, as the proposed list does not reflect the complete list of authors as presented in the PDF.

@mjpost
Copy link
Member Author

mjpost commented Dec 20, 2024

This LLM validation is not remotely correct! Everything about it is wrong.

mjpost added a commit that referenced this issue Dec 26, 2024
mjpost added a commit that referenced this issue Dec 26, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
@mjpost mjpost added the approved Used to note team approval of metadata requests label Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 27, 2024
mjpost added a commit that referenced this issue Dec 28, 2024
mjpost added a commit that referenced this issue Dec 28, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Dec 29, 2024
mjpost added a commit that referenced this issue Jan 1, 2025
mjpost added a commit that referenced this issue Jan 1, 2025
mjpost added a commit that referenced this issue Jan 1, 2025
mjpost added a commit that referenced this issue Jan 1, 2025
mjpost added a commit that referenced this issue Jan 1, 2025
@mjpost mjpost closed this as completed in 1579589 Jan 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment