docs: Reward model cookbook

[Asher-hss]

Description

Add a cookbook about Slackbot and the reward model.

Motivation and Context

Add a cookbook about Slackbot and the reward model.

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of example)

Implemented Tasks

• Subtask 1
• Subtask 2
• Subtask 3

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide. (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly. (required for a bug fix or a new feature)
• I have updated the documentation accordingly.

[Asher-hss]

There are some usage issues with the Slackbot section that require further discussion with other team members.

[Wendong-Fan]

Thanks @Asher-hss , could you add more detail about how will you implement reward model with slack bot?

[Wendong-Fan]

thanks @Asher-hss ! here is our cookbook guideline FYR https://github.com/camel-ai/camel/blob/master/CONTRIBUTING.md#contributing-to-the-cookbook-writing-

for the reward model cookbook, i think we can make a more systematic cookbook to showcase generation RLHF data as described in issue:#1216, if just to showcase the usage of reward model, the volumn of generated data could be reduced, and add more content for reward model filtering part, seems the current cookbook only has 20-30% content for reward model part

[Asher-hss]

thanks @Asher-hss ! here is our cookbook guideline FYR https://github.com/camel-ai/camel/blob/master/CONTRIBUTING.md#contributing-to-the-cookbook-writing-

for the reward model cookbook, i think we can make a more systematic cookbook to showcase generation RLHF data as described in issue:#1216, if just to showcase the usage of reward model, the volumn of generated data could be reduced, and add more content for reward model filtering part, seems the current cookbook only has 20-30% content for reward model part

Thanks @Wendong-Fan，I think the current cookbook can focus solely on demonstrating how to use the reward model, and an additional cookbook can be created to showcase the RLHF training process.

[Wendong-Fan]

1. please check out template, add url link to colab, star repo part etc.
2. conclusion part is not added
3. thresholds = {"helpfulness": 2.5, "correctness": 2.5} add description why we set thresholds like this, show and explain the filtered data, how well the reward model helped us do data filter

[Asher-hss]

https://colab.research.google.com/drive/15Y4iDw_yeskG7ZXzqy6pnJDVtPjl3RVA

[Wendong-Fan]

Thanks @Asher-hss , left some comments in the colab notebook, please refer to https://github.com/camel-ai/camel/blob/master/CONTRIBUTING.md#contributing-to-the-cookbook-writing- ,update docs/index.rst and docs/README.md

[koch3092]

Thanks @Asher-hss , left some comments

[koch3092]

Shall we update camel version to 0.2.14?

[Asher-hss]

Sure

[Asher-hss]

Thanks @koch3092，The latest paper mentioned in the previous Colab link has not yet been uploaded to the PR.

[koch3092]

clean up unused code

[koch3092]

Suggested change

	" alpaca_items = [n_item.item for n_item in\n",
	" AlpacaItemResponse.\n",
	" model_validate_json(response.msgs[0].content).items]\n",
	" n_items = AlpacaItemResponse.model_validate_json(response.msgs[0].content).items\n"
	" alpaca_items =[n_item.item for n_item in n_items]\n",

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB