feat: python interpreter #182

dandansamax · 2023-06-24T12:49:38Z

Description

Extract utils.py as a individual module camel.utils
Add PythonInterpreter class in camel.utils, an abstraction controlable python code executor. It contains several features than eval() like variable accessing control, module import, and fuzzy variable matching. It is also extendable and flexiable.
Change code executor of EmbodiedAgent from built-in function eval() to PythonInterpreter

Motivation and Context

#150

I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)
Example (update in the folder of example)

Implemented Tasks

Subtask 1
Subtask 2
Subtask 3

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.

…erpreter

dandansamax · 2023-06-24T18:20:38Z

The example in hugging_face_tool.py has been run successfully.

lightaime · 2023-06-24T20:08:10Z

Thanks, @dandansamax. Looks amazing. I will look into it soon!

Obs01ete

Overall looks cool. Please let us know if it is an original code or it was borrowed from somewhere.

camel/prompts/base.py

camel/utils/__init__.py

camel/utils/python_interpreter.py

Obs01ete · 2023-06-28T08:31:48Z

camel/utils/python_interpreter.py

+        self.state = self.action_space.copy()
+        self.fuzz_state = {}
+
+    def _execute_ast(self, expression: ast.AST) -> Any:


Is this whole file taken from somewhere?

The logic of this class was borrowed from hugging face agent. See python_interpreter.py. However, most of the codes are rewritten to fit our need.

@lightaime I think we should refer the original file since it is quite similar

Sorry, I'm not very familiar with these licenses and copyright things. As Apache 2.0 request, I should contain original license from HuggingFace team and add our modifications copyright. However, the license checker didn't allow me to do that. Also, I don't know if it is acceptable in our project.

Another solution may be to totally modify its implementation. What do you think?

Sure, we should add the link to the original hugging face implementation and their license to the docstring of PythonInterpreter, and mention that it is adapted from the hugging face implementation. We should also mention what is added and rewritten.

Thank you for your advisions. I have updated the docstring.

camel/agents/embodied_agent.py

lightaime

Awesome! Liked it. Left some comments. As we mention, we may also support other actions like file IO. But it is not in the scope of this PR. But we may support unsafe running mode or safe running mode with docker in the future. Thanks!

camel/utils/__init__.py

lightaime · 2023-07-02T19:45:36Z

camel/utils/python_interpreter.py

+        self.state = self.action_space.copy()
+        self.fuzz_state = {}
+
+    def _execute_ast(self, expression: ast.AST) -> Any:


Sure, we should add the link to the original hugging face implementation and their license to the docstring of PythonInterpreter, and mention that it is adapted from the hugging face implementation. We should also mention what is added and rewritten.

camel/utils/python_interpreter.py

lightaime · 2023-07-02T19:53:28Z

camel/utils/python_interpreter.py

+        Returns:
+            Any: An evaluation value of the last line in the code.
+        """
+        if state:


Suggested change

if state:

if state is not None:

test/utils/test_python_interpreter.py

camel/prompts/base.py

camel/utils/python_interpreter.py

test/utils/test_python_interpreter.py

Co-authored-by: Guohao Li <lightaime@gmail.com>

Obs01ete

Please resolve the remaining comment about the reference to the original code.

dandansamax · 2023-07-03T18:07:39Z

Please resolve the remaining comment about the reference to the original code.

Updated. Could you please review it?

dandansamax added 4 commits June 23, 2023 16:53

[WIP] Add python interpreter

fa41ca3

Finish python interpreter

0d15cc2

Change python code executor of EmbodiedAgent from eval() to PythonInt…

91374bd

…erpreter

Fix testcases and change exeception message

85d1984

dandansamax requested review from lightaime and Obs01ete June 24, 2023 17:39

Remove useless function

cfafc8a

lightaime requested a review from oserikov June 28, 2023 01:43

Obs01ete reviewed Jun 28, 2023

View reviewed changes

Fix empty list and dict as function default args

28a5af9

lightaime assigned dandansamax Jun 30, 2023

lightaime added the Prompt Related to camel prompts label Jun 30, 2023

Update document

dbbaed4

lightaime reviewed Jul 2, 2023

View reviewed changes

dandansamax and others added 9 commits July 3, 2023 16:21

Update test/prompts/test_prompt_base.py

53df74b

Co-authored-by: Guohao Li <lightaime@gmail.com>

Update camel/utils/__init__.py

4840ecb

Co-authored-by: Guohao Li <lightaime@gmail.com>

Update camel/utils/python_interpreter.py

88d4195

Co-authored-by: Guohao Li <lightaime@gmail.com>

Move functions to functions.py

d2c9a6b

Remove misuploaded image

2fe9687

Update "Returns" doc of function execution

4bd8a41

Update

f5d52ae

Merge branch 'master' into python_interpreter

2f398cf

Fix utils

4f33fc4

Benjamin-eecs changed the title ~~Python interpreter~~ feat: Python interpreter Jul 3, 2023

Benjamin-eecs changed the title ~~feat: Python interpreter~~ feat: python interpreter Jul 3, 2023

dandansamax added 3 commits July 4, 2023 00:57

update

1c392ef

Add keep state test for the import

d75af5a

Fix

995790f

Obs01ete reviewed Jul 3, 2023

View reviewed changes

Add original license and modifications in document

dc34cdd

Merge branch 'master' into python_interpreter

6f7b43c

lightaime approved these changes Jul 3, 2023

View reviewed changes

dandansamax merged commit abff2ea into master Jul 4, 2023

dandansamax deleted the python_interpreter branch July 4, 2023 02:20

This was referenced Jul 4, 2023

[Feature Request] Control code excution in CodePrompt.execute #150

Closed

fix: fix mypy errors in PythonInterpreter #200

Merged

lightaime mentioned this pull request Jul 29, 2023

[Roadmap] Add support for tools and APIs #57

Open

39 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: python interpreter #182

feat: python interpreter #182

dandansamax commented Jun 24, 2023

dandansamax commented Jun 24, 2023

lightaime commented Jun 24, 2023

Obs01ete left a comment

Obs01ete Jun 28, 2023

dandansamax Jun 28, 2023

Obs01ete Jun 28, 2023

dandansamax Jul 1, 2023

lightaime Jul 2, 2023

dandansamax Jul 3, 2023

lightaime left a comment

lightaime Jul 2, 2023

lightaime Jul 2, 2023

Obs01ete left a comment

dandansamax commented Jul 3, 2023

feat: python interpreter #182

feat: python interpreter #182

Conversation

dandansamax commented Jun 24, 2023

Description

Motivation and Context

Types of changes

Implemented Tasks

Checklist

dandansamax commented Jun 24, 2023

lightaime commented Jun 24, 2023

Obs01ete left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lightaime left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Obs01ete left a comment

Choose a reason for hiding this comment

dandansamax commented Jul 3, 2023