feat: Add Discord OAuth Flow #1129

AveryYay · 2024-10-29T06:00:42Z

Description

This PR introduces discord OAuth flow: Implements a Discord OAuth flow which allows users to authenticate with Discord, managing the entire OAuth process from generating the authorization URL, handling the authorization code exchange, to retrieving basic user information and an installation store to manage multi server installation.

Motivation and Context

Enables secure user authentication via Discord OAuth, enhancing the bot’s integration with Discord’s authentication system. And supports multi server management.

I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)
Example (update in the folder of example)

Implemented Tasks

Implemented OAuth flow for user authentication
Created a custom Discord bot with thread creation and history response capabilities
Added queuing for efficient message processing in the bot

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.

…threads and responding with the chat history features

koch3092 · 2024-10-29T07:24:33Z

Thanks @AveryYay , I will review the code and assist you in completing the PR integration

koch3092

Thanks @AveryYay , good code! left some comments below.

And in terms of functionality, I hope you refer to camel.bots.slack.slack_app.py and integrate the oauth process into camel.bots.discord.discord_app.py (you can create camel.bots.discord to store module content)

examples/bots/discord_OAuth_flow.py

Co-authored-by: koch3092 <108793340+koch3092@users.noreply.github.com>

koch3092

thanks @AveryYay , left some comments below.

koch3092 · 2024-11-04T13:55:39Z

camel/bots/discord/discord_app.py

+
+        user_data = await self.oauth_client.get_user_info(access_token)
+        logger.info(f"User data: {user_data}")
+        await self.shutdown_oauth_server()


Suggested change

await self.shutdown_oauth_server()

Server is a long-term service, so it's better to let users manually shut it down

koch3092 · 2024-11-04T14:06:48Z

examples/bots/discord_bot_public_thread_w_history.py

Example can be combined with the usage of chat_agent to integrate existing historical records into the memory of chat_agent.

Wendong-Fan

Thanks @AveryYay ! Overall LGTM, left some comments below, It would be better if we could also add docstring for DiscordOAuth
BTW, you can run pre-commit run --all-files before the push to ensure the code pass pre-commit check

Wendong-Fan · 2024-11-06T20:09:33Z

camel/bots/discord/discord_app.py

+        token: Optional[str] = None,
+        client_id: Optional[str] = None,
+        client_secret: Optional[str] = None,
+        redirect_uri: str = "http://localhost:8000/callback",


use REDIRECT_URI instead?

Wendong-Fan · 2024-11-06T20:22:52Z

camel/bots/discord/discord_app.py

+            response = await client.post(TOKEN_URL, data=data, headers=headers)
+            response_data = response.json()


before fetch the data, check the status code first like below?

if response.status_code != 200: logger.error("Failed to exchange code for token) return None

Wendong-Fan · 2024-11-06T20:26:44Z

camel/bots/discord/discord_app.py

+            user_response = await client.get(USER_URL, headers=headers)
+            return user_response.json()


same as above

Wendong-Fan · 2024-11-06T20:33:21Z

camel/bots/discord/discord_app.py

+            self.server.should_exit = True
+            logger.info("Shutting down the OAuth server.")


should we add await here to make sure the server is fully stopped?

Wendong-Fan

Thanks @AveryYay , left some comments below

Wendong-Fan · 2024-11-24T10:32:29Z

camel/bots/discord/discord_app.py

@@ -53,6 +62,9 @@ def __init__(
            token (Optional[str]): The Discord bot token for authentication.
                If not provided, the token will be retrieved from the
                environment variable `DISCORD_TOKEN`.
+            client_id (Optional[str]): The client ID for Discord OAuth.


same as other variables

Suggested change

client_id (Optional[str]): The client ID for Discord OAuth.

client_id (Optional[str]): The client ID for Discord OAuth. (default: :obj:`None`)

Suggested change

client_id (Optional[str]): The client ID for Discord OAuth.

client_id (str, optional): The client ID for Discord OAuth. (default: :obj:`None`)

Wendong-Fan · 2024-11-24T10:33:17Z

camel/bots/discord/discord_app.py

+        if not all([client_id, client_secret]):
+            raise ValueError(
+                "DISCORD_CLIENT_ID and DISCORD_CLIENT_SECRET must be set "
+                "for OAuth."
+            )


from camel.utils import api_keys_required use @api_keys_required

These aren't actually api keys, do I still use @api_keys_required?

Wendong-Fan · 2024-11-24T10:34:12Z

camel/bots/discord/discord_app.py

+        r"""Generate an OAuth URL to invite the bot to a Discord server.
+
+        This method returns a link that allows users to invite the bot to
+        their Discord server with specific permissions.
+        """


including args and returns in docstring

- Added DiscordAsyncInstallationStore to manage installations (using list right now, need to switch to database or file instead) - Added DiscordInstallation entity class - Implemented relevant functions in DiscordApp

…rdInstallation abd DiscordAsyncInstallationStore to __init__.py

liuxukun2000 · 2025-01-03T06:06:09Z

Hi @Wendong-Fan and @AveryYay, the current implementation of DiscordBaseInstallationStore looks great! For future scalability, though, I’d like to suggest considering the use of an ORM as an alternative. For example, you can check out Tortoise ORM. It simplifies the code and provides the flexibility to support nearly any type of database without requiring code modifications.

What are your thoughts on this approach? 😊

Asher-hss · 2025-01-03T08:06:43Z

你好@Wendong-Fan和@AveryYay，的当前实现DiscordBaseInstallationStore看起来很棒！不过，为了将来的可扩展性，我建议考虑使用 ORM 作为替代方案。例如，您可以查看Tortoise ORM。它简化了代码，并提供了灵活性，无需修改代码即可支持几乎任何类型的数据库。

你对这种方法有什么看法？😊

@koch3092

koch3092

Thanks @AveryYay , left some comments below

koch3092 · 2025-01-03T14:51:06Z

camel/bots/discord/discord_app.py

+                " here: `https://discord.com/developers/applications`."
+            )
+
+        intents = discord.Intents.default()


Let's add an intents parameter so that users can set their own bot permissions

koch3092 · 2025-01-03T14:51:39Z

camel/bots/discord/discord_app.py

+        r"""Retrieve user information using the access token.
+        Args:
+            access_token (str): The access token received from Discord.
+        Returns:
+            dict: The user information retrieved from Discord.
+        """


Suggested change

r"""Retrieve user information using the access token.

Args:

access_token (str): The access token received from Discord.

Returns:

dict: The user information retrieved from Discord.

"""

r"""Retrieve user information using the access token.

Args:

access_token (str): The access token received from Discord.

Returns:

dict: The user information retrieved from Discord.

"""

koch3092 · 2025-01-03T14:52:23Z

camel/bots/discord/discord_app.py

+
+        Returns:
+            None


Suggested change

Returns:

None

koch3092 · 2025-01-03T14:52:50Z

camel/bots/discord/discord_app.py

+
+        Returns:
+            None


Suggested change

Returns:

None

koch3092 · 2025-01-03T15:06:06Z

Hi @Wendong-Fan and @AveryYay, the current implementation of DiscordBaseInstallationStore looks great! For future scalability, though, I’d like to suggest considering the use of an ORM as an alternative. For example, you can check out Tortoise ORM. It simplifies the code and provides the flexibility to support nearly any type of database without requiring code modifications.

What are your thoughts on this approach? 😊

We are currently using SQLite as a database for lightweight storage. If ORM is introduced, it will make the entire functionality too cumbersome. If we specifically implement certain businesses, we can use ORM, but just doing OAuth and SQL would be more efficient. I have discussed this issue with @AveryYay . Avery's current implementation is based on the Slack Bot solution, which also provides the SQLite solution.

# Conflicts: # poetry.lock

Wendong-Fan

Thanks @AveryYay , left some comments below

Wendong-Fan · 2024-11-30T19:53:09Z

camel/bots/__init__.py

+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#


please check and clean the license info

Wendong-Fan · 2024-12-28T16:24:53Z

camel/bots/discord/__init__.py

+# =========== Copyright 2023 @ CAMEL-AI.org. All Rights Reserved. ===========
+# Licensed under the Apache License, Version 2.0 (the “License”);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0


Wendong-Fan · 2025-01-07T00:18:12Z

camel/bots/discord/discord_installation.py

+# =========== Copyright 2023 @ CAMEL-AI.org. All Rights Reserved. ===========
+# Licensed under the Apache License, Version 2.0 (the “License”);
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an “AS IS” BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# =========== Copyright 2023 @ CAMEL-AI.org. All Rights Reserved. ===========


Wendong-Fan · 2025-01-07T00:19:34Z

camel/bots/discord_app.py

this file need to be deleted

Wendong-Fan · 2025-01-07T00:20:34Z

camel/bots/discord/discord_installation.py

+class DiscordInstallation:
+    def __init__(
+        self,
+        guild_id: str,
+        access_token: str,
+        refresh_token: str,
+        installed_at: datetime,
+        token_expires_at: Optional[datetime] = None,
+    ):
+        self.guild_id = guild_id
+        self.access_token = access_token
+        self.refresh_token = refresh_token
+        self.installed_at = installed_at
+        self.token_expires_at = token_expires_at


add docstring

Wendong-Fan · 2025-01-07T00:27:30Z

camel/bots/discord/discord_app.py

+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)


use our own logger

from camel.logger import get_logger logger = get_logger(__name__)

koch3092

Thanks @AveryYay LGTM

Wendong-Fan

please resolve the conflict with master branch

# Conflicts: # poetry.lock # pyproject.toml

liuxukun2000

LGTM, but there are a few small points that could be improved.

liuxukun2000 · 2025-01-09T00:23:44Z

camel/bots/discord/discord_app.py

+            return None
+        headers = {"Authorization": f"Bearer {access_token}"}
+        async with httpx.AsyncClient() as client:
+            user_response = await client.get(USER_URL, headers=headers)


Error handling should be implemented here.

liuxukun2000 · 2025-01-09T00:24:24Z

camel/bots/discord/discord_app.py

+                "client_secret, or redirect_uri."
+            )
+            return None
+        assert self.installation_store is not None


It would be better to raise an error here.

I had a check before this that already checks self.installation_store is not None, this assertion here is just for it to pass pre commit.

liuxukun2000 · 2025-01-09T00:25:12Z

camel/bots/discord/discord_app.py

+                "client_secret, or redirect_uri."
+            )
+            return None
+        assert self.installation_store is not None


It would be better to raise an error here.

liuxukun2000 · 2025-01-09T00:25:52Z

camel/bots/discord/discord_app.py

+                "client_secret, or redirect_uri."
+            )
+            return None
+        assert self.installation_store is not None


It would be better to raise an error here.

Wendong-Fan · 2025-01-10T08:02:00Z

hey @AveryYay @liuxukun2000 @koch3092 , I think there's some error in cicd was not fixed but this PR was approved and merged. current error message in master branch:

=========================== short test summary info ============================
FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_init_with_token_from_env - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)
FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_run_bot - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)
============ 2 failed, 39 passed, 12 warnings in 166.87s (0:02:46) =============

I will do a hot fix, please pay more attention to the cicd check next time~

AveryYay · 2025-01-10T09:00:18Z

hey @AveryYay @liuxukun2000 @koch3092 , I think there's some error in cicd was not fixed but this PR was approved and merged. current error message in master branch:


=========================== short test summary info ============================

FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_init_with_token_from_env - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)

FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_run_bot - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)

============ 2 failed, 39 passed, 12 warnings in 166.87s (0:02:46) =============

I will do a hot fix, please pay more attention to the cicd check next time~

Sorry about this and thank you for the fix! I thought it was one of the certification issue I encountered since I forked the repo😭

Wendong-Fan · 2025-01-11T20:38:28Z

hey @AveryYay @liuxukun2000 @koch3092 , I think there's some error in cicd was not fixed but this PR was approved and merged. current error message in master branch:
=========================== short test summary info ============================

FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_init_with_token_from_env - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)

FAILED examples/test/bots/test_discord_app.py::TestDiscordApp::test_run_bot - ValueError: `DISCORD_TOKEN` not defined. Get it here: `[https://discord.com/developers/applications`.](https://discord.com/developers/applications%60.)

============ 2 failed, 39 passed, 12 warnings in 166.87s (0:02:46) =============
I will do a hot fix, please pay more attention to the cicd check next time~
Sorry about this and thank you for the fix! I thought it was one of the certification issue I encountered since I forked the repo😭

No worries~

Implemented Discord OAuth Flow && a Discord bot with creating public …

e16bac3

…threads and responding with the chat history features

koch3092 self-requested a review October 29, 2024 07:23

koch3092 assigned AveryYay Oct 29, 2024

koch3092 reviewed Oct 29, 2024

View reviewed changes

examples/bots/discord_OAuth_flow.py Outdated Show resolved Hide resolved

examples/bots/discord_OAuth_flow.py Outdated Show resolved Hide resolved

AveryYay and others added 5 commits October 29, 2024 11:14

Apply suggestions from code review

2307a7d

Co-authored-by: koch3092 <108793340+koch3092@users.noreply.github.com>

Integrated OAuth flow into discord_app.py

c7f04d1

Added init for camel.bots.discord

0938fb1

Delete examples/bots/discord_OAuth_flow.py

548c52e

Deleted unnecessary import in camel.bots.discord.__init__.py

6db7ba0

koch3092 reviewed Nov 4, 2024

View reviewed changes

Wendong-Fan changed the title ~~Add Discord OAuth Flow and Bot Public Thread with History Response Feature~~ feat: Add Discord OAuth Flow and Bot Public Thread with History Response Feature Nov 6, 2024

Wendong-Fan reviewed Nov 6, 2024

View reviewed changes

AveryYay and others added 7 commits November 23, 2024 19:54

Updated OAuth in DiscordApp

5350d8a

Merge branch 'camel-ai:master' into master

87f255f

Style fix && deleted the example discord bot

62a2f51

minor fix

ba84a25

Fixed oauth redirect logic

67c55ed

Changed permission data type

23da0ba

Merge branch 'master' into master

940188f

Wendong-Fan reviewed Nov 24, 2024

View reviewed changes

AveryYay added 2 commits November 28, 2024 05:03

Preliminary implementation of Discord OAuth flow

b414ec6

- Added DiscordAsyncInstallationStore to manage installations (using list right now, need to switch to database or file instead) - Added DiscordInstallation entity class - Implemented relevant functions in DiscordApp

Merge remote-tracking branch 'origin/master'

800ede5

AveryYay marked this pull request as draft November 28, 2024 21:18

AveryYay added 2 commits November 29, 2024 02:53

Changed the installation storage handling to aiosqlite && Added Disco…

4ca2ee8

…rdInstallation abd DiscordAsyncInstallationStore to __init__.py

pre-commit fixes

6ff54a6

AveryYay marked this pull request as ready for review November 29, 2024 08:08

AveryYay added 3 commits November 29, 2024 03:26

Resolved conflicts and merge

9354dee

Merge branch 'camel-ai-master' (resolved conflicts)

a6b9aae

Pre-commit: license updated

04e5d71

AveryYay and others added 3 commits December 29, 2024 10:33

Added aiosqlite to poetry lock

c53fe0c

Merge branch 'master' into master

61280bd

modified poetry lock

8b7bd0e

Wendong-Fan marked this pull request as ready for review December 29, 2024 16:05

Wendong-Fan requested review from koch3092 and liuxukun2000 January 2, 2025 07:57

koch3092 reviewed Jan 3, 2025

View reviewed changes

Added discord intents as a parameter && minor style change

7160208

AveryYay added 2 commits January 6, 2025 00:47

Merge branch 'tmp'

250cbc9

# Conflicts: # poetry.lock

Updated poetry

e3bbece

Wendong-Fan reviewed Jan 7, 2025

View reviewed changes

AveryYay added 4 commits January 7, 2025 13:07

Cleaned license && deleted unused discord_app.py

4047229

Added docstring for DiscordInstallation

2fc8364

pre-commit

e5be892

Switched logger && pre commit

dd166b7

koch3092 self-requested a review January 8, 2025 07:09

koch3092 approved these changes Jan 8, 2025

View reviewed changes

Wendong-Fan reviewed Jan 8, 2025

View reviewed changes

AveryYay added 2 commits January 8, 2025 13:31

Merge branch 'tmp'

28b9096

# Conflicts: # poetry.lock # pyproject.toml

Updated poetry lock

6f720c5

liuxukun2000 reviewed Jan 9, 2025

View reviewed changes

Added error handling in exchange_code_for_token_response

db686de

liuxukun2000 approved these changes Jan 10, 2025

View reviewed changes

AveryYay merged commit a4820b7 into camel-ai:master Jan 10, 2025
1 of 6 checks passed

		response = await client.post(TOKEN_URL, data=data, headers=headers)
		response_data = response.json()

		user_response = await client.get(USER_URL, headers=headers)
		return user_response.json()

		self.server.should_exit = True
		logger.info("Shutting down the OAuth server.")

	client_id (Optional[str]): The client ID for Discord OAuth.
	client_id (Optional[str]): The client ID for Discord OAuth. (default: :obj:`None`)

	client_id (Optional[str]): The client ID for Discord OAuth.
	client_id (str, optional): The client ID for Discord OAuth. (default: :obj:`None`)

		logging.basicConfig(level=logging.INFO)
		logger = logging.getLogger(__name__)

feat: Add Discord OAuth Flow #1129

feat: Add Discord OAuth Flow #1129

Conversation

AveryYay commented Oct 29, 2024 • edited Loading

Description

Motivation and Context

Types of changes

Implemented Tasks

Checklist

koch3092 commented Oct 29, 2024

koch3092 left a comment

Choose a reason for hiding this comment

koch3092 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wendong-Fan left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuxukun2000 commented Jan 3, 2025

Asher-hss commented Jan 3, 2025

koch3092 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koch3092 commented Jan 3, 2025

Wendong-Fan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

koch3092 left a comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

liuxukun2000 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Wendong-Fan commented Jan 10, 2025

AveryYay commented Jan 10, 2025

Wendong-Fan commented Jan 11, 2025

AveryYay commented Oct 29, 2024 •

edited

Loading

Wendong-Fan left a comment •

edited

Loading