diff --git a/.gitignore b/.gitignore index f984034..043d5d2 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ __pycache__ .vscode -*.bak \ No newline at end of file +*.bak +.idea \ No newline at end of file diff --git a/README.md b/README.md index 1da8ecf..cd63dbb 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,79 @@ ![CLOUDLINK 4 BANNER](https://user-images.githubusercontent.com/12957745/188282246-a221e66a-5d8a-4516-9ae2-79212b745d91.png) -##### CL4 banner(s) made by [@zedthehedgehog](https://github.com/zedthehedgehog) -# CloudLink -CloudLink is a free and open-source, websocket-powered API optimized for Scratch 3.0. CloudLink comes with several powerful utilities and features: -* Multicast and unicasting messages - Perfect for high-speed projects -* Friendly software suite for implementing project-specific features - Multi-project save files and more -* Advanced packet queuing system - Messages are handled browser-level so your Scratch code doesn't need to do extra work -* Support for sandboxed/unsandboxed extension modes -* Proven reliability - Extensively tested and utilized in [The Meower Project](https://github.com/meower-media-co/) -* Unique project identifiers to allow frictionless communication between projects and app servers over a single CloudLink Server - -CloudLink is now powered by [aaugustin/websockets!](https://github.com/aaugustin/websockets) CloudLink will perform better and more reliably than before. - -## Get started with CloudLink -For full documentation of CloudLink, please visit CloudLink's [Documentation](https://hackmd.io/g6BogABhT6ux1GA2oqaOXA) page. - -There are several publically-hosted CloudLink instances available, which can be found in [serverlist.json](https://github.com/MikeDev101/cloudlink/blob/master/serverlist.json) or through the Server List block. - -CloudLink was originally created for Scratch 3.0. You can view the latest version of CloudLink in any of these Scratch editors: -- [TurboWarp](https://turbowarp.org/editor?extension=https://mikedev101.github.io/cloudlink/S4-0-nosuite.js) -- [SheepTester's E 羊 icques](https://sheeptester.github.io/scratch-gui/?url=https://mikedev101.github.io/cloudlink/S4-0-nosuite.js) +# Cloudlink +CloudLink is a free and open-source websocket solution optimized for Scratch. +Originally created as a cloud variables alternative, it can function as a multipurpose websocket framework for other projects. + +# πŸ’‘ Features πŸ’‘ + +### πŸͺΆ Fast and lightweight +CloudLink can run on minimal resources. At least 25MB of RAM and any reasonably capable CPU can run a CloudLink server. + +### 🌐 Essential networking tools +* Unicast and multicast packets across clients +* Expandable functionality with a built-in method loader + +### πŸ“¦ Minimal dependencies +All dependencies below can be installed using `pip install -r requirements.txt`. +* 🐍 Python >=3.11 +* 🧡 asyncio (Built-in) +* πŸ“ƒ ["ujson" ultrajson](https://github.com/ultrajson/ultrajson) +* πŸ” [pyeve/cerberus](https://github.com/pyeve/cerberus) +* ❄️ ["snowflake-id" vd2org/snowflake](https://github.com/vd2org/snowflake) +* 🌐 [aaugustin/websockets](https://github.com/aaugustin/websockets) + +### πŸ”‹Batteries included +The CloudLink Python server comes with full support for the CL4 protocol and the Scratch cloud variable protocol. +Just download, setup, and start! + +### 🧱 Plug-and-play modularity +You can easily extend the functionality of the server using classes and decorators. +Here's an example of a simple plugin that displays "Foobar!" in the console +when a client sends the message `{ "cmd": "foo" }` to the server. + +```python +# Import the server +from cloudlink import server + +# Import default protocol +from cloudlink.server.protocols import clpv4 + +# Instantiate the server object +server = server() + +# Set logging level +server.logging.basicConfig( + level=server.logging.DEBUG +) + +# Load default CL protocol +clpv4 = clpv4(server) + +# Define the functions your plugin executes +class myplugin: + def __init__(self, server, protocol): + + # Example command - client sends { "cmd": "foo" } to the server, this function will execute + @server.on_command(cmd="foo", schema=protocol.schema) + async def foobar(client, message): + print("Foobar!") + +# Load the plugin! +myplugin(server, clpv4) + +# Start the server! +server.run() +``` + +# 🐈 A powerful extension for Scratch 3.0 +You can learn about the protocol using the original Scratch 3.0 client extension. +Feel free to test-drive the extension in any of these Scratch mods: + +- [TurboWarp](https://turbowarp.org/editor?extension=url=https://mikedev101.github.io/cloudlink/S4-1-nosuite.js) +- [SheepTester's E羊icques](https://sheeptester.github.io/scratch-gui/?url=https://mikedev101.github.io/cloudlink/S4-1-nosuite.js) - [Ogadaki's Adacraft](https://adacraft.org/studio/) - [Ogadaki's Adacraft (Beta)](https://beta.adacraft.org/studio/) -- [PenguinMod](https://studio.penguinmod.site/editor.html?extension=https://extensions.turbowarp.org/cloudlink.js) - -CloudLink is also available as a Python module, which comes bundled with the CloudLink Server. -There is even a web-friendly version of CloudLink available as CLJS. - -### [Discussion Forum (Archive)](https://scratch.mit.edu/discuss/topic/398473) -### [CloudLink JS "CLJS"](https://www.npmjs.com/package/@williamhorning/cloudlink) - -## FAQ -> Will my CloudLink 3.0/TURBO projects support CloudLink 4.0 Servers? - -Yes, there will be no compatibility-breaking changes to how CloudLink 4.0 handles messages. However, the new custom command handler will bind all custom commands to use: `{"cmd": "(custom command here)"}` instead of using the Direct command, `{"cmd": "direct", "val": {"cmd": "(custom command here)"}}` - -> Will my Server (v0.1.7.x and older) need to be rewritten entirely to support CloudLink 4.0? - -No, you will only need to rewrite your custom packet handlers as CloudLink 4.0 will reimplement custom commands. - -> Where can I find old versions of CloudLink? - -You can check the releases tab in Github for older versions, or you can download a complete archive of all old versions here (LINK TBD). - -> Will CloudLink 4.0 work with my project made for CloudLink TURBO? - -No, CloudLink 4.0 serves as a replacement of CloudLink TURBO. While CloudLink 4.0 is built upon CloudLink TURBO, it does not have the same blocks as CloudLink TURBO. In favor or retaining compatibility with CloudLink 3.0, CloudLink TURBO should not be used and will be retired. - -> Will my older projects (prior to CloudLink 3.0) work with CloudLink 4.0? - -No, only projects built with CloudLink 3.0 will work with CloudLink 4.0. - -> Does CloudLink 4.0 have the CloudLink Suite? - -Yes. CloudLink 4.0 will be a complete reimplementation of the original CloudLink Suite. - -> What is the CloudLink Suite? - -The Cloudlink Suite is a set of extra features built into the Cloudlink Extension. It provides extra features for Scratch developers to implement in projects that would normally add extra bloat, but can be implemented in a few blocks. These features include: -* CloudDisk: Completely free cloud storage (Up to 10 KB, or 10^4 Bytes, per account), and a cross-project, cross-platform save file system (Up to 1 KB, or 10^3 Bytes, per save file with a maximum of 10 save files). -* CloudCoin: Simple per-project, per-user currency system and supports cross-project trading. -* CloudAccount: Extremely easy-to-use username/password system as an alternative to the username block for user identification, and protects your CloudCoin and CloudDisk data from unwanted users. +- [PenguinMod](https://studio.penguinmod.site/editor.html?extension=url=https://mikedev101.github.io/cloudlink/S4-1-nosuite.js) -## Found an issue? -Please report any bugs, glitches, and/or security vulnerabilities [here](https://github.com/MikeDev101/cloudlink/issues). +# πŸ“ƒ The CloudLink Protocol πŸ“ƒ +Documentation of the CL4 protocol can be found in the CloudLink Repository's Wiki page. diff --git a/S4-0-nosuite.js b/S4-0-nosuite.js index ca774b3..5ea3a2e 100644 --- a/S4-0-nosuite.js +++ b/S4-0-nosuite.js @@ -1707,4 +1707,4 @@ class CloudLink { window.vm.extensionManager._loadedExtensions.set(extensionInstance.getInfo().id, serviceName); console.log("CloudLink 4.0 loaded. Detecting unsandboxed mode."); }; -})() \ No newline at end of file +})() diff --git a/S4-1-nosuite.js b/S4-1-nosuite.js new file mode 100644 index 0000000..54b2a4c --- /dev/null +++ b/S4-1-nosuite.js @@ -0,0 +1,1710 @@ +var servers = {}; ; // Server list +let mWS = null; + +// Get the server URL list +try { + fetch('https://mikedev101.github.io/cloudlink/serverlist.json').then(response => { + return response.text(); + }).then(data => { + servers = JSON.parse(data); + }).catch(err => { + console.log(err); + servers = {}; + }); +} catch(err) { + console.log(err); + servers = {}; +}; + +function find_id(ID, ulist) { + // Thanks StackOverflow! + if (jsonCheck(ID) && (!intCheck(ID))) { + return ulist.some(o => ((o.username === JSON.parse(ID).username) && (o.id == JSON.parse(ID).id))); + } else { + return ulist.some(o => ((o.username === String(ID)) || (o.id == ID))); + }; +} + +function jsonCheck(JSON_STRING) { + try { + JSON.parse(JSON_STRING); + return true; + } catch (err) { + return false; + } +} + +function intCheck(value) { + return !isNaN(value); +} + +function autoConvert(value) { + // Check if the value is JSON / Dict first + try { + JSON.parse(value); + return JSON.parse(value); + } catch (err) {}; + + // Check if the value is an array + try { + tmp = value; + tmp = tmp.replace(/'/g, '"'); + JSON.parse(tmp); + return JSON.parse(tmp); + } catch (err) {}; + + // Check if an int/float + if (!isNaN(value)) { + return Number(value); + }; + + // Leave as the original value if none of the above work + return value; +} + +class CloudLink { + constructor (runtime, extensionId) { + // Extension stuff + this.runtime = runtime; + this.cl_icon = 'data:image/svg+xml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHdpZHRoPSIyMjUuMzU0OCIgaGVpZ2h0PSIyMjUuMzU0OCIgdmlld0JveD0iMCwwLDIyNS4zNTQ4LDIyNS4zNTQ4Ij48ZyB0cmFuc2Zvcm09InRyYW5zbGF0ZSgtMTI3LjMyMjYsLTY3LjMyMjYpIj48ZyBkYXRhLXBhcGVyLWRhdGE9InsmcXVvdDtpc1BhaW50aW5nTGF5ZXImcXVvdDs6dHJ1ZX0iIHN0cm9rZT0ibm9uZSIgc3Ryb2tlLWxpbmVjYXA9ImJ1dHQiIHN0cm9rZS1saW5lam9pbj0ibWl0ZXIiIHN0cm9rZS1taXRlcmxpbWl0PSIxMCIgc3Ryb2tlLWRhc2hhcnJheT0iIiBzdHJva2UtZGFzaG9mZnNldD0iMCIgc3R5bGU9Im1peC1ibGVuZC1tb2RlOiBub3JtYWwiPjxwYXRoIGQ9Ik0xMjcuMzIyNiwxODBjMCwtNjIuMjMwMDEgNTAuNDQ3MzksLTExMi42Nzc0IDExMi42Nzc0LC0xMTIuNjc3NGM2Mi4yMzAwMSwwIDExMi42Nzc0LDUwLjQ0NzM5IDExMi42Nzc0LDExMi42Nzc0YzAsNjIuMjMwMDEgLTUwLjQ0NzM5LDExMi42Nzc0IC0xMTIuNjc3NCwxMTIuNjc3NGMtNjIuMjMwMDEsMCAtMTEyLjY3NzQsLTUwLjQ0NzM5IC0xMTIuNjc3NCwtMTEyLjY3NzR6IiBmaWxsPSIjMDBjMjhjIiBmaWxsLXJ1bGU9Im5vbnplcm8iIHN0cm9rZS13aWR0aD0iMCIvPjxnIGZpbGwtcnVsZT0iZXZlbm9kZCIgc3Ryb2tlLXdpZHRoPSIxIj48cGF0aCBkPSJNMjg2LjEyMDM3LDE1MC41NTc5NWMyMy4yNDA4NiwwIDQyLjA3ODksMTguODM5NDYgNDIuMDc4OSw0Mi4wNzg5YzAsMjMuMjM5NDQgLTE4LjgzODAzLDQyLjA3ODkgLTQyLjA3ODksNDIuMDc4OWgtOTIuMjQwNzRjLTIzLjI0MDg2LDAgLTQyLjA3ODksLTE4LjgzOTQ2IC00Mi4wNzg5LC00Mi4wNzg5YzAsLTIzLjIzOTQ0IDE4LjgzODAzLC00Mi4wNzg5IDQyLjA3ODksLTQyLjA3ODloNC4xODg4N2MxLjgxMTUzLC0yMS41NzA1NSAxOS44OTM1NywtMzguNTEyODkgNDEuOTMxNSwtMzguNTEyODljMjIuMDM3OTMsMCA0MC4xMTk5NywxNi45NDIzNCA0MS45MzE1LDM4LjUxMjg5eiIgZmlsbD0iI2ZmZmZmZiIvPjxwYXRoIGQ9Ik0yODkuMDg2NTUsMjEwLjM0MTE0djkuMDQ2NjdoLTI2LjkxNjYzaC05LjA0NjY3di05LjA0NjY3di01NC41MDMzOWg5LjA0NjY3djU0LjUwMzM5eiIgZmlsbD0iIzAwYzI4YyIvPjxwYXRoIGQ9Ik0yMjIuNDA5MjUsMjE5LjM4NzgxYy04LjM1MzIsMCAtMTYuMzY0MzEsLTMuMzE4MzQgLTIyLjI3MDksLTkuMjI0OTJjLTUuOTA2NjEsLTUuOTA2NTggLTkuMjI0OTEsLTEzLjkxNzY4IC05LjIyNDkxLC0yMi4yNzA4OWMwLC04LjM1MzIgMy4zMTgyOSwtMTYuMzY0MzEgOS4yMjQ5MSwtMjIuMjcwOWM1LjkwNjU5LC01LjkwNjYxIDEzLjkxNzcsLTkuMjI0OTEgMjIuMjcwOSwtOS4yMjQ5MWgyMS4xMDg5djguOTM0OThoLTIxLjEwODl2MC4xMDI1N2MtNS45NTYyOCwwIC0xMS42Njg2NCwyLjM2NjE2IC0xNS44ODAzNyw2LjU3Nzg5Yy00LjIxMTczLDQuMjExNzMgLTYuNTc3ODksOS45MjQwOCAtNi41Nzc4OSwxNS44ODAzN2MwLDUuOTU2MjggMi4zNjYxNiwxMS42Njg2NCA2LjU3Nzg5LDE1Ljg4MDM3YzQuMjExNzMsNC4yMTE3MyA5LjkyNDA4LDYuNTc3OTMgMTUuODgwMzcsNi41Nzc5M3YwLjEwMjUzaDIxLjEwODl2OC45MzQ5OHoiIGZpbGw9IiMwMGMyOGMiLz48L2c+PC9nPjwvZz48L3N2Zz48IS0tcm90YXRpb25DZW50ZXI6MTEyLjY3NzQwNDA4NDA4MzkyOjExMi42Nzc0MDQwODQwODQwMy0tPg=='; + this.cl_block = 'data:image/svg+xml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIHdpZHRoPSIxNzYuMzk4NTQiIGhlaWdodD0iMTIyLjY3MDY5IiB2aWV3Qm94PSIwLDAsMTc2LjM5ODU0LDEyMi42NzA2OSI+PGcgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTE1MS44MDA3MywtMTE4LjY2NDY2KSI+PGcgZGF0YS1wYXBlci1kYXRhPSJ7JnF1b3Q7aXNQYWludGluZ0xheWVyJnF1b3Q7OnRydWV9IiBmaWxsLXJ1bGU9ImV2ZW5vZGQiIHN0cm9rZT0ibm9uZSIgc3Ryb2tlLXdpZHRoPSIxIiBzdHJva2UtbGluZWNhcD0iYnV0dCIgc3Ryb2tlLWxpbmVqb2luPSJtaXRlciIgc3Ryb2tlLW1pdGVybGltaXQ9IjEwIiBzdHJva2UtZGFzaGFycmF5PSIiIHN0cm9rZS1kYXNob2Zmc2V0PSIwIiBzdHlsZT0ibWl4LWJsZW5kLW1vZGU6IG5vcm1hbCI+PGc+PHBhdGggZD0iTTI4Ni4xMjAzNywxNTcuMTc3NTVjMjMuMjQwODYsMCA0Mi4wNzg5LDE4LjgzOTQ2IDQyLjA3ODksNDIuMDc4OWMwLDIzLjIzOTQ0IC0xOC44MzgwMyw0Mi4wNzg5IC00Mi4wNzg5LDQyLjA3ODloLTkyLjI0MDc0Yy0yMy4yNDA4NiwwIC00Mi4wNzg5LC0xOC44Mzk0NiAtNDIuMDc4OSwtNDIuMDc4OWMwLC0yMy4yMzk0NCAxOC44MzgwMywtNDIuMDc4OSA0Mi4wNzg5LC00Mi4wNzg5aDQuMTg4ODdjMS44MTE1MywtMjEuNTcwNTUgMTkuODkzNTcsLTM4LjUxMjg5IDQxLjkzMTUsLTM4LjUxMjg5YzIyLjAzNzkzLDAgNDAuMTE5OTcsMTYuOTQyMzQgNDEuOTMxNSwzOC41MTI4OXoiIGZpbGw9IiNmZmZmZmYiLz48cGF0aCBkPSJNMjg5LjA4NjU1LDIxNi45NjA3NHY5LjA0NjY3aC0yNi45MTY2M2gtOS4wNDY2N3YtOS4wNDY2N3YtNTQuNTAzMzloOS4wNDY2N3Y1NC41MDMzOXoiIGZpbGw9IiMwMGMyOGMiLz48cGF0aCBkPSJNMjIyLjQwOTI1LDIyNi4wMDc0MWMtOC4zNTMyLDAgLTE2LjM2NDMxLC0zLjMxODM0IC0yMi4yNzA5LC05LjIyNDkyYy01LjkwNjYxLC01LjkwNjU4IC05LjIyNDkxLC0xMy45MTc2OCAtOS4yMjQ5MSwtMjIuMjcwODljMCwtOC4zNTMyIDMuMzE4MjksLTE2LjM2NDMxIDkuMjI0OTEsLTIyLjI3MDljNS45MDY1OSwtNS45MDY2MSAxMy45MTc3LC05LjIyNDkxIDIyLjI3MDksLTkuMjI0OTFoMjEuMTA4OXY4LjkzNDk4aC0yMS4xMDg5djAuMTAyNTdjLTUuOTU2MjgsMCAtMTEuNjY4NjQsMi4zNjYxNiAtMTUuODgwMzcsNi41Nzc4OWMtNC4yMTE3Myw0LjIxMTczIC02LjU3Nzg5LDkuOTI0MDggLTYuNTc3ODksMTUuODgwMzdjMCw1Ljk1NjI4IDIuMzY2MTYsMTEuNjY4NjQgNi41Nzc4OSwxNS44ODAzN2M0LjIxMTczLDQuMjExNzMgOS45MjQwOCw2LjU3NzkzIDE1Ljg4MDM3LDYuNTc3OTN2MC4xMDI1M2gyMS4xMDg5djguOTM0OTh6IiBmaWxsPSIjMDBjMjhjIi8+PC9nPjwvZz48L2c+PC9zdmc+PCEtLXJvdGF0aW9uQ2VudGVyOjg4LjE5OTI2OTk5OTk5OTk4OjYxLjMzNTM0NDk5OTk5OTk5LS0+'; + + // Socket data + this.socketData = { + "gmsg": [], + "pmsg": [], + "direct": [], + "statuscode": [], + "gvar": [], + "pvar": [], + "motd": "", + "client_ip": "", + "ulist": [], + "server_version": "" + }; + this.varData = { + "gvar": {}, + "pvar": {} + }; + + this.queueableCmds = ["gmsg", "pmsg", "gvar", "pvar", "direct", "statuscode"]; + this.varCmds = ["gvar", "pvar"]; + + // Listeners + this.socketListeners = {}; + this.socketListenersData = {}; + this.newSocketData = { + "gmsg": false, + "pmsg": false, + "direct": false, + "statuscode": false, + "gvar": false, + "pvar": false + }; + + // Edge-triggered hat blocks + this.connect_hat = 0; + this.packet_hat = 0; + this.close_hat = 0; + + // Status stuff + this.isRunning = false; + this.isLinked = false; + this.version = "S4.1"; + this.link_status = 0; + this.username = ""; + this.tmp_username = ""; + this.isUsernameSyncing = false; + this.isUsernameSet = false; + this.disconnectWasClean = false; + this.wasConnectionDropped = false; + this.didConnectionFail = false; + this.protocolOk = false; + + // Listeners stuff + this.enableListener = false; + this.setListener = ""; + + // Rooms stuff + this.enableRoom = false; + this.isRoomSetting = false; + this.selectRoom = ""; + + // Remapping stuff + this.menuRemap = { + "Global data": "gmsg", + "Private data": "pmsg", + "Global variables": "gvar", + "Private variables": "pvar", + "Direct data": "direct", + "Status code": "statuscode", + "All data": "all" + }; + } + + getInfo () { + return { + "id": 'cloudlink', + "name": 'CloudLink', + "blockIconURI": this.cl_block, + "menuIconURI": this.cl_icon, + "blocks": [ + { + "opcode": 'returnGlobalData', + "blockType": "reporter", + "text": "Global data" + }, + { + "opcode": 'returnPrivateData', + "blockType": "reporter", + "text": "Private data" + }, + { + "opcode": 'returnDirectData', + "blockType": "reporter", + "text": "Direct Data" + }, + { + "opcode": 'returnLinkData', + "blockType": "reporter", + "text": "Link status" + }, + { + "opcode": 'returnStatusCode', + "blockType": "reporter", + "text": "Status code" + }, + { + "opcode": 'returnUserListData', + "blockType": "reporter", + "text": "Usernames" + }, + { + "opcode": "returnUsernameData", + "blockType": "reporter", + "text": "My username" + }, + { + "opcode": "returnVersionData", + "blockType": "reporter", + "text": "Extension version" + }, + { + "opcode": "returnServerVersion", + "blockType": "reporter", + "text": "Server version" + }, + { + "opcode": "returnServerList", + "blockType": "reporter", + "text": "Server list" + }, + { + "opcode": "returnMOTD", + "blockType": "reporter", + "text": "Server MOTD" + }, + { + "opcode": "returnClientIP", + "blockType": "reporter", + "text": "My IP address" + }, + { + "opcode": 'returnListenerData', + "blockType": "reporter", + "text": "Response for listener [ID]", + "arguments": { + "ID": { + "type": "string", + "defaultValue": "example-listener", + }, + }, + }, + { + "opcode": "readQueueSize", + "blockType": "reporter", + "text": "Size of queue for [TYPE]", + "arguments": { + "TYPE": { + "type": "string", + "menu": "allmenu", + "defaultValue": "All data", + }, + }, + }, + { + "opcode": "readQueueData", + "blockType": "reporter", + "text": "Packet queue for [TYPE]", + "arguments": { + "TYPE": { + "type": "string", + "menu": "allmenu", + "defaultValue": "All data", + }, + }, + }, + { + "opcode": 'returnVarData', + "blockType": "reporter", + "text": "[TYPE] [VAR] data", + "arguments": { + "VAR": { + "type": "string", + "defaultValue": "Apple", + }, + "TYPE": { + "type": "string", + "menu": "varmenu", + "defaultValue": "Global variables", + }, + }, + }, + { + "opcode": 'parseJSON', + "blockType": "reporter", + "text": '[PATH] of [JSON_STRING]', + "arguments": { + "PATH": { + "type": "string", + "defaultValue": 'fruit/apples', + }, + "JSON_STRING": { + "type": "string", + "defaultValue": '{"fruit": {"apples": 2, "bananas": 3}, "total_fruit": 5}', + }, + }, + }, + { + "opcode": 'getFromJSONArray', + "blockType": "reporter", + "text": 'Get [NUM] from JSON array [ARRAY]', + "arguments": { + "NUM": { + "type": "number", + "defaultValue": 0, + }, + "ARRAY": { + "type": "string", + "defaultValue": '["foo","bar"]', + } + } + }, + { + "opcode": 'fetchURL', + "blockType": "reporter", + "blockAllThreads": "true", + "text": "Fetch data from URL [url]", + "arguments": { + "url": { + "type": "string", + "defaultValue": "https://mikedev101.github.io/cloudlink/fetch_test", + }, + }, + }, + { + "opcode": 'requestURL', + "blockType": "reporter", + "blockAllThreads": "true", + "text": 'Send request with method [method] for URL [url] with data [data] and headers [headers]', + "arguments": { + "method": { + "type": "string", + "defaultValue": 'GET', + }, + "url": { + "type": "string", + "defaultValue": 'https://mikedev101.github.io/cloudlink/fetch_test', + }, + "data": { + "type": "string", + "defaultValue": '{}' + }, + "headers": { + "type": "string", + "defaultValue": '{}' + }, + } + }, + { + "opcode": 'makeJSON', + "blockType": "reporter", + "text": 'Convert [toBeJSONified] to JSON', + "arguments": { + "toBeJSONified": { + "type": "string", + "defaultValue": '{"test": true}', + }, + } + }, + { + "opcode": 'onConnect', + "blockType": "hat", + "text": 'When connected', + "blockAllThreads": "true" + }, + { + "opcode": 'onClose', + "blockType": "hat", + "text": 'When disconnected', + "blockAllThreads": "true" + }, + { + "opcode": 'onListener', + "blockType": "hat", + "text": 'When I receive new packet with listener [ID]', + "blockAllThreads": "true", + "arguments": { + "ID": { + "type": "string", + "defaultValue": "example-listener", + }, + }, + }, + { + "opcode": 'onNewPacket', + "blockType": "hat", + "text": 'When I receive new [TYPE] packet', + "blockAllThreads": "true", + "arguments": { + "TYPE": { + "type": "string", + "menu": "almostallmenu", + "defaultValue": 'Global data' + }, + }, + }, + { + "opcode": 'onNewVar', + "blockType": "hat", + "text": 'When I receive new [TYPE] data for [VAR]', + "blockAllThreads": "true", + "arguments": { + "TYPE": { + "type": "string", + "menu": "varmenu", + "defaultValue": 'Global variables', + }, + "VAR": { + "type": "string", + "defaultValue": 'Apple', + }, + }, + }, + { + "opcode": 'getComState', + "blockType": "Boolean", + "text": 'Connected?', + }, + { + "opcode": 'getRoomState', + "blockType": "Boolean", + "text": 'Linked to rooms?', + }, + { + "opcode": 'getComLostConnectionState', + "blockType": "Boolean", + "text": 'Lost connection?', + }, + { + "opcode": 'getComFailedConnectionState', + "blockType": "Boolean", + "text": 'Failed to connnect?', + }, + { + "opcode": 'getUsernameState', + "blockType": "Boolean", + "text": 'Username synced?', + }, + { + "opcode": 'returnIsNewData', + "blockType": "Boolean", + "text": 'Got New [TYPE]?', + "arguments": { + "TYPE": { + "type": "string", + "menu": "datamenu", + "defaultValue": 'Global data', + }, + }, + }, + { + "opcode": 'returnIsNewVarData', + "blockType": "Boolean", + "text": 'Got New [TYPE] data for variable [VAR]?', + "arguments": { + "TYPE": { + "type": "string", + "menu": "varmenu", + "defaultValue": 'Global variables', + }, + "VAR": { + "type": "string", + "defaultValue": 'Apple', + }, + }, + }, + { + "opcode": 'returnIsNewListener', + "blockType": "Boolean", + "text": 'Got new packet with listener [ID]?', + "blockAllThreads": "true", + "arguments": { + "ID": { + "type": "string", + "defaultValue": "example-listener", + }, + }, + }, + { + "opcode": 'checkForID', + "blockType": "Boolean", + "text": 'ID [ID] connected?', + "arguments": { + "ID": { + "type": "string", + "defaultValue": 'Another name', + }, + }, + }, + { + "opcode": 'isValidJSON', + "blockType": "Boolean", + "text": 'Is [JSON_STRING] valid JSON?', + "arguments": { + "JSON_STRING": { + "type": "string", + "defaultValue": '{"fruit": {"apples": 2, "bananas": 3}, "total_fruit": 5}', + }, + }, + }, + { + "opcode": 'openSocket', + "blockType": "command", + "text": 'Connect to [IP]', + "blockAllThreads": "true", + "arguments": { + "IP": { + "type": "string", + "defaultValue": 'ws://127.0.0.1:3000/', + }, + }, + }, + { + "opcode": 'openSocketPublicServers', + "blockType": "command", + "text": 'Connect to server [ID]', + "blockAllThreads": "true", + "arguments": { + "ID": { + "type": "number", + "defaultValue": '', + }, + }, + }, + { + "opcode": 'closeSocket', + "blockType": "command", + "blockAllThreads": "true", + "text": 'Disconnect', + }, + { + "opcode": 'setMyName', + "blockType": "command", + "text": 'Set [NAME] as username', + "blockAllThreads": "true", + "arguments": { + "NAME": { + "type": "string", + "defaultValue": "A name", + }, + }, + }, + { + "opcode": 'createListener', + "blockType": "command", + "text": 'Attach listener [ID] to next packet', + "blockAllThreads": "true", + "arguments": { + "ID": { + "type": "string", + "defaultValue": "example-listener", + }, + }, + }, + { + "opcode": 'linkToRooms', + "blockType": "command", + "text": 'Link to room(s) [ROOMS]', + "blockAllThreads": "true", + "arguments": { + "ROOMS": { + "type": "string", + "defaultValue": '["test"]', + }, + } + }, + { + "opcode": 'selectRoomsInNextPacket', + "blockType": "command", + "text": 'Select room(s) [ROOMS] for next packet', + "blockAllThreads": "true", + "arguments": { + "ROOMS": { + "type": "string", + "defaultValue": '["test"]', + }, + }, + }, + { + "opcode": 'unlinkFromRooms', + "blockType": "command", + "text": 'Unlink from all rooms', + "blockAllThreads": "true" + }, + { + "opcode": 'sendGData', + "blockType": "command", + "text": 'Send [DATA]', + "blockAllThreads": "true", + "arguments": { + "DATA": { + "type": "string", + "defaultValue": 'Apple' + } + } + }, + { + "opcode": 'sendPData', + "blockType": "command", + "text": 'Send [DATA] to [ID]', + "blockAllThreads": "true", + "arguments": { + "DATA": { + "type": "string", + "defaultValue": 'Apple' + }, + "ID": { + "type": "string", + "defaultValue": 'Another name' + } + } + }, + { + "opcode": 'sendGDataAsVar', + "blockType": "command", + "text": 'Send variable [VAR] with data [DATA]', + "blockAllThreads": "true", + "arguments": { + "DATA": { + "type": "string", + "defaultValue": 'Banana' + }, + "VAR": { + "type": "string", + "defaultValue": 'Apple' + } + } + }, + { + "opcode": 'sendPDataAsVar', + "blockType": "command", + "text": 'Send variable [VAR] to [ID] with data [DATA]', + "blockAllThreads": "true", + "arguments": { + "DATA": { + "type": "string", + "defaultValue": 'Banana' + }, + "ID": { + "type": "string", + "defaultValue": 'Another name' + }, + "VAR": { + "type": "string", + "defaultValue": 'Apple' + } + } + }, + { + "opcode": 'runCMDnoID', + "blockType": "command", + "text": 'Send command without ID [CMD] [DATA]', + "blockAllThreads": "true", + "arguments": { + "CMD": { + "type": "string", + "defaultValue": 'direct' + }, + "DATA": { + "type": "string", + "defaultValue": 'val' + } + } + }, + { + "opcode": 'runCMD', + "blockType": "command", + "text": 'Send command [CMD] [ID] [DATA]', + "blockAllThreads": "true", + "arguments": { + "CMD": { + "type": "string", + "defaultValue": 'direct' + }, + "ID": { + "type": "string", + "defaultValue": 'id' + }, + "DATA": { + "type": "string", + "defaultValue": 'val' + } + } + }, + { + "opcode": 'resetNewData', + "blockType": "command", + "text": 'Reset got new [TYPE] status', + "blockAllThreads": "true", + "arguments": { + "TYPE": { + "type": "string", + "menu": "datamenu", + "defaultValue": 'Global data' + } + } + }, + { + "opcode": 'resetNewVarData', + "blockType": "command", + "text": 'Reset got new [TYPE] [VAR] status', + "blockAllThreads": "true", + "arguments": { + "TYPE": { + "type": "string", + "menu": "varmenu", + "defaultValue": 'Global variables' + }, + "VAR": { + "type": "string", + "defaultValue": 'Apple' + } + } + }, + { + "opcode": 'resetNewListener', + "blockType": "command", + "text": 'Reset got new [ID] listener status', + "blockAllThreads": "true", + "arguments": { + "ID": { + "type": "string", + "defaultValue": 'example-listener' + } + } + }, + { + "opcode": 'clearAllPackets', + "blockType": "command", + "text": "Clear all packets for [TYPE]", + "arguments": { + "TYPE": { + "type": "string", + "menu": "allmenu", + "defaultValue": "All data" + }, + }, + } + ], + "menus": { + "coms": { + "items": ["Connected", "Username synced"] + }, + "datamenu": { + "items": ['Global data', 'Private data', 'Direct data', 'Status code'] + }, + "varmenu": { + "items": ['Global variables', 'Private variables'] + }, + "allmenu": { + "items": ['Global data', 'Private data', 'Direct data', 'Status code', "Global variables", "Private variables", "All data"] + }, + "almostallmenu": { + "items": ['Global data', 'Private data', 'Direct data', 'Status code', "Global variables", "Private variables"] + }, + }, + }; + }; + + // Code for blocks go here + + returnGlobalData() { + if (this.socketData.gmsg.length != 0) { + + let data = (this.socketData.gmsg[this.socketData.gmsg.length - 1].val); + + if (typeof(data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + } else { + return ""; + }; + }; + + returnPrivateData() { + if (this.socketData.pmsg.length != 0) { + let data = (this.socketData.pmsg[this.socketData.pmsg.length - 1].val); + + if (typeof (data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + } else { + return ""; + }; + }; + + returnDirectData() { + if (this.socketData.direct.length != 0) { + let data = (this.socketData.direct[this.socketData.direct.length - 1].val); + + if (typeof (data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + } else { + return ""; + }; + }; + + returnLinkData() { + return String(this.link_status); + }; + + returnStatusCode() { + if (this.socketData.statuscode.length != 0) { + let data = (this.socketData.statuscode[this.socketData.statuscode.length - 1].code); + + if (typeof (data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + } else { + return ""; + }; + }; + + returnUserListData() { + return JSON.stringify(this.socketData.ulist); + }; + + returnUsernameData() { + let data = this.username; + + if (typeof (data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + }; + + returnVersionData() { + return String(this.version); + }; + + returnServerVersion() { + return String(this.socketData.server_version); + }; + + returnServerList() { + return JSON.stringify(servers); + }; + + returnMOTD() { + return String(this.socketData.motd); + }; + + returnClientIP() { + return String(this.socketData.client_ip); + }; + + returnListenerData({ID}) { + const self = this; + if ((this.isRunning) && (this.socketListeners.hasOwnProperty(String(ID)))) { + return JSON.stringify(this.socketListenersData[ID]); + } else { + return "{}"; + }; + }; + + readQueueSize({TYPE}) { + if (this.menuRemap[String(TYPE)] == "all") { + let tmp_size = 0; + tmp_size = tmp_size + this.socketData.gmsg.length; + tmp_size = tmp_size + this.socketData.pmsg.length; + tmp_size = tmp_size + this.socketData.direct.length; + tmp_size = tmp_size + this.socketData.statuscode.length; + tmp_size = tmp_size + this.socketData.gvar.length; + tmp_size = tmp_size + this.socketData.pvar.length; + return tmp_size; + } else { + return this.socketData[this.menuRemap[String(TYPE)]].length; + }; + }; + + readQueueData({TYPE}) { + if (this.menuRemap[String(TYPE)] == "all") { + let tmp_socketData = JSON.parse(JSON.stringify(this.socketData)); // Deep copy + + delete tmp_socketData.motd; + delete tmp_socketData.client_ip; + delete tmp_socketData.ulist; + delete tmp_socketData.server_version; + + return JSON.stringify(tmp_socketData); + } else { + return JSON.stringify(this.socketData[this.menuRemap[String(TYPE)]]); + }; + }; + + returnVarData({ TYPE, VAR }) { + if (this.isRunning) { + if (this.varData.hasOwnProperty(this.menuRemap[TYPE])) { + if (this.varData[this.menuRemap[TYPE]].hasOwnProperty(VAR)) { + return this.varData[this.menuRemap[TYPE]][VAR].value; + } else { + return ""; + }; + } else { + return ""; + }; + } else { + return ""; + }; + }; + + parseJSON({PATH, JSON_STRING}) { + try { + const path = PATH.toString().split('/').map(prop => decodeURIComponent(prop)); + if (path[0] === '') path.splice(0, 1); + if (path[path.length - 1] === '') path.splice(-1, 1); + let json; + try { + json = JSON.parse(' ' + JSON_STRING); + } catch (e) { + return e.message; + }; + path.forEach(prop => json = json[prop]); + if (json === null) return 'null'; + else if (json === undefined) return ''; + else if (typeof json === 'object') return JSON.stringify(json); + else return json.toString(); + } catch (err) { + return ''; + }; + }; + + getFromJSONArray({NUM, ARRAY}) { + var json_array = JSON.parse(ARRAY); + if (json_array[NUM] == "undefined") { + return ""; + } else { + let data = json_array[NUM]; + + if (typeof (data) == "object") { + data = JSON.stringify(data); // Make the JSON safe for Scratch + } + + return data; + } + }; + + fetchURL(args) { + return fetch(args.url, { + method: "GET" + }).then(response => response.text()); + }; + + requestURL(args) { + if (args.method == "GET" || args.method == "HEAD") { + return fetch(args.url, { + method: args.method, + headers: JSON.parse(args.headers) + }).then(response => response.text()); + } else { + return fetch(args.url, { + method: args.method, + headers: JSON.parse(args.headers), + body: JSON.parse(args.data) + }).then(response => response.text()); + } + }; + + isValidJSON({JSON_STRING}) { + return jsonCheck(JSON_STRING); + }; + + makeJSON({toBeJSONified}) { + if (typeof(toBeJSONified) == "string") { + try { + JSON.parse(toBeJSONified); + return String(toBeJSONified); + } catch(err) { + return "Not JSON!"; + } + } else if (typeof(toBeJSONified) == "object") { + return JSON.stringify(toBeJSONified); + } else { + return "Not JSON!"; + }; + }; + + onConnect() { + const self = this; + if (self.connect_hat == 0 && self.isRunning && self.protocolOk) { + self.connect_hat = 1; + return true; + } else { + return false; + }; + }; + + onClose() { + const self = this; + if (self.close_hat == 0 && !self.isRunning) { + self.close_hat = 1; + return true; + } else { + return false; + }; + }; + + onListener({ ID }) { + const self = this; + if ((this.isRunning) && (this.socketListeners.hasOwnProperty(String(ID)))) { + if (self.socketListeners[String(ID)]) { + self.socketListeners[String(ID)] = false; + return true; + } else { + return false; + }; + } else { + return false; + }; + }; + + onNewPacket({ TYPE }) { + const self = this; + if ((this.isRunning) && (this.newSocketData[this.menuRemap[String(TYPE)]])) { + self.newSocketData[this.menuRemap[String(TYPE)]] = false; + return true; + } else { + return false; + }; + }; + + onNewVar({ TYPE, VAR }) { + const self = this; + if (this.isRunning) { + if (this.varData.hasOwnProperty(this.menuRemap[TYPE])) { + if (this.varData[this.menuRemap[TYPE]].hasOwnProperty(VAR)) { + if (this.varData[this.menuRemap[TYPE]][VAR].isNew) { + self.varData[this.menuRemap[TYPE]][VAR].isNew = false; + return true; + } else { + return false; + } + } else { + return false; + }; + } else { + return false; + }; + } else { + return false; + }; + }; + + getComState(){ + return String((this.link_status == 2) || this.protocolOk); + }; + + getRoomState() { + return this.isLinked; + }; + + getComLostConnectionState() { + return this.wasConnectionDropped; + }; + + getComFailedConnectionState() { + return this.didConnectionFail; + }; + + getUsernameState(){ + return this.isUsernameSet; + }; + + returnIsNewData({TYPE}){ + if (this.isRunning) { + return this.newSocketData[this.menuRemap[String(TYPE)]]; + } else { + return false; + }; + }; + + returnIsNewVarData({ TYPE, VAR }) { + if (this.isRunning) { + if (this.varData.hasOwnProperty(this.menuRemap[TYPE])) { + if (this.varData[this.menuRemap[TYPE]].hasOwnProperty(VAR)) { + return this.varData[this.menuRemap[TYPE]][VAR].isNew; + } else { + return false; + }; + } else { + return false; + }; + } else { + return false; + }; + }; + + returnIsNewListener({ ID }) { + if (this.isRunning) { + if (this.socketListeners.hasOwnProperty(String(ID))) { + return this.socketListeners[ID]; + } else { + return false; + }; + } else { + return false; + }; + }; + + checkForID({ ID }) { + return find_id(ID, this.socketData.ulist); + }; + + openSocket({IP}) { + const self = this; + if (!self.isRunning) { + console.log("Starting socket."); + self.link_status = 1; + + self.disconnectWasClean = false; + self.wasConnectionDropped = false; + self.didConnectionFail = false; + + mWS = new WebSocket(String(IP)); + + mWS.onerror = function(){ + self.isRunning = false; + }; + + mWS.onopen = function(){ + self.isRunning = true; + self.packet_queue = {}; + self.link_status = 2; + + // Send the handshake request to get server to detect client protocol + mWS.send(JSON.stringify({"cmd": "handshake", "listener": "setprotocol"})) + + console.log("Successfully opened socket."); + }; + + mWS.onmessage = function(event){ + let tmp_socketData = JSON.parse(event.data); + console.log("RX:", tmp_socketData); + + if (self.queueableCmds.includes(tmp_socketData.cmd)) { + self.socketData[tmp_socketData.cmd].push(tmp_socketData); + } else { + if (tmp_socketData.cmd == "ulist") { + // ulist functionality has been changed in server 0.1.9 + if (tmp_socketData.hasOwnProperty("mode")) { + if (tmp_socketData.mode == "set") { + self.socketData["ulist"] = tmp_socketData.val; + } else if (tmp_socketData.mode == "add") { + if (!self.socketData.ulist.some(o => ((o.username === tmp_socketData.val.username) && (o.id == tmp_socketData.val.id)))) { + self.socketData["ulist"].push(tmp_socketData.val); + } else { + console.log("Could not perform ulist method add, client", tmp_socketData.val, "already exists"); + }; + } else if (tmp_socketData.mode == "remove") { + if (self.socketData.ulist.some(o => ((o.username === tmp_socketData.val.username) && (o.id == tmp_socketData.val.id)))) { + // This is by far the fugliest thing I have ever written in JS, or in any programming language... thanks I hate it + self.socketData["ulist"] = self.socketData["ulist"].filter(user => ((!(user.username === tmp_socketData.val.username)) && (!(user.id == tmp_socketData.val.id)))); + } else { + console.log("Could not perform ulist method remove, client", tmp_socketData.val, "was not found"); + }; + } else { + console.log("Could not understand ulist method:", tmp_socketData.mode); + }; + } else { + // Retain compatibility wtih existing servers + self.socketData["ulist"] = tmp_socketData.val; + }; + } else { + self.socketData[tmp_socketData.cmd] = tmp_socketData.val; + }; + }; + + if (self.newSocketData.hasOwnProperty(tmp_socketData.cmd)) { + self.newSocketData[tmp_socketData.cmd] = true; + }; + + if (self.varCmds.includes(tmp_socketData.cmd)) { + self.varData[tmp_socketData.cmd][tmp_socketData.name] = { + "value": tmp_socketData.val, + "isNew": true + }; + }; + if (tmp_socketData.hasOwnProperty("listener")) { + if (tmp_socketData.listener == "setusername") { + self.socketListeners["setusername"] = true; + if (tmp_socketData.code == "I:100 | OK") { + self.username = tmp_socketData.val; + self.isUsernameSyncing = false; + self.isUsernameSet = true; + console.log("Username was accepted by the server, and has been set to:", self.username); + } else { + console.warn("Username was rejected by the server. Error code:", String(tmp_socketData.code)); + self.isUsernameSyncing = false; + }; + } else if (tmp_socketData.listener == "roomLink") { + self.isRoomSetting = false; + self.socketListeners["roomLink"] = true; + if (tmp_socketData.code == "I:100 | OK") { + console.log("Linking to room(s) was accepted by the server!"); + self.isLinked = true; + } else { + console.warn("Linking to room(s) was rejected by the server. Error code:", String(tmp_socketData.code)); + self.enableRoom = false; + self.isLinked = false; + self.selectRoom = ""; + }; + } else if ((tmp_socketData.listener == "setprotocol") && (!this.protocolOk)) { + console.log("Server successfully set client protocol to cloudlink!"); + self.socketData.statuscode = []; + self.protocolOk = true; + self.socketListeners["setprotocol"] = true; + } else { + if (self.socketListeners.hasOwnProperty(tmp_socketData.listener)) { + self.socketListeners[tmp_socketData.listener] = true; + }; + }; + self.socketListenersData[tmp_socketData.listener] = tmp_socketData; + }; + self.packet_hat = 0; + }; + + mWS.onclose = function() { + self.isRunning = false; + self.connect_hat = 0; + self.packet_hat = 0; + self.protocolOk = false; + if (self.close_hat == 1) { + self.close_hat = 0; + }; + self.socketData = { + "gmsg": [], + "pmsg": [], + "direct": [], + "statuscode": [], + "gvar": [], + "pvar": [], + "motd": "", + "client_ip": "", + "ulist": [], + "server_version": "" + }; + self.newSocketData = { + "gmsg": false, + "pmsg": false, + "direct": false, + "statuscode": false, + "gvar": false, + "pvar": false + }; + self.socketListeners = {}; + self.username = ""; + self.tmp_username = ""; + self.isUsernameSyncing = false; + self.isUsernameSet = false; + self.enableListener = false; + self.setListener = ""; + self.enableRoom = false; + self.selectRoom = ""; + self.isLinked = false; + self.isRoomSetting = false; + + if (self.link_status != 1) { + if (self.disconnectWasClean) { + self.link_status = 3; + console.log("Socket closed."); + self.wasConnectionDropped = false; + self.didConnectionFail = false; + } else { + self.link_status = 4; + console.error("Lost connection to the server."); + self.wasConnectionDropped = true; + self.didConnectionFail = false; + }; + } else { + self.link_status = 4; + console.error("Failed to connect to server."); + self.wasConnectionDropped = false; + self.didConnectionFail = true; + }; + }; + } else { + console.warn("Socket is already open."); + }; + } + + openSocketPublicServers({ ID }){ + if (servers.hasOwnProperty(ID)) { + console.log("Connecting to:", servers[ID].url) + this.openSocket({"IP": servers[ID].url}); + }; + }; + + closeSocket(){ + const self = this; + if (this.isRunning) { + console.log("Closing socket..."); + mWS.close(1000,'script closure'); + self.disconnectWasClean = true; + } else { + console.warn("Socket is not open."); + }; + } + + setMyName({NAME}) { + const self = this; + if (this.isRunning) { + if (!this.isUsernameSyncing) { + if (!this.isUsernameSet){ + if (String(NAME) != "") { + if ((!(String(NAME).length > 20))) { + if (!(String(NAME) == "%CA%" || String(NAME) == "%CC%" || String(NAME) == "%CD%" || String(NAME) == "%MS%")){ + let tmp_msg = { + cmd: "setid", + val: String(NAME), + listener: "setusername" + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + self.tmp_username = String(NAME); + self.isUsernameSyncing = true; + + } else { + console.log("Blocking attempt to use reserved usernames"); + }; + } else { + console.log("Blocking attempt to use username larger than 20 characters, username is " + String(NAME).length + " characters long"); + }; + } else { + console.log("Blocking attempt to use blank username"); + }; + } else { + console.warn("Username already has been set!"); + }; + } else { + console.warn("Username is still syncing!"); + }; + }; + }; + + createListener({ ID }) { + self = this; + if (this.isRunning) { + if (!this.enableListener) { + self.enableListener = true; + self.setListener = String(ID); + } else { + console.warn("Listeners were already created!"); + }; + } else { + console.log("Cannot assign a listener to a packet while disconnected"); + }; + }; + + linkToRooms({ ROOMS }) { + const self = this; + + if (this.isRunning) { + if (!this.isRoomSetting) { + if (!(String(ROOMS).length > 1000)) { + let tmp_msg = { + cmd: "link", + val: autoConvert(ROOMS), + listener: "roomLink" + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + self.isRoomSetting = true; + + } else { + console.warn("Blocking attempt to send a room ID / room list larger than 1000 bytes (1 KB), room ID / room list is " + String(ROOMS).length + " bytes"); + }; + } else { + console.warn("Still linking to rooms!"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + selectRoomsInNextPacket({ROOMS}) { + const self = this; + if (this.isRunning) { + if (this.isLinked) { + if (!this.enableRoom) { + if (!(String(ROOMS).length > 1000)) { + self.enableRoom = true; + self.selectRoom = ROOMS; + } else { + console.warn("Blocking attempt to select a room ID / room list larger than 1000 bytes (1 KB), room ID / room list is " + String(ROOMS).length + " bytes"); + }; + } else { + console.warn("Rooms were already selected!"); + }; + } else { + console.warn("Not linked to any room(s)!"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + unlinkFromRooms() { + const self = this; + if (this.isRunning) { + if (this.isLinked) { + let tmp_msg = { + cmd: "unlink", + val: "" + }; + + if (this.enableListener) { + tmp_msg["listener"] = autoConvert(this.setListener); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + + self.isLinked = false; + } else { + console.warn("Not linked to any rooms!"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + sendGData({DATA}){ + const self = this; + if (this.isRunning) { + if (!(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: "gmsg", + val: autoConvert(DATA) + }; + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + + if (this.enableRoom) { + tmp_msg["rooms"] = autoConvert(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet larger than 1000 bytes (1 KB), packet is " + String(DATA).length + " bytes"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + sendPData({DATA, ID}) { + const self = this; + if (this.isRunning) { + if (!(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: "pmsg", + val: autoConvert(DATA), + id: autoConvert(ID) + } + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + if (this.enableRoom) { + tmp_msg["rooms"] = autoConvert(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet larger than 1000 bytes (1 KB), packet is " + String(DATA).length + " bytes"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + sendGDataAsVar({VAR, DATA }) { + const self = this; + if (this.isRunning) { + if (!(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: "gvar", + name: VAR, + val: autoConvert(DATA) + } + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + if (this.enableRoom) { + tmp_msg["rooms"] = autoConvert(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet larger than 1000 bytes (1 KB), packet is " + String(DATA).length + " bytes"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + sendPDataAsVar({VAR, ID, DATA}) { + const self = this; + if (this.isRunning) { + if (!(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: "pvar", + name: VAR, + val: autoConvert(DATA), + id: autoConvert(ID) + } + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + if (this.enableRoom) { + tmp_msg["rooms"] = autoConvert(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet larger than 1000 bytes (1 KB), packet is " + String(DATA).length + " bytes"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + runCMDnoID({CMD, DATA}) { + const self = this; + if (this.isRunning) { + if (!(String(CMD).length > 100) || !(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: String(CMD), + val: autoConvert(DATA) + } + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + if (this.enableRoom) { + tmp_msg["rooms"] = String(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet with questionably long arguments"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + runCMD({CMD, ID, DATA}) { + const self = this; + if (this.isRunning) { + if (!(String(CMD).length > 100) || !(String(ID).length > 20) || !(String(DATA).length > 1000)) { + let tmp_msg = { + cmd: String(CMD), + id: autoConvert(ID), + val: autoConvert(DATA) + } + + if (this.enableListener) { + tmp_msg["listener"] = String(this.setListener); + }; + if (this.enableRoom) { + tmp_msg["rooms"] = String(this.selectRoom); + }; + + console.log("TX:", tmp_msg); + mWS.send(JSON.stringify(tmp_msg)); + + if (this.enableListener) { + if (!self.socketListeners.hasOwnProperty(this.setListener)) { + self.socketListeners[this.setListener] = false; + }; + self.enableListener = false; + }; + if (this.enableRoom) { + self.enableRoom = false; + self.selectRoom = ""; + }; + + } else { + console.warn("Blocking attempt to send packet with questionably long arguments"); + }; + } else { + console.warn("Socket is not open."); + }; + }; + + resetNewData({TYPE}){ + const self = this; + if (this.isRunning) { + self.newSocketData[this.menuRemap[String(TYPE)]] = false; + }; + }; + + resetNewVarData({ TYPE, VAR }) { + const self = this; + if (this.isRunning) { + if (this.varData.hasOwnProperty(this.menuRemap[TYPE])) { + if (this.varData[this.menuRemap[TYPE]].hasOwnProperty(VAR)) { + self.varData[this.menuRemap[TYPE]][VAR].isNew = false; + }; + }; + }; + }; + + resetNewListener({ ID }) { + const self = this; + if (this.isRunning) { + if (this.socketListeners.hasOwnProperty(String(ID))) { + self.socketListeners[String(ID)] = false; + }; + }; + }; + + clearAllPackets({TYPE}){ + const self = this; + if (this.menuRemap[String(TYPE)] == "all") { + self.socketData.gmsg = []; + self.socketData.pmsg = []; + self.socketData.direct = []; + self.socketData.statuscode = []; + self.socketData.gvar = []; + self.socketData.pvar = []; + } else { + self.socketData[this.menuRemap[String(TYPE)]] = []; + }; + }; +}; + +(function() { + var extensionClass = CloudLink; + if (typeof window === "undefined" || !window.vm) { + Scratch.extensions.register(new extensionClass()); + console.log("CloudLink 4.0 loaded. Detecting sandboxed mode, performance will suffer. Please load CloudLink in Unsandboxed mode."); + } else { + var extensionInstance = new extensionClass(window.vm.extensionManager.runtime); + var serviceName = window.vm.extensionManager._registerInternalExtension(extensionInstance); + window.vm.extensionManager._loadedExtensions.set(extensionInstance.getInfo().id, serviceName); + console.log("CloudLink 4.0 loaded. Detecting unsandboxed mode."); + }; +})() \ No newline at end of file diff --git a/SECURITY.md b/SECURITY.md index 4965127..beca3e6 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -2,17 +2,17 @@ You are to keep your instance of Cloudlink as up-to-date as possible. You are to assume that support can be discontinued at any time, with or without reason. ## Supported Versions -| Version number | Supported? | Note | -|--------------|--------------|------| -| 0.1.9.x | 🟒 Yes | Latest release | -| 0.1.8.x | πŸ”΄ End of life | Pre-CL4 optimized. Should be upgraded. | -| 0.1.7.x and older | πŸ”΄ End of life | CL3/CL Legacy - EOL, should NOT be used | +| Version number | Supported? | Note | +|-------------------|------------|----------------------------------------------------------------------| +| 0.2.0 | 🟒 Yes | Latest version. | +| 0.1.9.x | πŸ”΄ No | CL4 Optimized. EOL. | +| 0.1.8.x | πŸ”΄ No | Pre-CL4 optimized. EOL. | +| 0.1.7.x and older | πŸ”΄ No | CL3/CL Legacy - EOL. | ### Notice for public server hosts Public server hosts should maintain the latest version. If a public server host has been found to be running on a Deprecated release, or a version that has not been upgraded in over 30 days, your public host will be removed from the public server list and you will be notified to update your server. ## Reporting vulnerabilities - In the event that a vulnerability has been found, please use the following format to report said vulnerability: 1. A title of the vulnerability - Should be less than 20 words diff --git a/client-test-async.py b/client-test-async.py deleted file mode 100644 index 08250e9..0000000 --- a/client-test-async.py +++ /dev/null @@ -1,97 +0,0 @@ -from cloudlink import cloudlink - - -class example_events: - def __init__(self): - pass - - async def on_connect(self, client): - print(f"Client {client.obj_id} connected") - await client.set_username(str(client.obj_id)) - await client.send_gmsg("test") - - async def on_close(self, client): - print(f"Client {client.obj_id} disconnected") - - async def username_set(self, client): - print(f"Client {client.obj_id}'s username was set!") - - async def on_gmsg(self, client, message, listener): - print(f"Client {client.obj_id} got gmsg {message['val']}") - - -if __name__ == "__main__": - # Initialize Cloudlink. You will only need to initialize one instance of the main cloudlink module. - cl = cloudlink() - - # Create examples for various ways to extend the functionality of Cloudlink Server. - example = example_events() - - # Example - Multiple clients. - multi_client = cl.multi_client(async_client=True, logs=True) - - # Spawns 5 clients. - for x in range(5): - # Create a new client object. This supports initializing many clients at once. - client = multi_client.spawn(x, "ws://127.0.0.1:3000/") - - # Binding events - This example binds functions to certain events - - # When a client connects, all functions bound to this event will fire. - client.bind_event( - client.events.on_connect, - example.on_connect - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_close, - example.on_close - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_username_set, - example.username_set - ) - - # Binding callbacks for commands - This example binds an event when a gmsg packet is handled. - client.bind_callback_method(client.cl_methods.gmsg, example.on_gmsg) - - print("Waking up now") - multi_client.run() - input("All clients are ready. Press enter to shutdown.") - multi_client.stop() - input("All clients have shut down. Press enter to exit.") - - # Example - Singular clients. - - # Create a new client object. - client = cl.client(async_client=True, logs=True) - client.obj_id = "Test" - - # Binding events - This example binds functions to certain events - - # When a client connects, all functions bound to this event will fire. - client.bind_event( - client.events.on_connect, - example.on_connect - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_close, - example.on_close - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_username_set, - example.username_set - ) - - # Binding callbacks for commands - This example binds an event when a gmsg packet is handled. - client.bind_callback_method(client.cl_methods.gmsg, example.on_gmsg) - - # Run the client. - client.run("ws://127.0.0.1:3000/") diff --git a/client-test-old.py b/client-test-old.py deleted file mode 100644 index 639eee8..0000000 --- a/client-test-old.py +++ /dev/null @@ -1,97 +0,0 @@ -from cloudlink import cloudlink - - -class example_events: - def __init__(self): - pass - - def on_connect(self, client): - print(f"Client {client.obj_id} connected") - client.set_username(str(client.obj_id)) - client.send_gmsg("test") - - def on_close(self, client): - print(f"Client {client.obj_id} disconnected") - - def username_set(self, client): - print(f"Client {client.obj_id}'s username was set!") - - def on_gmsg(self, client, message, listener): - print(f"Client {client.obj_id} got gmsg {message['val']}") - - -if __name__ == "__main__": - # Initialize Cloudlink. You will only need to initialize one instance of the main cloudlink module. - cl = cloudlink() - - # Create examples for various ways to extend the functionality of Cloudlink Server. - example = example_events() - - # Example - Multiple clients. - multi_client = cl.multi_client(async_client=False, logs=True) - - # Spawns 5 clients. - for x in range(5): - # Create a new client object. This supports initializing many clients at once. - client = multi_client.spawn(x, "ws://127.0.0.1:3000/") - - # Binding events - This example binds functions to certain events - - # When a client connects, all functions bound to this event will fire. - client.bind_event( - client.events.on_connect, - example.on_connect - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_close, - example.on_close - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_username_set, - example.username_set - ) - - # Binding callbacks for commands - This example binds an event when a gmsg packet is handled. - client.bind_callback_method(client.cl_methods.gmsg, example.on_gmsg) - - print("Waking up now") - multi_client.run() - input("All clients are ready. Press enter to shutdown.") - multi_client.stop() - input("All clients have shut down. Press enter to exit.") - - # Example - Singular clients. - client = cl.client(async_client=False, logs=True) - - # Object IDs - Sets a friendly name to a specific client object. - client.obj_id = "Test" - - # Binding events - This example binds functions to certain events - - # When a client connects, all functions bound to this event will fire. - client.bind_event( - client.events.on_connect, - example.on_connect - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_close, - example.on_close - ) - - # When a client disconnects, all functions bound to this event will fire. - client.bind_event( - client.events.on_username_set, - example.username_set - ) - - # Binding callbacks for commands - This example binds an event when a gmsg packet is handled. - client.bind_callback_method(client.cl_methods.gmsg, example.on_gmsg) - - # Run the client. - client.run("ws://127.0.0.1:3000/") diff --git a/client_example.py b/client_example.py new file mode 100644 index 0000000..5885fe5 --- /dev/null +++ b/client_example.py @@ -0,0 +1,48 @@ +from cloudlink import client + +if __name__ == "__main__": + # Initialize the client + client = client() + + # Configure logging settings + client.logging.basicConfig( + level=client.logging.DEBUG + ) + + # Use this decorator to handle established connections. + @client.on_connect + async def on_connect(): + print("Connected!") + + # Ask for a username + await client.protocol.set_username(input("Please give me a username... ")) + + # Whenever a client is connected, you can call this function to gracefully disconnect. + # client.disconnect() + + # Use this decorator to handle disconnects. + @client.on_disconnect + async def on_disconnect(): + print("Disconnected!") + + # Use this decorator to handle username being set events. + @client.on_username_set + async def on_username_set(id, name, uuid): + print(f"My username has been set! ID: {id}, Name: {name}, UUID: {uuid}") + + # Example message-specific event handler. You can use different kinds of message types, + # such as pmsg, gvar, pvar, and more. + @client.on_gmsg + async def on_gmsg(message): + print(f"I got a global message! It says: \"{message['val']}\".") + + # Example use of on_command functions within the client. + @client.on_command(cmd="gmsg") + async def on_gmsg(message): + client.send_packet({"cmd": "direct", "val": "Hello, server!"}) + + # Enable SSL support (if you use self-generated SSL certificates) + #client.enable_ssl(certfile="cert.pem") + + # Start the client + client.run(host="ws://127.0.0.1:3000/") diff --git a/cloudlink/__init__.py b/cloudlink/__init__.py index 83c28c3..29f3165 100644 --- a/cloudlink/__init__.py +++ b/cloudlink/__init__.py @@ -1 +1,2 @@ -from .cloudlink import * \ No newline at end of file +from .server import server +from .client import client diff --git a/cloudlink/__main__.py b/cloudlink/__main__.py deleted file mode 100644 index d72d331..0000000 --- a/cloudlink/__main__.py +++ /dev/null @@ -1,23 +0,0 @@ -from .cloudlink import cloudlink - - -class example_events: - def __init__(self): - pass - - async def on_close(self, client): - print("Client", client.id, "disconnected.") - - async def on_connect(self, client): - print("Client", client.id, "connected.") - - -if __name__ == "__main__": - cl = cloudlink() - server = cl.server(logs=True) - events = example_events() - server.set_motd("CL4 Demo Server", True) - server.bind_event(server.events.on_connect, events.on_connect) - server.bind_event(server.events.on_close, events.on_close) - print("Welcome to Cloudlink 4! See https://github.com/mikedev101/cloudlink for more info. Now running server on ws://127.0.0.1:3000/!") - server.run(ip="localhost", port=3000) \ No newline at end of file diff --git a/cloudlink/async_client/__init__.py b/cloudlink/async_client/__init__.py deleted file mode 100644 index 5b2fe66..0000000 --- a/cloudlink/async_client/__init__.py +++ /dev/null @@ -1 +0,0 @@ -from .async_client import * \ No newline at end of file diff --git a/cloudlink/async_client/async_client.py b/cloudlink/async_client/async_client.py deleted file mode 100644 index ceab1c2..0000000 --- a/cloudlink/async_client/async_client.py +++ /dev/null @@ -1,632 +0,0 @@ -import websockets -import asyncio -import json -from copy import copy -import threading - - -# Multi client -class multi_client: - def __init__(self, parent, logs: bool = True): - self.shutdown_flag = False - self.threads = set() - self.logs = logs - self.parent = parent - self.supporter = self.parent.supporter(self) - - self.clients_counter = int() - self.clients_present = set() - self.clients_dead = set() - - # Display version - self.supporter.log(f"Cloudlink asyncio multi client v{self.parent.version}") - - async def on_connect(self, client): - self.clients_present.add(client) - self.clients_dead.discard(client) - while not self.shutdown_flag: - await client.asyncio.sleep(0) - await client.shutdown() - - async def on_disconnect(self, client): - self.clients_present.discard(client) - self.clients_dead.add(client) - - def spawn(self, client_id: any, url: str = "ws://127.0.0.1:3000/"): - _client = client(self.parent, logs=self.logs, multi_silent=True) - _client.obj_id = client_id - _client.bind_event(_client.events.on_connect, self.on_connect) - _client.bind_event(_client.events.on_close, self.on_disconnect) - _client.bind_event(_client.events.on_fail, self.on_disconnect) - self.threads.add(threading.Thread( - target=_client.run, - args=[url], - daemon=True - )) - self.clients_counter = len(self.threads) - return _client - - def run(self): - self.supporter.log("Initializing all clients...") - for thread in self.threads: - thread.start() - while (len(self.clients_present) + len(self.clients_dead)) != self.clients_counter: - pass - self.supporter.log("All clients have initialized.") - - def stop(self): - self.shutdown_flag = True - self.clients_counter = self.clients_counter - len(self.clients_dead) - self.supporter.log("Waiting for all clients to shutdown...") - while len(self.clients_present) != 0: - pass - self.supporter.log("All clients have shut down.") - self.clients_present.clear() - self.clients_dead.clear() - self.threads.clear() - - -# Cloudlink Client -class client: - def __init__(self, parent, logs: bool = True, multi_silent:bool = False): - self.version = parent.version - - # Locally define the client object - self.client = None - - # Declare loop - self.loop = None - - # Initialize required libraries - self.asyncio = asyncio - - self.websockets = websockets - self.copy = copy - self.json = json - - # Other - self.enable_logs = logs - self.client_id = dict() - self.client_ip = str() - self.running = False - self.username_set = False - - # Built-in listeners - self.handshake_listener = "protocolset" - self.setid_listener = "usernameset" - self.ping_listener = "ping" - - # Storage of server stuff - self.motd_message = str() - self.server_version = str() - - # Managing methods - self.custom_methods = custom_methods() - self.disabled_methods = set() - self.method_callbacks = dict() - self.listener_callbacks = dict() - self.safe_methods = set() - - # Managing events - self.events = events() - self.event_callbacks = dict() - - # Initialize supporter - self.supporter = parent.supporter(self) - - # Initialize attributes from supporter component - self.log = self.supporter.log - self.validate = self.supporter.validate - self.load_custom_methods = self.supporter.load_custom_methods - self.detect_listener = self.supporter.detect_listener - self.generate_statuscode = self.supporter.generate_statuscode - - # Initialize rooms storage - self.rooms = rooms(self) - - # Initialize methods - self.cl_methods = cl_methods(self) - - # Load safe CLPv4 methods to mitigate vulnerabilities - self.supporter.init_builtin_cl_methods() - - if not multi_silent: - # Display version - self.supporter.log(f"Cloudlink asyncio client v{parent.version}") - - # == Public API functionality == - - # Runs the client. - def run(self, ip: str = "ws://127.0.0.1:3000/"): - try: - self.loop = self.asyncio.new_event_loop() - self.asyncio.run(self.__session__(ip)) - except KeyboardInterrupt: - pass - except: - print(self.supporter.full_stack()) - - # Disconnects the client. - async def shutdown(self): - if not self.client: - return - - if not self.client.open: - return - - await self.client.close() - - # Sends packets, you should use other methods. - async def send_packet(self, cmd: str, val: any = None, listener: str = None, room_id: str = None, - quirk: str = "quirk_embed_val"): - if not self.client: - return - - if not self.client.open: - return - - # Manage specific message quirks - message = {"cmd": cmd} - if val: - if quirk == self.supporter.quirk_update_msg: - message.update(val) - elif quirk == self.supporter.quirk_embed_val: - message["val"] = val - else: - raise TypeError("Unknown message quirk!") - - # Attach a listener request - if listener: - message["listener"] = listener - - # Attach the rooms key - if room_id: - message["rooms"] = room_id - - # Send payload - try: - await self.client.send(self.json.dumps(message)) - except self.websockets.exceptions.ConnectionClosedError: - self.log(f"Failed to send packet: Connection closed unexpectedly") - - # Sets the client's username and enables pmsg, pvar, direct, and link/unlink commands. - async def set_username(self, username): - if not self.client: - return - - if not self.client.open: - return - - if self.username_set: - return - - await self.send_packet(cmd="setid", val=username, listener=self.setid_listener, - quirk=self.supporter.quirk_embed_val) - - # Gives a ping, gets a pong. Keeps connections alive and healthy. This is recommended for servers with tunnels/reverse proxies that have connection timeouts. - async def ping(self, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - if listener: - await self.send_packet(cmd="ping", listener=listener) - else: - await self.send_packet(cmd="ping", listener=self.ping_listener) - - # Sends global message packets. - async def send_gmsg(self, value, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - await self.send_packet(cmd="gmsg", val=value, listener=listener, quirk=self.supporter.quirk_embed_val) - - # Sends global variable packets. This will sync with all Scratch cloud variables. - async def send_gvar(self, name, value, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - payload = { - "name": name, - "val": value - } - - await self.send_packet(cmd="gvar", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends private message packets. - async def send_pmsg(self, recipient, value, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - payload = { - "id": recipient, - "val": value - } - - await self.send_packet(cmd="pmsg", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends private variable packets. This does not sync with Scratch. - async def send_pvar(self, name, recipient, value, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - payload = { - "name": name, - "id": recipient, - "val": value - } - - await self.send_packet(cmd="pvar", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends direct variable packets. - async def send_direct(self, recipient: any = None, value: any = None, listener: any = None): - if not self.client: - return - - if not self.client.open: - return - - payload = dict() - if recipient: - payload["id"] = recipient - - if value: - payload.update(value) - - await self.send_packet(cmd="direct", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Binding method callbacks - Provides a programmer-friendly interface to run extra code after a cloudlink method has been executed. - def bind_callback_method(self, callback_method: type, function: type): - if hasattr(self.cl_methods, callback_method.__name__) or hasattr(self.custom_methods, callback_method.__name__): - if callback_method.__name__ not in self.method_callbacks: - self.method_callbacks[callback_method.__name__] = set() - self.method_callbacks[callback_method.__name__].add(function) - - # Binding listener callbacks - Provides a programmer-friendly interface to run extra code when a message listener was detected. - def bind_callback_listener(self, listener_str: str, function: type): - if listener_str not in self.listener_callbacks: - self.listener_callbacks[listener_str] = set() - self.listener_callbacks[listener_str].add(function) - - # Binding events - Provides a programmer-friendly interface to detect client connects, disconnects, and errors. - def bind_event(self, event_method: type, function: type): - if hasattr(self.events, event_method.__name__): - if event_method.__name__ not in self.event_callbacks: - self.event_callbacks[event_method.__name__] = set() - self.event_callbacks[event_method.__name__].add(function) - - # == Client functionality == - - def __fire_method_callbacks__(self, callback_method, message, listener): - if callback_method.__name__ in self.method_callbacks: - for _method in self.method_callbacks[callback_method.__name__]: - self.asyncio.create_task(_method(self, message, listener)) - - def __fire_method_listeners__(self, listener_str, message): - if listener_str in self.listener_callbacks: - for _method in self.listener_callbacks[listener_str]: - self.asyncio.create_task(_method(self, message)) - - def __fire_event__(self, event_method: type): - if event_method.__name__ in self.event_callbacks: - for _method in self.event_callbacks[event_method.__name__]: - self.asyncio.create_task(_method(self)) - - async def __method_handler__(self, message): - # Check if the message contains the cmd key, with a string datatype. - if self.validate({"cmd": str}, message) != self.supporter.valid: - return - - # Detect listeners - listener = self.detect_listener(message) - - # Detect and convert CLPv3 custom responses to CLPv4 - if (message["cmd"] == "direct") and ("val" in message): - if self.validate({"cmd": str, "val": self.supporter.keydefaults["val"]}, - message["val"]) == self.supporter.valid: - tmp_msg = { - "cmd": message["val"]["cmd"], - "val": message["val"]["val"] - } - message = self.copy(tmp_msg) - - # Check if the command is disabled - if message["cmd"] in self.disabled_methods: - return - - method = None - - # Check if the command method exists - if hasattr(self.cl_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.cl_methods, message["cmd"]) - - elif hasattr(self.custom_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.custom_methods, message["cmd"]) - - if method: - # Run the method - await method(message, listener) - self.__fire_event__(self.events.on_msg) - - async def __session__(self, ip): - try: - async with self.websockets.connect(ip) as self.client: - await self.send_packet(cmd="handshake", listener=self.handshake_listener) - try: - while self.client.open: - try: - message = json.loads(await self.client.recv()) - await self.__method_handler__(message) - except: - pass - finally: - self.__fire_event__(self.events.on_close) - except Exception as e: - self.log(f"Closing connection handler due to error: {e}") - self.__fire_event__(self.events.on_fail) - finally: - pass - - -# Class for binding events -class events: - def __init__(self): - pass - - def on_connect(self): - pass - - def on_msg(self): - pass - - def on_error(self): - pass - - def on_close(self): - pass - - def on_fail(self): - pass - - def on_username_set(self): - pass - - def on_pong(self): - pass - - -# Class for managing the Cloudlink Protocol -class cl_methods: - def __init__(self, parent): - self.parent = parent - self.supporter = parent.supporter - self.copy = parent.copy - self.rooms = parent.rooms - self.log = parent.log - self.loop = None - - async def gmsg(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.global_data_value = message["val"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.gmsg, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - async def pmsg(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.private_data_value["val"] = message["val"] - room_data.private_data_value["origin"] = message["origin"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.pmsg, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - async def gvar(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.global_vars[message["name"]] = message["val"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.gvar, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - async def pvar(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.private_vars[message["name"]] = { - "origin": message["origin"], - "val": message["val"] - } - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.pvar, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - async def direct(self, message, listener): - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.direct, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - async def ulist(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Interpret and execute ulist method - if "mode" in message: - if message["mode"] in ["set", "add", "remove"]: - match message["mode"]: - case "set": - room_data.userlist = message["val"] - case "add": - if not message["val"] in room_data.userlist: - room_data.userlist.append(message["val"]) - case "remove": - if message["val"] in room_data.userlist: - room_data.userlist.remove(message["val"]) - else: - self.log(f"Could not understand ulist method: {message['mode']}") - else: - # Assume old userlist method - room_data.userlist = set(message["val"]) - - # ulist will never return a listener - self.parent.__fire_method_callbacks__(self.ulist, message, listener) - - async def server_version(self, message, listener): - self.parent.server_version = message['val'] - - # server_version will never return a listener - self.parent.__fire_method_callbacks__(self.server_version, message, listener) - - async def motd(self, message, listener): - self.parent.motd_message = message['val'] - - # motd will never return a listener - self.parent.__fire_method_callbacks__(self.motd, message, listener) - - async def client_ip(self, message, listener): - self.parent.client_ip = message['val'] - - # client_ip will never return a listener - self.parent.__fire_method_callbacks__(self.client_ip, message, listener) - - async def statuscode(self, message, listener): - if listener: - if listener in [self.parent.handshake_listener, self.parent.setid_listener]: - human_ok, machine_ok = self.supporter.generate_statuscode("OK") - match listener: - case self.parent.handshake_listener: - if (message["code"] == human_ok) or (message["code_id"] == machine_ok): - self.parent.__fire_event__(self.parent.events.on_connect) - else: - await self.parent.shutdown() - case self.parent.setid_listener: - if (message["code"] == human_ok) or (message["code_id"] == machine_ok): - self.parent.username_set = True - self.parent.client_id = message["val"] - self.parent.__fire_event__(self.parent.events.on_username_set) - case self.parent.ping_listener: - self.parent.__fire_event__(self.parent.events.on_pong) - else: - self.parent.__fire_method_callbacks__(self.statuscode, message, listener) - self.parent.__fire_method_listeners__(listener) - else: - self.parent.__fire_method_callbacks__(self.statuscode, message, listener) - - -# Class to store custom methods -class custom_methods: - def __init__(self): - pass - - -# Class to store room data -class rooms: - def __init__(self, parent): - self.default = self.__room__() - self.__parent__ = parent - - def get_all(self): - tmp = self.__parent__.copy(self.__dict__) - - # Remove attributes that aren't client objects - del tmp["__parent__"] - - return tmp - - def exists(self, room_id: str): - return hasattr(self, str(room_id)) - - def create(self, room_id: str): - if not self.exists(str(room_id)): - setattr(self, str(room_id), self.__room__()) - - def delete(self, room_id: str): - if self.exists(str(room_id)): - delattr(self, str(room_id)) - - def get(self, room_id: str): - if self.exists(str(room_id)): - return getattr(self, str(room_id)) - else: - return None - - class __room__: - def __init__(self): - # Global data stream current value - self.global_data_value = str() - - # Private data stream current value - self.private_data_value = { - "origin": str(), - "val": None - } - - # Storage of all global variables / Scratch Cloud Variables - self.global_vars = dict() - - # Storage of all private variables - self.private_vars = dict() - - # User management - self.userlist = list() diff --git a/cloudlink/async_iterables.py b/cloudlink/async_iterables.py new file mode 100644 index 0000000..a882a59 --- /dev/null +++ b/cloudlink/async_iterables.py @@ -0,0 +1,29 @@ +""" +async_iterable - converts a list or set of methods into an asyncio iterable +which can be used in the async for function. + +to use, init the class with the server parent and the list/set of functions. + +import async_iterable +... +async for event in async_iterable(parent, [foo, bar]): + await event() +""" + + +class async_iterable: + def __init__(self, iterables): + self.iterator = 0 + self.iterable = list(iterables) + + def __aiter__(self): + return self + + async def __anext__(self): + if self.iterator >= len(self.iterable): + self.iterator = 0 + raise StopAsyncIteration + + self.iterator += 1 + + return self.iterable[self.iterator - 1] diff --git a/cloudlink/client/__init__.py b/cloudlink/client/__init__.py new file mode 100644 index 0000000..00a869b --- /dev/null +++ b/cloudlink/client/__init__.py @@ -0,0 +1,574 @@ +# Core components of the CloudLink client +import asyncio +import ssl +import cerberus +import logging +import time +from copy import copy + +# Import websockets and SSL support +import websockets + +# Import shared module +from ..async_iterables import async_iterable + +# Import JSON library - Prefer UltraJSON but use native JSON if failed +try: + import ujson +except Exception as e: + print(f"Client failed to import UltraJSON, failing back to native JSON library. Exception code: {e}") + import json as ujson + +# Import required CL4 client protocol +from . import protocol, schema + + +# Define server exceptions +class exceptions: + class EmptyMessage(Exception): + """This exception is raised when a client receives an empty packet.""" + pass + + class UnknownCommand(Exception): + """This exception is raised when the server sends a command that the client does not recognize.""" + pass + + class JSONError(Exception): + """This exception is raised when the client fails to parse the server message's JSON.""" + pass + + class ValidationError(Exception): + """This exception is raised when the server sends a message that fails validation before execution.""" + pass + + class InternalError(Exception): + """This exception is raised when an unexpected and/or unhandled exception is raised.""" + pass + + class ListenerExists(Exception): + """This exception is raised when attempting to process a listener that already has an existing listener instance.""" + pass + + +# Main server +class client: + def __init__(self): + self.version = "0.2.0" + + # Logging + self.logging = logging + self.logger = self.logging.getLogger(__name__) + + # Asyncio + self.asyncio = asyncio + + # Configure websocket framework + self.ws = websockets + self.client = None + + # Components + self.ujson = ujson + self.validator = cerberus.Validator + self.async_iterable = async_iterable + self.exceptions = exceptions + self.copy = copy + + # Create event managers + self.on_initial_connect_events = set() + self.on_full_connect_events = set() + self.on_message_events = set() + self.on_disconnect_events = set() + self.on_error_events = set() + self.exception_handlers = dict() + self.listener_events_await_specific = dict() + self.listener_events_decorator_specific = dict() + self.listener_responses = dict() + self.on_username_set_events = set() + + # Prepare command event handlers + self.protocol_command_handlers = dict() + for cmd in [ + "ping", + "gmsg", + "gvar", + "pmsg", + "pvar", + "statuscode", + "client_obj", + "client_ip", + "server_version", + "ulist", + "direct" + ]: + self.protocol_command_handlers[cmd] = set() + + # Create method handlers + self.command_handlers = dict() + + # Configure framework logging + self.suppress_websocket_logs = True + + # Configure SSL support + self.ssl_enabled = False + self.ssl_context = None + + # Load built-in protocol + self.schema = schema.schema + self.protocol = protocol.clpv4(self) + + # Enables SSL support + def enable_ssl(self, certfile): + try: + self.ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + self.ssl_context.load_verify_locations(certfile) + self.ssl_enabled = True + self.logger.info(f"SSL support initialized!") + except Exception as e: + self.logger.error(f"Failed to initialize SSL support! {e}") + + # Runs the client. + def run(self, host="ws://127.0.0.1:3000"): + try: + # Startup message + self.logger.info(f"CloudLink {self.version} - Now connecting to {host}") + + # Suppress websocket library logging + if self.suppress_websocket_logs: + self.logging.getLogger('asyncio').setLevel(self.logging.ERROR) + self.logging.getLogger('asyncio.coroutines').setLevel(self.logging.ERROR) + self.logging.getLogger('websockets.client').setLevel(self.logging.ERROR) + self.logging.getLogger('websockets.protocol').setLevel(self.logging.ERROR) + + # Start client + self.asyncio.run(self.__run__(host)) + + except KeyboardInterrupt: + pass + + # Event binder for on_command events + def on_command(self, cmd): + def bind_event(func): + + # Create command event handler + if cmd not in self.command_handlers: + self.command_handlers[cmd] = set() + + # Add function to the command handler + self.command_handlers[cmd].add(func) + + # End on_command binder + return bind_event + + # Credit to @ShowierData9978 for this: Listen for messages containing specific "listener" keys + async def wait_for_listener(self, listener_id): + # Prevent listener collision + if listener_id in self.listener_events_await_specific: + raise self.exceptions.ListenerExists(f"The listener {listener_id} is already being awaited. Please use a different listener ID.") + + # Create a new event object. + event = self.asyncio.Event() + + # Register the event so that the client can continue listening for messages. + self.listener_events_await_specific[listener_id] = event + + # Create the waiter task. + task = self.asyncio.create_task( + self.listener_waiter( + listener_id, + event + ) + ) + + # Wait for the waiter task to finish. + await task + + # Get the response + response = self.copy(self.listener_responses[listener_id]) + + # Remove from the listener events dict. + self.listener_events_await_specific.pop(listener_id) + + # Free up listener responses + self.listener_responses.pop(listener_id) + + # Return the response + return response + + def on_username_set(self, func): + self.on_username_set_events.add(func) + + # Version of the wait for listener tool for decorator usage. + def on_listener(self, listener_id): + def bind_event(func): + + # Create listener event handler + if listener_id not in self.listener_events_decorator_specific: + self.listener_events_decorator_specific[listener_id] = set() + + # Add function to the listener handler + self.listener_events_decorator_specific[listener_id].add(func) + + # End on_listener binder + return bind_event + + # Event binder for on_error events with specific schemas/exception types + def on_exception(self, exception_type): + def bind_event(func): + + # Create error event handler + if exception_type not in self.exception_handlers: + self.exception_handlers[exception_type] = set() + + # Add function to the error command handler + self.exception_handlers[exception_type].add(func) + + # End on_error_specific binder + return bind_event + + # Event binder for on_message events + def on_message(self, func): + self.on_message_events.add(func) + + # Event binder for starting up the client. + def on_initial_connect(self, func): + self.on_initial_connect_events.add(func) + + # Event binder for on_connect events. + def on_connect(self, func): + self.on_full_connect_events.add(func) + + # Event binder for on_disconnect events. + def on_disconnect(self, func): + self.on_disconnect_events.add(func) + + # Event binder for on_error events. + def on_error(self, func): + self.on_error_events.add(func) + + # CL4 client-specific command events + + # Event binder for gmsg events. + def on_gmsg(self, func): + self.logger.debug(f"Binding function {func.__name__} to gmsg command event manager") + self.protocol_command_handlers["gmsg"].add(func) + + # Event binder for pmsg events. + def on_pmsg(self, func): + self.logger.debug(f"Binding function {func.__name__} to pmsg command event manager") + self.protocol_command_handlers["pmsg"].add(func) + + # Event binder for gvar events. + def on_gvar(self, func): + self.logger.debug(f"Binding function {func.__name__} to gvar command event manager") + self.protocol_command_handlers["gvar"].add(func) + + # Event binder for pvar events. + def on_pvar(self, func): + self.logger.debug(f"Binding function {func.__name__} to pvar command event manager") + self.protocol_command_handlers["pvar"].add(func) + + # Event binder for direct events. + def on_direct(self, func): + self.logger.debug(f"Binding function {func.__name__} to direct command event manager") + self.protocol_command_handlers["direct"].add(func) + + # Event binder for statuscode events. + def on_statuscode(self, func): + self.logger.debug(f"Binding function {func.__name__} to statuscode command event manager") + self.protocol_command_handlers["statuscode"].add(func) + + # Event binder for client_obj events. + def on_client_obj(self, func): + self.logger.debug(f"Binding function {func.__name__} to client_obj command event manager") + self.protocol_command_handlers["client_obj"].add(func) + + # Event binder for client_ip events. + def on_client_ip(self, func): + self.logger.debug(f"Binding function {func.__name__} to client_ip command event manager") + self.protocol_command_handlers["client_ip"].add(func) + + # Event binder for server_version events. + def on_server_version(self, func): + self.logger.debug(f"Binding function {func.__name__} to server_version command event manager") + self.protocol_command_handlers["server_version"].add(func) + + # Event binder for ulist events. + def on_ulist(self, func): + self.logger.debug(f"Binding function {func.__name__} to ulist command event manager") + self.protocol_command_handlers["ulist"].add(func) + + # Send message + def send_packet(self, message): + self.asyncio.create_task(self.execute_send(message)) + + # Send message and wait for a response + async def send_packet_and_wait(self, message): + self.logger.debug(f"Sending message containing listener {message['listener']}...") + await self.execute_send(message) + response = await self.wait_for_listener(message["listener"]) + return response + + # Close the connection + def disconnect(self, code=1000, reason=""): + self.asyncio.create_task(self.execute_disconnect(code, reason)) + + # Message processor + async def message_processor(self, message): + + # Empty packet + if not len(message): + self.logger.debug(f"Server sent empty message ") + + # Fire on_error events + asyncio.create_task(self.execute_on_error_events(self.exceptions.EmptyMessage)) + + # Fire exception handling events + self.asyncio.create_task( + self.execute_exception_handlers( + exception_type=self.exceptions.EmptyMessage, + details="Empty message" + ) + ) + + # End message_processor coroutine + return + + # Parse JSON in message and convert to dict + try: + message = self.ujson.loads(message) + + except Exception as error: + self.logger.debug(f"Server sent invalid JSON: {error}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(error)) + + # Fire exception handling events + if self.client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + exception_type=self.exceptions.JSONError, + details=error + ) + ) + + else: + # Close the connection + self.send_packet("Invalid JSON") + self.close_connection(reason="Invalid JSON") + + # End message_processor coroutine + return + + # Begin validation + validator = self.validator(self.schema.default, allow_unknown=True) + if not validator.validate(message): + errors = validator.errors + + # Log failed validation + self.logger.debug(f"Server sent message that failed validation: {errors}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(errors)) + + # Fire exception handling events + if self.client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + exception_type=self.exceptions.ValidationError, + details=errors + ) + ) + + # End message_processor coroutine + return + + # Check if command exists + if message["cmd"] not in self.command_handlers: + + # Log invalid command + self.logger.debug(f"Server sent an unknown command \"{message['cmd']}\"") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events("Unknown command")) + + # Fire exception handling events + if self.client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + exception_type=self.exceptions.InvalidCommand, + details=message["cmd"] + ) + ) + + # End message_processor coroutine + return + + # Check if the message contains listeners + if "listener" in message: + if message["listener"] in self.listener_events_await_specific: + # Fire awaiting listeners + self.logger.debug(f"Received message containing listener {message['listener']}!") + self.listener_responses[message["listener"]] = message + self.listener_events_await_specific[message["listener"]].set() + + elif message["listener"] in self.listener_events_decorator_specific: + # Fire all decorator-based listeners + self.asyncio.create_task( + self.execute_on_listener_events( + message + ) + ) + + # Fire on_command events + self.asyncio.create_task( + self.execute_on_command_events( + message + ) + ) + + # Fire on_message events + self.asyncio.create_task( + self.execute_on_message_events( + message + ) + ) + + # Connection handler + async def connection_handler(self): + + # Startup client attributes + self.client.snowflake = str() + self.client.protocol = None + self.client.protocol_set = False + self.client.rooms = set() + self.client.username_set = False + self.client.username = str() + self.client.handshake = False + + # Begin tracking the lifetime of the client + self.client.birth_time = time.monotonic() + + # Fire on_connect events + self.asyncio.create_task(self.execute_on_initial_connect_events()) + + self.logger.debug(f"Client connected") + + # Run connection loop + await self.connection_loop() + + # Fire on_disconnect events + self.asyncio.create_task(self.execute_on_disconnect_events()) + + self.logger.debug( + f"Client disconnected: Total lifespan of {time.monotonic() - self.client.birth_time} seconds.") + + # Connection loop - Redefine for use with another outside library + async def connection_loop(self): + # Primary asyncio loop for the lifespan of the websocket connection + try: + async for message in self.client: + # Start keeping track of processing time + start = time.perf_counter() + self.logger.debug(f"Now processing message from server...") + + # Process the message + await self.message_processor(message) + + # Log processing time + self.logger.debug( + f"Done processing message from server. Processing took {time.perf_counter() - start} seconds.") + + # Handle unexpected disconnects + except self.ws.exceptions.ConnectionClosedError: + pass + + # Handle OK disconnects + except self.ws.exceptions.ConnectionClosedOK: + pass + + # Catch any unexpected exceptions + except Exception as e: + self.logger.critical(f"Unexpected exception was raised: {e}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(f"Unexpected exception was raised: {e}")) + + # Fire exception handling events + self.asyncio.create_task( + self.execute_exception_handlers( + exception_type=self.exceptions.InternalError, + details=f"Unexpected exception was raised: {e}" + ) + ) + + # WebSocket-specific server loop + async def __run__(self, host): + async with self.ws.connect(host) as self.client: + await self.connection_handler() + + # Asyncio event-handling coroutines + + async def execute_on_username_set_events(self, id, username, uuid): + events = [event(id, username, uuid) for event in self.on_username_set_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_disconnect_events(self): + events = [event() for event in self.on_disconnect_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_initial_connect_events(self): + events = [event() for event in self.on_initial_connect_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_full_connect_events(self): + events = [event() for event in self.on_full_connect_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_message_events(self, message): + events = [event(message) for event in self.on_message_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_command_events(self, message): + events = [event(message) for event in self.command_handlers[message["cmd"]]] + group = self.asyncio.gather(*events) + await group + + async def execute_on_listener_events(self, message): + events = [event(message) for event in self.listener_events_decorator_specific[message["listener"]]] + group = self.asyncio.gather(*events) + await group + + async def execute_on_error_events(self, errors): + events = [event(errors) for event in self.on_error_events] + group = self.asyncio.gather(*events) + await group + + async def execute_exception_handlers(self, exception_type, details): + # Guard clauses + if exception_type not in self.exception_handlers: + return + + # Fire events + events = [event(details) for event in self.exception_handlers[exception_type]] + group = self.asyncio.gather(*events) + await group + + async def listener_waiter(self, listener_id, event): + await event.wait() + + # WebSocket-specific coroutines + + async def execute_disconnect(self, code=1000, reason=""): + await self.client.close(code, reason) + + async def execute_send(self, message): + # Convert dict to JSON + if type(message) == dict: + message = self.ujson.dumps(message) + await self.client.send(message) diff --git a/cloudlink/client/protocol.py b/cloudlink/client/protocol.py new file mode 100644 index 0000000..ce6a23b --- /dev/null +++ b/cloudlink/client/protocol.py @@ -0,0 +1,178 @@ +""" +This is the default protocol used for the CloudLink client. +The CloudLink 4.1 Protocol retains full support for CLPv4. + +Each packet format is compliant with UPLv2 formatting rules. + +Documentation for the CLPv4.1 protocol can be found here: +https://hackmd.io/@MikeDEV/HJiNYwOfo +""" + + +class clpv4: + def __init__(self, parent): + + # Define various status codes for the protocol. + class statuscodes: + # Code type character + info = "I" + error = "E" + + # Error / info codes as tuples + test = (info, 0, "Test") + echo = (info, 1, "Echo") + ok = (info, 100, "OK") + syntax = (error, 101, "Syntax") + datatype = (error, 102, "Datatype") + id_not_found = (error, 103, "ID not found") + id_not_specific = (error, 104, "ID not specific enough") + internal_error = (error, 105, "Internal server error") + empty_packet = (error, 106, "Empty packet") + id_already_set = (error, 107, "ID already set") + refused = (error, 108, "Refused") + invalid_command = (error, 109, "Invalid command") + disabled_command = (error, 110, "Command disabled") + id_required = (error, 111, "ID required") + id_conflict = (error, 112, "ID conflict") + too_large = (error, 113, "Too large") + json_error = (error, 114, "JSON error") + room_not_joined = (error, 115, "Room not joined") + + # Generate a user object + def generate_user_object(): + # Username set + if parent.client.username_set: + return { + "id": parent.client.snowflake, + "username": parent.client.username, + "uuid": str(parent.client.id) + } + + # Username not set + return { + "id": parent.client.snowflake, + "uuid": str(parent.client.id) + } + + # Expose username object generator function for extension usage + self.generate_user_object = generate_user_object + + async def set_username(username): + parent.logger.debug(f"Setting username to {username}...") + + # Send the set username request with a listener and wait for a response + response = await parent.send_packet_and_wait({ + "cmd": "setid", + "val": username, + "listener": "init_username" + }) + + if response["code_id"] == statuscodes.ok[1]: + # Log the successful connection + parent.logger.info(f"Successfully set username to {username}.") + + # Fire all on_connect events + val = response["val"] + parent.asyncio.create_task( + parent.execute_on_username_set_events(val["id"], val["username"], val["uuid"]) + ) + + else: + # Log the connection error + parent.logger.error(f"Failed to set username. Got response code: {response['code']}") + + # Expose the username set command + self.set_username = set_username + + # The CLPv4 command set + @parent.on_initial_connect + async def on_initial_connect(): + parent.logger.debug("Performing handshake with the server...") + + # Send the handshake request with a listener and wait for a response + response = await parent.send_packet_and_wait({ + "cmd": "handshake", + "val": { + "language": "Python", + "version": parent.version + }, + "listener": "init_handshake" + }) + + if response["code_id"] == statuscodes.ok[1]: + # Log the successful connection + parent.logger.info("Successfully connected to the server.") + + # Fire all on_connect events + parent.asyncio.create_task( + parent.execute_on_full_connect_events() + ) + + else: + # Log the connection error + parent.logger.error(f"Failed to connect to the server. Got response code: {response['code']}") + + # Disconnect + parent.asyncio.create_task( + parent.disconnect() + ) + + @parent.on_command(cmd="ping") + async def on_ping(message): + events = [event(message) for event in parent.protocol_command_handlers["ping"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="gmsg") + async def on_gmsg(message): + events = [event(message) for event in parent.protocol_command_handlers["gmsg"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="pmsg") + async def on_pmsg(message): + events = [event(message) for event in parent.protocol_command_handlers["pmsg"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="gvar") + async def on_gvar(message): + events = [event(message) for event in parent.protocol_command_handlers["gvar"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="pvar") + async def on_pvar(message): + events = [event(message) for event in parent.protocol_command_handlers["pvar"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="statuscode") + async def on_statuscode(message): + events = [event(message) for event in parent.protocol_command_handlers["statuscode"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="client_obj") + async def on_client_obj(message): + parent.logger.info(f"This client is known as ID {message['val']['id']} with UUID {message['val']['uuid']}.") + + @parent.on_command(cmd="client_ip") + async def on_client_ip(message): + parent.logger.debug(f"Client IP address is {message['val']}") + + @parent.on_command(cmd="server_version") + async def on_server_version(message): + parent.logger.info(f"Server is running Cloudlink v{message['val']}.") + + @parent.on_command(cmd="ulist") + async def on_ulist(message): + events = [event(message) for event in parent.protocol_command_handlers["ulist"]] + group = parent.asyncio.gather(*events) + await group + + @parent.on_command(cmd="direct") + async def on_direct(message): + events = [event(message) for event in parent.protocol_command_handlers["direct"]] + group = parent.asyncio.gather(*events) + await group diff --git a/cloudlink/client/schema.py b/cloudlink/client/schema.py new file mode 100644 index 0000000..ddfabbe --- /dev/null +++ b/cloudlink/client/schema.py @@ -0,0 +1,352 @@ +# Schema for interpreting the Cloudlink protocol v4.0 (CLPv4) command set +class schema: + + # Required - Defines the keyword to use to define the command + command_key = "cmd" + + # Required - Defines the default schema to test against + default = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": False, + }, + "mode": { + "type": "string", + "required": False + }, + "code": { + "type": "string", + "required": False + }, + "code_id": { + "type": "integer", + "required": False + }, + "name": { + "type": "string", + "required": False + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": False + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + linking = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True, + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + setid = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": "string", + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + gmsg = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + gvar = { + "cmd": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + pmsg = { + "cmd": { + "type": "string", + "required": True + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + direct = { + "cmd": { + "type": "string", + "required": True + }, + "id": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + pvar = { + "cmd": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": True + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } \ No newline at end of file diff --git a/cloudlink/cloudlink.py b/cloudlink/cloudlink.py deleted file mode 100644 index bc9968b..0000000 --- a/cloudlink/cloudlink.py +++ /dev/null @@ -1,52 +0,0 @@ -from .supporter import supporter - -""" -CloudLink 4.0 Server and Client - -CloudLink is a free and open-source, websocket-powered API optimized for Scratch 3.0. -For documentation, please visit https://hackmd.io/g6BogABhT6ux1GA2oqaOXA - -Cloudlink is built upon https://github.com/aaugustin/websockets. - -Please see https://github.com/MikeDev101/cloudlink for more details. - -Cloudlink's dependencies are: -* websockets (for server and asyncio client) -* websocket-client (for non-asyncio client) - -These dependencies are built-in to Python. -* copy -* asyncio -* traceback -* datetime -* json -""" - - -class cloudlink: - def __init__(self): - self.version = "0.1.9.2" - self.supporter = supporter - - def server(self, logs: bool = False): - # Initialize Cloudlink server - from .server import server - return server(self, logs) - - def client(self, logs: bool = False, async_client: bool = True): - # Initialize Cloudlink client - if async_client: - from .async_client import async_client - return async_client.client(self, logs) - else: - from .old_client import old_client - return old_client.client(self, logs) - - def multi_client(self, logs: bool = False, async_client: bool = True): - # Initialize Cloudlink client - if async_client: - from .async_client import async_client - return async_client.multi_client(self, logs) - else: - from .old_client import old_client - return old_client.multi_client(self, logs) diff --git a/cloudlink/docs/docs.txt b/cloudlink/docs/docs.txt deleted file mode 100644 index c7f9328..0000000 --- a/cloudlink/docs/docs.txt +++ /dev/null @@ -1 +0,0 @@ -https://hackmd.io/g6BogABhT6ux1GA2oqaOXA \ No newline at end of file diff --git a/cloudlink/old_client/__init__.py b/cloudlink/old_client/__init__.py deleted file mode 100644 index 3193e87..0000000 --- a/cloudlink/old_client/__init__.py +++ /dev/null @@ -1 +0,0 @@ -from .old_client import * \ No newline at end of file diff --git a/cloudlink/old_client/old_client.py b/cloudlink/old_client/old_client.py deleted file mode 100644 index 5ce66e2..0000000 --- a/cloudlink/old_client/old_client.py +++ /dev/null @@ -1,601 +0,0 @@ -import websocket as old_websockets -import json -from copy import copy -import time -import threading - - -# Multi client -class multi_client: - def __init__(self, parent, logs: bool = True): - self.shutdown_flag = False - self.threads = set() - self.logs = logs - self.parent = parent - self.supporter = self.parent.supporter(self) - - self.clients_counter = int() - self.clients_present = set() - self.clients_dead = set() - - # Display version - self.supporter.log(f"Cloudlink non-asyncio multi client v{self.parent.version}") - - def on_connect(self, client): - self.clients_present.add(client) - self.clients_dead.discard(client) - while not self.shutdown_flag: - time.sleep(0) - client.shutdown() - - def on_disconnect(self, client): - self.clients_present.discard(client) - self.clients_dead.add(client) - - def spawn(self, client_id: any, url: str = "ws://127.0.0.1:3000/"): - _client = client(self.parent, logs=self.logs, multi_silent=True) - _client.obj_id = client_id - _client.bind_event(_client.events.on_connect, self.on_connect) - _client.bind_event(_client.events.on_close, self.on_disconnect) - _client.bind_event(_client.events.on_fail, self.on_disconnect) - self.threads.add(threading.Thread( - target=_client.run, - args=[url], - daemon=True - )) - self.clients_counter = len(self.threads) - return _client - - def run(self): - self.supporter.log("Initializing all clients...") - for thread in self.threads: - thread.start() - while (len(self.clients_present) + len(self.clients_dead)) != self.clients_counter: - pass - self.supporter.log("All clients have initialized.") - - def stop(self): - self.shutdown_flag = True - self.clients_counter = self.clients_counter - len(self.clients_dead) - self.supporter.log("Waiting for all clients to shutdown...") - while len(self.clients_present) != 0: - pass - self.supporter.log("All clients have shut down.") - self.clients_present.clear() - self.clients_dead.clear() - self.threads.clear() - - -# Cloudlink Client -class client: - def __init__(self, parent, logs: bool = True, multi_silent:bool = False): - self.version = parent.version - - # Locally define the client object - self.client = None - - self.websockets = old_websockets - self.copy = copy - self.json = json - - # Other - self.enable_logs = logs - self.client_id = dict() - self.client_ip = str() - self.running = False - self.username_set = False - - # Built-in listeners - self.handshake_listener = "protocolset" - self.setid_listener = "usernameset" - self.ping_listener = "ping" - - # Storage of server stuff - self.motd_message = str() - self.server_version = str() - - # Managing methods - self.custom_methods = custom_methods() - self.disabled_methods = set() - self.method_callbacks = dict() - self.listener_callbacks = dict() - self.safe_methods = set() - - # Managing events - self.events = events() - self.event_callbacks = dict() - - # Initialize supporter - self.supporter = parent.supporter(self) - - # Initialize attributes from supporter component - self.log = self.supporter.log - self.validate = self.supporter.validate - self.load_custom_methods = self.supporter.load_custom_methods - self.detect_listener = self.supporter.detect_listener - self.generate_statuscode = self.supporter.generate_statuscode - - # Initialize rooms storage - self.rooms = rooms(self) - - # Initialize methods - self.cl_methods = cl_methods(self) - - # Load safe CLPv4 methods to mitigate vulnerabilities - self.supporter.init_builtin_cl_methods() - - if not multi_silent: - # Display version - self.supporter.log(f"Cloudlink non-asyncio client v{parent.version}") - - # == Public API functionality == - - # Runs the client. - def run(self, ip: str = "ws://127.0.0.1:3000/"): - try: - self.client = old_websockets.WebSocketApp( - ip, - on_message=self.__session_on_message__, - on_error=self.__session_on_error__, - on_open=self.__session_on_connect__, - on_close=self.__session_on_close__ - ) - self.client.run_forever() - except Exception as e: - self.log(f"{e}: {self.supporter.full_stack()}") - - # Disconnects the client. - def shutdown(self): - if not self.client: - return - - self.client.close() - - # Sends packets, you should use other methods. - def send_packet(self, cmd: str, val: any = None, listener: str = None, room_id: str = None, - quirk: str = "quirk_embed_val"): - if not self.client: - return - - # Manage specific message quirks - message = {"cmd": cmd} - if val: - if quirk == self.supporter.quirk_update_msg: - message.update(val) - elif quirk == self.supporter.quirk_embed_val: - message["val"] = val - else: - raise TypeError("Unknown message quirk!") - - # Attach a listener request - if listener: - message["listener"] = listener - - # Attach the rooms key - if room_id: - message["rooms"] = room_id - - # Send payload - try: - self.client.send(self.json.dumps(message)) - except Exception as e: - self.log(f"Failed to send packet: {e}") - - # Sets the client's username and enables pmsg, pvar, direct, and link/unlink commands. - def set_username(self, username): - if not self.client: - return - - if self.username_set: - return - - self.send_packet(cmd="setid", val=username, listener=self.setid_listener, quirk=self.supporter.quirk_embed_val) - - # Gives a ping, gets a pong. Keeps connections alive and healthy. This is recommended for servers with tunnels/reverse proxies that have connection timeouts. - def ping(self, listener: any = None): - if not self.client: - return - - if listener: - self.send_packet(cmd="ping", listener=listener) - else: - self.send_packet(cmd="ping", listener=self.ping_listener) - - # Sends global message packets. - def send_gmsg(self, value, listener: any = None): - if not self.client: - return - - if listener: - self.send_packet(cmd="gmsg", val=value, listener=listener, quirk=self.supporter.quirk_embed_val) - else: - self.send_packet(cmd="gmsg", val=value, quirk=self.supporter.quirk_embed_val) - - # Sends global variable packets. This will sync with all Scratch cloud variables. - def send_gvar(self, name, value, listener: any = None): - if not self.client: - return - - payload = { - "name": name, - "val": value - } - - self.send_packet(cmd="gvar", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends private message packets. - def send_pmsg(self, recipient, value, listener: any = None): - if not self.client: - return - - payload = { - "id": recipient, - "val": value - } - - self.send_packet(cmd="pmsg", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends private variable packets. This does not sync with Scratch. - def send_pvar(self, name, recipient, value, listener: any = None): - if not self.client: - return - - payload = { - "name": name, - "id": recipient, - "val": value - } - - self.send_packet(cmd="pvar", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Sends direct variable packets. - def send_direct(self, recipient: any = None, value: any = None, listener: any = None): - if not self.client: - return - - payload = dict() - if recipient: - payload["id"] = recipient - - if value: - payload.update(value) - - self.send_packet(cmd="direct", val=payload, listener=listener, quirk=self.supporter.quirk_update_msg) - - # Binding method callbacks - Provides a programmer-friendly interface to run extra code after a cloudlink method has been executed. - def bind_callback_method(self, callback_method: type, function: type): - if hasattr(self.cl_methods, callback_method.__name__) or hasattr(self.custom_methods, callback_method.__name__): - if callback_method.__name__ not in self.method_callbacks: - self.method_callbacks[callback_method.__name__] = set() - self.method_callbacks[callback_method.__name__].add(function) - - # Binding listener callbacks - Provides a programmer-friendly interface to run extra code when a message listener was detected. - def bind_callback_listener(self, listener_str: str, function: type): - if listener_str not in self.listener_callbacks: - self.listener_callbacks[listener_str] = set() - self.listener_callbacks[listener_str].add(function) - - # Binding events - Provides a programmer-friendly interface to detect client connects, disconnects, and errors. - def bind_event(self, event_method: type, function: type): - if hasattr(self.events, event_method.__name__): - if event_method.__name__ not in self.event_callbacks: - self.event_callbacks[event_method.__name__] = set() - self.event_callbacks[event_method.__name__].add(function) - - # == Client functionality == - - def __fire_method_callbacks__(self, callback_method, message, listener): - if callback_method.__name__ in self.method_callbacks: - for _method in self.method_callbacks[callback_method.__name__]: - threading.Thread(target=_method, args=[self, message, listener], daemon=True).start() - - def __fire_method_listeners__(self, listener_str, message): - if listener_str in self.listener_callbacks: - for _method in self.listener_callbacks[listener_str]: - threading.Thread(target=_method, args=[self, message], daemon=True).start() - - def __fire_event__(self, event_method: type): - if event_method.__name__ in self.event_callbacks: - for _method in self.event_callbacks[event_method.__name__]: - threading.Thread(target=_method, args=[self], daemon=True).start() - - def __method_handler__(self, message): - # Check if the message contains the cmd key, with a string datatype. - if self.validate({"cmd": str}, message) != self.supporter.valid: - return - - # Detect listeners - listener = self.detect_listener(message) - - # Detect and convert CLPv3 custom responses to CLPv4 - if (message["cmd"] == "direct") and ("val" in message): - if self.validate({"cmd": str, "val": self.supporter.keydefaults["val"]}, - message["val"]) == self.supporter.valid: - tmp_msg = { - "cmd": message["val"]["cmd"], - "val": message["val"]["val"] - } - message = self.copy(tmp_msg) - - # Check if the command is disabled - if message["cmd"] in self.disabled_methods: - return - - method = None - - # Check if the command method exists - if hasattr(self.cl_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.cl_methods, message["cmd"]) - - elif hasattr(self.custom_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.custom_methods, message["cmd"]) - - if method: - # Run the method - method(message, listener) - self.__fire_event__(self.events.on_msg) - - def __session_on_message__(self, ws, message): - try: - message = json.loads(message) - self.__method_handler__(message) - except: - pass - - def __session_on_connect__(self, ws): - self.send_packet(cmd="handshake", listener=self.handshake_listener) - - def __session_on_error__(self, ws, error): - self.log(f"Connection handler encountered an error: {error}") - self.__fire_event__(self.events.on_error) - - def __session_on_close__(self, ws, close_status_code, close_msg): - self.__fire_event__(self.events.on_close) - -# Class for binding events -class events: - def __init__(self): - pass - - def on_connect(self): - pass - - def on_msg(self): - pass - - def on_error(self): - pass - - def on_close(self): - pass - - def on_fail(self): - pass - - def on_username_set(self): - pass - - def on_pong(self): - pass - - -# Class for managing the Cloudlink Protocol -class cl_methods: - def __init__(self, parent): - self.parent = parent - self.supporter = parent.supporter - self.copy = parent.copy - self.rooms = parent.rooms - self.log = parent.log - - def gmsg(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.global_data_value = message["val"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.gmsg, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - def pmsg(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.private_data_value["val"] = message["val"] - room_data.private_data_value["origin"] = message["origin"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.pmsg, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - def gvar(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.global_vars[message["name"]] = message["val"] - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.gvar, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - def pvar(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Update the room data - room_data.private_vars[message["name"]] = { - "origin": message["origin"], - "val": message["val"] - } - - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.pvar, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - def direct(self, message, listener): - # Fire callbacks for method and it's listener - self.parent.__fire_method_callbacks__(self.direct, message, listener) - if listener: - self.parent.__fire_method_listeners__(listener) - - def ulist(self, message, listener): - room_data = None - if "rooms" in message: - # Automatically create room and update the global data value - self.rooms.create(message["rooms"]) - room_data = self.rooms.get(message["rooms"]) - else: - # Assume a gmsg with no "rooms" value is the default room - room_data = self.rooms.get("default") - - # Interpret and execute ulist method - if "mode" in message: - if message["mode"] in ["set", "add", "remove"]: - match message["mode"]: - case "set": - room_data.userlist = message["val"] - case "add": - if not message["val"] in room_data.userlist: - room_data.userlist.append(message["val"]) - case "remove": - if message["val"] in room_data.userlist: - room_data.userlist.remove(message["val"]) - else: - self.log(f"Could not understand ulist method: {message['mode']}") - else: - # Assume old userlist method - room_data.userlist = set(message["val"]) - - # ulist will never return a listener - self.parent.__fire_method_callbacks__(self.ulist, message, listener) - - def server_version(self, message, listener): - self.parent.server_version = message['val'] - - # server_version will never return a listener - self.parent.__fire_method_callbacks__(self.server_version, message, listener) - - def motd(self, message, listener): - self.parent.motd_message = message['val'] - - # motd will never return a listener - self.parent.__fire_method_callbacks__(self.motd, message, listener) - - def client_ip(self, message, listener): - self.parent.client_ip = message['val'] - - # client_ip will never return a listener - self.parent.__fire_method_callbacks__(self.client_ip, message, listener) - - def statuscode(self, message, listener): - if listener: - if listener in [self.parent.handshake_listener, self.parent.setid_listener]: - human_ok, machine_ok = self.supporter.generate_statuscode("OK") - match listener: - case self.parent.handshake_listener: - if (message["code"] == human_ok) or (message["code_id"] == machine_ok): - self.parent.__fire_event__(self.parent.events.on_connect) - else: - self.parent.shutdown() - case self.parent.setid_listener: - if (message["code"] == human_ok) or (message["code_id"] == machine_ok): - self.parent.username_set = True - self.parent.client_id = message["val"] - self.parent.__fire_event__(self.parent.events.on_username_set) - case self.parent.ping_listener: - self.parent.__fire_event__(self.parent.events.on_pong) - else: - self.parent.__fire_method_callbacks__(self.statuscode, message, listener) - self.parent.__fire_method_listeners__(listener) - else: - self.parent.__fire_method_callbacks__(self.statuscode, message, listener) - - -# Class to store custom methods -class custom_methods: - def __init__(self): - pass - - -# Class to store room data -class rooms: - def __init__(self, parent): - self.default = self.__room__() - self.__parent__ = parent - - def get_all(self): - tmp = self.__parent__.copy(self.__dict__) - - # Remove attributes that aren't client objects - del tmp["__parent__"] - - return tmp - - def exists(self, room_id: str): - return hasattr(self, str(room_id)) - - def create(self, room_id: str): - if not self.exists(str(room_id)): - setattr(self, str(room_id), self.__room__()) - - def delete(self, room_id: str): - if self.exists(str(room_id)): - delattr(self, str(room_id)) - - def get(self, room_id: str): - if self.exists(str(room_id)): - return getattr(self, str(room_id)) - else: - return None - - class __room__: - def __init__(self): - # Global data stream current value - self.global_data_value = str() - - # Private data stream current value - self.private_data_value = { - "origin": str(), - "val": None - } - - # Storage of all global variables / Scratch Cloud Variables - self.global_vars = dict() - - # Storage of all private variables - self.private_vars = dict() - - # User management - self.userlist = list() diff --git a/cloudlink/server/__init__.py b/cloudlink/server/__init__.py index 2787d2b..8067c72 100644 --- a/cloudlink/server/__init__.py +++ b/cloudlink/server/__init__.py @@ -1 +1,730 @@ -from .server import * \ No newline at end of file +# Core components of the CloudLink server +import asyncio +import cerberus +import logging +import time +from copy import copy +from snowflake import SnowflakeGenerator + +# Import websockets and SSL support +import websockets +import ssl + +# Import shared modules +from ..async_iterables import async_iterable + +# Import server-specific modules +from .modules.clients_manager import clients_manager +from .modules.rooms_manager import rooms_manager + +# Import JSON library - Prefer UltraJSON but use native JSON if failed +try: + import ujson +except Exception as e: + print(f"Server failed to import UltraJSON, failing back to native JSON library. Exception code: {e}") + import json as ujson + + +# Define server exceptions +class exceptions: + class EmptyMessage(Exception): + """This exception is raised when a client sends an empty packet.""" + pass + + class InvalidCommand(Exception): + """This exception is raised when a client sends an invalid command for it's determined protocol.""" + pass + + class JSONError(Exception): + """This exception is raised when the server fails to parse a message's JSON.""" + pass + + class ValidationError(Exception): + """This exception is raised when a client with a known protocol sends a message that fails validation before commands can execute.""" + pass + + class InternalError(Exception): + """This exception is raised when an unexpected and/or unhandled exception is raised.""" + pass + + class Overloaded(Exception): + """This exception is raised when the server believes it is overloaded.""" + pass + + +# Main server +class server: + def __init__(self): + self.version = "0.2.0" + + # Logging + self.logging = logging + self.logger = self.logging.getLogger(__name__) + + # Asyncio + self.asyncio = asyncio + + # Configure websocket framework + self.ws = websockets + + # Components + self.ujson = ujson + self.gen = SnowflakeGenerator(42) + self.validator = cerberus.Validator + self.async_iterable = async_iterable + self.copy = copy + self.clients_manager = clients_manager(self) + self.rooms_manager = rooms_manager(self) + self.exceptions = exceptions() + + # Dictionary containing protocols as keys and sets of commands as values + self.disabled_commands = dict() + + # Create event managers + self.on_connect_events = set() + self.on_message_events = set() + self.on_disconnect_events = set() + self.on_error_events = set() + self.exception_handlers = dict() + self.disabled_commands_handlers = dict() + self.protocol_identified_events = dict() + self.protocol_disconnect_events = dict() + + # Create method handlers + self.command_handlers = dict() + + # Configure framework logging + self.suppress_websocket_logs = True + + # Set to -1 to allow as many client as possible + self.max_clients = -1 + + # Configure SSL support + self.ssl_enabled = False + self.ssl_context = None + + # Enables SSL support + def enable_ssl(self, certfile, keyfile): + try: + self.ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + self.ssl_context.load_cert_chain(certfile=certfile, keyfile=keyfile) + self.ssl_enabled = True + self.logger.info(f"SSL support initialized!") + except Exception as e: + self.logger.error(f"Failed to initialize SSL support! {e}") + + # Runs the server. + def run(self, ip="127.0.0.1", port=3000): + try: + # Validate config before startup + if type(self.max_clients) != int: + raise TypeError( + "The max_clients value must be a integer value set to -1 (unlimited clients) or greater than zero!" + ) + + if self.max_clients < -1 or self.max_clients == 0: + raise ValueError( + "The max_clients value must be a integer value set to -1 (unlimited clients) or greater than zero!" + ) + + # Startup message + self.logger.info(f"CloudLink {self.version} - Now listening to {ip}:{port}") + + # Suppress websocket library logging + if self.suppress_websocket_logs: + self.logging.getLogger('asyncio').setLevel(self.logging.ERROR) + self.logging.getLogger('asyncio.coroutines').setLevel(self.logging.ERROR) + self.logging.getLogger('websockets.server').setLevel(self.logging.ERROR) + self.logging.getLogger('websockets.protocol').setLevel(self.logging.ERROR) + + # Start server + self.asyncio.run(self.__run__(ip, port)) + + except KeyboardInterrupt: + pass + + def unbind_command(self, cmd, schema): + if schema not in self.command_handlers: + raise ValueError + if cmd not in self.command_handlers[schema]: + raise ValueError + self.logger.debug(f"Unbinding command {cmd} from {schema.__qualname__} command event manager") + self.command_handlers[schema].pop(cmd) + + # Event binder for on_command events + def on_command(self, cmd, schema): + + def bind_event(func): + self.bind_callback(cmd, schema, func) + + # End on_command binder + return bind_event + + # Event binder for on_error events with specific shemas/exception types + def on_exception(self, exception_type, schema): + def bind_event(func): + + # Create schema category for error event manager + if schema not in self.exception_handlers: + self.logger.debug(f"Creating protocol {schema.__qualname__} exception event manager") + self.exception_handlers[schema] = dict() + + # Create error event handler + if exception_type not in self.exception_handlers[schema]: + self.exception_handlers[schema][exception_type] = set() + + # Add function to the error command handler + self.logger.debug(f"Binding function {func.__name__} to exception {exception_type.__name__} in {schema.__qualname__} exception event manager") + self.exception_handlers[schema][exception_type].add(func) + + # End on_error_specific binder + return bind_event + + # Event binder for invalid command events with specific shemas/exception types + def on_disabled_command(self, schema): + def bind_event(func): + # Create disabled command event manager + if schema not in self.disabled_commands_handlers: + self.logger.debug(f"Creating disabled command event manager {schema.__qualname__}") + self.disabled_commands_handlers[schema] = set() + + # Add function to the error command handler + self.logger.debug(f"Binding function {func.__name__} to {schema.__qualname__} disabled command event manager") + self.disabled_commands_handlers[schema].add(func) + + # End on_error_specific binder + return bind_event + + def on_protocol_identified(self, schema): + def bind_event(func): + # Create protocol identified event manager + if schema not in self.protocol_identified_events: + self.logger.debug(f"Creating protocol identified event manager {schema.__qualname__}") + self.protocol_identified_events[schema] = set() + + # Add function to the protocol identified event manager + self.logger.debug(f"Binding function {func.__name__} to {schema.__qualname__} protocol identified event manager") + self.protocol_identified_events[schema].add(func) + + # End on_protocol_identified binder + return bind_event + + def on_protocol_disconnect(self, schema): + def bind_event(func): + # Create protocol disconnect event manager + if schema not in self.protocol_disconnect_events: + self.logger.debug(f"Creating protocol disconnect event manager {schema.__qualname__}") + self.protocol_disconnect_events[schema] = set() + + # Add function to the protocol disconnect event manager + self.logger.debug(f"Binding function {func.__name__} to {schema.__qualname__} protocol disconnected event manager") + self.protocol_disconnect_events[schema].add(func) + + # End on_protocol_disconnect binder + return bind_event + + # Event binder for on_message events + def on_message(self, func): + self.logger.debug(f"Binding function {func.__name__} to on_message events") + self.on_message_events.add(func) + + # Event binder for on_connect events. + def on_connect(self, func): + self.logger.debug(f"Binding function {func.__name__} to on_connect events") + self.on_connect_events.add(func) + + # Event binder for on_disconnect events. + def on_disconnect(self, func): + self.logger.debug(f"Binding function {func.__name__} to on_disconnect events") + self.on_disconnect_events.add(func) + + # Event binder for on_error events. + def on_error(self, func): + self.logger.debug(f"Binding function {func.__name__} to on_error events") + self.on_error_events.add(func) + + # Friendly version of send_packet_unicast / send_packet_multicast + def send_packet(self, obj, message): + if type(obj) in [list, set]: + self.asyncio.create_task(self.execute_multicast(obj, message)) + else: + self.asyncio.create_task(self.execute_unicast(obj, message)) + + # Send message to a single client + def send_packet_unicast(self, client, message): + # Create unicast task + self.asyncio.create_task(self.execute_unicast(client, message)) + + # Send message to multiple clients + def send_packet_multicast(self, clients, message): + # Create multicast task + self.asyncio.create_task(self.execute_multicast(clients, message)) + + # Close the connection to client(s) + def close_connection(self, obj, code=1000, reason=""): + if type(obj) in [list, set]: + self.asyncio.create_task(self.execute_close_multi(obj, code, reason)) + else: + self.asyncio.create_task(self.execute_close_single(obj, code, reason)) + + # Command disabler + def disable_command(self, cmd, schema): + # Check if the schema has no disabled commands + if schema not in self.disabled_commands: + self.disabled_commands[schema] = set() + + # Check if the command isn't already disabled + if cmd in self.disabled_commands[schema]: + raise ValueError( + f"The command {cmd} is already disabled in protocol {schema.__qualname__}, or was enabled beforehand.") + + # Disable the command + self.disabled_commands[schema].add(cmd) + self.logger.debug(f"Disabled command {cmd} in protocol {schema.__qualname__}") + + # Command enabler + def enable_command(self, cmd, schema): + # Check if the schema has disabled commands + if schema not in self.disabled_commands: + raise ValueError(f"There are no commands to disable in protocol {schema.__qualname__}.") + + # Check if the command is disabled + if cmd not in self.disabled_commands[schema]: + raise ValueError( + f"The command {cmd} is already enabled in protocol {schema.__qualname__}, or wasn't disabled beforehand.") + + # Enable the command + self.disabled_commands[schema].remove(cmd) + self.logger.debug(f"Enabled command {cmd} in protocol {schema.__qualname__}") + + # Free up unused disablers + if not len(self.disabled_commands[schema]): + self.disabled_commands.pop(schema) + + # Message processor + async def message_processor(self, client, message): + + # Empty packet + if not len(message): + self.logger.debug(f"Client {client.snowflake} sent empty message ") + + # Fire on_error events + asyncio.create_task(self.execute_on_error_events(client, self.exceptions.EmptyMessage)) + + # Fire exception handling events + if client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + client=client, + exception_type=self.exceptions.EmptyMessage, + schema=client.protocol, + details="Empty message" + ) + ) + else: + # Close the connection + self.send_packet(client, "Empty message") + self.close_connection(client, reason="Empty message") + + # End message_processor coroutine + return + + # Parse JSON in message and convert to dict + try: + message = self.ujson.loads(message) + + except Exception as error: + self.logger.debug(f"Client {client.snowflake} sent invalid JSON: {error}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(client, error)) + + # Fire exception handling events + if client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + client=client, + exception_type=self.exceptions.JSONError, + schema=client.protocol, + details=error + ) + ) + + else: + # Close the connection + self.send_packet(client, "Invalid JSON") + self.close_connection(client, reason="Invalid JSON") + + # End message_processor coroutine + return + + # Begin validation + valid = False + selected_protocol = None + + # Client protocol is unknown + if not client.protocol: + self.logger.debug(f"Trying to identify client {client.snowflake}'s protocol") + + # Identify protocol + errorlist = list() + + for schema in self.command_handlers: + validator = self.validator(schema.default, allow_unknown=True) + if validator.validate(message): + valid = True + selected_protocol = schema + break + else: + errorlist.append(validator.errors) + + if not valid: + # Log failed identification + self.logger.debug(f"Could not identify protocol used by client {client.snowflake}: {errorlist}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(client, "Unable to identify protocol")) + + # Close the connection + self.send_packet(client, "Unable to identify protocol") + self.close_connection(client, reason="Unable to identify protocol") + + # End message_processor coroutine + return + + # Log known protocol + self.logger.debug(f"Client {client.snowflake} is using protocol {selected_protocol.__qualname__}") + + # Make the client's protocol known + self.clients_manager.set_protocol(client, selected_protocol) + + # Fire protocol identified events + self.asyncio.create_task(self.execute_protocol_identified_events(client, selected_protocol)) + + else: + self.logger.debug( + f"Validating message from {client.snowflake} using protocol {client.protocol.__qualname__}") + + # Validate message using known protocol + selected_protocol = client.protocol + + validator = self.validator(selected_protocol.default, allow_unknown=True) + + + if not validator.validate(message): + errors = validator.errors + + # Log failed validation + self.logger.debug(f"Client {client.snowflake} sent message that failed validation: {errors}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(client, errors)) + + # Fire exception handling events + if client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + client=client, + exception_type=self.exceptions.ValidationError, + schema=client.protocol, + details=errors + ) + ) + + # End message_processor coroutine + return + + # Check if command exists + if message[selected_protocol.command_key] not in self.command_handlers[selected_protocol]: + + # Log invalid command + self.logger.debug( + f"Client {client.snowflake} sent an invalid command \"{message[selected_protocol.command_key]}\" in protocol {selected_protocol.__qualname__}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(client, "Invalid command")) + + # Fire exception handling events + if client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + client=client, + exception_type=self.exceptions.InvalidCommand, + schema=client.protocol, + details=message[selected_protocol.command_key] + ) + ) + + # End message_processor coroutine + return + + # Check if the command is disabled + if selected_protocol in self.disabled_commands: + if message[selected_protocol.command_key] in self.disabled_commands[selected_protocol]: + self.logger.debug( + f"Client {client.snowflake} sent a disabled command \"{message[selected_protocol.command_key]}\" in protocol {selected_protocol.__qualname__}") + + # Fire disabled command event + self.asyncio.create_task( + self.execute_disabled_command_events( + client, + selected_protocol, + message[selected_protocol.command_key] + ) + ) + + # End message_processor coroutine + return + + # Fire on_command events + self.asyncio.create_task( + self.execute_on_command_events( + client, + message, + selected_protocol + ) + ) + + # Fire on_message events + self.asyncio.create_task( + self.execute_on_message_events( + client, + message + ) + ) + + # Connection handler + async def connection_handler(self, client): + + # Limit the amount of clients connected + if self.max_clients != -1: + if len(self.clients_manager) >= self.max_clients: + self.logger.warning("Server full: Refused a new connection") + self.send_packet(client, "Server is full!") + self.close_connection(client, reason="Server is full!") + return + + # Startup client attributes + client.snowflake = str(next(self.gen)) + client.protocol = None + client.protocol_set = False + client.rooms = set() + client.username_set = False + client.username = str() + client.handshake = False + + # Begin tracking the lifetime of the client + client.birth_time = time.monotonic() + + # Add to clients manager + self.clients_manager.add(client) + + # Fire on_connect events + self.asyncio.create_task(self.execute_on_connect_events(client)) + + self.logger.debug(f"Client {client.snowflake} connected") + + # Run connection loop + await self.connection_loop(client) + + # Remove from clients manager + self.clients_manager.remove(client) + + # Fire on_disconnect events + self.asyncio.create_task(self.execute_on_disconnect_events(client)) + + # Execute all protocol-specific disconnect events + if client.protocol_set: + self.asyncio.create_task( + self.execute_protocol_disconnect_events(client, client.protocol) + ) + + self.logger.debug( + f"Client {client.snowflake} disconnected: Total lifespan of {time.monotonic() - client.birth_time} seconds.") + + # Connection loop - Redefine for use with another outside library + async def connection_loop(self, client): + # Primary asyncio loop for the lifespan of the websocket connection + try: + async for message in client: + # Start keeping track of processing time + start = time.perf_counter() + self.logger.debug(f"Now processing message from client {client.snowflake}...") + + # Process the message + await self.message_processor(client, message) + + # Log processing time + self.logger.debug( + f"Done processing message from client {client.snowflake}. Processing took {time.perf_counter() - start} seconds.") + + # Handle unexpected disconnects + except self.ws.exceptions.ConnectionClosedError: + pass + + # Handle OK disconnects + except self.ws.exceptions.ConnectionClosedOK: + pass + + # Catch any unexpected exceptions + except Exception as e: + self.logger.critical(f"Unexpected exception was raised: {e}") + + # Fire on_error events + self.asyncio.create_task(self.execute_on_error_events(client, f"Unexpected exception was raised: {e}")) + + # Fire exception handling events + if client.protocol_set: + self.asyncio.create_task( + self.execute_exception_handlers( + client=client, + exception_type=self.exceptions.InternalError, + schema=client.protocol, + details=f"Unexpected exception was raised: {e}" + ) + ) + + # WebSocket-specific server loop + async def __run__(self, ip, port): + if self.ssl_enabled: + # Run with SSL support + async with self.ws.serve(self.connection_handler, ip, port, ssl=self.ssl_context): + await self.asyncio.Future() + else: + # Run without SSL support + async with self.ws.serve(self.connection_handler, ip, port): + await self.asyncio.Future() + + # Asyncio event-handling coroutines + + async def execute_on_disconnect_events(self, client): + events = [event(client) for event in self.on_disconnect_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_connect_events(self, client): + events = [event(client) for event in self.on_connect_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_message_events(self, client, message): + events = [event(client, message) for event in self.on_message_events] + group = self.asyncio.gather(*events) + await group + + async def execute_on_command_events(self, client, message, schema): + events = [event(client, message) for event in self.command_handlers[schema][message[schema.command_key]]] + group = self.asyncio.gather(*events) + await group + + async def execute_on_error_events(self, client, errors): + events = [event(client, errors) for event in self.on_error_events] + group = self.asyncio.gather(*events) + await group + + async def execute_exception_handlers(self, client, exception_type, schema, details): + # Guard clauses + if schema not in self.exception_handlers: + return + if exception_type not in self.exception_handlers[schema]: + return + + # Fire events + events = [event(client, details) for event in self.exception_handlers[schema][exception_type]] + group = self.asyncio.gather(*events) + await group + + async def execute_disabled_command_events(self, client, schema, cmd): + # Guard clauses + if schema not in self.disabled_commands_handlers: + return + + # Fire events + events = [event(client, cmd) for event in self.disabled_commands_handlers[schema]] + group = self.asyncio.gather(*events) + await group + + async def execute_protocol_identified_events(self, client, schema): + # Guard clauses + if schema not in self.protocol_identified_events: + return + + # Fire events + events = [event(client) for event in self.protocol_identified_events[schema]] + group = self.asyncio.gather(*events) + await group + + async def execute_protocol_disconnect_events(self, client, schema): + # Guard clauses + if schema not in self.protocol_disconnect_events: + return + + # Fire events + events = [event(client) for event in self.protocol_disconnect_events[schema]] + group = self.asyncio.gather(*events) + await group + + # WebSocket-specific coroutines + + async def execute_unicast(self, client, message): + # Guard clause + if type(message) not in [dict, str]: + raise TypeError(f"Supported datatypes for messages are dicts and strings, got type {type(message)}.") + + # Convert dict to JSON + if type(message) == dict: + message = self.ujson.dumps(message) + + # Attempt to send the packet + try: + await client.send(message) + except Exception as e: + self.logger.critical( + f"Unexpected exception was raised while sending message to client {client.snowflake}: {e}" + ) + + async def execute_multicast(self, clients, message): + # Multicast the message + events = [self.execute_unicast(client, message) for client in clients] + group = self.asyncio.gather(*events) + await group + + async def execute_close_single(self, client, code=1000, reason=""): + try: + await client.close(code, reason) + except Exception as e: + self.logger.critical( + f"Unexpected exception was raised while closing connection to client {client.snowflake}: {e}" + ) + + async def execute_close_multi(self, clients, code=1000, reason=""): + events = [self.execute_close_single(client, code, reason) for client in clients] + group = self.asyncio.gather(*events) + await group + + # Deprecated. Provides semi-backwards compatibility for callback functions from 0.1.9.2. + def bind_callback(self, cmd, schema, method): + # Create schema category for command event manager + if schema not in self.command_handlers: + self.logger.debug(f"Creating protocol {schema.__qualname__} command event manager") + self.command_handlers[schema] = dict() + + # Create command event handler + if cmd not in self.command_handlers[schema]: + self.command_handlers[schema][cmd] = set() + + # Add function to the command handler + self.logger.debug(f"Binding function {method.__name__} to command {cmd} in {schema.__qualname__} command event manager") + self.command_handlers[schema][cmd].add(method) + + # Deprecated. Provides semi-backwards compatibility for event functions from 0.1.9.2. + def bind_event(self, event, func): + match (event): + case self.on_connect: + self.on_connect(func) + case self.on_disconnect: + self.on_disconnect(func) + case self.on_error: + self.on_error(func) + case self.on_message: + self.on_message(func) diff --git a/cloudlink/server/modules/__init__.py b/cloudlink/server/modules/__init__.py new file mode 100644 index 0000000..81421e8 --- /dev/null +++ b/cloudlink/server/modules/__init__.py @@ -0,0 +1,2 @@ +from .clients_manager import * +from .rooms_manager import * diff --git a/cloudlink/server/modules/clients_manager.py b/cloudlink/server/modules/clients_manager.py new file mode 100644 index 0000000..2202985 --- /dev/null +++ b/cloudlink/server/modules/clients_manager.py @@ -0,0 +1,158 @@ +""" +clients_manager - Provides tools to search, add, and remove clients from the server. +""" + + +class exceptions: + class ClientDoesNotExist(Exception): + """This exception is raised when a client object does not exist""" + pass + + class ClientAlreadyExists(Exception): + """This exception is raised when attempting to add a client object that is already present""" + pass + + class ClientUsernameAlreadySet(Exception): + """This exception is raised when a client attempts to set their friendly username, but it was already set.""" + pass + + class ClientUsernameNotSet(Exception): + """This exception is raised when a client object has not yet set it's friendly username.""" + pass + + class NoResultsFound(Exception): + """This exception is raised when there are no results for a client search request.""" + pass + + class ProtocolAlreadySet(Exception): + """This exception is raised when attempting to change a client's protocol.""" + pass + + +class clients_manager: + def __init__(self, parent): + # Inherit parent + self.parent = parent + + # Create attributes for storage/searching + self.clients = set() + self.snowflakes = dict() + self.protocols = dict() + self.usernames = dict() + self.uuids = dict() + + # Init exceptions + self.exceptions = exceptions() + + # Init logger + self.logging = parent.logging + self.logger = self.logging.getLogger(__name__) + + def __len__(self): + return len(self.clients) + + def get_snowflakes(self): + return set(obj.snowflake for obj in self.clients) + + def get_uuids(self): + return set(str(obj.id) for obj in self.clients) + + def exists(self, obj): + return obj in self.clients + + def add(self, obj): + if self.exists(obj): + raise self.exceptions.ClientAlreadyExists + + # Add object to set + self.clients.add(obj) + + # Add to snowflakes + self.snowflakes[obj.snowflake] = obj + + # Add to UUIDs + self.uuids[str(obj.id)] = obj + + def remove(self, obj): + if not self.exists(obj): + raise self.exceptions.ClientDoesNotExist + + # Remove from all clients set + self.clients.remove(obj) + + # Remove from snowflakes + self.snowflakes.pop(obj.snowflake) + + # Remove from UUIDs + self.uuids.pop(str(obj.id)) + + # Remove from protocol references + if obj.protocol_set: + + # Remove reference to protocol object + if obj in self.protocols[obj.protocol]: + self.protocols[obj.protocol].remove(obj) + + # Clean up unused protocol references + if not len(self.protocols[obj.protocol]): + del self.protocols[obj.protocol] + + # Remove client from username references + if obj.username_set: + + # Remove reference to username object + if obj.username in self.usernames: + self.usernames[obj.username].remove(obj) + + # Clean up unused usernames + if not len(self.usernames[obj.username]): + del self.usernames[obj.username] + + def set_username(self, obj, username): + if not self.exists(obj): + raise self.exceptions.ClientDoesNotExist + + if obj.username_set: + raise self.exceptions.ClientUsernameAlreadySet + + # Create username reference + if username not in self.usernames: + self.usernames[username] = set() + + # Create reference to object from its username + self.usernames[username].add(obj) + + # Finally set attributes + obj.username_set = True + obj.username = username + + def set_protocol(self, obj, schema): + if not self.exists(obj): + raise self.exceptions.ClientDoesNotExist + + if obj.protocol_set: + raise self.exceptions.ProtocolAlreadySet + + # If the protocol was not specified beforehand, create it + if schema not in self.protocols: + self.protocols[schema] = set() + + # Add client to the protocols identifier + self.protocols[schema].add(obj) + + # Set client protocol + obj.protocol = schema + obj.protocol_set = True + + def find_obj(self, query): + if type(query) not in [str, dict]: + raise TypeError("Clients can only be usernames (str), snowflakes (str), UUIDs (str), or Objects (dict).") + + if query in self.usernames: + return self.usernames[query] + elif query in self.get_uuids(): + return self.uuids[query] + elif query in self.get_snowflakes(): + return self.snowflakes[query] + else: + raise self.exceptions.NoResultsFound diff --git a/cloudlink/server/modules/rooms_manager.py b/cloudlink/server/modules/rooms_manager.py new file mode 100644 index 0000000..8d28405 --- /dev/null +++ b/cloudlink/server/modules/rooms_manager.py @@ -0,0 +1,285 @@ +""" +rooms_manager - Provides tools to search, add, and remove rooms from the server. +""" + + +class exceptions: + class RoomDoesNotExist(Exception): + """This exception is raised when a client accesses a room that does not exist""" + pass + + class RoomAlreadyExists(Exception): + """This exception is raised when attempting to create a room that already exists""" + pass + + class RoomNotEmpty(Exception): + """This exception is raised when attempting to delete a room that is not empty""" + pass + + class NoResultsFound(Exception): + """This exception is raised when there are no results for a room search request.""" + pass + + class RoomUnsupportedProtocol(Exception): + """This exception is raised when a room does not support a client's protocol.""" + pass + + +class rooms_manager: + def __init__(self, parent): + # Inherit parent + self.parent = parent + + # Storage of rooms + self.rooms = dict() + + # Init exceptions + self.exceptions = exceptions() + + # Init logger + self.logging = parent.logging + self.logger = self.logging.getLogger(__name__) + + def get(self, room_id): + try: + return self.find_room(room_id) + except self.exceptions.NoResultsFound: + # Return default dict + return { + "clients": dict(), + "global_vars": dict(), + "private_vars": dict() + } + + def create(self, room_id): + # Rooms may only have string names + if type(room_id) != str: + raise TypeError("Room IDs only support strings!") + + # Prevent re-declaring a room + if self.exists(room_id): + raise self.exceptions.RoomAlreadyExists + + # Create the room + self.rooms[room_id] = { + "clients": dict(), + "global_vars": dict(), + "private_vars": dict() + } + + # Log creation + self.parent.logger.debug(f"Created room {room_id}") + + def delete(self, room_id): + # Rooms may only have string names + if type(room_id) != str: + raise TypeError("Room IDs only support strings!") + + # Check if the room exists + if not self.exists(room_id): + raise self.exceptions.RoomDoesNotExist + + # Prevent deleting a room if it's not empty + if len(self.rooms[room_id]["clients"]): + raise self.exceptions.RoomNotEmpty + + # Delete the room + self.rooms.pop(room_id) + + # Log deletion + self.parent.logger.debug(f"Deleted room {room_id}") + + def exists(self, room_id) -> bool: + # Rooms may only have string names + if type(room_id) != str: + raise TypeError("Room IDs only support strings!") + + return room_id in self.rooms + + def subscribe(self, obj, room_id): + # Rooms may only have string names + if type(room_id) != str: + raise TypeError("Room IDs only support strings!") + + # Check if room exists + if not self.exists(room_id): + self.create(room_id) + + room = self.rooms[room_id] + + # Create room protocol categories + if obj.protocol not in room["clients"]: + room["clients"][obj.protocol] = { + "all": set(), + "uuids": dict(), + "snowflakes": dict(), + "usernames": dict() + } + + room_objs = self.rooms[room_id]["clients"][obj.protocol] + + # Exit if client has subscribed to the room already + if str(obj.id) in room_objs["uuids"]: + return + + # Add to room + room_objs["all"].add(obj) + room_objs["uuids"][str(obj.id)] = obj + room_objs["snowflakes"][obj.snowflake] = obj + + # Create room username reference + if obj.username not in room_objs["usernames"]: + room_objs["usernames"][obj.username] = set() + + # Add to usernames reference + room_objs["usernames"][obj.username].add(obj) + + # Add room to client object + obj.rooms.add(room_id) + + # Log room subscribe + self.parent.logger.debug(f"Subscribed client {obj.snowflake} to room {room_id}") + + def unsubscribe(self, obj, room_id): + # Rooms may only have string names + if type(room_id) != str: + raise TypeError("Room IDs only support strings!") + + # Check if room exists + if not self.exists(room_id): + raise self.exceptions.RoomDoesNotExist + + room = self.rooms[room_id]["clients"][obj.protocol] + + # Check if a client has subscribed to a room + if str(obj.id) not in room["uuids"]: + return + + # Remove from room + room["all"].remove(obj) + room["uuids"].pop(str(obj.id)) + room["snowflakes"].pop(obj.snowflake) + if obj in room["usernames"][obj.username]: + room["usernames"][obj.username].remove(obj) + + # Remove empty username reference set + if not len(room["usernames"][obj.username]): + room["usernames"].pop(obj.username) + + room = self.rooms[room_id]["clients"] + + # Clean up room protocol categories + if not len(room[obj.protocol]["all"]): + room.pop(obj.protocol) + + # Remove room from client object + obj.rooms.remove(room_id) + + # Log room unsubscribe + self.parent.logger.debug(f"Unsubscribed client {obj.snowflake} from room {room_id}") + + # Delete empty room + if not len(room): + self.parent.logger.debug(f"Deleting emptied room {room_id}...") + self.delete(room_id) + + def find_room(self, query): + # Rooms may only have string names + if type(query) != str: + raise TypeError("Searching for room objects requires a string for the query.") + if query in (room for room in self.rooms): + return self.rooms[query] + else: + raise self.exceptions.NoResultsFound + + def find_obj(self, query, room): + # Prevent accessing clients with usernames not being set + if not len(query): + raise self.exceptions.NoResultsFound + + # Locate client objects in room + if query in room["usernames"]: + return room["usernames"][query] # returns set of client objects + elif query in self.get_uuids(room): + return room["uuids"][query] # returns client object + elif query in self.get_snowflakes(room): + return room["snowflakes"][query] # returns client object + else: + raise self.exceptions.NoResultsFound + + def generate_userlist(self, room_id, protocol) -> list: + userlist = list() + + room = self.get(room_id)["clients"][protocol]["all"] + + for obj in room: + if not obj.username_set: + continue + + userlist.append({ + "id": obj.snowflake, + "username": obj.username, + "uuid": str(obj.id) + }) + + return userlist + + def get_snowflakes(self, room) -> set: + return set(obj for obj in room["snowflakes"]) + + def get_uuids(self, room) -> set: + return set(obj for obj in room["uuids"]) + + async def get_all_in_rooms(self, rooms, protocol) -> set: + obj_set = set() + + # Validate types + if type(rooms) not in [list, set, str]: + raise TypeError(f"Gathering all user objects in rooms requires using a list, set, or string! Got {type(rooms)}.") + + # Convert to set + if type(rooms) == str: + rooms = {rooms} + if type(rooms) == list: + rooms = set(rooms) + + # Collect all user objects in rooms + async for room in self.parent.async_iterable(rooms): + if protocol not in self.get(room)["clients"]: + continue + obj_set.update(self.get(room)["clients"][protocol]["all"]) + + return obj_set + + async def get_specific_in_room(self, room, protocol, queries) -> set: + obj_set = set() + + # Validate types + if type(room) != str: + raise TypeError(f"Gathering specific clients in a room only supports strings for room IDs.") + if type(queries) not in [list, set, str]: + raise TypeError(f"Gathering all user objects in a room requires using a list, set, or string! Got {type(queries)}.") + + # Just return an empty set if the room doesn't exist + if not self.exists(room): + return set() + + room = self.get(room)["clients"][protocol] + + # Convert queries to set + if type(queries) == str: + queries = {queries} + if type(queries) == list: + queries = set(queries) + + async for query in self.parent.async_iterable(queries): + try: + obj = self.find_obj(query, room) + if type(obj) == set: + obj_set.update(obj) + else: + obj_set.add(obj) + except self.exceptions.NoResultsFound: + continue + + return obj_set diff --git a/cloudlink/server/protocols/__init__.py b/cloudlink/server/protocols/__init__.py index b19fa2e..72fbef9 100644 --- a/cloudlink/server/protocols/__init__.py +++ b/cloudlink/server/protocols/__init__.py @@ -1,2 +1,2 @@ -from .cl_methods import * -from .scratch_methods import * \ No newline at end of file +from .clpv4.clpv4 import * +from .scratch.scratch import * diff --git a/cloudlink/server/protocols/cl_methods.py b/cloudlink/server/protocols/cl_methods.py deleted file mode 100644 index d76988e..0000000 --- a/cloudlink/server/protocols/cl_methods.py +++ /dev/null @@ -1,647 +0,0 @@ -class cl_methods: - def __init__(self, parent): - self.parent = parent - self.supporter = parent.supporter - self.copy = parent.copy - self.rooms = parent.rooms - self.get_rooms = parent.supporter.get_rooms - self.clients = parent.clients - - # Various ways to send messages - self.send_packet_unicast = parent.send_packet_unicast - self.send_packet_multicast = parent.send_packet_multicast - self.send_packet_multicast_variable = parent.send_packet_multicast_variable - self.send_code = parent.send_code - - # Get protocol types to allow cross-protocol data sync - self.proto_scratch_cloud = self.supporter.proto_scratch_cloud - self.proto_cloudlink = self.supporter.proto_cloudlink - - # Packet check definitions - Specifies required keys, their datatypes, and optional keys - self.validator = { - self.gmsg: { - "required": { - "val": self.supporter.keydefaults["val"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["listener", "rooms"], - "sizes": { - "val": 1000 - } - }, - self.pmsg: { - "required": { - "val": self.supporter.keydefaults["val"], - "id": self.supporter.keydefaults["id"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["listener", "rooms"], - "sizes": { - "val": 1000 - } - }, - self.gvar: { - "required": { - "val": self.supporter.keydefaults["val"], - "name": self.supporter.keydefaults["name"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["listener", "rooms"], - "sizes": { - "val": 1000 - } - }, - self.pvar: { - "required": { - "val": self.supporter.keydefaults["val"], - "name": self.supporter.keydefaults["name"], - "id": self.supporter.keydefaults["id"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["listener", "rooms"], - "sizes": { - "val": 1000 - } - }, - self.setid: { - "required": { - "val": self.supporter.keydefaults["val"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["listener", "rooms"], - "sizes": { - "val": 1000 - } - }, - self.link: { - "required": { - "val": self.supporter.keydefaults["rooms"], - "listener": self.supporter.keydefaults["listener"] - }, - "optional": ["listener"], - "sizes": { - "val": 1000 - } - }, - self.unlink: { - "required": { - "val": self.supporter.keydefaults["rooms"], - "listener": self.supporter.keydefaults["listener"] - }, - "optional": ["val", "listener"], - "sizes": { - "val": 1000 - } - }, - self.direct: { - "required": { - "val": self.supporter.keydefaults["val"], - "id": self.supporter.keydefaults["id"], - "listener": self.supporter.keydefaults["listener"], - "rooms": self.supporter.keydefaults["rooms"] - }, - "optional": ["id", "listener", "rooms"], - "sizes": { - "val": 1000 - } - } - } - - async def __auto_validate__(self, validator, client, message, listener): - validation = self.supporter.validate( - keys=validator["required"], - payload=message, - optional=validator["optional"], - sizes=validator["sizes"] - ) - - match validation: - case self.supporter.invalid: - # Command datatype is invalid - await self.parent.send_code(client, "DataType", listener=listener) - return False - case self.supporter.missing_key: - # Command syntax is invalid - await self.parent.send_code(client, "Syntax", listener=listener) - return False - case self.supporter.too_large: - # Payload size overload - await self.parent.send_code(client, "TooLarge", listener=listener) - return False - - return True - - async def handshake(self, client, message, listener): - # Validation is not needed since this command takes no arguments - - if self.parent.check_ip_addresses: - # Report client's IP - await self.send_packet_unicast( - client=client, - cmd="client_ip", - val=client.full_ip, - quirk=self.supporter.quirk_embed_val - ) - - # Report server version - await self.send_packet_unicast( - client=client, - cmd="server_version", - val=self.parent.version, - quirk=self.supporter.quirk_embed_val - ) - - # Report server MOTD - if self.parent.enable_motd: - await self.send_packet_unicast( - client=client, - cmd="motd", - val=self.parent.motd_message, - quirk=self.supporter.quirk_embed_val - ) - - # Report the current userlist - room_data = self.parent.rooms.get("default") - await self.send_packet_unicast( - client=client, - cmd="ulist", - val={ - "mode": "set", - "val": room_data.userlist - }, - room_id="default", - quirk=self.supporter.quirk_update_msg - ) - - # Report the cached gmsg value - await self.send_packet_unicast( - client=client, - cmd="gmsg", - val=room_data.global_data_value, - room_id="default", - quirk=self.supporter.quirk_embed_val - ) - - # Tell the client that the cloudlink protocol was selected - await self.send_code( - client=client, - code="OK", - listener=listener - ) - - async def ping(self, client, message, listener): - # Validation is not needed since this command takes no arguments - # Command remains here for compatibility. - # CL4's websocket server apparently has a built-in keepalive, so this is somewhat redundant... - # Return ping - await self.send_code( - client=client, - code="OK", - listener=listener - ) - - async def gmsg(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.gmsg], client, message, listener): - return - - # Get rooms - rooms = self.get_rooms(client, message) - - # Send to all rooms specified - exclude_client = None - if listener: - exclude_client = client - - # Cache the room's gmsg value - for room in self.copy(rooms): - self.rooms.get(room).global_data_value = message["val"] - - # Send to all rooms specified - for room in rooms: - await self.send_packet_multicast( - cmd="gmsg", - val=message["val"], - room_id=room, - exclude_client=exclude_client, - quirk=self.supporter.quirk_embed_val, - ) - - # Handle listeners - if listener: - await self.send_packet_unicast( - client=client, - cmd="gmsg", - val=message["val"], - room_id=room, - listener=listener - ) - - async def pmsg(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.pmsg], client, message, listener): - return - - # Check if the client has set their username - if not client.username_set: - await self.send_code(client, "IDRequired", listener=listener) - return - - # Get rooms - rooms = self.get_rooms(client, message) - - # Get the origin of the request - origin = self.clients.convert_json(client) - - # Locate clients to send message to - clients = set() - for room in rooms: - clients.update(self.clients.find_multi_obj(message["id"], room)) - - # Can't send message since no clients were found - if len(clients) == 0: - await self.send_code(client, "IDNotFound", listener=listener) - return - - # Send to all rooms specified - for room in rooms: - await self.send_packet_multicast( - cmd="pmsg", - val={ - "val": message["val"], - "origin": origin, - "rooms": room - }, - clients=clients, - quirk=self.supporter.quirk_update_msg, - ) - - # Tell the client the message was sent OK - await self.send_code(client, "OK", listener=listener) - - async def gvar(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.gvar], client, message, listener): - return - - # Get rooms - rooms = self.get_rooms(client, message) - - # Send to all rooms specified - exclude_client = None - if listener: - exclude_client = client - - # Cache the room's current gvar value - for room in self.copy(rooms): - room_data = self.rooms.get(room) - if message["name"] in room_data.global_vars: - room_data.global_vars[message["name"]] = message["val"] - else: - room_data.global_vars[message["name"]] = message["val"] - - # Send to all rooms specified (auto convert to scratch format) - for room in rooms: - room_data = self.parent.rooms.get(room) - - await self.send_packet_multicast_variable( - cmd="gvar", - name=message["name"], - val=message["val"], - room_id=room, - exclude_client=exclude_client - ) - - # Handle listeners - if listener and (len(rooms) != 0): - await self.send_packet_unicast( - client=client, - cmd="gvar", - val={ - "name": message["name"], - "val": message["val"] - }, - quirk=self.supporter.quirk_update_msg, - listener=listener - ) - - async def pvar(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.pvar], client, message, listener): - return - - # Check if the client has set their username - if not client.username_set: - await self.send_code(client, "IDRequired", listener=listener) - return - - # Get rooms - rooms = self.get_rooms(client, message) - - # Get the origin of the request - origin = self.clients.convert_json(client) - - # Locate clients to send message to - clients = set() - for room in rooms: - clients.update(self.clients.find_multi_obj(message["id"], room)) - - # Can't send message since no clients were found - if len(clients) == 0: - await self.send_code(client, "IDNotFound", listener=listener) - return - - # Send to all rooms specified - for room in rooms: - await self.send_packet_multicast( - cmd="pvar", - val={ - "val": message["val"], - "name": message["name"], - "origin": origin, - "rooms": room - }, - clients=clients, - quirk=self.supporter.quirk_update_msg, - ) - - # Tell the client the message was sent OK - await self.send_code(client, "OK", listener=listener) - - async def setid(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.setid], client, message, listener): - return - - if client.username_set: - await self.send_code(client, "IDSet", listener=listener) - return - - result = self.parent.clients.set_username(client, message["val"]) - if result == self.supporter.username_set: - await self.send_code(client, "OK", extra_data={"val": self.parent.clients.convert_json(client)}, - listener=listener) - else: - await self.send_code(client, "IDConflict", listener=listener) - return - - # Refresh room - self.parent.rooms.refresh(client, "default") - - # Update all userlists - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "add", - "val": self.parent.clients.convert_json(client) - }, - quirk=self.supporter.quirk_update_msg, - room_id="default" - ) - - async def link(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.link], client, message, listener): - return - - if not client.username_set: - await self.send_code(client, "IDRequired", listener=listener) - return - - # Prepare rooms - rooms = set() - if type(message["val"]) == str: - message["val"] = [message["val"]] - rooms.update(set(message["val"])) - - # Client cannot be linked to no rooms - if len(rooms) == 0: - rooms.update(["default"]) - - # Manage existing rooms - old_rooms = self.copy(client.rooms) - if ("default" in client.rooms) and ("default" not in rooms): - self.parent.rooms.unlink(client, "default") - for room in old_rooms: - self.parent.rooms.unlink(client, room) - - # Create rooms if they do not exist - for room in rooms: - if not self.parent.rooms.exists(room): - self.parent.rooms.create(room) - - # Link client to new rooms - for room in rooms: - self.parent.rooms.link(client, room) - new_rooms = self.copy(client.rooms) - - # Tell the client they have been linked - await self.send_code(client, "OK", listener=listener) - - # Update old userlist - for room in old_rooms: - # Prevent duplication - if room in new_rooms: - continue - - # Get the room data - room_data = self.parent.rooms.get(room) - if not room_data: - continue - - # Update the state - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "set", - "val": room_data.userlist - }, - quirk=self.supporter.quirk_update_msg, - room_id=room - ) - - # Update new userlist - for room in new_rooms: - # Get the room data - room_data = self.parent.rooms.get(room) - if not room_data: - continue - - # Update the state - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "set", - "val": room_data.userlist - }, - quirk=self.supporter.quirk_update_msg, - room_id=room - ) - - # Sync the global variable state - for tmp_room in client.rooms: - room_data = self.parent.rooms.get(tmp_room) - if len(room_data.global_vars.keys()) != 0: - # Update the client's state - for var in room_data.global_vars.keys(): - await self.send_packet_unicast( - client=client, - cmd="gvar", - val={ - "name": var, - "val": room_data.global_vars[var] - }, - quirk=self.supporter.quirk_update_msg, - room_id=tmp_room - ) - - # Report the room's cached gmsg value - await self.send_packet_unicast( - client=client, - cmd="gmsg", - val=room_data.global_data_value, - quirk=self.supporter.quirk_embed_val, - room_id=tmp_room - ) - - async def unlink(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.unlink], client, message, listener): - return - - if not client.username_set: - await self.send_code(client, "IDRequired", listener=listener) - return - - # Prepare the rooms - rooms = set() - if ("val" not in message) or (("val" in message) and (len(message["val"]) == 0)): - # Unlink all rooms - rooms.update(self.copy(client.rooms)) - else: - # Unlink from a single room, or many rooms - if type(message["val"]) == list: - rooms.update(set(message["val"])) - else: - rooms.update(message["val"]) - - # Unlink client from rooms - old_rooms = self.copy(client.rooms) - for room in rooms: - self.parent.rooms.unlink(client, room) - if len(client.rooms) == 0: - # Reset to default room - self.parent.rooms.link(client, "default") - new_rooms = self.copy(client.rooms) - - # Tell the client they have been unlinked - await self.send_code(client, "OK", listener=listener) - - # Update old userlist - for room in old_rooms: - # Prevent duplication - if room in new_rooms: - continue - - # Get the room data - room_data = self.parent.rooms.get(room) - if not room_data: - continue - - # Update the state - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "set", - "val": room_data.userlist - }, - quirk=self.supporter.quirk_update_msg, - room_id=room - ) - - # Update new userlist - for room in new_rooms: - # Get the room data - room_data = self.parent.rooms.get(room) - if not room_data: - continue - - # Update the state - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "set", - "val": room_data.userlist - }, - quirk=self.supporter.quirk_update_msg, - room_id=room - ) - - # Sync the global variable state - for tmp_room in client.rooms: - room_data = self.parent.rooms.get(tmp_room) - if len(room_data.global_vars.keys()) != 0: - # Update the client's state - for var in room_data.global_vars.keys(): - await self.send_packet_unicast( - client=client, - cmd="gvar", - val={ - "name": var, - "val": room_data.global_vars[var] - }, - room_id=tmp_room, - quirk=self.supporter.quirk_update_msg - ) - - # Report the room's cached gmsg value - await self.send_packet_unicast( - client=client, - cmd="gmsg", - val=room_data.global_data_value, - quirk=self.supporter.quirk_embed_val, - room_id=tmp_room - ) - - async def direct(self, client, message, listener): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.direct], client, message, listener): - return - - # Legacy command formatting will be automatically converted to new format. - # This function defaults to doing nothing unless an ID is specified - - if "id" in message: - # Attempting to send a packet directly to someone/something - if not client.username_set: - await self.send_code(client, "IDRequired", listener=listener) - return - - # Get the origin of the request - origin = self.clients.convert_json(client) - - # Locate all clients to send the direct data to - clients = self.clients.find_multi_obj(message["id"], None) - - if not clients: - await self.send_code(client, "IDNotFound", listener=listener) - return - - if (type(clients) in [list, set]) and (len(clients) == 0): - await self.send_code(client, "IDNotFound", listener=listener) - return - - await self.send_packet_multicast( - cmd="direct", - val={ - "val": message["val"], - "origin": origin - }, - clients=clients, - quirk=self.supporter.quirk_update_msg - ) diff --git a/cloudlink/server/protocols/clpv4/__init__.py b/cloudlink/server/protocols/clpv4/__init__.py new file mode 100644 index 0000000..a9e2996 --- /dev/null +++ b/cloudlink/server/protocols/clpv4/__init__.py @@ -0,0 +1 @@ +from .clpv4 import * diff --git a/cloudlink/server/protocols/clpv4/clpv4.py b/cloudlink/server/protocols/clpv4/clpv4.py new file mode 100644 index 0000000..131f7f8 --- /dev/null +++ b/cloudlink/server/protocols/clpv4/clpv4.py @@ -0,0 +1,800 @@ +from .schema import cl4_protocol + +""" +This is the default protocol used for the CloudLink server. +The CloudLink 4.1 Protocol retains full support for CLPv4. + +Each packet format is compliant with UPLv2 formatting rules. + +Documentation for the CLPv4.1 protocol can be found here: +https://github.com/MikeDev101/cloudlink/wiki/The-CloudLink-Protocol +""" + + +class clpv4: + def __init__(self, server): + + """ + Configuration settings + + warn_if_multiple_username_matches: Boolean, Default: True + If True, the server will warn users if they are resolving multiple clients for a username search. + + enable_motd: Boolean, Default: False + If True, whenever a client sends the handshake command or whenever the client's protocol is identified, + the server will send the Message-Of-The-Day from whatever value motd_message is set to. + + motd_message: String, Default: Blank string + If enable_mod is True, this string will be sent as the server's Message-Of-The-Day. + + real_ip_header: String, Default: None + If you use CloudLink behind a tunneling service or reverse proxy, set this value to whatever + IP address-fetching request header to resolve valid IP addresses. When set to None, it will + utilize the host's incoming network for resolving IP addresses. + + Examples include: + * x-forwarded-for + * cf-connecting-ip + + """ + self.warn_if_multiple_username_matches = True + self.enable_motd = False + self.motd_message = str() + self.real_ip_header = None + + # Exposes the schema of the protocol. + self.schema = cl4_protocol + self.__qualname__ = "clpv4" + + # Define various status codes for the protocol. + class statuscodes: + # Code type character + info = "I" + error = "E" + + # Error / info codes as tuples + test = (info, 0, "Test") + echo = (info, 1, "Echo") + ok = (info, 100, "OK") + syntax = (error, 101, "Syntax") + datatype = (error, 102, "Datatype") + id_not_found = (error, 103, "ID not found") + id_not_specific = (error, 104, "ID not specific enough") + internal_error = (error, 105, "Internal server error") + empty_packet = (error, 106, "Empty packet") + id_already_set = (error, 107, "ID already set") + refused = (error, 108, "Refused") + invalid_command = (error, 109, "Invalid command") + disabled_command = (error, 110, "Command disabled") + id_required = (error, 111, "ID required") + id_conflict = (error, 112, "ID conflict") + too_large = (error, 113, "Too large") + json_error = (error, 114, "JSON error") + room_not_joined = (error, 115, "Room not joined") + + @staticmethod + def generate(code: tuple): + return f"{code[0]}:{code[1]} | {code[2]}", code[1] + + # Expose statuscodes class for extension usage + self.statuscodes = statuscodes + + # Identification of a client's IP address + def get_client_ip(client): + # Grab IP address using headers + if self.real_ip_header: + if self.real_ip_header in client.request_headers: + return client.request_headers.get(self.real_ip_header) + + # Use system identified IP address + if type(client.remote_address) == tuple: + return str(client.remote_address[0]) + + # Expose get_client_ip for extension usage + self.get_client_ip = get_client_ip + + # valid(message, schema): Used to verify messages. + def valid(client, message, schema, allow_unknown=True): + + validator = server.validator(schema, allow_unknown=allow_unknown) + + if validator.validate(message): + return True + + # Alert the client that the schema was invalid + send_statuscode(client, statuscodes.syntax, details=dict(validator.errors)) + return False + + # Expose validator function for extension usage + self.valid = valid + + # Simplify sending error/info messages + def send_statuscode(client, code, details=None, message=None, val=None): + # Generate a statuscode + code_human, code_id = statuscodes.generate(code) + + # Template the message + tmp_message = { + "cmd": "statuscode", + "code": code_human, + "code_id": code_id + } + + if details: + tmp_message["details"] = details + + if message: + if "listener" in message: + tmp_message["listener"] = message["listener"] + + if val: + tmp_message["val"] = val + + # Send the code + server.send_packet(client, tmp_message) + + # Expose the statuscode generator for extension usage + self.send_statuscode = send_statuscode + + # Send messages with automatic listener attaching + def send_message(client, payload, message=None): + if message: + if "listener" in message: + payload["listener"] = message["listener"] + + # Send the code + server.send_packet(client, payload) + + # Expose the message sender for extension usage + self.send_message = send_message + + # Simplify alerting users that a command requires a username to be set + def require_username_set(client, message): + if not client.username_set: + send_statuscode( + client, + statuscodes.id_required, + details="This command requires setting a username.", + message=message + ) + + return client.username_set + + # Expose username requirement function for extension usage + self.require_username_set = require_username_set + + # Tool for gathering client rooms + def gather_rooms(client, message): + if "rooms" in message: + # Read value from message + rooms = message["rooms"] + + # Convert to set + if type(rooms) == str: + rooms = {rooms} + if type(rooms) == list: + rooms = set(rooms) + + return rooms + else: + # Use all subscribed rooms + return client.rooms + + # Expose rooms gatherer for extension usage + self.gather_rooms = gather_rooms + + # Generate a user object + def generate_user_object(obj): + # Username set + if obj.username_set: + return { + "id": obj.snowflake, + "username": obj.username, + "uuid": str(obj.id) + } + + # Username not set + return { + "id": obj.snowflake, + "uuid": str(obj.id) + } + + # Expose username object generator function for extension usage + self.generate_user_object = generate_user_object + + # If the client has not explicitly used the handshake command, send them the handshake data + async def notify_handshake(client): + # Don't execute this if handshake was already done + if client.handshake: + return + client.handshake = True + + # Send client IP address + server.send_packet(client, { + "cmd": "client_ip", + "val": get_client_ip(client) + }) + + # Send server version + server.send_packet(client, { + "cmd": "server_version", + "val": server.version + }) + + # Send Message-Of-The-Day + if self.enable_motd: + server.send_packet(client, { + "cmd": "motd", + "val": self.motd_message + }) + + # Send client's Snowflake ID + server.send_packet(client, { + "cmd": "client_obj", + "val": generate_user_object(client) + }) + + # Send userlists of rooms + async for room in server.async_iterable(client.rooms): + server.send_packet(client, { + "cmd": "ulist", + "mode": "set", + "val": server.rooms_manager.generate_userlist(room, cl4_protocol), + "rooms": room + }) + + # Exception handlers + + @server.on_exception(exception_type=server.exceptions.ValidationError, schema=cl4_protocol) + async def validation_failure(client, details): + send_statuscode(client, statuscodes.syntax, details=dict(details)) + + @server.on_exception(exception_type=server.exceptions.InvalidCommand, schema=cl4_protocol) + async def invalid_command(client, details): + send_statuscode( + client, + statuscodes.invalid_command, + details=f"{details} is an invalid command." + ) + + @server.on_disabled_command(schema=cl4_protocol) + async def disabled_command(client, details): + send_statuscode( + client, + statuscodes.disabled_command, + details=f"{details} is a disabled command." + ) + + @server.on_exception(exception_type=server.exceptions.JSONError, schema=cl4_protocol) + async def json_exception(client, details): + send_statuscode( + client, + statuscodes.json_error, + details=f"A JSON error was raised: {details}" + ) + + @server.on_exception(exception_type=server.exceptions.EmptyMessage, schema=cl4_protocol) + async def empty_message(client): + send_statuscode( + client, + statuscodes.empty_packet, + details="Your client has sent an empty message." + ) + + # Protocol identified event + @server.on_protocol_identified(schema=cl4_protocol) + async def protocol_identified(client): + server.logger.debug(f"Adding client {client.snowflake} to default room.") + server.rooms_manager.subscribe(client, "default") + + @server.on_protocol_disconnect(schema=cl4_protocol) + async def protocol_disconnect(client): + server.logger.debug(f"Removing client {client.snowflake} from rooms...") + + # Unsubscribe from all rooms + async for room_id in server.async_iterable(server.copy(client.rooms)): + server.rooms_manager.unsubscribe(client, room_id) + + # Don't bother with notifying if client username wasn't set + if not client.username_set: + continue + + # Notify rooms of removed client + clients = await server.rooms_manager.get_all_in_rooms(room_id, cl4_protocol) + clients = server.copy(clients) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "remove", + "val": generate_user_object(client), + "rooms": room_id + }) + + # The CLPv4 command set + + @server.on_command(cmd="handshake", schema=cl4_protocol) + async def on_handshake(client, message): + await notify_handshake(client) + send_statuscode(client, statuscodes.ok, message=message) + + @server.on_command(cmd="ping", schema=cl4_protocol) + async def on_ping(client, message): + send_statuscode(client, statuscodes.ok, message=message) + + @server.on_command(cmd="gmsg", schema=cl4_protocol) + async def on_gmsg(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.gmsg): + return + + # Gather rooms to send to + rooms = gather_rooms(client, message) + + # Broadcast to all subscribed rooms + async for room in server.async_iterable(rooms): + + # Prevent accessing rooms not joined + if room not in client.rooms: + send_statuscode( + client, + statuscodes.room_not_joined, + details=f'Attempted to access room {room} while not joined.', + message=message + ) + + # Stop gmsg command + return + + clients = await server.rooms_manager.get_all_in_rooms(room, cl4_protocol) + clients = server.copy(clients) + + # Attach listener (if present) and broadcast + if "listener" in message: + + # Remove originating client from broadcast + clients.remove(client) + + # Define the message to broadcast + tmp_message = { + "cmd": "gmsg", + "val": message["val"] + } + + # Broadcast message + server.send_packet(clients, tmp_message) + + # Define the message to send + tmp_message = { + "cmd": "gmsg", + "val": message["val"], + "listener": message["listener"], + "rooms": room + } + + # Unicast message + server.send_packet(client, tmp_message) + else: + # Broadcast message + server.send_packet(clients, { + "cmd": "gmsg", + "val": message["val"], + "rooms": room + }) + + @server.on_command(cmd="pmsg", schema=cl4_protocol) + async def on_pmsg(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.pmsg): + return + + # Require sending client to have set their username + if not require_username_set(client, message): + return + + # Gather rooms + rooms = gather_rooms(client, message) + + # Search and send to all specified clients in rooms + any_results_found = False + async for room in server.async_iterable(rooms): + + # Prevent accessing rooms not joined + if room not in client.rooms: + send_statuscode( + client, + statuscodes.room_not_joined, + details=f'Attempted to access room {room} while not joined.', + message=message + ) + + # Stop pmsg command + return + + clients = await server.rooms_manager.get_specific_in_room(room, cl4_protocol, message['id']) + + # Continue if no results are found + if not len(clients): + continue + + # Mark the full search OK + if not any_results_found: + any_results_found = True + + # Warn if multiple matches are found (mainly for username queries) + if self.warn_if_multiple_username_matches and len(clients) >> 1: + send_statuscode( + client, + statuscodes.id_not_specific, + details=f'Multiple matches found for {message["id"]}, found {len(clients)} matches. Please use Snowflakes, UUIDs, or client objects instead.', + message=message + ) + + # Stop pmsg command + return + + # Send message + tmp_message = { + "cmd": "pmsg", + "val": message["val"], + "origin": generate_user_object(client), + "rooms": room + } + server.send_packet(clients, tmp_message) + + if not any_results_found: + send_statuscode( + client, + statuscodes.id_not_found, + details=f'No matches found: {message["id"]}', + message=message + ) + + # End pmsg command handler + return + + # Results were found and sent successfully + send_statuscode( + client, + statuscodes.ok, + message=message + ) + + @server.on_command(cmd="gvar", schema=cl4_protocol) + async def on_gvar(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.gvar): + return + + # Gather rooms to send to + rooms = gather_rooms(client, message) + + # Broadcast to all subscribed rooms + async for room in server.async_iterable(rooms): + + # Prevent accessing rooms not joined + if room not in client.rooms: + send_statuscode( + client, + statuscodes.room_not_joined, + details=f'Attempted to access room {room} while not joined.', + message=message + ) + + # Stop gvar command + return + + clients = await server.rooms_manager.get_all_in_rooms(room, cl4_protocol) + clients = server.copy(clients) + + # Define the message to send + tmp_message = { + "cmd": "gvar", + "name": message["name"], + "val": message["val"], + "rooms": room + } + + # Attach listener (if present) and broadcast + if "listener" in message: + clients.remove(client) + server.send_packet(clients, tmp_message) + tmp_message["listener"] = message["listener"] + server.send_packet(client, tmp_message) + else: + server.send_packet(clients, tmp_message) + + @server.on_command(cmd="pvar", schema=cl4_protocol) + async def on_pvar(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.pvar): + return + + # Require sending client to have set their username + if not require_username_set(client, message): + return + + # Gather rooms + rooms = gather_rooms(client, message) + + # Search and send to all specified clients in rooms + any_results_found = False + async for room in server.async_iterable(rooms): + + # Prevent accessing rooms not joined + if room not in client.rooms: + send_statuscode( + client, + statuscodes.room_not_joined, + details=f'Attempted to access room {room} while not joined.', + message=message + ) + + # Stop pvar command + return + + clients = await server.rooms_manager.get_specific_in_room(room, cl4_protocol, message['id']) + clients = server.copy(clients) + + # Continue if no results are found + if not len(clients): + continue + + # Mark the full search OK + if not any_results_found: + any_results_found = True + + # Warn if multiple matches are found (mainly for username queries) + if self.warn_if_multiple_username_matches and len(clients) >> 1: + send_statuscode( + client, + statuscodes.id_not_specific, + details=f'Multiple matches found for {message["id"]}, found {len(clients)} matches. Please use Snowflakes, UUIDs, or client objects instead.', + message=message + ) + + # Stop pvar command + return + + # Send message + tmp_message = { + "cmd": "pvar", + "name": message["name"], + "val": message["val"], + "origin": generate_user_object(client), + "rooms": room + } + server.send_packet(clients, tmp_message) + + if not any_results_found: + send_statuscode( + client, + statuscodes.id_not_found, + details=f'No matches found: {message["id"]}', + message=message + ) + + # End pmsg command handler + return + + # Results were found and sent successfully + send_statuscode( + client, + statuscodes.ok, + message=message + ) + + @server.on_command(cmd="setid", schema=cl4_protocol) + async def on_setid(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.setid): + return + + # Prevent setting the username more than once + if client.username_set: + server.logger.error(f"Client {client.snowflake} attempted to set username again!") + send_statuscode( + client, + statuscodes.id_already_set, + val=generate_user_object(client), + message=message + ) + + # Exit setid command + return + + # Leave default room + server.rooms_manager.unsubscribe(client, "default") + + # Set the username + server.clients_manager.set_username(client, message['val']) + + # Re-join default room + server.rooms_manager.subscribe(client, "default") + + # Broadcast userlist state to existing members + clients = await server.rooms_manager.get_all_in_rooms("default", cl4_protocol) + clients = server.copy(clients) + clients.remove(client) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "add", + "val": generate_user_object(client), + "rooms": "default" + }) + + # Notify client of current room state + server.send_packet(client, { + "cmd": "ulist", + "mode": "set", + "val": server.rooms_manager.generate_userlist("default", cl4_protocol), + "rooms": "default" + }) + + # Attach listener (if present) and broadcast + send_statuscode( + client, + statuscodes.ok, + val=generate_user_object(client), + message=message + ) + + @server.on_command(cmd="link", schema=cl4_protocol) + async def on_link(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.linking): + return + + # Require sending client to have set their username + if not require_username_set(client, message): + return + + # Convert to set + if type(message["val"]) in [list, str]: + if type(message["val"]) == list: + message["val"] = set(message["val"]) + if type(message["val"]) == str: + message["val"] = {message["val"]} + + # Unsubscribe from default room if not mentioned + if not "default" in message["val"]: + server.rooms_manager.unsubscribe(client, "default") + + # Broadcast userlist state to existing members + clients = await server.rooms_manager.get_all_in_rooms("default", cl4_protocol) + clients = server.copy(clients) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "remove", + "val": generate_user_object(client), + "rooms": "default" + }) + + async for room in server.async_iterable(message["val"]): + server.rooms_manager.subscribe(client, room) + + # Broadcast userlist state to existing members + clients = await server.rooms_manager.get_all_in_rooms(room, cl4_protocol) + clients = server.copy(clients) + clients.remove(client) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "add", + "val": generate_user_object(client), + "rooms": room + }) + + # Notify client of current room state + server.send_packet(client, { + "cmd": "ulist", + "mode": "set", + "val": server.rooms_manager.generate_userlist(room, cl4_protocol), + "rooms": room + }) + + # Attach listener (if present) and broadcast + send_statuscode( + client, + statuscodes.ok, + message=message + ) + + @server.on_command(cmd="unlink", schema=cl4_protocol) + async def on_unlink(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.linking): + return + + # Require sending client to have set their username + if not require_username_set(client, message): + return + + # If blank, assume all rooms + if type(message["val"]) == str and not len(message["val"]): + message["val"] = client.rooms + + # Convert to set + if type(message["val"]) in [list, str]: + if type(message["val"]) == list: + message["val"] = set(message["val"]) + if type(message["val"]) == str: + message["val"] = {message["val"]} + + async for room in server.async_iterable(message["val"]): + server.rooms_manager.unsubscribe(client, room) + + # Broadcast userlist state to existing members + clients = await server.rooms_manager.get_all_in_rooms(room, cl4_protocol) + clients = server.copy(clients) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "remove", + "val": generate_user_object(client), + "rooms": room + }) + + # Re-link to default room if no rooms are joined + if not len(client.rooms): + server.rooms_manager.subscribe(client, "default") + + # Broadcast userlist state to existing members + clients = await server.rooms_manager.get_all_in_rooms("default", cl4_protocol) + clients = server.copy(clients) + clients.remove(client) + server.send_packet(clients, { + "cmd": "ulist", + "mode": "add", + "val": generate_user_object(client), + "rooms": "default" + }) + + # Notify client of current room state + server.send_packet(client, { + "cmd": "ulist", + "mode": "set", + "val": server.rooms_manager.generate_userlist("default", cl4_protocol), + "rooms": "default" + }) + + # Attach listener (if present) and broadcast + send_statuscode( + client, + statuscodes.ok, + message=message + ) + + @server.on_command(cmd="direct", schema=cl4_protocol) + async def on_direct(client, message): + # Validate schema + if not valid(client, message, cl4_protocol.direct): + return + + try: + tmp_client = server.clients_manager.find_obj(message["id"]) + + tmp_msg = { + "cmd": "direct", + "val": message["val"] + } + + if client.username_set: + tmp_msg["origin"] = generate_user_object(client) + + else: + tmp_msg["origin"] = { + "id": client.snowflake, + "uuid": str(client.id) + } + + if "listener" in message: + tmp_msg["listener"] = message["listener"] + + server.send_packet_unicast(tmp_client, tmp_msg) + + except server.clients_manager.exceptions.NoResultsFound: + send_statuscode( + client, + statuscodes.id_not_found, + message=message + ) + + # Stop direct command + return diff --git a/cloudlink/server/protocols/clpv4/schema.py b/cloudlink/server/protocols/clpv4/schema.py new file mode 100644 index 0000000..3592a67 --- /dev/null +++ b/cloudlink/server/protocols/clpv4/schema.py @@ -0,0 +1,340 @@ +# Schema for interpreting the Cloudlink protocol v4.0 (CLPv4) command set +class cl4_protocol: + + # Required - Defines the keyword to use to define the command + command_key = "cmd" + + # Required - Defines the default schema to test against + default = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": False, + }, + "name": { + "type": "string", + "required": False + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": False + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + linking = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True, + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + setid = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": "string", + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + gmsg = { + "cmd": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + gvar = { + "cmd": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + pmsg = { + "cmd": { + "type": "string", + "required": True + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } + + direct = { + "cmd": { + "type": "string", + "required": True + }, + "id": { + "type": "string", + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + } + } + + pvar = { + "cmd": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": True + }, + "id": { + "type": [ + "string", + "dict", + "list", + "set" + ], + "required": True + }, + "val": { + "type": [ + "string", + "integer", + "float", + "number", + "boolean", + "dict", + "list", + "set", + ], + "required": True + }, + "listener": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number" + ], + "required": False + }, + "rooms": { + "type": [ + "string", + "integer", + "float", + "boolean", + "number", + "list", + "set" + ], + "required": False + } + } diff --git a/cloudlink/server/protocols/scratch/__init__.py b/cloudlink/server/protocols/scratch/__init__.py new file mode 100644 index 0000000..c2ce79b --- /dev/null +++ b/cloudlink/server/protocols/scratch/__init__.py @@ -0,0 +1 @@ +from .scratch import * \ No newline at end of file diff --git a/cloudlink/server/protocols/scratch/schema.py b/cloudlink/server/protocols/scratch/schema.py new file mode 100644 index 0000000..9dd9990 --- /dev/null +++ b/cloudlink/server/protocols/scratch/schema.py @@ -0,0 +1,110 @@ +# Schema for interpreting the Cloud Variables protocol used in Scratch 3.0 +class scratch_protocol: + + # Required - Defines the keyword to use to define the command + command_key = "method" + + # Required - Defines the default schema to test against + default = { + "method": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": False + }, + "new_name": { + "type": "string", + "required": False + }, + "value": { + "type": [ + "string", + "integer", + "float", + "boolean", + "dict", + "list", + "set" + ], + "required": False + }, + "project_id": { + "type": [ + "string", + "integer", + "float", + "boolean" + ], + "required": False, + }, + "user": { + "type": "string", + "required": False, + "minlength": 1, + "maxlength": 20 + } + } + + handshake = { + "method": { + "type": "string", + "required": True + }, + "project_id": { + "type": [ + "string", + "integer", + "float", + "boolean" + ], + "required": True, + }, + "user": { + "type": "string", + "required": True, + } + } + + method = { + "method": { + "type": "string", + "required": True + }, + "name": { + "type": "string", + "required": True + }, + "new_name": { + "type": "string", + "required": False + }, + "value": { + "type": [ + "string", + "integer", + "float", + "boolean", + "dict", + "list", + "set" + ], + "required": False + }, + "project_id": { + "type": [ + "string", + "integer", + "float", + "boolean" + ], + "required": True, + }, + "user": { + "type": "string", + "required": False, + "minlength": 1, + "maxlength": 20 + } + } diff --git a/cloudlink/server/protocols/scratch/scratch.py b/cloudlink/server/protocols/scratch/scratch.py new file mode 100644 index 0000000..7adc9f1 --- /dev/null +++ b/cloudlink/server/protocols/scratch/scratch.py @@ -0,0 +1,250 @@ +from .schema import scratch_protocol + +""" +This is a FOSS reimplementation of Scratch's Cloud Variable protocol. +See https://github.com/TurboWarp/cloud-server/blob/master/doc/protocol.md for details. +""" + + +class scratch: + def __init__(self, server): + self.__qualname__ = "scratch" + + # Define various status codes for the protocol. + class statuscodes: + connection_error = 4000 + username_error = 4002 + overloaded = 4003 + unavailable = 4004 + refused_security = 4005 + + # Exposes the schema of the protocol. + self.schema = scratch + + # valid(message, schema): Used to verify messages. + def valid(client, message, schema, allow_unknown=True): + validator = server.validator(schema, allow_unknown=allow_unknown) + + if validator.validate(message): + return True + else: + errors = validator.errors + server.logger.warning(f"Error: {errors}") + server.send_packet_unicast(client, f"Validation failed: {dict(errors)}") + server.close_connection(client, code=statuscodes.connection_error, reason=f"Validation failed") + return False + + @server.on_protocol_disconnect(schema=scratch_protocol) + async def protocol_disconnect(client): + server.logger.debug(f"Removing client {client.snowflake} from rooms...") + + # Unsubscribe from all rooms + async for room_id in server.async_iterable(server.copy(client.rooms)): + server.rooms_manager.unsubscribe(client, room_id) + + @server.on_command(cmd="handshake", schema=scratch_protocol) + async def handshake(client, message): + + # Don't execute this command if handshake was already done + if client.handshake: + return + client.handshake = True + + # Safety first + if ("scratchsessionsid" in client.request_headers) or ("scratchsessionsid" in client.response_headers): + + # Log the hiccup + server.logger.critical(f"Client {client.id} sent scratchsessionsid header(s) - Aborting connection!") + + # Tell the client they are doing a no-no + server.send_packet_unicast(client, "The cloud data library you are using is putting your Scratch account at risk by sending your login token for no reason. Change your Scratch password immediately, then contact the maintainers of that library for further information. This connection is being closed to protect your security.") + + # Abort the connection + server.close_connection(client, code=statuscodes.refused_security, reason=f"Connection closed for security reasons") + + # End this guard clause + return + + # Validate schema + if not valid(client, message, scratch_protocol.handshake): + return + + # Set username + server.logger.debug(f"Scratch client {client.snowflake} declares username {message['user']}.") + + # Set client username + server.clients_manager.set_username(client, message['user']) + + # Subscribe to room + server.rooms_manager.subscribe(client, message["project_id"]) + + # Get values + room_data = server.rooms_manager.get(message["project_id"]) + + # Sync project ID variable state + server.logger.debug(f"Synchronizing room {message['project_id']} state to client {client.id}") + async for variable in server.async_iterable(room_data["global_vars"]): + server.send_packet_unicast(client, { + "method": "set", + "name": variable, + "value": room_data["global_vars"][variable] + }) + + @server.on_command(cmd="create", schema=scratch_protocol) + async def create_variable(client, message): + + # Don't execute this command if handshake wasn't already done + if not client.handshake: + return + + # Validate schema + if not valid(client, message, scratch_protocol.method): + return + + # Guard clause - Room must exist before adding to it + if not server.rooms_manager.exists(message["project_id"]): + server.logger.warning(f"Error: room {message['project_id']} does not exist yet") + + # Abort the connection + server.close_connection(client, code=statuscodes.unavailable, reason=f"Invalid room ID: {message['project_id']}") + + server.logger.debug(f"Creating global variable {message['name']} in {message['project_id']}") + + # Get values + room_data = server.rooms_manager.get(message["project_id"]) + + # Create variable + room_data["global_vars"][message['name']] = message["value"] + + # Broadcast the variable state + server.send_packet_multicast(room_data["clients"][scratch_protocol]["all"], { + "method": "create", + "name": message['name'], + "value": room_data["global_vars"][message['name']] + }) + + @server.on_command(cmd="rename", schema=scratch_protocol) + async def rename_variable(client, message): + + # Don't execute this command if handshake wasn't already done + if not client.handshake: + return + + # Validate schema + if not valid(client, message, scratch_protocol.method): + return + + # Guard clause - Room must exist before deleting values from it + if not server.rooms_manager.exists(message["project_id"]): + server.logger.warning(f"Error: room {message['project_id']} does not exist yet") + + # Abort the connection + server.close_connection( + client, + code=statuscodes.unavailable, + reason=f"Invalid room ID: {message['project_id']}" + ) + return + + server.logger.debug(f"Renaming global variable {message['name']} to {message['new_name']} in {message['project_id']}") + + # Get values + room_data = server.rooms_manager.get(message["project_id"]) + + if message["name"] in room_data["global_vars"]: + # Copy variable + room_data["global_vars"][message["new_name"]] = server.copy(room_data["global_vars"][message["name"]]) + + # Delete old variable + room_data["global_vars"].pop(message['name']) + else: + # Create new variable (renamed from a value in a deleted room) + room_data["global_vars"][message["new_name"]] = str() + + # Broadcast the variable state + server.send_packet_multicast(room_data["clients"][scratch_protocol]["all"], { + "method": "rename", + "name": message['name'], + "new_name": message['new_name'] + }) + + @server.on_command(cmd="delete", schema=scratch_protocol) + async def create_variable(client, message): + + # Don't execute this command if handshake wasn't already done + if not client.handshake: + return + + # Validate schema + if not valid(client, message, scratch_protocol.method): + return + + # Guard clause - Room must exist before deleting values from it + if not server.rooms_manager.exists(message["project_id"]): + server.logger.warning(f"Error: room {message['project_id']} does not exist yet") + + # Abort the connection + server.close_connection( + client, + code=statuscodes.unavailable, + reason=f"Invalid room ID: {message['project_id']}" + ) + return + + server.logger.debug(f"Deleting global variable {message['name']} in {message['project_id']}") + + # Get values + room_data = server.rooms_manager.get(message["project_id"]) + + # Delete variable + room_data["global_vars"].pop(message['name']) + + # Broadcast the variable state + server.send_packet_multicast(room_data["clients"][scratch_protocol]["all"], { + "method": "delete", + "name": message['name'] + }) + + @server.on_command(cmd="set", schema=scratch_protocol) + async def set_value(client, message): + + # Don't execute this command if handshake wasn't already done + if not client.handshake: + return + + # Validate schema + if not valid(client, message, scratch_protocol.method): + return + + # Guard clause - Room must exist before adding to it + if not server.rooms_manager.exists(message["project_id"]): + server.logger.warning(f"Error: room {message['project_id']} does not exist yet") + + # Abort the connection + server.close_connection( + client, + code=statuscodes.unavailable, + reason=f"Invalid room ID: {message['project_id']}" + ) + return + + # Get values + room_data = server.rooms_manager.get(message["project_id"]) + + # Don't re-broadcast values that are identical + if message["name"] in room_data["global_vars"]: + if room_data["global_vars"][message['name']] == message["value"]: + server.logger.debug(f"Not going to rebroadcast global variable {message['name']} in {message['project_id']}") + return + + server.logger.debug(f"Updating global variable {message['name']} in {message['project_id']} to value {message['value']}") + + # Update variable + room_data["global_vars"][message['name']] = message["value"] + + # Broadcast the variable state + server.send_packet_multicast(room_data["clients"][scratch_protocol]["all"], { + "method": "set", + "name": message['name'], + "value": room_data["global_vars"][message['name']] + }) diff --git a/cloudlink/server/protocols/scratch_methods.py b/cloudlink/server/protocols/scratch_methods.py deleted file mode 100644 index a76ff79..0000000 --- a/cloudlink/server/protocols/scratch_methods.py +++ /dev/null @@ -1,258 +0,0 @@ -class scratch_methods: - def __init__(self, parent): - self.parent = parent - self.supporter = parent.supporter - self.json = parent.json - self.copy = parent.copy - self.log = parent.supporter.log - - # Various ways to send messages - self.send_packet_unicast = parent.send_packet_unicast - self.send_packet_multicast = parent.send_packet_multicast - self.send_packet_multicast_variable = parent.send_packet_multicast_variable - - # Get protocol types to allow cross-protocol data sync - self.proto_scratch_cloud = self.supporter.proto_scratch_cloud - self.proto_cloudlink = self.supporter.proto_cloudlink - - # Packet check definitions - Specifies required keys, their datatypes, and optional keys - self.validator = { - self.handshake: { - "required": { - "method": self.supporter.keydefaults["method"], - "project_id": self.supporter.keydefaults["project_id"], - "user": self.supporter.keydefaults["user"] - }, - "optional": [], - "sizes": { - "project_id": 100, - "user": 20 - } - }, - self.set: { - "required": { - "method": self.supporter.keydefaults["method"], - "name": self.supporter.keydefaults["name"], - "value": self.supporter.keydefaults["value"] - }, - "optional": [], - "sizes": { - "name": 21, - "value": 1000 - } - }, - self.create: { - "required": { - "method": self.supporter.keydefaults["method"], - "name": self.supporter.keydefaults["name"], - "value": self.supporter.keydefaults["value"] - }, - "optional": [], - "sizes": { - "name": 21, - "value": 1000 - } - }, - self.delete: { - "required": { - "method": self.supporter.keydefaults["method"], - "name": self.supporter.keydefaults["name"] - }, - "optional": [], - "sizes": { - "name": 21 - } - }, - self.rename: { - "required": { - "method": self.supporter.keydefaults["method"], - "name": self.supporter.keydefaults["name"], - "new_name": self.supporter.keydefaults["name"] - }, - "optional": [], - "sizes": { - "name": 21, - "new_name": 21 - } - } - } - - async def __auto_validate__(self, validator, client, message): - validation = self.supporter.validate( - keys=validator["required"], - payload=message, - optional=validator["optional"], - sizes=validator["sizes"] - ) - - match validation: - case self.supporter.invalid: - # Command datatype is invalid - await client.close(code=self.supporter.connection_error, reason="Invalid datatype error") - return False - case self.supporter.missing_key: - # Command syntax is invalid - await client.close(code=self.supporter.connection_error, reason="Syntax error") - return False - case self.supporter.too_large: - # Payload size overload - await client.close(code=self.supporter.connection_error, reason="Contents too large") - return False - - return True - - async def handshake(self, client, message): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.handshake], client, message): - return - - # According to the Scratch cloud variable spec, if the handshake is successful then there will be no response - # Handshake request will be determined "failed" if the server terminates the connection - - if not len(message["user"]) in range(1, 21): - await client.close(code=self.supporter.connection_error, reason=f"Invalid username: {message['user']}") - return - - if not len(message["project_id"]) in range(1, 101): - await client.close(code=self.supporter.connection_error, reason=f"Invalid room ID: {message['project_id']}") - return - - if not len(message["project_id"]) in range(1, 101): - await client.close(code=self.supporter.connection_error, reason=f"Invalid room ID: {message['project_id']}") - return - - if ("scratchsessionsid" in client.request_headers) or ("scratchsessionsid" in client.response_headers): - await client.send( - "The cloud data library you are using is putting your Scratch account at risk by sending us your login token for no reason. Change your Scratch password immediately, then contact the maintainers of that library for further information. This connection is being refused to protect your security.") - await self.parent.asyncio.sleep(0.1) - await client.close(code=self.supporter.refused_security, reason=f"Connection closed for security reasons") - return - - # Create the project room - if not self.parent.rooms.exists(message["project_id"]): - self.parent.rooms.create(message["project_id"]) - - # Get the room data - room = self.parent.rooms.get(message["project_id"]) - - # Add the user to the room - room.users.add(client) - - # Configure the client - client.rooms = [message["project_id"]] - - result = self.parent.clients.set_username(client, message["user"]) - if result != self.supporter.username_set: - await client.close(code=self.supporter.username_error, reason=f"Username conflict") - return - - if client.friendly_username in room.usernames: - await client.close(code=self.supporter.username_error, reason=f"Username conflict") - return - - room.usernames.add(client.friendly_username) - - # Sync the global variable state - room = self.parent.rooms.get(client.rooms[0]) - if len(room.global_vars.keys()) != 0: - # Wait for client to finish processing new state - await self.parent.asyncio.sleep(0.1) - - # Update the client's state - for var in room.global_vars.keys(): - message = { - "method": "set", - "name": var, - "value": room.global_vars[var] - } - await client.send(self.json.dumps(message)) - - async def set(self, client, message): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.set], client, message): - return - - room = self.parent.rooms.get(client.rooms[0]) - if not room: - await client.close(code=self.supporter.connection_error, reason="No room set up yet") - return - else: - # Don't update the value if it's already set - if message["name"] in room.global_vars: - if room.global_vars[message["name"]] == message["value"]: - return - - room.global_vars[message["name"]] = message["value"] - await self.send_packet_multicast_variable( - cmd="set", - name=message["name"], - val=message["value"], - room_id=client.rooms[0] - ) - - async def create(self, client, message): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.create], client, message): - return - - room = self.parent.rooms.get(client.rooms[0]) - if not room: - await client.close(code=self.supporter.connection_error, reason="No room set up yet") - return - else: - if message["name"] in room.global_vars: - return - - room.global_vars[message["name"]] = message["value"] - await self.send_packet_multicast_variable( - cmd="create", - name=message["name"], - val=message["value"], - room_id=client.rooms[0] - ) - - async def delete(self, client, message): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.delete], client, message): - return - - room = self.parent.rooms.get(client.rooms[0]) - if not room: - await client.close(code=self.supporter.connection_error, reason="No room set up yet") - return - else: - if not message["name"] in room.global_vars: - return - - del room.global_vars[message["name"]] - await self.send_packet_multicast_variable( - cmd="delete", - name=message["name"], - room_id=client.rooms[0] - ) - - async def rename(self, client, message): - # Validate the message syntax and datatypes - if not await self.__auto_validate__(self.validator[self.rename], client, message): - return - - room = self.parent.rooms.get(client.rooms[0]) - - if not room: - await client.close(code=self.supporter.connection_error, reason="No room set up yet") - return - else: - if not message["name"] in room.global_vars: - await client.close(code=self.supporter.connection_error, reason="Variable does not exist") - return - - # Copy the old room data to new one, and then delete - room.global_vars[message["new_name"]] = self.copy(room.global_vars[message["name"]]) - del room.global_vars[message["name"]] - - await self.send_packet_multicast_variable( - cmd="rename", - name=message["name"], - new_name=message["new_name"], - room_id=client.rooms[0] - ) diff --git a/cloudlink/server/server.py b/cloudlink/server/server.py deleted file mode 100644 index 4534ba5..0000000 --- a/cloudlink/server/server.py +++ /dev/null @@ -1,953 +0,0 @@ -from .protocols import cl_methods, scratch_methods - -import websockets -import asyncio -import json -from copy import copy - - -class server: - def __init__(self, parent, logs: bool = True): - self.version = parent.version - - # Initialize required libraries - self.asyncio = asyncio - self.websockets = websockets - self.copy = copy - self.json = json - - # Other - self.enable_logs = logs - self.id_counter = 0 - self.ip_blocklist = [] - - # Config - self.reject_clients = False - self.enable_scratch_support = True - self.check_ip_addresses = False - self.enable_motd = False - self.motd_message = str() - - # Managing methods - self.custom_methods = custom_methods() - self.disabled_methods = set() - self.method_callbacks = dict() - self.listener_callbacks = dict() - self.safe_methods = set() - - # Managing events - self.events = events() - self.event_callbacks = dict() - - # Initialize supporter - self.supporter = parent.supporter(self) - - # Initialize attributes from supporter component - This will also become part of the server's public API - self.log = self.supporter.log - self.validate = self.supporter.validate - self.load_custom_methods = self.supporter.load_custom_methods - self.disable_methods = self.supporter.disable_methods - self.is_json = self.supporter.is_json - self.get_client_ip = self.supporter.get_client_ip - self.detect_listener = self.supporter.detect_listener - self.generate_statuscode = self.supporter.generate_statuscode - - # Client and room management - self.rooms = rooms(self) - self.clients = clients(self) - - # Load Cloudlink methods - self.cl_methods = cl_methods(self) - - # Load safe CLPv4 methods to mitigate vulnerabilities - self.supporter.init_builtin_cl_methods() - - # Load Scratch cloud variable methods - self.scratch_methods = scratch_methods(self) - - # Display version - self.supporter.log(f"Cloudlink server v{parent.version}") - - # == Public API functionality == - - # Runs the server. - def run(self, ip: str = "localhost", port: int = 3000): - try: - self.asyncio.run(self.__run__(ip, port)) - except KeyboardInterrupt: - pass - - # Set the server's Message-of-the-Day. - def set_motd(self, message: str, enable: bool = True): - self.enable_motd = enable - self.motd_message = message - - # Sets the client's username and enables private messages/variables, room link/unlink and direct functionality. - def set_client_username(self, client: type, username: str): - result = self.clients.set_username(client, username) - if result: - self.rooms.refresh(client, "default") - - # Links clients to rooms. Supports strings and lists/sets. - def link_to_rooms(self, client: type, rooms_to_link=[]): - # Prepare rooms - rooms = set() - if type(rooms_to_link) == str: - rooms_to_link = [rooms_to_link] - rooms.update(set(rooms_to_link)) - - # Client cannot be linked to no rooms - if len(rooms) == 0: - rooms.update(["default"]) - - # Manage existing rooms - old_rooms = self.copy(client.rooms) - if ("default" in client.rooms) and ("default" not in rooms): - self.parent.rooms.unlink(client, "default") - for room in old_rooms: - self.parent.rooms.unlink(client, room) - - # Create rooms if they do not exist - for room in rooms: - if not self.parent.rooms.exists(room): - self.parent.rooms.create(room) - - # Link client to new rooms - for room in rooms: - self.parent.rooms.link(client, room) - new_rooms = self.copy(client.rooms) - - # Unlinks clients from rooms. Supports strings and lists/sets. - def unlink_from_rooms(self, client: type, rooms_to_unlink=None): - # Prepare the rooms - rooms = set() - if (not rooms_to_unlink) or (rooms_to_unlink and (len(rooms_to_unlink) == 0)): - # Unlink all rooms - rooms.update(self.copy(client.rooms)) - else: - # Unlink from a single room, or many rooms - if type(rooms_to_unlink) == list: - rooms.update(set(rooms_to_unlink)) - else: - rooms.update(rooms_to_unlink) - - # Unlink client from rooms - old_rooms = self.copy(client.rooms) - for room in rooms: - self.parent.rooms.unlink(client, room) - if len(client.rooms) == 0: - # Reset to default room - self.parent.rooms.link(client, "default") - new_rooms = self.copy(client.rooms) - - # Binding callbacks - Provides a programmer-friendly interface to run extra code after a cloudlink method has been executed. - def bind_callback(self, callback_method: type, function: type): - if hasattr(self.cl_methods, callback_method.__name__) or hasattr(self.custom_methods, callback_method.__name__): - if callback_method.__name__ not in self.method_callbacks: - self.method_callbacks[callback_method.__name__] = set() - self.method_callbacks[callback_method.__name__].add(function) - - # Binding events - Provides a programmer-friendly interface to detect client connects, disconnects, and errors. - def bind_event(self, event_method: type, function: type): - if hasattr(self.events, event_method.__name__): - if event_method.__name__ not in self.event_callbacks: - self.event_callbacks[event_method.__name__] = set() - self.event_callbacks[event_method.__name__].add(function) - - # Multicasting variables - Automatically translates Scratch and Cloudlink. Used only for gvar. - async def send_packet_multicast_variable(self, cmd: str, name: str, val: any = None, clients: type = None, - exclude_client: any = None, room_id: str = None, new_name: str = None): - # Get all clients present - tmp_clients = None - if clients: - tmp_clients = set(clients) - else: - tmp_clients = set(self.copy(self.rooms.get(room_id).users)) - if not tmp_clients: - return - - if exclude_client: - tmp_clients.discard(exclude_client) - clients_cl = set() - clients_scratch = set() - - # Filter clients by protocol type - for client in tmp_clients: - match client.protocol: - case self.supporter.proto_cloudlink: - clients_cl.add(client) - case self.supporter.proto_scratch_cloud: - clients_scratch.add(client) - - # Methods that are marked with NoneType will not be translated - translate_cl = { - "set": "gvar", - "create": "gvar", - "delete": None, - "rename": None - } - translate_scratch = { - "gvar": "set", - "pvar": None - } - - # Translate between Scratch and Cloudlink - if cmd in translate_cl: - # Send message to all scratch protocol clients - message = {"method": cmd, "name": name} - tmp = self.copy(message) - - if val: - message["value"] = val - - # Prevent crashing Scratch clients if the val is a dict / JSON object - tmp = self.copy(message) - if type(tmp["value"]) == dict: - tmp["value"] = self.json.dumps(tmp["value"]) - - if new_name: - message["new_name"] = new_name - - self.websockets.broadcast(clients_scratch, self.json.dumps(tmp)) - - if (translate_cl[cmd]) and (len(clients_cl) != 0): - # Send packet to only cloudlink protocol clients - message = {"cmd": translate_cl[cmd], "name": name, "rooms": room_id} - if val: - message["val"] = val - - # Translate JSON string to dict - if type(message["val"]) == str: - try: - message["val"] = self.json.loads(message["val"]) - except: - pass - - self.websockets.broadcast(clients_cl, self.json.dumps(message)) - - elif cmd in translate_scratch: - # Send packet to only cloudlink protocol clients - message = {"cmd": cmd, "name": name, "val": val, "rooms": room_id} - - # Translate JSON string to dict - if type(message["val"]) == str: - try: - message["val"] = self.json.loads(message["val"]) - except: - pass - - self.websockets.broadcast(clients_cl, self.json.dumps(message)) - - if (translate_scratch[cmd]) and (len(clients_scratch) != 0): - # Send message to all scratch protocol clients - message = {"method": translate_scratch[cmd], "name": name, "value": val} - - # Prevent crashing Scratch clients if the val is a dict / JSON object - if type(message["value"]) == dict: - message["value"] = self.json.dumps(message["value"]) - - self.websockets.broadcast(clients_scratch, self.json.dumps(message)) - else: - raise TypeError("Command is not translatable!") - - # Multicast data - Used for gmsg, gvar, or multicasted direct messages. - async def send_packet_multicast(self, cmd: str, val: any, clients: type = None, exclude_client: any = None, - room_id: str = None, quirk: str = "quirk_embed_val"): - # Get all clients present - tmp_clients = None - if clients: - tmp_clients = set(clients) - else: - tmp_clients = set(self.copy(self.rooms.get(room_id).users)) - if not tmp_clients: - return - - # Remove individual client - if exclude_client: - tmp_clients.discard(exclude_client) - - # Purge clients that aren't cloudlink - for client in self.copy(tmp_clients): - if client.protocol == self.supporter.proto_scratch_cloud: - tmp_clients.discard(client) - - # Send packet to only cloudlink protocol clients - message = {"cmd": cmd} - if quirk == self.supporter.quirk_update_msg: - message.update(val) - elif quirk == self.supporter.quirk_embed_val: - message["val"] = val - else: - raise ValueError("Unknown message quirk!") - - # Attach the rooms key - if room_id: - message["rooms"] = room_id - - self.supporter.log_debug(f"Multicasting payload: {message}") - - # Send payload - self.websockets.broadcast(tmp_clients, self.json.dumps(message)) - - # Unicast data - Used for pmsg, pvar, or unicasted direct messages. - async def send_packet_unicast(self, client: type, cmd: str, val: any, listener: str = None, room_id: str = None, - quirk: str = "quirk_embed_val"): - # Check client protocol - if client.protocol == self.supporter.proto_unset: - raise Exception("Cannot send packet to a client with an unset protocol!") - if client.protocol != self.supporter.proto_cloudlink: - raise TypeError("Unsupported protocol type!") - - # Manage specific message quirks - message = {"cmd": cmd} - if quirk == self.supporter.quirk_update_msg: - message.update(val) - elif quirk == self.supporter.quirk_embed_val: - message["val"] = val - else: - raise TypeError("Unknown message quirk!") - - # Attach a listener response - if listener: - message["listener"] = listener - - # Attach the rooms key - if room_id: - message["rooms"] = room_id - - self.supporter.log_debug(f"Unicasting payload: {message}") - - # Send payload - try: - await client.send(self.json.dumps(message)) - except self.websockets.exceptions.ConnectionClosedError: - self.supporter.log_error(f"Failed to send packet to client {client.id}: Connection closed unexpectedly") - - # Unicast status codes - Only used for statuscode. - async def send_code(self, client: type, code: str, extra_data: dict = None, listener: str = None): - # Check client protocol - if client.protocol == self.supporter.proto_unset: - raise Exception("Cannot send codes to a client with an unset protocol!") - if client.protocol != self.supporter.proto_cloudlink: - raise TypeError("Unsupported protocol type!") - - # Prepare message - human_code, machine_code = self.generate_statuscode(code) - message = { - "cmd": "statuscode", - "code": human_code, - "code_id": machine_code - } - - # Attach extra data - if extra_data: - message.update(extra_data) - - # Attach a listener response - if listener: - message["listener"] = listener - - self.supporter.log_debug(f"Sending payload: {message}") - - # Send payload - try: - await client.send(self.json.dumps(message)) - except self.websockets.exceptions.ConnectionClosedError: - self.supporter.log_error(f"Failed to send status code to client {client.id}: Connection closed unexpectedly") - - # == Server functionality == - - async def __run__(self, ip, port): - # Main event loop - async with self.websockets.serve(self.__handler__, ip, port): - await self.asyncio.Future() - - async def reject_client(self, client, reason): - await client.close(code=1001, reason=reason) - - def __fire_callbacks__(self, callback_method, client, message, listener): - if callback_method.__name__ in self.method_callbacks: - for _method in self.method_callbacks[callback_method.__name__]: - self.asyncio.create_task(_method(client, message, listener)) - - def __fire_event__(self, event_method, client): - if event_method.__name__ in self.event_callbacks: - for _method in self.event_callbacks[event_method.__name__]: - self.asyncio.create_task(_method(client)) - - async def __cl_method_handler__(self, client: type, message: dict): - # Check if the message contains the cmd key, with a string datatype. - match self.validate({"cmd": str}, message): - case self.supporter.invalid: - return self.supporter.invalid - case self.supporter.missing_key: - return self.supporter.missing_key - case self.supporter.not_a_dict: - raise TypeError - - # Detect and convert CLPv3 custom requests to CLPv4 - if (message["cmd"] == "direct") and ("val" in message): - if self.validate({"cmd": str, "val": self.supporter.keydefaults["val"]}, - message["val"]) == self.supporter.valid: - tmp_msg = { - "cmd": message["val"]["cmd"], - "val": message["val"]["val"] - } - message = self.copy(tmp_msg) - - # Detect listeners - listener = self.detect_listener(message) - - # Check if the command is disabled - if message["cmd"] in self.disabled_methods: - return self.supporter.disabled_method - - # Check if the command method exists - # Custom methods override / take precedence over builtin methods - if hasattr(self.custom_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.custom_methods, message["cmd"]) - - await method(client, message, listener) - self.__fire_callbacks__(method, client, message, listener) - - return self.supporter.valid - - # Run builtin method - elif hasattr(self.cl_methods, message["cmd"]) and (message["cmd"] in self.safe_methods): - method = getattr(self.cl_methods, message["cmd"]) - - await method(client, message, listener) - self.__fire_callbacks__(method, client, message, listener) - - return self.supporter.valid - - # Method not found - else: - return self.supporter.unknown_method - - async def __scratch_method_handler__(self, client: type, message: dict): - # Check if the message contains the method key, with a string datatype. - match self.validate({"method": str}, message): - case self.supporter.invalid: - return self.supporter.invalid - case self.supporter.missing_key: - return self.supporter.missing_key - case self.supporter.not_a_dict: - raise TypeError - - # Check if the command method exists - if not hasattr(self.scratch_methods, message["method"]): - return self.supporter.unknown_method - - # Run the method - await getattr(self.scratch_methods, message["method"])(client, message) - return self.supporter.valid - - async def __run_method__(self, client: type, message: dict): - # Detect connection protocol (supports Cloudlink CLPv4 or Scratch Cloud Variables Protocol) - if client.protocol == self.supporter.proto_unset: - if "cmd" in message: - # Update the client's protocol type and update the clients iterable - client.protocol = self.supporter.proto_cloudlink - self.clients.set_protocol(client, self.supporter.proto_cloudlink) - - # Link client to the default room - self.rooms.link(client, "default") - - return await self.__cl_method_handler__(client, message) - - elif "method" in message: - if not self.enable_scratch_support: - await client.close(code=1000, reason="Scratch protocol is disabled") - return - - # Update the client's protocol type and update the clients iterable - client.protocol = self.supporter.proto_scratch_cloud - self.clients.set_protocol(client, self.supporter.proto_scratch_cloud) - - return await self.__scratch_method_handler__(client, message) - else: - # Reject the client because the server does not understand the protocol being used - return self.supporter.unknown_protocol - - elif client.protocol == self.supporter.proto_cloudlink: - # Interpret and process CL commands - return await self.__cl_method_handler__(client, message) - - elif client.protocol == self.supporter.proto_scratch_cloud: - # Interpret and process Scratch commands - return await self.__scratch_method_handler__(client, message) - - else: - raise TypeError(f"Unknown protocol type: {client.protocol}") - - async def __handler__(self, client): - if self.check_ip_addresses: - # Get the IP address of client - client.full_ip = self.get_client_ip(client) - - rejected = False - if self.reject_clients: - rejected = True - await client.close(code=1013, reason="Reject mode is enabled") - self.supporter.log(f"Client disconnected in reject mode: {client.full_ip}") - elif self.check_ip_addresses and (client.full_ip in self.ip_blocklist): - rejected = True - self.supporter.log(f"Client rejected: IP address {client.full_ip} blocked") - await client.close(code=1008, reason="IP blocked") - - # Do absolutely nothing if the client was rejected - if not rejected: - # Set the initial protocol type - client.protocol = self.supporter.proto_unset - - # Assign an ID to the client - client.id = self.id_counter - self.id_counter += 1 - - # Register the client - self.clients.create(client) - - # Configure client - client.rooms = set() - client.username_set = False - client.linked = False - client.friendly_username = None - - # Log event - if self.check_ip_addresses: - self.supporter.log(f"Client {client.id} connected: {client.full_ip}") - else: - self.supporter.log(f"Client {client.id} connected") - - # Fire events - self.__fire_event__(self.events.on_connect, client) - - # Handle requests from the client - try: - async for tmp_msg in client: - # Handle empty payloads - if len(tmp_msg) == 0: - if client.protocol == self.supporter.proto_cloudlink: - await self.send_code(client, "EmptyPacket") - continue - elif client.protocol == self.supporter.proto_scratch_cloud: - await client.close(code=self.supporter.connection_error, reason="Empty message") - else: - await client.close(code=1002, reason="Empty message") - - # Convert/sanity check JSON - message = None - try: - message = self.json.loads(tmp_msg) - except: - if client.protocol == self.supporter.proto_cloudlink: - await self.send_code(client, "Syntax") - continue - elif client.protocol == self.supporter.proto_scratch_cloud: - await client.close(code=self.supporter.connection_error, reason="Corrupt/malformed JSON") - else: - await client.close(code=1002, reason="Corrupt/malformed JSON") - - # Fire events - self.__fire_event__(self.events.on_error, client) - - # Run handlers - if message: - # Convert keys in the packet to proper JSON (Primarily for Scratch-based clients) - for key in message.keys(): - if type(message[key]) == str: - if self.supporter.is_json(message[key]): - message[key] = self.json.loads(message[key]) - - result = await self.__run_method__(client, message) - match result: - case self.supporter.disabled_method: - listener = self.supporter.detect_listener(message) - await self.send_code(client, "Disabled", listener=listener) - - case self.supporter.invalid: - if client.protocol == self.supporter.proto_cloudlink: - listener = self.supporter.detect_listener(message) - await self.send_code(client, "Syntax", listener=listener) - - elif client.protocol == self.proto_scratch_cloud: - await client.close(code=self.supporter.connection_error, reason="Bad method") - - case self.supporter.missing_key: - if client.protocol == self.supporter.proto_cloudlink: - listener = self.supporter.detect_listener(message) - await self.send_code(client, "Syntax", listener=listener) - - elif client.protocol == self.proto_scratch_cloud: - await client.close(code=self.supporter.connection_error, - reason="Missing method key in JSON") - - case self.supporter.unknown_method: - if client.protocol == self.supporter.proto_cloudlink: - listener = self.supporter.detect_listener(message) - await self.send_code(client, "Invalid", listener=listener) - - elif client.protocol == self.proto_scratch_cloud: - await client.close(code=self.supporter.connection_error, reason="Invalid method") - - case self.supporter.unknown_protocol: - await client.close(code=1002, reason="Unknown protocol") - - case _: - self.__fire_event__(self.events.on_msg, client) - - # Handle unexpected disconnects - except self.websockets.exceptions.ConnectionClosedError: - pass - - # Handle OK disconnects - except self.websockets.exceptions.ConnectionClosedOK: - pass - - # Handle unexpected exceptions - except Exception as e: - self.supporter.log_error(f"Exception was raised: \"{e}\"\n{self.supporter.full_stack()}") - await client.close(code=1011, reason="Unexpected exception was raised") - - # Gracefully shutdown the handler - finally: - if client.username_set and (client.protocol == self.supporter.proto_cloudlink): - # Alert clients that a client has disconnected - rooms = self.copy(client.rooms) - for room in rooms: - self.rooms.unlink(client, room) - room_data = self.rooms.get(room) - if not room_data: - continue - - # Update the state - await self.send_packet_multicast( - cmd="ulist", - val={ - "mode": "remove", - "val": self.clients.convert_json(client) - }, - quirk=self.supporter.quirk_update_msg, - room_id=room - ) - - # Dispose the client - self.clients.delete(client) - - # Fire events - self.__fire_event__(self.events.on_close, client) - - # Log event - if self.check_ip_addresses: - self.supporter.log(f"Client {client.id} disconnected: {client.full_ip} - Code {client.close_code} and reason \"{client.close_reason}\"") - else: - self.supporter.log(f"Client {client.id} disconnected: Code {client.close_code} and reason \"{client.close_reason}\"") - - -# Class to store custom methods -class custom_methods: - def __init__(self): - pass - - -# Clients management -class clients: - def __init__(self, parent): - self.__all_cl__ = set() - self.__all_scratch__ = set() - self.__proto_unset__ = parent.supporter.proto_unset - self.__proto_cloudlink__ = parent.supporter.proto_cloudlink - self.__proto_scratch_cloud__ = parent.supporter.proto_scratch_cloud - self.__parent__ = parent - self.__usernames__ = dict() - - def get_all_scratch(self): - return self.__all_scratch__ - - def get_all_cloudlink(self): - return self.__all_cl__ - - def get_all_usernames(self): - return self.__usernames__ - - def get_all(self): - tmp = self.__parent__.copy(self.__dict__) - - # Remove attributes that aren't client objects - del tmp["__all_cl__"] - del tmp["__all_scratch__"] - del tmp["__proto_unset__"] - del tmp["__proto_cloudlink__"] - del tmp["__proto_scratch_cloud__"] - del tmp["__usernames__"] - del tmp["__parent__"] - - return tmp - - def set_protocol(self, client: dict, protocol: str): - match protocol: - case self.__proto_cloudlink__: - self.__all_cl__.add(client) - case self.__proto_scratch_cloud__: - self.__all_scratch__.add(client) - case _: - raise TypeError(f"Unsupported protocol ID: {protocol}") - - if self.exists(client): - self.get(client).protocol = protocol - - else: - raise ValueError - - def exists(self, client: dict): - return hasattr(self, str(client.id)) - - def create(self, client: dict): - if not self.exists(client): - setattr(self, str(client.id), client) - - def set_username(self, client: dict, username: str): - # Abort if the client is invalid - if not self.exists(client): - return self.__parent__.supporter.username_not_set - - # Create new username set if not present - if not str(username) in self.__usernames__: - self.__usernames__[str(username)] = set() - - # Add pointer to client object - self.__usernames__[str(username)].add(client) - - # Client username has been set successfully - client.friendly_username = str(username) - client.username_set = True - return self.__parent__.supporter.username_set - - def delete(self, client: dict): - if self.exists(client): - # Remove user from client type lists - match self.get(client).protocol: - case self.__proto_cloudlink__: - self.__all_cl__.discard(client) - case self.__proto_scratch_cloud__: - self.__all_scratch__.discard(client) - case self.__proto_unset__: - pass - case _: - raise TypeError(f"Unsupported protocol ID: {self.get(client).protocol}") - - # Remove the username from the userlist - if client.username_set: - if str(client.friendly_username) in self.__usernames__: - # Remove client from the shared username set - if client in self.__usernames__[str(client.friendly_username)]: - self.__usernames__[str(client.friendly_username)].remove(client) - - # Delete the username if there are no more users present with that name - if len(self.__usernames__[str(client.friendly_username)]) == 0: - del self.__usernames__[str(client.friendly_username)] - - # Clean up rooms - if hasattr(client, "rooms") and client.rooms: - for room in self.__parent__.copy(client.rooms): - self.__parent__.rooms.unlink(client, room) - - # Dispose the client - delattr(self, str(client.id)) - - def get(self, client: dict): - if self.exists(client): - return getattr(self, str(client.id)) - else: - return None - - def convert_json(self, client: any): - return {"username": client.friendly_username, "id": client.id} - - def find_multi_obj(self, clients_to_find, rooms="default"): - if not type(rooms) in [set, list]: - rooms = {rooms} - - tmp = set() - - # Check if the input is iterable - if type(clients_to_find) in [list, set]: - for room in rooms: - for client in clients_to_find: - res = self.find_obj(client, room) - if not res: - continue - if type(res) in [list, set]: - tmp.update(res) - else: - tmp.add(res) - else: - for room in rooms: - res = self.find_obj(clients_to_find, room) - if not res: - continue - if type(res) in [list, set]: - tmp.update(res) - else: - tmp.add(res) - - return tmp - - def find_obj(self, client_to_find, room_id=None): - room_user_objs = set() - room_user_names = dict() - if room_id: - if not self.__parent__.rooms.exists(room_id): - return None - - room_user_objs = self.__parent__.rooms.get(room_id).clients - room_user_names = self.__parent__.rooms.get(room_id).usernames_searchable - else: - room_user_objs = self.get_all() - room_user_names = self.get_all_usernames() - - if type(client_to_find) == str: - if client_to_find in room_user_names: - return room_user_names[client_to_find] - - elif type(client_to_find) == dict: - if "id" in client_to_find: - if str(client_to_find["id"]) in room_user_objs: - return room_user_objs[str(client_to_find["id"])] - - elif type(client_to_find) == int: - if str(client_to_find) in room_user_objs: - return room_user_objs[str(client_to_find)] - - else: - return None - - -# Rooms management -class rooms: - def __init__(self, parent): - self.default = self.__room__() - self.__parent__ = parent - - def get_all(self): - tmp = self.__parent__.copy(self.__dict__) - - # Remove attributes that aren't client objects - del tmp["__parent__"] - - return tmp - - def exists(self, room_id: str): - return hasattr(self, str(room_id)) - - def create(self, room_id: str): - if not self.exists(str(room_id)): - setattr(self, str(room_id), self.__room__()) - - def delete(self, room_id: str): - if self.exists(str(room_id)): - delattr(self, str(room_id)) - - def get(self, room_id: str): - if self.exists(str(room_id)): - return getattr(self, str(room_id)) - else: - return None - - def unlink(self, client, room_id): - room_data = self.get(room_id) - if room_data: - # Remove the user from the room - if client in room_data.users: - room_data.users.remove(client) - - if client.username_set: - # Remove the user from the room's JSON-friendly userlist - user_json = self.__parent__.clients.convert_json(client) - if user_json in room_data.userlist: - room_data.userlist.remove(user_json) - - if client.friendly_username in room_data.usernames: - room_data.usernames.remove(client.friendly_username) - - if (not room_id == "default") and (len(room_data.users) == 0): - self.delete(room_id) - - if str(client.id) in room_data.clients: - del room_data.clients[str(client.id)] - - if client.friendly_username in room_data.usernames_searchable: - del room_data.usernames_searchable[client.friendly_username] - - if room_id in client.rooms: - client.rooms.remove(room_id) - - def link(self, client, room_id): - # Get the room data - room_data = self.get(room_id) - if room_data: - # Add the user to the room - room_data.users.add(client) - - if room_id not in client.rooms: - client.rooms.add(room_id) - - if client.username_set: - # Add the user to the room's JSON-friendly userlist - user_json = self.__parent__.clients.convert_json(client) - room_data.userlist.append(user_json) - - if client.friendly_username not in room_data.usernames: - room_data.usernames.add(client.friendly_username) - - if str(client.id) not in room_data.clients: - room_data.clients[str(client.id)] = client - - if client.friendly_username not in room_data.usernames_searchable: - room_data.usernames_searchable[client.friendly_username] = set() - - if client not in room_data.usernames_searchable[client.friendly_username]: - room_data.usernames_searchable[client.friendly_username].add(client) - - def refresh(self, client, room_id): - if self.exists(room_id): - self.unlink(client, room_id) - self.link(client, room_id) - - class __room__: - def __init__(self): - # Global data stream current value - self.global_data_value = str() - - # Storage of all global variables / Scratch Cloud Variables - self.global_vars = dict() - - # User management - self.users = set() - self.usernames = set() - self.userlist = list() - - # Client management - self.clients = dict() - - # Used only for string-based user lookup - self.usernames_searchable = dict() - - -# Class for binding events -class events: - def __init__(self): - pass - - def on_connect(self): - pass - - def on_error(self): - pass - - def on_close(self): - pass - - def on_msg(self): - pass diff --git a/cloudlink/supporter.py b/cloudlink/supporter.py deleted file mode 100644 index 0760cf7..0000000 --- a/cloudlink/supporter.py +++ /dev/null @@ -1,274 +0,0 @@ -import sys -import traceback -from datetime import datetime -import logging - -class supporter: - def __init__(self, parent): - self.parent = parent - - # Use logging library - self.logger = logging - - # Reconfigure when needed - self.logger.basicConfig(format="[%(asctime)s | %(created)f] (%(thread)d - %(threadName)s) %(levelname)s: %(message)s", level=self.logger.INFO) - - # Define protocol types - self.proto_unset = "proto_unset" - self.proto_cloudlink = "proto_cloudlink" - self.proto_scratch_cloud = "proto_scratch_cloud" - - # Multicasting message quirks - self.quirk_embed_val = "quirk_embed_val" - self.quirk_update_msg = "quirk_update_msg" - - # Case codes - self.valid = 0 - self.invalid = 1 - self.missing_key = 2 - self.not_a_dict = 3 - self.unknown_method = 4 - self.unknown_protocol = 5 - self.username_set = 6 - self.username_not_set = 7 - self.disabled_method = 8 - self.too_large = 9 - - # Scratch error codes - self.connection_error = 4000 - self.username_error = 4002 - self.overloaded = 4003 - self.unavailable = 4004 - self.refused_security = 4005 - - # Status codes - self.info = "I" - self.error = "E" - self.codes = { - "Test": (self.info, 0, "Test"), - "OK": (self.info, 100, "OK"), - "Syntax": (self.error, 101, "Syntax"), - "DataType": (self.error, 102, "Datatype"), - "IDNotFound": (self.error, 103, "ID not found"), - "IDNotSpecific": (self.error, 104, "ID not specific enough"), - "InternalServerError": (self.error, 105, "Internal server error"), - "EmptyPacket": (self.error, 106, "Empty packet"), - "IDSet": (self.error, 107, "ID already set"), - "Refused": (self.error, 108, "Refused"), - "Invalid": (self.error, 109, "Invalid command"), - "Disabled": (self.error, 110, "Command disabled"), - "IDRequired": (self.error, 111, "ID required"), - "IDConflict": (self.error, 112, "ID conflict"), - "TooLarge": (self.error, 113, "Too large") - } - - # Method default keys and permitted datatypes - self.keydefaults = { - "val": [str, int, float, list, dict], - "id": [str, int, dict, set, list], - "listener": [str, dict, float, int], - "rooms": [str, list], - "name": str, - "user": str, - "project_id": str, - "method": str, - "cmd": str, - "value": [str, int, float] - } - - # New and improved version of the message sanity checker. - def validate(self, keys: dict, payload: dict, optional=None, sizes: dict = None): - # Check if input datatypes are valid - if optional is None: - optional = [] - if (type(keys) != dict) or (type(payload) != dict): - return self.not_a_dict - - self.log_debug(f"Running validator: {keys}, {payload}, {optional}, {sizes}") - - for key in keys.keys(): - # Check if a key is present - if (key in payload) or (key in optional): - # Bypass checks if a key is optional and not present - if (key not in payload) and (key in optional): - self.log_debug(f"Validator: Payload {payload} key {key} is optional") - continue - - # Check if there are multiple supported datatypes for a key - if type(keys[key]) == list: - # Validate key datatype - if not type(payload[key]) in keys[key]: - self.log_debug(f"Validator: Payload {payload} key {key} value is invalid type. Expecting {keys[key]}, got {type(payload[key])}") - return self.invalid - - # Check if the size of the payload is too large - if sizes: - if (key in sizes.keys()) and (len(str(payload[key])) > sizes[key]): - self.log_debug(f"Validator: Payload {payload} key {key} value is too large") - return self.too_large - - else: - # Validate key datatype - if type(payload[key]) != keys[key]: - self.log_debug(f"Validator: Payload {payload} key {key} value is invalid type. Expecting {keys[key]}, got {type(payload[key])}") - return self.invalid - - # Check if the size of the payload is too large - if sizes: - if (key in sizes.keys()) and (len(str(payload[key])) > sizes[key]): - self.log_debug(f"Validator: Payload {payload} key {key} value is too large") - return self.too_large - else: - self.log_debug(f"Validator: Payload {payload} is missing key {key}") - return self.missing_key - - # Hooray, the message is sane - self.log_debug(f"Validator: Payload {payload} is valid") - return self.valid - - def full_stack(self): - exc = sys.exc_info()[0] - if exc is not None: - f = sys.exc_info()[-1].tb_frame.f_back - stack = traceback.extract_stack(f) - else: - stack = traceback.extract_stack()[:-1] - trc = 'Traceback (most recent call last):\n' - stackstr = trc + ''.join(traceback.format_list(stack)) - if exc is not None: - stackstr += ' ' + traceback.format_exc().lstrip(trc) - return stackstr - - def is_json(self, json_str): - is_valid_json = False - try: - if type(json_str) == dict: - is_valid_json = True - elif type(json_str) == str: - json_str = self.json.loads(json_str) - is_valid_json = True - except: - is_valid_json = False - return is_valid_json - - def get_client_ip(self, client: dict): - if "x-forwarded-for" in client.request_headers: - return client.request_headers.get("x-forwarded-for") - elif "cf-connecting-ip" in client.request_headers: - return client.request_headers.get("cf-connecting-ip") - else: - if type(client.remote_address) == tuple: - return str(client.remote_address[0]) - else: - return client.remote_address - - def generate_statuscode(self, code: str): - if code in self.codes: - c_type, c_code, c_msg = self.codes[code] - return f"{c_type}:{c_code} | {c_msg}", c_code - else: - raise ValueError - - # Determines if a method has a listener - def detect_listener(self, message): - validation = self.validate( - { - "listener": self.keydefaults["listener"] - }, - message - ) - - match validation: - case self.invalid: - return None - case self.missing_key: - return None - - return message["listener"] - - # Internal usage only, not for use in Public API - def get_rooms(self, client, message): - rooms = set() - if "rooms" not in message: - rooms.update(client.rooms) - else: - if type(message["rooms"]) == str: - message["rooms"] = [message["rooms"]] - rooms.update(set(message["rooms"])) - - # Filter rooms client doesn't have access to - for room in self.copy(rooms): - if room not in client.rooms: - rooms.remove(room) - return rooms - - # Disables methods. Supports disabling built-in methods for monkey-patching or for custom reimplementation. - def disable_methods(self, functions: list): - if type(functions) != list: - raise TypeError - - for function in functions: - if type(function) != str: - continue - - if function not in self.parent.disabled_methods: - self.parent.disabled_methods.add(function) - - self.parent.safe_methods.discard(function) - - # Support for loading custom methods. Automatically selects safe methods. - def load_custom_methods(self, _class): - for function in dir(_class): - # Ignore loading private methods - if "__" in function: - continue - - # Do not initialize server methods - if hasattr(self.parent, function): - continue - if hasattr(self.parent.supporter, function): - continue - - # Ignore loading commands marked as ignore - if hasattr(_class, "importer_ignore_functions"): - if function in _class.importer_ignore_functions: - continue - - setattr(self.parent.custom_methods, function, getattr(_class, function)) - self.parent.safe_methods.add(function) - - # This initializes methods that are guaranteed safe to use. This mitigates the possibility of clients accessing - # private or sensitive methods. - def init_builtin_cl_methods(self): - for function in dir(self.parent.cl_methods): - # Ignore loading private methods - if "__" in function: - continue - - # Do not initialize server methods - if hasattr(self.parent, function): - continue - if hasattr(self.parent.supporter, function): - continue - - # Ignore loading commands marked as ignore - if hasattr(self.parent.cl_methods, "importer_ignore_functions"): - if function in self.parent.cl_methods.importer_ignore_functions: - continue - - self.parent.safe_methods.add(function) - - def log(self, msg): - self.logger.info(msg) - - def log_debug(self, msg): - self.logger.debug(msg) - - def log_warning(self, msg): - self.logger.warning(msg) - - def log_critical(self, msg): - self.logger.critical(msg) - - def log_error(self, msg): - self.logger.error(msg) \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 9f0a10d..52ed0a6 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,2 +1,4 @@ -websockets -websocket-client \ No newline at end of file +snowflake-id>=0.0.2 +cerberus>=1.3.4 +websockets>=10.4 +ujson>=5.7.0 diff --git a/server_example.py b/server_example.py index d79f10f..a6a07ed 100644 --- a/server_example.py +++ b/server_example.py @@ -1,24 +1,45 @@ -from cloudlink import cloudlink +from cloudlink import server +from cloudlink.server.protocols import clpv4, scratch +import asyncio class example_callbacks: def __init__(self, parent): self.parent = parent - async def test1(self, client, message, listener): + async def test1(self, client, message): print("Test1!") - await self.parent.asyncio.sleep(1) + await asyncio.sleep(1) print("Test1 after one second!") - - async def test2(self, client, message, listener): + + async def test2(self, client, message): print("Test2!") - await self.parent.asyncio.sleep(1) + await asyncio.sleep(1) print("Test2 after one second!") - - async def test3(self, client, message, listener): + + async def test3(self, client, message): print("Test3!") +class example_commands: + def __init__(self, parent, protocol): + + # Creating custom commands - This example adds a custom command called "foobar". + @server.on_command(cmd="foobar", schema=protocol.schema) + async def foobar(client, message): + print("Foobar!") + + # Reading the IP address of the client is as easy as calling get_client_ip from the clpv4 protocol object. + print(protocol.get_client_ip(client)) + + # In case you need to report a status code, use send_statuscode. + protocol.send_statuscode( + client=client, + code=protocol.statuscodes.ok, + message=message + ) + + class example_events: def __init__(self): pass @@ -27,82 +48,45 @@ async def on_close(self, client): print("Client", client.id, "disconnected.") async def on_connect(self, client): - print("Client", client.id, "connected.") - - -class example_commands: - def __init__(self, parent): - self.parent = parent - self.supporter = parent.supporter - - # If you want to have commands with very specific formatting, use the validate() function. - self.validate = parent.validate - - # Various ways to send messages - self.send_packet_unicast = parent.send_packet_unicast - self.send_packet_multicast = parent.send_packet_multicast - self.send_packet_multicast_variable = parent.send_packet_multicast_variable - self.send_code = parent.send_code - - async def foobar(self, client, message, listener): - print("Foobar!") - - # Reading the IP address of the client is as easy as calling get_client_ip from the server object. - print(self.parent.get_client_ip(client)) - - # In case you need to report a status code, use send_code. - await self.send_code( - client=client, - code="OK", - listener=listener - ) + print("Client", client.id, "connected.") if __name__ == "__main__": - # Initialize Cloudlink. You will only need to initialize one instance of the main cloudlink module. - cl = cloudlink() - - # Create a new server object. This supports initializing many servers at once. - server = cl.server(logs=True) - - # Create examples for various ways to extend the functionality of Cloudlink Server. + # Initialize the server + server = server() + + # Configure logging settings + server.logging.basicConfig( + level=server.logging.DEBUG + ) + + # Load protocols + clpv4 = clpv4(server) + scratch = scratch(server) + + # Load examples callbacks = example_callbacks(server) - commands = example_commands(server) + commands = example_commands(server, clpv4) events = example_events() - # Set the message-of-the-day. - server.set_motd("CL4 Optimized! Gotta Go Fast!", True) - - # Here are some extra parameters you can specify to change the functionality of the server. - - # Defaults to empty list. Requires having check_ip_addresses set to True. - # server.ip_blocklist = ["127.0.0.1"] - - # Defaults to False. If True, the server will refuse all connections until False. - # server.reject_clients = False - - # Defaults to False. If True, client IP addresses will be resolved and stored until a client disconnects. - # server.check_ip_addresses = True - - # Defaults to True. If True, the server will support Scratch's cloud variable protocol. - # server.enable_scratch_support = False - # Binding callbacks - This example binds the "handshake" command with example callbacks. # You can bind as many functions as you want to a callback, but they must use async. # To bind callbacks to built-in methods (example: gmsg), see cloudlink.cl_methods. - server.bind_callback(server.cl_methods.handshake, callbacks.test1) - server.bind_callback(server.cl_methods.handshake, callbacks.test2) + server.bind_callback(cmd="handshake", schema=clpv4.schema, method=callbacks.test1) + server.bind_callback(cmd="handshake", schema=clpv4.schema, method=callbacks.test2) # Binding events - This example will print a client connect/disconnect message. # You can bind as many functions as you want to an event, but they must use async. # To see all possible events for the server, see cloudlink.events. - server.bind_event(server.events.on_connect, events.on_connect) - server.bind_event(server.events.on_close, events.on_close) - - # Creating custom commands - This example adds a custom command "foobar" from example_commands - # and then binds the callback test3 to the new command. - server.load_custom_methods(commands) - server.bind_callback(commands.foobar, callbacks.test3) - - # Run the server. - server.run(ip="localhost", port=3000) + server.bind_event(server.on_connect, events.on_connect) + server.bind_event(server.on_disconnect, events.on_close) + + # You can also bind an event to a custom command. We'll bind callbacks.test3 to our + # foobar command from earlier. + server.bind_callback(cmd="foobar", schema=clpv4.schema, method=callbacks.test3) + + # Initialize SSL support + # server.enable_ssl(certfile="cert.pem", keyfile="privkey.pem") + + # Start the server + server.run(ip="127.0.0.1", port=3000)