Quality-of-Life updates

- disco.api.client: define a repr for Disco's APIClient object;
- disco.bot.bot: define a repr for Disco's Bot object;
- disco.bot.bot: log invalid plugin paths as errors instead of exceptions;
- disco.gateway.client: define a repr for Disco's GatewayClient object;
- disco.gateway.client: use the websocket's ping to calculate latency instead of running our own calculation;
- disco.voice.client: use the websocket's ping to calculate latency instead of running our own calculation;
- disco.voice.client: keep list of VoiceClient.experiments as sent from the gateway;
- disco.voice.client: define a repr for Disco's VoiceClient object;
- disco.voice.client: log invalid voice gateway packets as errors instead of exceptions;
- disco.voice.packets: map `CLIENT_CONNECT` as OP 11 (not 18), `CLIENT_FLAGS` as OP 18;
- disco.voice.udp: fix `UDPVoiceClient.connect()` IP stringification;
- disco.client: define a repr for Disco's Client object;
- disco.state: define a repr for Disco's State object;
- disco.state: do not cache webhooks in `User` cache on `MessageCreate` events;
- pyproject.toml: what could possibly go wrong?;
- README.md: there's a meme in here somewhere?;

README.md

@@ -1,29 +1,33 @@
 # BetterDisco
+_BetterDisco_ is an up-to-date modernized fork of Disco, a library witten by b1nzy, the creator of Discord's API, iirc. 
+_Disco_ is a _library_, written in Python 3 to interface with [Discord's API](https://discord.com/developers/docs/intro) as _efficiently_ and _effectively_ as possible. 
+Disco is _expressive_, and contains a _functional interface_. 
+Disco is built for _performance_ and _efficiency_. 
+Disco is _scalable_ and works well in large and small deployments. 
+Disco is _configurable_ and _modular_. 
+Disco contains evented network and IO pipes, courtesy of `gevent`. 
+_Buzzwords **100**._ WYSIWYG. 
 
-Disco is an extensive and extendable Python 3.x library for the [Discord API](https://discord.com/developers/docs/intro). Disco boasts the following major features:
-
-- Expressive, functional interface that gets out of the way
-- Built for high-performance and efficiency
-- Configurable and modular, take the bits you need
-- Full support for Python 3.x
-- Evented networking and IO using Gevent
 
 ## Installation
+Disco is designed to run both as a generic-use library, and as a standalone bot toolkit. Installing disco is as easy as running `pip install betterdisco-py --upgrade --no-cache-dir`, however, additional options are available for extended features, performance, and support:
 
-Disco was built to run both as a generic-use library, and a standalone bot toolkit. Installing disco is as easy as running `pip install disco-py`, however some extra packages are recommended for power-users, namely:
+| _This_                        | Installs _these_                                            | _Why?_                                                                         |
+|-------------------------------|-------------------------------------------------------------|--------------------------------------------------------------------------------|
+| `betterdisco-py`              | `gevent`, `requests`, `websocket-client`                    | Required for base Disco functionality.                                         |
+| `betterdisco-py[http]`        | `flask`                                                     | Useful for hosting an API to interface with your bot.                          |
+| `betterdisco-py[performance]` | `erlpack`, `isal`, `regex`, `pylibyaml`, `ujson`, `wsaccel` | Useful for performance improvement in several areas. _I am speed._             |
+| `betterdisco-py[sharding]`    | `gipc`, `dill`                                              | Required for auto-sharding and inter-process communication.                    |
+| `betterdisco-py[voice]`       | `libnacl`                                                   | Required for VC connectivity and features.                                     |
+| `betterdisco-py[yaml]`        | `pyyaml`                                                    | Required for YAML support, particularly if using `config.yaml`.                |
+| `betterdisco-py[all]`         | _**All of the above**, unless otherwise noted._             | **All additional packages**, for the poweruser that _absolutely needs it all_. |
 
-|Name|Reason|
-|----|------|
-|requests[security]|adds packages for a proper SSL implementation|
-|earl-etf (3.x)|ETF parser run with the --encoder=etf flag|
-|gipc|Gevent IPC, required for autosharding|
 
 ## Examples
-
-Simple bot using the builtin bot authoring tools:
+Simple bot using the built-in bot authoring tools:
 
 ```python
-from disco.bot import Bot, Plugin
+from disco.bot import Plugin
 
 
 class SimplePlugin(Plugin):
@@ -35,18 +39,20 @@ class SimplePlugin(Plugin):
     # They also provide an easy-to-use command component
     @Plugin.command('ping')
     def on_ping_command(self, event):
-        event.msg.reply('Pong!')
+        event.reply('Pong!')
 
     # Which includes command argument parsing
     @Plugin.command('echo', '<content:str...>')
     def on_echo_command(self, event, content):
-        event.msg.reply(content)
+        event.reply(content)
 ```
 
 Using the default bot configuration, we can now run this script like so:
 
 `python -m disco.cli --token="MY_DISCORD_TOKEN" --run-bot --plugin simpleplugin`
 
-And commands can be triggered by mentioning the bot (configured by the BotConfig.command\_require\_mention flag):
+And commands can be triggered by mentioning the bot (configured by the BotConfig.command_require_mention flag):
 
 ![](http://i.imgur.com/Vw6T8bi.png)
+
+### For further information and configuration options, please refer to our documentation first and foremost.

disco/__init__.py

@@ -1 +0,0 @@
-VERSION = '1.0.0'

disco/api/client.py

@@ -69,6 +69,9 @@ def __init__(self, token, client=None):
 
         self._captures = local()
 
+    def __repr__(self):
+        return '<Disco APIClient{}>'.format(f' shard_id={self.client.config.shard_id}' if self.client else '')
+
     def _after_requests(self, response):
         if not hasattr(self._captures, 'responses'):
             return

disco/bot/bot.py

@@ -208,6 +208,9 @@ def __init__(self, client, config=None):
             level = int(level) if str(level).isdigit() else get_enum_value_by_name(CommandLevels, level)
             self.config.levels[entity_id] = level
 
+    def __repr__(self):
+        return f'<DiscoBot bot_id={self.client.state.me.id} shard_id={self.client.config.shard_id}>'
+
     @classmethod
     def from_cli(cls, *plugins):
         """
@@ -577,7 +580,7 @@ def add_plugin_module(self, path, config=None):
             self.add_plugin(plugin, config)
 
         if not loaded:
-            raise Exception(f'Could not find any plugins to load within module {path}')
+            self.log.error(f'Could not find plugins to load within module {path}')
 
     def load_plugin_config(self, cls):
         name = cls.__name__.lower()

disco/client.py

@@ -111,6 +111,9 @@ def __init__(self, config):
                                                localf=lambda: self.manhole_locals)
             self.manhole.start()
 
+    def __repr__(self):
+        return '<DiscoClient{}{}>'.format(f' bot_id={self.state.me.id}' if self.state and self.state.me else '', f' shard_id={self.config.shard_id}')
+
     def update_presence(self, status, game=None, afk=False, since=0.0):
         """
         Updates the current client's presence.

disco/gateway/client.py

@@ -1,7 +1,7 @@
 from gevent import sleep as gevent_sleep, spawn as gevent_spawn
 from gevent.event import Event as GeventEvent
 from platform import system as platform_system
-from time import perf_counter as time_perf_counter, perf_counter_ns as time_perf_counter_ns
+from time import time, perf_counter_ns as time_perf_counter_ns
 from websocket import ABNF, WebSocketConnectionClosedException, WebSocketTimeoutException
 from zlib import decompress as zlib_decompress, decompressobj as zlib_decompressobj
 
@@ -76,6 +76,9 @@ def __init__(self, client, max_reconnects=5, encoder='json', zlib_stream_enabled
         self._last_heartbeat = 0
         self.latency = -1
 
+    def __repr__(self):
+        return f'<GatewayClient shard_id={self.client.config.shard_id} endpoint={self._cached_gateway_url}>'
+
     def send(self, op, data):
         if not self.ws.is_closed:
             self.limiter.check()
@@ -98,7 +101,7 @@ def heartbeat_task(self, interval):
                 self.ws.close(status=1000)
                 self.client.gw.on_close(0, 'HEARTBEAT failure')
                 return
-            self._last_heartbeat = time_perf_counter()
+            self._last_heartbeat = time()
 
             self._send(OPCode.HEARTBEAT, self.seq)
             self._heartbeat_acknowledged = False
@@ -125,7 +128,7 @@ def handle_heartbeat(self, _):
     def handle_heartbeat_acknowledge(self, _):
         self.log.debug('Received HEARTBEAT_ACK')
         self._heartbeat_acknowledged = True
-        self.latency = float('{:.2f}'.format((time_perf_counter() - self._last_heartbeat) * 1000))
+        self.latency = self.ws.last_pong_tm and float('{:.2f}'.format((self.ws.last_pong_tm - self.ws.last_ping_tm) * 1000))
 
     def handle_reconnect(self, _):
         self.log.warning('Received RECONNECT request; resuming')
@@ -175,7 +178,7 @@ def connect_and_run(self, gateway_url=None):
         self.ws.emitter.on('on_close', self.on_close)
         self.ws.emitter.on('on_message', self.on_message)
 
-        self.ws.run_forever()
+        self.ws.run_forever(ping_interval=60, ping_timeout=5)
 
     def on_message(self, msg):
         if self.zlib_stream_enabled:

disco/state.py

@@ -140,6 +140,9 @@ def __init__(self, client, config):
         self.listeners = []
         self.bind()
 
+    def __repr__(self):
+        return f'<DiscoState bot_id={self.me.id} shard_id={self.client.config.shard_id}>'
+
     def unbind(self):
         """
         Unbinds all bound event listeners for this state object.
@@ -172,7 +175,7 @@ def on_user_update(self, event):
         self.me.inplace_update(event.user)
 
     def on_message_create(self, event):
-        if self.config.cache_users and event.message.author.id not in self.users:
+        if self.config.cache_users and event.message.author.id not in self.users and not event.message.webhook_id:
             self.users[event.message.author.id] = event.message.author
 
         if self.config.sync_guild_members and event.message.member:

disco/voice/client.py

@@ -1,5 +1,5 @@
 from gevent import sleep as gevent_sleep, spawn as gevent_spawn
-from time import perf_counter as time_perf_counter, time
+from time import time
 
 from collections import namedtuple as namedtuple
 from websocket import WebSocketConnectionClosedException, WebSocketTimeoutException
@@ -87,6 +87,9 @@ def __init__(self, client, server_id, is_dm=False, max_reconnects=5, encoder='js
         self.video_enabled = video_enabled
         self.media = None
 
+        self.deaf = False
+        self.mute = False
+
         self.proxy = None
 
         # Set the VoiceClient in the state's voice clients
@@ -117,14 +120,15 @@ def __init__(self, client, server_id, is_dm=False, max_reconnects=5, encoder='js
         self.ip = None
         self.port = None
         self.enc_modes = None
-        self.experiments = None
+        self.experiments = []
         # self.streams = None
         self.sdp = None
         self.mode = None
         self.udp = None
         self.audio_codec = None
         self.video_codec = None
         self.transport_id = None
+        self.keyframe_interval = None
         self.secure_frames_version = None
         self.seq = -1
 
@@ -137,6 +141,8 @@ def __init__(self, client, server_id, is_dm=False, max_reconnects=5, encoder='js
         self._heartbeat_acknowledged = True
         self._identified = False
         self._safe_reconnect_state = False
+        self._creation_time = time()
+        self._ws_creation_time = None
 
         # Latency
         self._last_heartbeat = 0
@@ -147,11 +153,8 @@ def __init__(self, client, server_id, is_dm=False, max_reconnects=5, encoder='js
         self.video_ssrcs = {}
         self.rtx_ssrcs = {}
 
-        self.deaf = False
-        self.mute = False
-
     def __repr__(self):
-        return '<VoiceClient guild_id={} channel_id={}>'.format(self.server_id, self.channel_id)
+        return f'<VoiceClient guild_id={self.server_id} channel_id={self.channel_id} endpoint={self.endpoint}>'
 
     @cached_property
     def guild(self):
@@ -219,7 +222,7 @@ def connect_and_run(self, gateway_url=None):
         self.ws.emitter.on('on_error', self.on_error)
         self.ws.emitter.on('on_close', self.on_close)
         self.ws.emitter.on('on_message', self.on_message)
-        self.ws.run_forever()
+        self.ws.run_forever(ping_interval=60, ping_timeout=5)
 
     def heartbeat_task(self, interval):
         while True:
@@ -229,7 +232,7 @@ def heartbeat_task(self, interval):
                 self.ws.close(status=4000)
                 self.on_close(0, 'HEARTBEAT failure')
                 return
-            self._last_heartbeat = time_perf_counter()
+            self._last_heartbeat = time()
 
             self.send(VoiceOPCode.HEARTBEAT, {'seq_ack': self.seq, 't': int(time())})
             self._heartbeat_acknowledged = False
@@ -241,7 +244,7 @@ def handle_heartbeat(self, _):
     def handle_heartbeat_acknowledge(self, _):
         self.log.debug('[{}] Received WS HEARTBEAT_ACK'.format(self.channel_id))
         self._heartbeat_acknowledged = True
-        self.latency = float('{:.2f}'.format((time_perf_counter() - self._last_heartbeat) * 1000))
+        self.latency = self.ws.last_pong_tm and float('{:.2f}'.format((self.ws.last_pong_tm - self.ws.last_ping_tm) * 1000))
 
     def set_speaking(self, voice=False, soundshare=False, priority=False, delay=0):
         value = SpeakingFlags.NONE
@@ -302,12 +305,14 @@ def on_voice_codecs(self, data):
         self.video_codec = data['video_codec']
         if 'media_session_id' in data.keys():
             self.transport_id = data['media_session_id']
+        if 'keyframe_interval' in data.keys():
+            self.keyframe_interval = data['keyframe_interval']
 
         # Set the UDP's RTP Audio Header's Payload Type
-        self.udp.set_audio_codec(data['audio_codec'])
+        # self.udp.set_audio_codec(data['audio_codec'])  # bypass because the audio codec will always be opus
 
     def on_voice_hello(self, packet):
-        self.log.info('[{}] Received Voice HELLO payload, starting heartbeater'.format(self.channel_id))
+        self.log.info('[{}] Received HELLO payload, starting heartbeater'.format(self.channel_id))
         self._heartbeat_task = gevent_spawn(self.heartbeat_task, packet['heartbeat_interval'])
         self.set_state(VoiceState.AUTHENTICATED)
 
@@ -349,7 +354,7 @@ def on_voice_ready(self, data):
             codecs.append({
                 'name': codec,
                 'payload_type': RTPPayloadTypes.get(codec).value,
-                'priority': idx,
+                'priority': 1000 + idx,
                 'type': 'audio',
             })
 
@@ -362,7 +367,7 @@ def on_voice_ready(self, data):
                         'encode': False,
                         'name': codec,
                         'payload_type': ptype.value,
-                        'priority': idx,
+                        'priority': 1000 * idx,
                         'rtxPayloadType': ptype.value + 1,
                         'type': 'video',
                     })
@@ -376,7 +381,7 @@ def on_voice_ready(self, data):
                 'mode': self.mode,
             },
             'codecs': codecs,
-            'experiments': [],
+            'experiments': self.experiments,
         })
         self.send(VoiceOPCode.CLIENT_CONNECT, {
             'audio_ssrc': self.ssrc,
@@ -396,17 +401,19 @@ def on_voice_sdp(self, sdp):
 
         self.mode = sdp['mode']  # UDP-only, does not apply to webRTC
         self.audio_codec = sdp['audio_codec']
-        if self.video_enabled:
-            self.video_codec = sdp['video_codec']
         self.transport_id = sdp['media_session_id']  # analytics
         self.secure_frames_version = sdp['secure_frames_version']
-        # self.sdp = sdp['sdp']  # webRTC only
-        # self.keyframe_interval = sdp['keyframe_interval']
+        if 'sdp' in sdp.keys():
+            self.sdp = sdp['sdp']  # webRTC only
 
         # Set the UDP's RTP Audio Header's Payload Type
         self.udp.set_audio_codec(sdp['audio_codec'])
+
         if self.video_enabled:
+            self.video_codec = sdp['video_codec']
             self.udp.set_video_codec(sdp['video_codec'])
+        if 'keyframe_interval' in sdp.keys():
+            self.keyframe_interval = sdp['keyframe_interval']
 
         # Create a secret box for encryption/decryption
         self.udp.setup_encryption(bytes(bytearray(sdp['secret_key'])))  # UDP only
@@ -496,10 +503,10 @@ def on_message(self, msg):
         try:
             data = self.encoder.decode(msg)
             self.packets.emit(data['op'], data['d'])
-            if 'seq' in data:
+            if 'seq' in data.keys():
                 self.seq = data['seq']
         except Exception:
-            self.log.exception('Failed to parse voice gateway message: ')
+            self.log.error('Failed to parse voice gateway message: ')
 
     def on_error(self, error):
         if isinstance(error, WebSocketTimeoutException):
@@ -602,6 +609,7 @@ def connect(self, channel_id, timeout=10, **kwargs):
             self.disconnect()
             raise VoiceException('Failed to connect to voice', self)
         else:
+            self._ws_creation_time = time()
             return self
 
     def disconnect(self):

disco/voice/packets.py

@@ -9,12 +9,13 @@ class VoiceOPCode:
     RESUME = 7
     HELLO = 8
     RESUMED = 9
+    CLIENT_CONNECT = 11
     VIDEO = 12
     CLIENT_DISCONNECT = 13
     CODECS = 14
     MEDIA_SINK_WANTS = 15
     VOICE_BACKEND_VERSION = 16
     CHANNEL_OPTIONS_UPDATE = 17
-    CLIENT_CONNECT = 18
+    CLIENT_FLAGS = 18
     SPEED_TEST = 19
     PLATFORM = 20

disco/voice/udp.py

@@ -366,7 +366,7 @@ def connect(self, host, port, timeout=10, addrinfo=None):
                 return None, None
 
             # Read IP and port
-            ip = str(data[8:]).split('\x00', 1)[0]
+            ip = str(data[8:].split(b'\x00', 1)[0], "utf=8")
             port = struct_unpack('<H', data[-2:])[0]  # little endian, unsigned short
 
         # Spawn read thread so we don't max buffers