Ensure contractions learn correctly.

By tokenizing sentences instead of splitting by spaces. See #29.
tomaarsen · Nov 7, 2022 · 4c33740 · 4c33740
1 parent 5238492
commit 4c33740
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/MarkovChainBot.py b/MarkovChainBot.py
@@ -206,7 +206,7 @@ def message_handler(self, m: Message):
 
                     for sentence in sentences:
                         # Get all seperate words
-                        words = sentence.split(" ")
+                        words = tokenize(sentence)
                         # Double spaces will lead to invalid rules. We remove empty words here
                         if "" in words:
                             words = [word for word in words if word]