Sends the daily video "Tagesschau in 100 Sekunden auf Arabisch" (German news in 100 seconds in Arabic) to a WhatsApp group with refugees.
Why? Because the ones I know are - at the moment - not all able to understand complex German sentences and should still be informed about what is going on in Germany and the rest of our world.
Nevertheless it should be easy to switch to the German version of the "Tagesschau in 100 Sekunden" or even the English one.
Why did you call this project "bottomnews"? Because it creates a robot named "Tom" that can send news automatically directly from the (or abbreviated "ft.", like in music titles) source / the bottom.
What else can this program do? It answers private (but no group) messages in a realistic style. The visible steps are:
- Receive all messages
- Come online
- Read new messages for every contact
- Write
- "Check for mistakes" (stop typing)
- Echo every last message (to prevent a ban)
- Go offline
What is left to do? The code is a modified version of yowsup's cli demo. It contains a lot of unused functions and other unneeded leftovers which can be removed. Also the variables every user has to set (see Usage / Private Information) spread across different files and could be packed together into a single one.
This project builds on top of yowsup, an unofficial WhatsApp API written in Python. To install it follow these installation instructions.
First of all: WhatsApp does not allow the usage of yowsup. So here's a ...
Warning: Do not use your primary phone number with unofficial WhatsApp APIs!
You risk a ban of your phone number which makes it impossible for your friends to contact you and a lot of work for you, e. g. if you've enabled 2 factor authentification on any website.
Of course I did not include my real phone number, password and contact list in the source. You have to replace it with your data which can be obtained from yowsup's registration instructions.
Files that contain asterisks and need personal configuration:
The main executable is bottomnews.py which accepts the --verbose argument.
It took me a long time to get yowsup running. There were several problems that did not only originate from yowsup itself. The following paragraphs describe the fixes I applied and offer a downloadable solution.
For everybody who does not know, here you can find FFVideo's source.
At the time of development (Q3 2016) FFVideo's extremely outdated version 0.0.13 could not be installed on my Raspberry Pi 2B running Raspbian Jessie via pip without errors and warnings. Ignoring the warnings I found out that the source's string r_frame_rate was not supported any more. This could be related to my installed versions of FFmpeg and its extensions, so you might not have this problem.
You can see my steps to the solution in my comments under issue 1689 and a downloadable fix in my GitHub repository Dargmuesli/FFVideo. Though, I do not guarantee endless availability.
If you want to fix it yourself replace all occurrences of the string r_frame_rate with avg_frame_rate in the source file FFVideo.c and install that.
Yowsup gave me two main problems that were pretty hard to fix as I was completely new to Python.
This problem stands somehow in relation to FFVideo, but is definitly a problem within yowsup. It seems like importlib's import_module does not like my fixed install of FFVideo (or something else). It complains that it can't be imported while a simple import FFVideo in the python command shell does work fine.
The fix is admittedly dirty, but it works. To /yowsup/common/tools.py add one import and change the VideoTools class:
from ffvideo import VideoStream
class VideoTools:
@staticmethod
def getVideoProperties(videoFile):
s = VideoStream(videoFile)
return s.width, s.height, s.bitrate, s.duration
@staticmethod
def generatePreviewFromVideo(videoFile):
fd, path = tempfile.mkstemp('.jpg')
stream = VideoStream(videoFile)
stream.get_frame_at_sec(0).image().save(path)
preview = ImageTools.generatePreviewFromImage(path)
os.remove(path)
return previewI simply removed this is both functions:
with FFVideoOptionalModule() as imp:
VideoStream = imp("VideoStream")The act of sending a video via yowsup is currently doomed to failure. Whoever tries gets the error:
AttributeError: "type object 'DownloadableMediaMessageProtocolEntity' has no attribute 'fromFilePath'"No wonder, that's because DownloadableMediaMessageProtocolEntity actually has no attribute 'fromFilePath' because it's not a VideoDownloadableMediaMessageProtocolEntity (note the "Video"). Who could've thought! I believe that is caused by the fact that - while yowsup is not really being discontinued - its pull requests haven't been merged for a long time now. The one that fixes this problem is #1564 labeled "Builder support for audio and video upload". Nice.
Either replace the corresponding original files in /yowsup/layers/protocol_media/protocolentities/ with a copy of the fixed audio file and/or the fixed video file by tanquetav or install my GitHub repository Dargmuesli/yowsup-mediafix the usual way.
Receiving emoticons resulted in errors similar to the following:
UnicodeEncodeError: "'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)"I added a simple .encode('utf-8') to layer.py L622 and
reload(sys)
sys.setdefaultencoding('utf8')to L32 f.. In L47 I added just one u:
MESSAGE_FORMAT = u"[{FROM}({TIME})]:[{MESSAGE_ID}]\t {MESSAGE}"Once I thought this bot was well tested enough I added it to the WhatsApp group with my friends. Well, it turned out that yowsup has problems handling some media message types like audio, location, document and url. So I quickly headed over to jlguardi's yowsup fork and searched all commits for media related ones. I changed several files, but finally got it working.
Add the following two imports:
from .protocolentities import DocumentDownloadableMediaMessageProtocolEntity
from .protocolentities import UrlMediaMessageProtocolEntityAppend these elif-checks to the large if-statement in the middle:
elif mediaNode.getAttributeValue("type") == "url":
entity = UrlMediaMessageProtocolEntity.fromProtocolTreeNode(node)
self.toUpper(entity)
elif mediaNode.getAttributeValue("type") == "document":
entity = DocumentDownloadableMediaMessageProtocolEntity.fromProtocolTreeNode(node)
self.toUpper(entity)This is a fairly simple addition:
from .message_media_downloadable_document import DocumentDownloadableMediaMessageProtocolEntity
from .message_media_url import UrlMediaMessageProtocolEntityThis is a whole new file. Copy it from my fork.
The same applies to this file. Copy it from my fork too.
Again, append these elif-checks to the large if-statement in the middle:
elif m.HasField("video_message"):
handled = True
self.handleVideoMessage(node, m.video_message)
elif m.HasField("audio_message"):
handled = True
self.handleAudioMessage(node, m.audio_message)Then add the appropriate message handler functions:
def handleAudioMessage(self, originalEncNode, audioMessage):
messageNode = copy.deepcopy(originalEncNode)
messageNode["type"] = "media"
mediaNode = ProtocolTreeNode("media", {
"type": "audio",
"filehash": audioMessage.file_sha256,
"size": str(audioMessage.file_length),
"url": audioMessage.url,
"mimetype": audioMessage.mime_type,
"duration": str(audioMessage.duration),
"seconds": str(audioMessage.duration),
"encoding": "raw",
"file": "enc",
"ip": "0",
"mediakey": audioMessage.media_key
})
messageNode.addChild(mediaNode)
self.toUpper(messageNode)
def handleVideoMessage(self, originalEncNode, videoMessage):
messageNode = copy.deepcopy(originalEncNode)
messageNode["type"] = "media"
mediaNode = ProtocolTreeNode("media", {
"type": "video",
"filehash": videoMessage.file_sha256,
"size": str(videoMessage.file_length),
"url": videoMessage.url,
"mimetype": videoMessage.mime_type,
"duration": str(videoMessage.duration),
"seconds": str(videoMessage.duration),
"caption": videoMessage.caption,
"encoding": "raw",
"file": "enc",
"ip": "0",
"mediakey": videoMessage.media_key
}, data = videoMessage.jpeg_thumbnail)
messageNode.addChild(mediaNode)
self.toUpper(messageNode)If you want you can also fix the already present but empty message handler functions for url and document messages a little bit further down:
def handleUrlMessage(self, originalEncNode, urlMessage):
messageNode = copy.deepcopy(originalEncNode)
messageNode["type"] = "media"
mediaNode = ProtocolTreeNode("media", {
"type": "url",
"text": urlMessage.text,
"match": urlMessage.matched_text,
"url": urlMessage.canonical_url,
"description": urlMessage.description,
"title": urlMessage.title
}, data = urlMessage.jpeg_thumbnail)
messageNode.addChild(mediaNode)
self.toUpper(messageNode)
def handleDocumentMessage(self, originalEncNode, documentMessage):
messageNode = copy.deepcopy(originalEncNode)
messageNode["type"] = "media"
mediaNode = ProtocolTreeNode("media", {
"type": "document",
"url": documentMessage.url,
"mimetype": documentMessage.mime_type,
"title": documentMessage.title,
"filehash": documentMessage.file_sha256,
"size": str(documentMessage.file_length),
"pages": str(documentMessage.page_count),
"mediakey": documentMessage.media_key
}, data = documentMessage.jpeg_thumbnail)
messageNode.addChild(mediaNode)
self.toUpper(messageNode)The last step is to correct a spelling mistake. Search for degress_longitude and replace it with degrees_longitude.
This file needs to change almost in its entirety. Copy the whole file in my fork.
The same applies to this file. Copy the whole file in my fork too.
The deployment worked out well, without any issues. After several questions about the identity of BotTom News arose, I decided to ask the refugees whether they'd like to keep receiving daily news videos in our WhatsApp group. The answer an unmistakable "No":
Thus, I discontinued the bot's activity.
Eventhough my offer was rejected, I learned much from reading and understanding yowsup's mostly undocumented source code and I'm glad that the refugees I know are completely honest with me.