Skip to content

Commit

Permalink
Minor update
Browse files Browse the repository at this point in the history 
SYNC CHANGES FROM MLTB
- Sync function using await.
- Important to avoid error path is invalid.
- Add zst file decompressor.
- Fix rare issue in same directory download.
- Escaping dirpath for combine images
- Fix thumbnail for user setting.
- Add -doc and -med args for leech type to use them as task option.
- Force thumbnail layout on yt-dlp videos.
- Add thumbnail layout option.
- Fix name sub and change usage - use ffprobe to know document type first.

Signed-off-by: Dawn India <dawn-in@z-mirror.eu.org>
  • Loading branch information
@Dawn-India
Dawn-India committed Sep 25, 2024
1 parent 849cb7b commit e1e44a5
Show file tree
Hide file tree
Showing 94 changed files with 4,064 additions and 3,574 deletions.
29 changes: 16 additions & 13 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ programming in Python.
## qBittorrent

- External access to webui, so you can remove files or edit settings. Then you can sync settings in database with sync button in bsetting
- Select files from a Torrent before and during download using mltb file selector (Requires Base URL) (task option)
- Select files from a Torrent before and during download using ZEE file selector (Requires Base URL) (task option)
- Seed torrents to a specific ratio and time (task option)
- Edit Global Options while the bot is running from bot settings (global option)

Expand All @@ -37,7 +37,7 @@ programming in Python.
## Sabnzbd

- External access to web interface, so you can remove files or edit settings. Then you can sync settings in database with sync button in bsetting
- Remove files from job before and during download using mltb file selector (Requires Base URL) (task option)
- Remove files from job before and during download using ZEE file selector (Requires Base URL) (task option)
- Edit Global Options while the bot is running from bot settings (global option)
- Servers menu to edit/add/remove usenet servers

Expand Down Expand Up @@ -70,6 +70,7 @@ programming in Python.
- Download restricted messages (document or link) by tg private/public/super links (task option)
- Choose transfer by bot or user session in case you have a premium plan (global, user option and task option)
- Mix upload between user and bot session with respect to file size (global, user option and task option)
- Upload with custom layout multiple thubnmail (global, user option and task option)

## Google Drive

Expand Down Expand Up @@ -144,6 +145,7 @@ programming in Python.

- Extract splits with or without password
- Zip file/folder with or without password
- Decompress zst files
- Using 7-zip tool to extract with or without password all supported types:

> ZIP, RAR, TAR, 7z, ISO, WIM, CAB, GZIP, BZIP2, APM, ARJ, CHM, CPIO, CramFS, DEB, DMG, FAT, HFS, LZH, LZMA, LZMA2,MBR,
Expand Down Expand Up @@ -175,6 +177,7 @@ programming in Python.
- Bulk download from telegram txt file or text message contains links seperated by new line (task option)
- Join splitted files that have splitted before by split(linux pkg) (task option)
- Sample video Generator (task option)
- Screenshots Generator (task option)
- Ability to cancel upload/clone/archive/extract/split/queue (task option)
- Cancel all buttons for choosing specific tasks status to cancel (global option)
- Convert videos and audios to specific format with filter (task option)
Expand Down Expand Up @@ -371,20 +374,20 @@ programming in Python.
- `USER_TRANSMISSION`: Upload/Download by user session. Only in superChat. Default is `False`. `Bool`
- `MIXED_LEECH`: Upload by user and bot session with respect to file size. Only in superChat. Default is `False`. `Bool`
- `USER_LEECH_DESTINATION`: ID or USERNAME or PM(private message) to where files would be uploaded. `Int`|`Str`. Add `-100` before channel/superGroup id.
- `NAME_SUBSTITUTE`: Add word/letter/sentense/pattern to remove or replace with other words with sensitive case or without.**Notes**:
- `NAME_SUBSTITUTE`: Add word/letter/character/sentense/pattern to remove or replace with other words with sensitive case or without. **Notes**:
1. Seed will get disbaled while using this option
2. Before any character you must add \, those are the characters: `\^$.|?*+()[]{}-`
* Example-1: `text : code : s | mirror : leech | tea : : s | clone`
- text will get replaced by code with sensitive case
- mirror will get replaced by leech
- tea will get removed with sensitive case
- clone will get removed
* Example-2: `\(text\) | \[test\] : test | \\text\\ : text : s`
- `(text)` will get removed
- `[test]` will get replaced by test
- `\text\` will get replaced by text with sensitive case
2. Before any character you must add `\BACKSLASH`, those are the characters: `\^$.|?*+()[]{}-`
* Example: script/code/s | mirror/leech | tea/ /s | clone | cpu/ | \[ZEE\]/ZEE | \\text\\/text/s
- script will get replaced by code with sensitive case
- mirror will get replaced by leech
- tea will get replaced by space with sensitive case
- clone will get removed
- cpu will get replaced by space
- [ZEE] will get replaced by ZEE
- \text\ will get replaced by text with sensitive case
- `METADATA_TXT`: Edit metadata of the video. `Str`
- `META_ATTACHMENT`: Add attachment to the metadata. `Str`
- `THUMBNAIL_LAYOUT`: Thumbnail layout (widthxheight, 2x2, 3x3, 2x4, 4x4, ...) of how many photo arranged for the thumbnail.`Str`

**12. Super Group Features**

Expand Down
130 changes: 66 additions & 64 deletions bot/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
from collections import OrderedDict
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from asyncio import (
Lock,
get_running_loop,
new_event_loop,
set_event_loop
)
from aria2p import (
API as ariaAPI,
Client as ariaClient
)
from asyncio import (
Lock,
get_event_loop
)
from concurrent.futures import ThreadPoolExecutor
from collections import OrderedDict
from dotenv import (
load_dotenv,
dotenv_values
)
from logging import (
INFO,
ERROR,
getLogger,
FileHandler,
StreamHandler,
INFO,
basicConfig,
error as log_error,
info as log_info,
warning as log_warning,
ERROR,
)
from nekozee import Client as TgClient
from os import (
remove,
path as ospath,
environ
)
from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi
from nekozee import (
Client as tgClient
)
from qbittorrentapi import Client as qbClient
from sabnzbdapi import sabnzbdClient
from qbittorrentapi import Client as QbClient
from sabnzbdapi import SabnzbdClient
from shutil import rmtree
from socket import setdefaulttimeout
from subprocess import (
Popen,
Expand All @@ -60,10 +60,11 @@
getLogger("pymongo").setLevel(ERROR)
getLogger("nekozee").setLevel(ERROR)

botStartTime = time()
bot_loop = get_event_loop()
THREADPOOL = ThreadPoolExecutor(max_workers=99999)
bot_loop.set_default_executor(THREADPOOL)
bot_start_time = time()

bot_loop = new_event_loop()
set_event_loop(bot_loop)


basicConfig(
format="%(levelname)s | From %(name)s -> %(module)s line no: %(lineno)d | %(message)s",
Expand All @@ -81,20 +82,20 @@
override=True
)

Intervals = {
intervals = {
"status": {},
"qb": "",
"jd": "",
"nzb": "",
"stopAll": False
}
QbTorrents = {}
qb_torrents = {}
jd_downloads = {}
nzb_jobs = {}
DRIVES_NAMES = []
DRIVES_IDS = []
INDEX_URLS = []
GLOBAL_EXTENSION_FILTER = [
drives_names = []
drives_ids = []
index_urls = []
global_extension_filter = [
"aria2",
"!qB"
]
Expand Down Expand Up @@ -124,6 +125,7 @@
jd_lock = Lock()
cpu_eater_lock = Lock()
subprocess_lock = Lock()
same_directory_lock = Lock()
status_dict = {}
task_dict = {}
rss_dict = {}
Expand All @@ -140,7 +142,7 @@
log_error("BOT_TOKEN variable is missing! Exiting now")
exit(1)

bot_id = BOT_TOKEN.split(
BOT_ID = BOT_TOKEN.split(
":",
1
)[0]
Expand All @@ -160,10 +162,10 @@
)
db = conn.zee
current_config = dict(dotenv_values("config.env"))
old_config = db.settings.deployConfig.find_one({"_id": bot_id})
old_config = db.settings.deployConfig.find_one({"_id": BOT_ID})
if old_config is None:
db.settings.deployConfig.replace_one(
{"_id": bot_id},
{"_id": BOT_ID},
current_config,
upsert=True
)
Expand All @@ -174,15 +176,15 @@
and old_config != current_config
):
db.settings.deployConfig.replace_one(
{"_id": bot_id},
{"_id": BOT_ID},
current_config,
upsert=True
)
elif config_dict := db.settings.config.find_one({"_id": bot_id}):
elif config_dict := db.settings.config.find_one({"_id": BOT_ID}):
del config_dict["_id"]
for key, value in config_dict.items():
environ[key] = str(value)
if pf_dict := db.settings.files.find_one({"_id": bot_id}):
if pf_dict := db.settings.files.find_one({"_id": BOT_ID}):
del pf_dict["_id"]
for key, value in pf_dict.items():
if value:
Expand All @@ -195,27 +197,13 @@
"wb+"
) as f:
f.write(value)
if file_ == "cfg.zip":
run([
"rm",
"-rf",
"/JDownloader/cfg"
])
run([
"7z",
"x",
"-bso0",
"cfg.zip",
"-o/JDownloader"
])
remove("cfg.zip")
if a2c_options := db.settings.aria2c.find_one({"_id": bot_id}):
if a2c_options := db.settings.aria2c.find_one({"_id": BOT_ID}):
del a2c_options["_id"]
aria2_options = a2c_options
if qbit_opt := db.settings.qbittorrent.find_one({"_id": bot_id}):
if qbit_opt := db.settings.qbittorrent.find_one({"_id": BOT_ID}):
del qbit_opt["_id"]
qbit_options = qbit_opt
if nzb_opt := db.settings.nzb.find_one({"_id": bot_id}):
if nzb_opt := db.settings.nzb.find_one({"_id": BOT_ID}):
if ospath.exists("sabnzbd/SABnzbd.ini.bak"):
remove("sabnzbd/SABnzbd.ini.bak")
del nzb_opt["_id"]
Expand All @@ -228,7 +216,7 @@
"BOT_TOKEN",
""
)
bot_id = BOT_TOKEN.split(
BOT_ID = BOT_TOKEN.split(
":",
1
)[0]
Expand All @@ -241,6 +229,20 @@
else:
config_dict = {}

if ospath.exists("cfg.zip"):
if ospath.exists("/JDownloader/cfg"):
rmtree(
"/JDownloader/cfg",
ignore_errors=True
)
run([
"7z",
"x",
"cfg.zip",
"-o/JDownloader"
])
remove("cfg.zip")

if not ospath.exists(".netrc"):
with open(
".netrc",
Expand Down Expand Up @@ -286,7 +288,7 @@
)
if len(USER_SESSION_STRING) != 0:
try:
user = tgClient(
user = TgClient(
"zeeu",
TELEGRAM_API,
TELEGRAM_HASH,
Expand Down Expand Up @@ -376,7 +378,7 @@
fx = EXTENSION_FILTER.split()
for x in fx:
x = x.lstrip(".")
GLOBAL_EXTENSION_FILTER.append(x.strip().lower())
global_extension_filter.append(x.strip().lower())

JD_EMAIL = environ.get(
"JD_EMAIL",
Expand Down Expand Up @@ -1065,6 +1067,9 @@
MEGA_PASSWORD = ""


THUMBNAIL_LAYOUT = environ.get("THUMBNAIL_LAYOUT", "")
THUMBNAIL_LAYOUT = "" if len(THUMBNAIL_LAYOUT) == 0 else THUMBNAIL_LAYOUT

config_dict = {
"AS_DOCUMENT": AS_DOCUMENT,
"AUTHORIZED_CHATS": AUTHORIZED_CHATS,
Expand Down Expand Up @@ -1145,6 +1150,7 @@
"TELEGRAM_API": TELEGRAM_API,
"TELEGRAM_HASH": TELEGRAM_HASH,
"TORRENT_LIMIT": TORRENT_LIMIT,
"THUMBNAIL_LAYOUT": THUMBNAIL_LAYOUT,
"TORRENT_TIMEOUT": TORRENT_TIMEOUT,
"TOKEN_TIMEOUT": TOKEN_TIMEOUT,
"USER_TRANSMISSION": USER_TRANSMISSION,
Expand All @@ -1162,9 +1168,9 @@
config_dict = OrderedDict(sorted(config_dict.items()))

if GDRIVE_ID:
DRIVES_NAMES.append("Main")
DRIVES_IDS.append(GDRIVE_ID)
INDEX_URLS.append(INDEX_URL)
drives_names.append("Main")
drives_ids.append(GDRIVE_ID)
index_urls.append(INDEX_URL)

KEY = ("@Z_Mirror")

Expand All @@ -1176,12 +1182,12 @@
lines = f.readlines()
for line in lines:
temp = line.strip().split()
DRIVES_IDS.append(temp[1])
DRIVES_NAMES.append(temp[0].replace("_", " "))
drives_ids.append(temp[1])
drives_names.append(temp[0].replace("_", " "))
if len(temp) > 2:
INDEX_URLS.append(temp[2])
index_urls.append(temp[2])
else:
INDEX_URLS.append("")
index_urls.append("")

if ospath.exists("buttons.txt"):
with open(
Expand Down Expand Up @@ -1221,11 +1227,7 @@

if ospath.exists("accounts.zip"):
if ospath.exists("accounts"):
run([
"rm",
"-rf",
"accounts"
])
rmtree("accounts")
run([
"7z",
"x",
Expand All @@ -1246,7 +1248,7 @@
config_dict["USE_SERVICE_ACCOUNTS"] = False


qbittorrent_client = qbClient(
qbittorrent_client = QbClient(
host="localhost",
port=8090,
VERIFY_WEBUI_CERTIFICATE=False,
Expand All @@ -1260,7 +1262,7 @@

BASE += ("G8k7bAblAEkiZDyAAjM6a")

sabnzbd_client = sabnzbdClient(
sabnzbd_client = SabnzbdClient(
host="http://localhost",
api_key="zee",
port="8070",
Expand All @@ -1283,7 +1285,7 @@
"server-stat-of",
]

bot = tgClient(
bot = TgClient(
"zeeb",
TELEGRAM_API,
TELEGRAM_HASH,
Expand Down
Loading

0 comments on commit e1e44a5

Please sign in to comment.