Merge pull request #76 from Ljzd-PRO/devel

Bump to v0.5.0
Ljzd-PRO · Mar 24, 2024 · 0a74118 · 0a74118
2 parents 5d6a297 + bad64f1
commit 0a74118
Show file tree

Hide file tree

Showing 20 changed files with 722 additions and 570 deletions.
diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml
@@ -4,21 +4,7 @@ on:
   workflow_dispatch:
   workflow_call:
   push:
-    branches:
-      - master
-      - devel
-      - pure-py
-    paths:
-      - ".github/actions/setup-python/**"
-      - ".github/actions/pyinstaller/**"
-      - ".github/workflows/python-package.yml"
-      - "pyproject.toml"
   pull_request:
-    paths:
-      - ".github/actions/setup-python/**"
-      - ".github/actions/pyinstaller/**"
-      - ".github/workflows/python-package.yml"
-      - "pyproject.toml"
 
 jobs:
   build-wheels:

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,15 +1,61 @@
 ## Changes
 
-- Removed `--update-from` flag
-- Set `LoggerConfiguration.path` defaults to `None` (KToolBox will no longer output logs to files by default)
-- Fixed the issue where downloading files with the same name but different suffix simultaneously would fail.
-- Documents update (Coomer)
+### 🐍 Fix
+
+- Fixed download failure when server returns an invalid filename (`Attachment.name`) (#73)
+
+### 💡 Feature
+
+- Add support for local storage bucket mode (#74) (@Nacosia)
+  - Edit `KTOOLBOX_DOWNLOADER__USE_BUCKET`, `KTOOLBOX_DOWNLOADER_BUCKET_PATH` in `prod.env` or environment variables to set this option
+  - 📖More information: [Configuration-Reference-DownloaderConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.DownloaderConfiguration)
+
+- Add support for customizing the post directory name format (#45, #46)
+  - Edit `KTOOLBOX_JOB__POST_DIRNAME_FORMAT` in `prod.env` or environment variables to set this option
+  - 📖More information: [Configuration-Reference-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
+   ```dotenv
+   # It will create directories like `[2024-1-1]HelloWorld`
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="[{published}]{title}"
+   ```
+   ```dotenv
+   # It will create directories like `2024-1-1_12345_112233`
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="{published}_{user}_{id}"
+   ```
+   ```dotenv
+   # Default value. It will create directories like `HelloWorld`
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="{title}"
+   ```
+
+- Marked `JobConfiguration.post_id_as_path` as deprecated, use `JobConfiguration.post_dirname_format` instead
 
 - - -
 
-- 删除了 `--update-from` 标志
-- 将 `LoggerConfiguration.path` 的默认值设置为 `None`（KToolBox 默认不再将日志输出到文件）
-- 修复了同时下载具有相同名称但不同后缀的文件会失败的问题。
-- 文档更新（Coomer）
+### 🐍 修复
+
+- 修复当服务器返回的文件名不合法时下载出错的问题 (`Attachment.name`) (#73)
+
+### 💡 新特性
+
+- 增加本地存储桶模式的存储支持 (#74) (@Nacosia)
+  - 在 `prod.env` 或环境变量中编辑 `KTOOLBOX_DOWNLOADER__USE_BUCKET`, `KTOOLBOX_DOWNLOADER_BUCKET_PATH` 以设置该选项
+  - 📖更多信息: [Configuration-Reference-DownloaderConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.DownloaderConfiguration)
+
+- 增加支持自定义作品目录名格式 (#45, #46)
+  - 在 `prod.env` 或环境变量中编辑 `KTOOLBOX_JOB__POST_DIRNAME_FORMAT` 以设置该选项
+  - 📖更多信息: [Configuration-Reference-JobConfiguration](https://ktoolbox.readthedocs.io/latest/configuration/reference/#ktoolbox.configuration.JobConfiguration)
+   ```dotenv
+   # 将会创建例如 `[2024-1-1]HelloWorld` 的目录名
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="[{published}]{title}"
+   ```
+   ```dotenv
+   # 将会创建例如 `2024-1-1_12345_112233` 的目录名
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="{published}_{user}_{id}"
+   ```
+   ```dotenv
+   # 默认值。 将会创建例如 `HelloWorld` 的目录名
+   KTOOLBOX_JOB__POST_DIRNAME_FORMAT="{title}"
+   ```
+
+- 将 `JobConfiguration.post_id_as_path` 标记为已弃用, 请用 `JobConfiguration.post_dirname_format` 取代
 
-**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.3.6...v0.4.0
+**Full Changelog**: https://github.com/Ljzd-PRO/KToolBox/compare/v0.4.0...v0.5.0
diff --git a/README.md b/README.md
@@ -56,7 +56,7 @@
 
 ## Dev Plan
 
-- [ ] Add a Fluent Design style GUI
+- [ ] GUI
 - [x] Add uvloop support for Unix platform
 
 ## Tutorial
@@ -97,28 +97,26 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
 If some files failed to download, you can try to execute the command line again, 
 the downloaded files will be **skipped**.
 
-#### ⬇️🖌️ Download all posts from a creator
+#### ⬇️🖌️ Download posts from a creator
 ```bash
+# Download all posts of the creator/artist
 ktoolbox sync-creator https://kemono.su/fanbox/user/9016
-```
-
-> By default, you will get a `creator-indices.ktoolbox` file in the creator directory, 
-> you can use it to update the directory anytime.
-
 
-#### 🔄️ Update a downloaded creator directory
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+# Download posts from the creator/artist from 2024-1-1 to 2024-3-1
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --start-time=2024-1-1 --end-time=2024-3-1
 ```
 
-The `creator-indices.ktoolbox` file contains the information and filepath of posts inside the directory.
-
 ### iOS Shortcuts
 
 Goto [Shortcuts for iOS](https://ktoolbox.readthedocs.io/latest/shortcut/) page for more details.
 
 ### Configuration
 
+- Download 10 files at the same time
+- Rename attachments in numerical order
+- Prefix the post directory name with its release/publish date
+- ...
+
 Goto [Configuration-Guide](https://ktoolbox.readthedocs.io/latest/configuration/guide/) page for more details.
 
 ## Other Branches

diff --git a/README_zh-CN.md b/README_zh-CN.md
@@ -57,7 +57,7 @@
 
 ## 开发计划
 
-- [ ] 增加 Fluent Design 风格的 UI 界面
+- [ ] GUI
 - [x] 对 Unix 平台增加 uvloop 支持
 
 ## 使用方法
@@ -99,25 +99,24 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
 
 #### ⬇️🖌️ 下载作者的所有作品
 ```bash
+# 下载作者/画师的所有作品
 ktoolbox sync-creator https://kemono.su/fanbox/user/9016
-```
-
-> 默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，你可以用它来更新目录。
-
 
-#### 🔄️ 更新一个作者目录
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+# 下载作者/画师从 2024-1-1 到 2024-3-1 的作品
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --start-time=2024-1-1 --end-time=2024-3-1
 ```
 
-`creator-indices.ktoolbox` 包含目录下的所有作品的信息和路径。
-
 ### iOS 快捷指令
 
 前往 [iOS 快捷指令](https://ktoolbox.readthedocs.io/latest/zh/shortcut/) 页面查看更多详情。
 
 ### 配置
 
+- 同时下载10个文件
+- 按照数字顺序重命名附件, 例如 `1.png`, `2.png`, ...
+- 将发布日期作为作品目录名的开头，例如 `[2024-1-1]HelloWorld`
+- ...
+
 前往 [配置-向导](https://ktoolbox.readthedocs.io/latest/zh/configuration/guide/) 页面查看更多详情。
 
 ## 其他分支

diff --git a/docs/en/commands/guide.md b/docs/en/commands/guide.md
@@ -34,31 +34,22 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
 `sync-creator`
 
 ```bash
+# Download all posts of the creator/artist
 ktoolbox sync-creator https://kemono.su/fanbox/user/9016
 ```
-??? info "Output"
+??? info "About `creator-indices.ktoolbox` file"
     By default, you will get a `creator-indices.ktoolbox` file in the creator directory, 
-    you can use it to update the directory anytime.
-
-
-## Update a downloaded creator directory
-
-`sync-creator`
+    it contains the information and filepath of posts inside the directory.
 
-- `--update-with`
-
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
-```
-??? info "About `creator-indices.ktoolbox` file"
-    The `creator-indices.ktoolbox` file contains the information and filepath of posts inside the directory.
+??? tip "Update creator directory"
+    You can rerun the command, files with the same filename will be skipped.
 
 ## Download posts that published within the specified time range
 
 `sync-creator`
 
-- `--start-time`
-- `--end-time`
+- `--start-time`: Start time of the published time range for posts downloading.
+- `--end-time`: End time of the published time range for posts downloading.
 
 ```bash
 # From 2023-8-5 to 2023-12-6

diff --git a/docs/en/configuration/guide.md b/docs/en/configuration/guide.md
@@ -11,17 +11,20 @@
 # Download 10 files at the same time.
 KTOOLBOX_JOB__COUNT=10
 
-# Allocate 102400 Bytes as buffer for each download job
-KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
-
 # Set post attachments directory path as `./`, it means to save all attachments files in post directory
 # without making a new sub directory to storage them
 KTOOLBOX_JOB__POST_STRUCTURE__ATTACHMENTS=./
 
+# Rename attachments in numerical order, e.g. `1.png`, `2.png`, ...
+KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
+
+# Prefix the post directory name with its release/publish date, e.g. `[2024-1-1]HelloWorld`
+KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{title}
+
+# Allocate 102400 Bytes as buffer for each download job
+KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
+
 # Disable SSL certificate verification for Kemono API server and download server
 # It's useful when certificate on Kemono server expired. (SSL: CERTIFICATE_VERIFY_FAILED)
 KTOOLBOX_SSL_VERIFY=False
-
-# Rename attachments in numerical order, e.g. `1.png`, `2.png`, ...
-KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
 ```
diff --git a/docs/en/index.md b/docs/en/index.md
@@ -88,18 +88,14 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
     If some files failed to download, you can try to execute the command line again, 
     the downloaded files will be **skipped**.
 
-#### ⬇️🖌️ Download all posts from a creator
+#### ⬇️🖌️ Download posts from a creator
 ```bash
+# Download all posts of the creator/artist
 ktoolbox sync-creator https://kemono.su/fanbox/user/9016
-```
-??? info "Output"
-    By default, you will get a `creator-indices.ktoolbox` file in the creator directory, 
-    you can use it to update the directory anytime.
-
 
-#### 🔄️ Update a downloaded creator directory
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+# Download posts from the creator/artist from 2024-1-1 to 2024-3-1
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --start-time=2024-1-1 --end-time=2024-3-1
 ```
 ??? info "About `creator-indices.ktoolbox` file"
-    The `creator-indices.ktoolbox` file contains the information and filepath of posts inside the directory.
+    By default, you will get a `creator-indices.ktoolbox` file in the creator directory, 
+    it contains the information and filepath of posts inside the directory.
diff --git a/docs/zh/commands/guide.md b/docs/zh/commands/guide.md
@@ -33,30 +33,21 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
 `sync-creator`
 
 ```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/9016
-```
-??? info "输出"
-    默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，你可以用它来更新目录。
-
-
-## 更新一个作者目录
-
-`sync-creator`
-
-- `--update-with`
-
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+# 下载作者/画师的所有作品
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --offset=10 --length=5
 ```
 ??? info "关于 `creator-indices.ktoolbox` 文件"
-    `creator-indices.ktoolbox` 包含目录下的所有作品的信息和路径。
+    默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，它包含目录下的所有作品的信息和路径。
+
+??? tip "更新作者目录"
+    你可以再次运行命令，文件名相同的文件将会被跳过。
 
 ## 下载在指定时间范围内发布的作品
 
 `sync-creator`
 
-- `--start-time`
-- `--end-time`
+- `--start-time`：下载指定开始时间范围内的作品
+- `--end-time`：下载指定结束时间范围内的作品
 
 ```bash
 # 从 2023-8-5 到 2023-12-6

diff --git a/docs/zh/configuration/guide.md b/docs/zh/configuration/guide.md
@@ -11,17 +11,20 @@
 # 可同时下载10个文件
 KTOOLBOX_JOB__COUNT=10
 
-# 为每个下载任务分配 102400 字节内存作为缓冲区
-KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
-
 # 设置作品附件目录为 `./`, 这意味着所有附件将直接保存在作品目录下
 # 而不会创建一个子目录来储存
 KTOOLBOX_JOB__POST_STRUCTURE__ATTACHMENTS=./
 
+# 按照数字顺序重命名附件, 例如 `1.png`, `2.png`, ...
+KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
+
+# 将发布日期作为作品目录名的开头，例如 `[2024-1-1]HelloWorld`
+KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{title}
+
+# 为每个下载任务分配 102400 字节内存作为缓冲区
+KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
+
 # 为Kemono API服务器和下载服务器禁用SSL证书检查
 # 在Kemono服务器的证书过期时很有用 （SSL: CERTIFICATE_VERIFY_FAILED）
 KTOOLBOX_SSL_VERIFY=False
-
-# 按照数字顺序重命名附件, 例如 `1.png`, `2.png`, ...
-KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
 ```
diff --git a/docs/zh/index.md b/docs/zh/index.md
@@ -89,15 +89,11 @@ ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
 
 #### ⬇️🖌️ 下载作者的所有作品
 ```bash
+# 下载作者/画师的所有作品
 ktoolbox sync-creator https://kemono.su/fanbox/user/9016
-```
-??? info "输出"
-    默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，你可以用它来更新目录。
-
 
-#### 🔄️ 更新一个作者目录
-```bash
-ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+# 下载作者/画师从 2024-1-1 到 2024-3-1 的作品
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016 --start-time=2024-1-1 --end-time=2024-3-1
 ```
 ??? info "关于 `creator-indices.ktoolbox` 文件"
-    `creator-indices.ktoolbox` 包含目录下的所有作品的信息和路径。
+    默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，它包含目录下的所有作品的信息和路径。
diff --git a/example.env b/example.env
@@ -1,16 +1,19 @@
 # Download 10 files at the same time.
 KTOOLBOX_JOB__COUNT=10
 
-# Allocate 102400 Bytes as buffer for each download job
-KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
-
 # Set post attachments directory path as `./`, it means to save all attachments files in post directory
 # without making a new sub directory to storage them
 KTOOLBOX_JOB__POST_STRUCTURE__ATTACHMENTS=./
 
+# Rename attachments in numerical order, e.g. `1.png`, `2.png`, ...
+KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
+
+# Prefix the post directory name with its release/publish date, e.g. `[2024-1-1]HelloWorld`
+KTOOLBOX_JOB__POST_DIRNAME_FORMAT=[{published}]{title}
+
+# Allocate 102400 Bytes as buffer for each download job
+KTOOLBOX_DOWNLOADER__BUFFER_SIZE=102400
+
 # Disable SSL certificate verification for Kemono API server and download server
 # It's useful when certificate on Kemono server expired. (SSL: CERTIFICATE_VERIFY_FAILED)
 KTOOLBOX_SSL_VERIFY=False
-
-# Rename attachments in numerical order, e.g. `1.png`, `2.png`, ...
-KTOOLBOX_JOB__SEQUENTIAL_FILENAME=True
diff --git a/ktoolbox/__init__.py b/ktoolbox/__init__.py
@@ -1,4 +1,4 @@
 __title__ = "KToolBox"
 # noinspection SpellCheckingInspection
 __description__ = "A useful CLI tool for downloading posts in Kemono.party / .su"
-__version__ = "0.4.0"
+__version__ = "0.5.0"