Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow custom file extensions for directory journal #1873

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/journal-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,7 @@ use any extension. `jrnl` will automatically create the file when you save
your first entry.

## Folder
The folder journal format organizes your entries into subfolders for the year
and month and `.txt` files for each day. If there are multiple entries in a day,
they all appear in the same `.txt` file.
The folder journal format organizes your entries into subfolders for the year and month, and a new file for each day. If there are multiple entries in a day, they all appear in the same file. The file extension (`.txt` by default) can be configured with the `extension` configuration key.

The directory tree structure is in this format: `YYYY/MM/DD.txt`. For instance, if
you have an entry on May 5th, 2021 in a folder journal at `~/folderjournal`, it will
Expand Down
9 changes: 9 additions & 0 deletions docs/reference-config-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,15 @@ for details.
If `true`, encrypts your journal using AES. Do not change this
value for journals that already have data in them.

### extension

For [folder journals](journal-types.md#folder), control the extension of the day
files. By default it is `.txt`.

!!! warning
Changing the extension after a folder journal has been created will prevent
`jrnl` from finding past entries.

### template
The path to a text file to use as a template for new entries. Only works when you
have the `editor` field configured. If you use a template, the editor's
Expand Down
28 changes: 17 additions & 11 deletions jrnl/journals/FolderJournal.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
DIGIT_PATTERN = "[0123456789]"
YEAR_PATTERN = DIGIT_PATTERN * 4
MONTH_PATTERN = "[01]" + DIGIT_PATTERN
DAY_PATTERN = "[0123]" + DIGIT_PATTERN + ".txt"
DAY_PATTERN = "[0-3][0-9]"

DEFAULT_EXTENSION = ".txt"


class Folder(Journal):
Expand All @@ -34,7 +36,8 @@ def open(self) -> "Folder":
self.entries = []

if os.path.exists(self.config["journal"]):
filenames = Folder._get_files(self.config["journal"])
file_extension = self.config.get("extension", DEFAULT_EXTENSION)
filenames = Folder._get_files(self, self.config["journal"], file_extension)
for filename in filenames:
with codecs.open(filename, "r", "utf-8") as f:
journal = f.read()
Expand All @@ -45,6 +48,8 @@ def open(self) -> "Folder":

def write(self) -> None:
"""Writes only the entries that have been modified into proper files."""
file_extension = self.config.get("extension", DEFAULT_EXTENSION)

# Create a list of dates of modified entries. Start with diff_entry_dates
modified_dates = self._diff_entry_dates
seen_dates = set(self._diff_entry_dates)
Expand All @@ -63,7 +68,7 @@ def write(self) -> None:
self.config["journal"],
d.strftime("%Y"),
d.strftime("%m"),
d.strftime("%d") + ".txt",
d.strftime("%d") + file_extension,
)
dirname = os.path.dirname(filename)
# create directory if it doesn't exist
Expand All @@ -81,7 +86,7 @@ def write(self) -> None:
journal_file.write(journal)
# look for and delete empty files
filenames = []
filenames = Folder._get_files(self.config["journal"])
filenames = Folder._get_files(self, self.config["journal"], file_extension)
for filename in filenames:
if os.stat(filename).st_size <= 0:
os.remove(filename)
Expand Down Expand Up @@ -121,12 +126,12 @@ def parse_editable_str(self, edited: str) -> None:
self.entries = mod_entries

@staticmethod
def _get_files(journal_path: str) -> list[str]:
def _get_files(self, journal_path: str, extension: str) -> list[str]:
"""Searches through sub directories starting with journal_path and find all text
files that look like entries"""
for year_folder in Folder._get_year_folders(pathlib.Path(journal_path)):
for month_folder in Folder._get_month_folders(year_folder):
yield from Folder._get_day_files(month_folder)
yield from Folder._get_day_files(month_folder, extension)

@staticmethod
def _get_year_folders(path: pathlib.Path) -> list[pathlib.Path]:
Expand All @@ -143,15 +148,16 @@ def _get_month_folders(path: pathlib.Path) -> list[pathlib.Path]:
return

@staticmethod
def _get_day_files(path: pathlib.Path) -> list[str]:
for child in path.glob(DAY_PATTERN):
def _get_day_files(path: pathlib.Path, extension: str) -> list[str]:
for child in path.glob(DAY_PATTERN + extension):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the only thing left on this PR is supporting a lack of dots in the extension. For instance, I'd expect this to work the same whether my config value is extension: md or extension: .md

day = str(child.name).replace(extension, "")
if (
int(child.stem) > 0
and int(child.stem) <= 31
int(day) > 0
and int(day) <= 31
and time.is_valid_date(
year=int(path.parent.name),
month=int(path.name),
day=int(child.stem),
day=int(day),
)
and child.is_file()
):
Expand Down
11 changes: 10 additions & 1 deletion tests/bdd/features/file_storage.feature
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,22 @@ Feature: Journals iteracting with the file system in a way that users can see
And the journal directory should contain
2013/07/23.txt

@skip_win # https://github.com/jrnl-org/jrnl/issues/1894
Scenario: Adding entries to a Folder journal with a custom extension should generate date files
Given we use the config "empty_folder_with_extension.yaml"
When we run "jrnl 23 July 2013: Testing folder journal."
Then we should get no error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congratulations, your test uncovered an existing bug! 🎉 I filed it in #1894.

Looks like empty_folder.yaml got past this issue by having an "empty" file in it: tests/data/journals/empty_folder/empty

Feel free to copy that "empty" file to tests/data/journals/empty_folder_with_extension/empty to get this working.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't replicate this. I'm not sure what might have caused the issue.

I re-ran the tests without the empty file in test/data/journals/empty_folder and the tests still passed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like bug #1894 is Windows-only, and this is combined with a limitation of git: since we can only commit files (not directories), that empty_folder doesn't exist on the remote repository until we push a file in it.

And the journal directory should contain
2013/07/23.md

Scenario: Adding multiple entries to a Folder journal should generate multiple date files
Given we use the config "empty_folder.yaml"
When we run "jrnl 23 July 2013: Testing folder journal."
And we run "jrnl 3/7/2014: Second entry of journal."
And we run "jrnl 5/3/2014: Second entry of journal."
Then we should get no error
And the journal directory should contain
2013/07/23.txt
2014/05/03.txt

Scenario: If the journal and its parent directory don't exist, they should be created
Given we use the config "missing_directory.yaml"
Expand Down
13 changes: 13 additions & 0 deletions tests/data/configs/empty_folder_with_extension.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
default_hour: 9
default_minute: 0
editor: ''
template: false
encrypt: false
extension: .md
highlight: true
journals:
default: features/journals/empty_folder_with_extension/
linewrap: 80
tagsymbols: '@'
timeformat: '%Y-%m-%d %H:%M'
indent_character: "|"
2 changes: 1 addition & 1 deletion tests/unit/test_journals_folder_journal.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def test_get_day_files_expected_filtering(inputs_and_outputs):
mock.patch("pathlib.Path.glob", return_value=glob_files),
mock.patch.object(pathlib.Path, "is_file", return_value=True),
):
actual_output = list(Folder._get_day_files(year_month_path))
actual_output = list(Folder._get_day_files(year_month_path, ".txt"))
actual_output.sort()

expected_output.sort()
Expand Down