Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache: Don't use checksum as filename #72

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

r4f4
Copy link
Collaborator

@r4f4 r4f4 commented Aug 14, 2019

In some cases (e.g in Pungi), the metadata is changing all the time, fus
is run multiple times a day and the cache just grows. So instead of
using checksum we use the reponame passed in the command line invocation
and the metadata type to create a filename so that only one copy exists
for that reponame. Therefore the cache layout now is:

$CACHEDIR/fus/$reponame/repodata/repomd.xml
$CACHEDIR/fus/$reponame/repodata/primary.xml.gz
$CACHEDIR/fus/$reponame/repodata/modules.xml.gz
$CACHEDIR/fus/$reponame/repodata/group_gz.x86_64.xml.xz
$CACHEDIR/fus/$reponame/repodata/filelists.xml.gz

Fixes #71

Signed-off-by: Rafael Fonseca r4f4rfs@gmail.com

In some cases (e.g in Pungi), the metadata is changing all the time, fus
is run multiple times a day and the cache just grows. So instead of
using checksum we use the reponame passed in the command line invocation
and the metadata type to create a filename so that only one copy exists
for that reponame. Therefore the cache layout now is:

$CACHEDIR/fus/$reponame/$chksum.solv
$CACHEDIR/fus/$reponame/$chksum.solvx
$CACHEDIR/fus/$reponame/repodata/repomd.xml
$CACHEDIR/fus/$reponame/repodata/primary.xml.gz
$CACHEDIR/fus/$reponame/repodata/modules.xml.gz
$CACHEDIR/fus/$reponame/repodata/group_gz.x86_64.xml.xz
$CACHEDIR/fus/$reponame/repodata/filelists.xml.gz

Fixes fedora-modularity#71

Signed-off-by: Rafael Fonseca <r4f4rfs@gmail.com>
Copy link
Collaborator

@lubomir lubomir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this and did not see any change in solving results. The cache files are renamed and the total size will not grow forever anymore.

@@ -668,11 +690,15 @@ filelist_loadcb (Pool *pool,
return 1;
}

/* Cleanup old libsolv cache files (if any) */
remove_files_by_ext (cachedir, ".solvx");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is really bad idea. that means, if you use even same repo with same metadata, you always remove stuff which is unrelated. I think we should not do this…

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I can load previous libsolv cache, I don't execute this code (see previous if with return 1).
I just remove the solv files in case I couldn't load previous cache for the same metadata, in which case the metadata has changed. If I don't do that, the libsolv cache will only grow because we'll create a new file every time the metadata changes for that repo.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless it's not a problem that .solv[x] files are never removed.

@ignatenkobrain
Copy link
Member

Looks good except of removal of .solv/.solvx files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cleanup ~/.cache/fus metadata before fus exits.
3 participants