What happens when multiple juicefs processes in different containers are mounted to the same directory #1054
-
|
Hi @davies, What happens when multiple juicefs processes in different containers on the same host are mounted to the same directory? Processes can use each other's caches or downloading the same data repeatedly？ |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
@chenshibin-vip When the processes use the same cache directories, they can share the cached data in theory. Because there is index of cached blocks in their memory, so they don't know whether a block a cached by other process or not. The process will scan the cached blocks in every N hours (hardcoded), after the re-scan, they can know the cached blocks from other processes. |
Beta Was this translation helpful? Give feedback.
-
|
@davies Processes in different Docker containers，use the same hostPath directory by volume， can they also share the cache? As you say juicefs process scan in every N hours，repeat downloads may occur during this time，is it possible to mess with data？Can multiple processes simultaneously download a data and write to the same directory for consistency? |
Beta Was this translation helpful? Give feedback.
-
|
When hostPath is used, these containers can share the data cache in kernel and in cache-dir. When you use CSI to share a PV for different PODs, they can share the cached blocks. When multiple JuiceFS processes write the blcoks into cache-dir, they write data into temporary file first, then rename it once the write is successful. So it should be safe to multiple process write the same block into disk at the same time. |
Beta Was this translation helpful? Give feedback.
-
|
@davies Each docker container starts a juicefs process and mounted the same fs to the same volume dir by hostPath. Will the above problems work？ What might go wrong with this usage scenario？ |
Beta Was this translation helpful? Give feedback.
-
|
It will have separate page cache (more memory), and more cache-miss and more requests to object storage. |
Beta Was this translation helpful? Give feedback.
@chenshibin-vip When the processes use the same cache directories, they can share the cached data in theory. Because there is index of cached blocks in their memory, so they don't know whether a block a cached by other process or not. The process will scan the cached blocks in every N hours (hardcoded), after the re-scan, they can know the cached blocks from other processes.