DFS-Lib provides a simple Scala interface for the HDFS filesystem API.
Right now and because work is in progress DFS-Lib does not aim to be a complete replacement for the org.apache.hadoop.fs
. That being said, DFS-Lib should provide all necessary functions to perform basic and advanced file operations.
It creates a file to the specified path with default permissions. Keep in mind, that any missing parent directory found in the path is also created (default behavior brought by the java API, it might change in the future). You can decide to (i) overwrite the file, set (ii) the hadoop replication factor, (iii) the hadoop block size, (iv) the writing buffer size. It returns true
when operation is successful:
val fs = yourHadoopClusterInstance.getFileSystem()
val pathFile = "parent/directory/test_file01.txt"
dfs.touch(fs = fs, path = pathFile)
I would like to thank @lihaoyi for his fabulous scala libraries and notably the OS-Lib tool he developed. I got heavely inspired by the way code is written there.