Skip to content

Commit

Permalink
Bigger blocks, less loopy
Browse files Browse the repository at this point in the history
  • Loading branch information
jelmervdl committed Sep 18, 2023
1 parent 8d0ad71 commit 5347f1b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion opuscleaner/download.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def get_monolingual_dataset(entry:RemoteEntry, path:str) -> None:

def _extract(path:str, name:str, dest:str) -> str:
with ZipFile(path) as archive, archive.open(name) as fin, gzip.open(dest, 'wb') as fout:
copyfileobj(fin, fout)
copyfileobj(fin, fout, length=2**24) # 16MB blocks
return dest


Expand Down

0 comments on commit 5347f1b

Please sign in to comment.