Ability to sink lazy datasets to STDOUT or to files

[aaronsteers]

Description

I am reading in a massive jsonl file (several GB) - and I want to send a transformed version of it to STDOUT (or to another file buffer). It is larger-than memory, so I'm hoping to use the lazy methods to process it.

It doesn't look like this is possible today. The write_*() methods explicitly accept a file-like object, and I found previous issues in the repo discussing how those can be used to send data to STDOUT. However, the sink_*() methods expect str | Path, without an allowance or acceptance for a file-like object to write to.

Is it possible that the sink_*() methods could also support file-like outputs?

Update:

• Logged related question in StackOverflow with more context: https://stackoverflow.com/questions/79008061/proper-way-to-process-larger-than-memory-datasets-in-polars