Today, we write many small parquet files, which isn't ideal. Instead, consider a mode where we write one parquet file per run, and trap things like `SIG*`/exit to write the parquet footer.
Today, we write many small parquet files, which isn't ideal.
Instead, consider a mode where we write one parquet file per run, and trap things like
SIG*/exit to write the parquet footer.