[Bug] Empty partition directories persist on OSS after data file expiration

[qifengdao]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

1.4

Compute Engine

spark 3.5

Minimal reproduce step

After the partition table data on OSS expires, the files within the partition are successfully deleted, but the directory itself still remains.
For example, with the following configuration, data from more than 14 days ago is deleted, but the partition directories persist:
"end-input.check-partition-expire" = "true",
"partition.expiration-time" = "14 d",
"partition.timestamp-pattern" = "$day",
"partition.timestamp-formatter" = "yyyy-MM-dd"

What doesn't meet your expectations?

the files within the partition are successfully deleted, but the directory itself still remains.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!