google-cloud-storage

Hi. First of all, kudos to you for the VFS cache - it's really brilliant. I have one request though.

For a little background, as of now I'm using the following settings:

--vfs-cache-mode full \
  --vfs-cache-max-size 500G \
  --vfs-cache-max-age 5000h \
  --vfs-cache-poll-interval 5m \
  --vfs-read-ahead 2G \
  --bwlimit-file 32M \
  --vfs-read-chunk-size=64M \
  --vfs-read-chunk-s

These are all functions/structs for configuring the same thing:

https://github.com/thanos-io/thanos/blob/5bda2d48b9ef0bf138af6132641eb20102223edb/pkg/objstore/azure/helpers.go#L121
https://github.com/thanos-io/thanos/blob/46ac4d590e783368fc4ec7676f29ff97cd0bd630/pkg/objstore/cos/cos.go#L95
https://github.com/thanos-io/thanos/blob/bf963237e0b95402aeea4093dce465abde055675/pkg/objstore/s3/s3.go#

What

being able to take a data object (or prefix, like a partition) and get back the commit that added/modified it.

Why

This is valuable lineage information that is currently available in lakeFS but not exposed easily, and mimics the behavior of git blame

How

Given the lakeFS API already supports listing the log of commits for an object or prefix (🎉), this could be a `

Currently upload and download throttling is done at the provider level, which means it's not uniformly supported. For example filesystem, webdav and sftp do not support it, while all 4 cloud providers do support it.

We can make this more uniform by implementing a wrapper provider (like readonly, logging, retrying, etc.), which would do rate limiting in terms of bytes sent and received bef

This is motivated by #5703. Applications may need to persist the state of pagination iterators, for example, because they are implementing a web service that returns pages of results.

I suggested this API:

struct Page {
  std::vector<std::string> bucket_names;
  std::string reader_state;
};

Page BucketsPage(gcs::Client client, std::string reader_state) {
  auto reader = client.

Is your feature request related to a problem? Please describe.
I'm using drone-cache for a Rust project to cache the target directory, which contains compiled modules for all of my project's dependencies, so that the pipeline doesn't have to rebuild everything from scratch (it takes a looong time!). Unfortunately the target directory tends to grow very large (often several GBs), so I'd li