Skip to content

v6: Collection handle + ReadResult — DataFrame flows #41

Description

@Pandys

The flagship ergonomic surface (dataframes.py): cf.data.collection(ref) holding (client, collection).

  • read(where=, fields=, limit=) -> DataFrame; where/fields/limit are client-side sugar compiled to a $match/$project/$limit pipeline over AggregateStream (the API never sees these kwargs). Kwarg is where=, never query= — a Query is the saved server-side entity
  • fetch(...) -> ReadResult (iteration + to_pandas(); read() == fetch().to_pandas()). ReadResult is the seam future to_polars()/to_arrow()/__arrow_c_stream__ land on — batch-wise internals, one batch per gRPC chunk
  • iter_batches(batch_size=) streaming, memory-bounded
  • aggregate(pipeline) — caller-supplied pipeline, passed through untouched (Elastic-backed collections use this; the sugar emits Mongo syntax only)
  • Writes: append(df) (InsertMany bidi, chunked, progress callback), update(df, on="_id") (UpdateMany), upsert(df, on=key) (SyncManyByField), replace_where(filter, set_values) (UpdateManyByQuery), delete(where=|oids=), clear()
  • Round-trip contract: read() keeps _id as a regular column so update(df) just works

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestpriority: highDo first — core pathv6v6 rewrite (Major/6)

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions