findMany returns an array and sorts it before applying skip and limit. For long sequential reads, use the collection API's _id cursor so application memory is bounded by a batch:
type Schema = { events: { type: string; payload: unknown } };
const db = new SencilloDB<Schema>({ folder: "./data", sharding: true });
const events = db.collection("events");
for await (const event of events.stream({ limit: 500 })) {
await send(event);
}Each batch is a separate read-only transaction. Writers can run between batches, and newly inserted documents with higher IDs may appear later. The stream is not a historical snapshot.
const first = await events.page({ limit: 100 });
const second = await events.page({ limit: 100, cursor: first.nextCursor });page() fetches one extra row to determine whether nextCursor exists. page and stream use ascending _id order and reject custom sort functions. limit is the page or batch size.
Store reads and writes use Node's native JSON and gzip facilities. A single JSON file must fit comfortably in memory while it is parsed or serialized. Folder mode bounds this to one loaded collection; sharded mode bounds it to loaded shard payloads plus collection metadata. Use another database when an individual shard is too large to materialize safely.
Writes use a unique temporary file, flush its contents, rename it over the destination, and attempt to flush the containing directory. This protects the previous complete file from partial replacement; it does not make a multi-file folder commit atomic across every file.