Roadmap

No-reflection, trim-safe typed collection API via CSharpDB.Generators with GetGeneratedCollectionAsync<T>(), GeneratedCollection<T>, generated field metadata, binary direct payloads for supported shapes, and NativeAOT-friendly model registration.

Separated collection write probes from the read-side B-tree routing-cache, reused traversal scratch during insert/replace, and buffered catalog mutation bookkeeping inside explicit transactions.

Recovered covered composite-index lookup optimization for queries that can be answered entirely from the index without touching the base table.

Configurable durable commit batch window to coalesce WAL fsync calls across concurrent transactions for higher write throughput.

Deduplicate SELECT output with DISTINCT. Multi-column indexes for broader query coverage.

Use eligible single-column INTEGER and single-column ordered TEXT indexes for <, >, <=, >=, and BETWEEN predicates; REAL indexes remain equality-only.

Cache parsed ASTs and query plans to avoid re-parsing identical SQL statements.

Open a database fully in memory, load from disk, and save committed snapshots back to disk.

Nested scalar, array-element, nested array-object, Guid, temporal, and ordered text path indexes.

Merge underflowed pages on delete to reclaim space via borrow/merge with interior collapse.

Maintenance report, REINDEX, VACUUM/compact, fragmentation analysis, and database size report.

CSharpDB.Daemon host with full gRPC coverage for SQL, schema, procedures, collections, and maintenance.

Incremental/sliced auto-checkpointing to move work off the triggering commit path.

Lazy-resident durable storage with on-demand page loading and gRPC tunable file-cache.

ANALYZE command with persisted row counts, column NDV/min/max, and initial stats-guided index selection.

BackupAsync / RestoreAsync as first-class operations across direct, HTTP, gRPC, CLI, and Admin.

Native .csdbtable snapshots with fast Admin Import / Export, download or server-path destinations, CREATE EXTERNAL TABLE, sys.external_tables, read-only scans/joins, and embedded primary-key lookup indexes.

Validate/apply maintenance workflow that rewrites existing child tables with persisted FK metadata across direct, HTTP, gRPC, CLI, and Admin.

Visual banded-report designer with grouping, sorting, expressions, aggregate functions, page settings, and printable preview.

The bounded SQL implementation and its automated functional qualification gate are complete across the public reference, regression suite, versioned migration capability catalog, replayable EF SQL corpus, historical-database reopen coverage, fault-injected recovery, and a canonical typed parity workload plus feature-specific direct/ADO.NET/HTTP/gRPC coverage where applicable. Status remains In Progress until the release commit records two clean Windows, Linux, and macOS passes, two passing GitHub-hosted comparisons of the 18 stable master-table rows, and two passing sequential local durable-write comparisons on one idle fixed-SSD Windows machine using the same hash-verified harness and valid, stable raw evidence. The local gate must publish its success status on that exact release commit. Supplemental performance suites remain report-only or manual diagnostics. Advanced SQL work is tracked separately below and does not keep the bounded implementation slices partially complete.

Done for the trusted in-process model: host-registered C# scalar functions, common SQL/Admin built-ins, trusted commands, Admin Forms/Reports/pipeline hooks, declarative form action sequences, and local Admin Forms C# code modules. Untrusted sandboxed UDF execution is intentionally out of scope.

Opt-in writable external table registrations over mutable .csdbx files, backed by CSharpDB B+tree storage and limited to INSERT, UPDATE, and DELETE in v1 while .csdbtable archives remain read-only.

Done as a bounded in-memory slice: ranking and common aggregate windows with partitioning, LAG/LEAD/FIRST_VALUE/LAST_VALUE, explicit ROWS frames, case-insensitive named windows, compatible shared ordering across different frames, deterministic NULL/peer behavior, prepared execution, cancellation, and configurable limits with explicit ResourceLimitExceeded failures. Disk spill is deferred and is not required for completion because memory growth is bounded; incompatible partition/order specifications and later SQL window forms remain explicitly unsupported.

Done for the bounded provider contract: a frozen, independently replayable three-version Up/Down SQL corpus covers empty and populated migrations, upgrade/downgrade paths, defaults/checks, composite keys, named relationships and immediate referential actions, primary-key changes, supported column rewrites, table/column/index rename chains, rollback, reopen, runtime CRUD, and ADO.NET schema inspection. Unsupported migration operations covered by this contract fail during SQL generation with stable CDBEF2001 diagnostics. EF migration execution remains embedded/direct; ordinary SQL and metadata transport parity is qualified separately.

Done for the bounded physical-plan contract: structural EXPLAIN and executing EXPLAIN ANALYZE rowsets distinguish estimates from actual rows, loops, and elapsed time; expose operator, access-path, index/join, predicate, and relative-cost metadata across direct, ADO.NET, HTTP, and gRPC; preserve normal DML transaction semantics; redact literal and prepared-parameter values from predicate metadata; and attach bounded partial diagnostics on cancellation or errors where safe.

Done for the bounded literal and deterministic row-local slice: persisted literal defaults, DEFAULT markers, DEFAULT VALUES, named/unnamed column and table checks, stable CHECK identities, write enforcement, reopen persistence, catalogs, EF SQL, and metadata transport are qualified.

Done for the bounded ALTER COLUMN slice: exact INTEGER/REAL conversion with eligible index rebuilding, strict dependency-free UTF-8 TEXT/BLOB conversion, and indexed TEXT collation changes use transactional shadow-root rewrites; literal-default and nullability changes use validated transactional catalog updates. The rewrite paths preserve row ids, revalidate CHECK expressions and rebuilt uniqueness, roll back atomically after injected WAL failures, persist across reopen, and are emitted with the catalog operations in EF Core Up/Down migrations.

Done for the bounded key slice: named/unnamed single and composite INTEGER/TEXT primary/unique keys support persistence, enforcement, ordered catalogs, and EF migrations. Standalone primary-key add/drop covers validated logical-key changes with engine-owned backing-index creation/removal; adding a key to a populated table additionally supports bounded single-INTEGER physical rekeying with atomic eligible relational-index and complete ready full-text-owned-store rebuilding.

Done for immediate MATCH SIMPLE: column/table scalar and composite foreign keys have stable identities, candidate-key/type/collation validation, and the full RESTRICT/NO ACTION/CASCADE/SET NULL/SET DEFAULT matrix for deletes and referenced-key updates. Nested mutations preserve normal row semantics and transactional rollback across persistence, metadata, transports, archives, and tooling. EF-generated DDL covers the bounded relationships; SET DEFAULT and mutating ON UPDATE actions require explicit migration operations rather than relationship-model scaffolding.

Statement-time and computed default expressions plus broader safe check-expression forms beyond the completed deterministic row-local contract.

Indexed TEXT/BLOB conversion, broader dependency-aware rewrites, additional key shapes and index families, and ordered/range REAL access beyond the completed bounded rewrite and single-INTEGER rekey paths.

Deferrable constraints and MATCH FULL/MATCH PARTIAL, separate from the completed immediate MATCH SIMPLE action matrix.

Additional third-party ORM suites and further normalized metadata surfaces beyond the qualified EF Core and ADO.NET contracts.

The automated GitHub gate blocks publishing on two clean full-suite passes for each supported operating system covering durable, in-memory, reopen/recovery configurations, historical-database upgrades, transport parity, the EF corpus, executable documentation, bounded property tests, and stable diagnostics for intentionally unsupported SQL. It also runs two balanced paired previous-release comparisons for the 18 persistent-read and in-memory master-table rows that are stable on hosted Windows runners. Disk-sensitive performance qualification is a required pre-tag local gate: two sequential balanced paired passes qualify each of the remaining ten durable SQL/collection single and batch write rows independently on one idle fixed-SSD Windows machine. Every exact row uses adjacent previous/candidate measurements, pass two reverses the starting order, and every individual measurement must retain at least 30 measured seconds and 10,000 latency samples within the bounded measurement cap. Each logical side has one predeclared attempt; evidence is not discarded, replaced, or silently retried. Both revisions use symmetrically conditioned artifacts with the same hash-recorded candidate benchmark harness; raw runs, recomputed aggregates, artifact closures, and pass reports are hash bound. After post-build quiescence, a required one-second Windows monitor covers every declared measurement. Five consecutive samples above 8% observable external process CPU, 0.5 CPU-core equivalent, or 4,194,304 observable external process I/O bytes per second contaminate the pass; named external build, test, installer, or update processes contaminate immediately. Missing monitor evidence, unavailable allowed runner-tree CPU, a coverage gap above five seconds, invalid benchmark evidence, unstable, order-sensitive, or regressed blocking assessments, and missing, unverifiable, or changed Windows Installer, Application event-log, or pending-file evidence fail closed; unavailable external-process counters remain explicit diagnostics. The expected local runtime is 3.5–4.5 hours. Stable comparisons enforce a 15% throughput limit and fail P95 only when regression exceeds both 25% and 0.05 ms; P99 remains diagnostic. Only the canonical durable-v3 local policy can publish a policy-bound status on the exact candidate commit. Its description is exactly policy=durable-v3; baseline=<40 lowercase hex>; design=<8 uppercase hex>; reports=<8 uppercase>/<8 uppercase>; older policy descriptions are rejected, and the release workflow requires the status from the configured attestor before publishing. Broader performance coverage remains in the existing scheduled report-only guardrails and manual supplemental suite diagnostics. This item closes when all automated functional and hosted-stable jobs and both local durable-write passes succeed for the release commit.

CSharpDB.Daemon now hosts the existing REST/HTTP /api surface and gRPC from one long-running process backed by the same warm daemon-hosted client. Standalone CSharpDB.Api remains supported for REST-only hosting.

Opt-in API-key mode protects REST /api/* and daemon gRPC calls with constant-time key comparison while keeping default no-auth behavior for compatibility.

Authorization, protected admin endpoint scopes, JWT/RBAC options, and TLS/mTLS deployment helpers for remote HTTP and gRPC access.

CSharpDB.Daemon can be packaged as a persistent background service across systemd, Windows Service, and launchd.

Self-contained daemon archives and install scripts ship for Windows, Linux, and macOS; dotnet tool, Docker, Homebrew, and winget distribution remain future work.

DbConnection.GetSchema() now exposes standard metadata collections for tooling and ORM schema discovery.

BINARY, NOCASE, NOCASE_AI, and ICU:<locale> collation work across SQL schema/query semantics, metadata, ordered SQL text indexes, and collection path indexes.

Scalar subqueries, IN/EXISTS (including correlated), UNION, UNION ALL, INTERSECT, and EXCEPT across SELECT results. INTERSECT ALL and EXCEPT ALL remain unsupported.

Admin query builder with source canvas, join editing, design grid, SQL preview, and saved layouts.

Inverted index support with tokenization, stemming, and relevance ranking.

Current phase is complete: opt-in generated models provide GetGeneratedCollectionAsync<T>, generated descriptors/index bindings, binary direct payloads for supported shapes, JSON fallback for unsupported shapes, and trim/NativeAOT smoke coverage.

Streamline NuGet/analyzer packaging, templates, onboarding docs, and project setup for the opt-in generated collection path.

Expand generator support beyond the current scalar, scalar collection, nested scalar, and nested collection-scalar shapes.

Internal row-batch transport serves as the batch-first SQL execution foundation across batch-capable result boundaries, scans, joins, and generic aggregates.

Follow writable .csdbx storage with broader external-table indexes, planner costing, and multi-column lookup/range support beyond the current archive primary-key point-lookup path.

Deep engine/page compression remains planned; application-level payload compression is available as a sample/SDK pattern without changing the storage format.

Encrypt database and WAL files with passphrase-based key management and explicit plaintext/encrypted migration/export paths; implementation must meet the database-encryption plan entry criteria before shipping.

Current phase is complete: ANALYZE-driven stats-guided costing uses internal histograms, heavy hitters, composite-prefix summaries, skew-aware estimates, correlation-aware filters/joins, non-unique lookup costing, hash build-side choice, and bounded DP join reordering.

Current phase is complete: opt-in adaptive join execution can switch eligible index nested-loop joins to hash joins and flip inner hash build sides at safe pre-emission boundaries.

Stable SQL-first diagnostics expose sys.planner_histograms, sys.planner_heavy_hitters, sys.planner_index_prefix_stats, and EXPLAIN ESTIMATE FOR <query>.

Current phase is complete: WAL frame-chunk writes, chunked checkpoint page copies, shared snapshot/export batching, reusable B-tree copy utilities, and the close-out audit cover the main storage and maintenance write paths.

Advisory planner-stat persistence can stay deferred without weakening committed-row durability, and sys.table_stats.row_count_is_exact makes exact versus estimated row-count semantics explicit.

Opt-in UseDurableCommitBatchWindow(...) batches durable WAL flushes across contending in-process transactions — an expert measure-first knob rather than default behavior.

Explicit WriteTransaction conflict-detected retry flow, shared auto-commit non-insert isolation, and opt-in ConcurrentWriteTransactions for shared implicit inserts.

Opt-in concurrent write transactions now reserve shared row-id ranges and rebase hot right-edge insert pages against pending WAL images for improved insert fan-in.

Route-aware client, REST, gRPC, daemon, and ADO.NET surfaces can target single-shard operations across multiple warm CSharpDB database files using explicit keyspace/shard-key context and stable virtual-bucket ownership.

Retained commit-log change feeds and reactive query subscriptions for read replicas, live Admin views, and event-driven applications.