You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(database): integrate CompactResultDataset into SQL drivers to eliminate 3x memory copies and string allocations #1355
lib/core/database/compact_result_dataset.dart implements a columnar dataset CompactResultDataset with typed arrays (Int64List, Float64List), bit-packed null bitmaps, string interning, and lazy on-demand string generation via asLazyRowList(). It is fully tested in test/core/database/compact_result_dataset_test.dart, but is currently completely unused in production code.
Instead, PostgreSQL, MySQL, and SQLite drivers eagerly convert all query results into a full List<List<String>> matrix:
Driver raw records (Result / taken.items / results).
Intermediate rawRows (List<List<Object?>>).
Eagerly stringified outRows (List<List<String>>).
For a 10,000–50,000 row result set across 30 columns, this creates up to 1,500,000 individual String objects on the Dart VM heap, taking 150–300 MB of RAM and causing large GC pauses, instead of 10–20 MB when packed in columnar typed data.
Proposed Solution
Integrate CompactResultDataset into SqlExecutionResult, or construct CompactResultDataset.fromRawRows directly in PostgresSqlExecutionDelegate, MySqlSqlExecutionDelegate, and SqliteSqlExecutionDelegate.
Provide rows to VirtualResultGrid via dataset.asLazyRowList(), so only visible cells in the viewport are stringified.
Acceptance Criteria
Memory footprint for queries returning $\ge 10,000$ rows is reduced by $\ge 60%$.
Intermediate List<List<Object?>> and List<List<String>> allocations are eliminated in favor of columnar typed storage.
Resolved in #1464. CompactResultDataset integrated into SQLite, PostgreSQL, and MySQL execution delegates; SqlExecutionResult.fromDataset provides lazy viewport row access and maintains backwards compatibility.
Activity
perf(database): integrate CompactResultDataset into SQL drivers to el…
ZhuchkaTriplesix commentedon Oct 11, 2026
Resolved in #1464. CompactResultDataset integrated into SQLite, PostgreSQL, and MySQL execution delegates; SqlExecutionResult.fromDataset provides lazy viewport row access and maintains backwards compatibility.