Repository navigation
convert: loom, import and export - #57
Conversation
Import (loompy, velocyto, kallisto|bustools; spec 2 and 3) follows anndata.read_loom: X is the spliced layer when there is one, the main matrix kept as layers["matrix"]; attributes become obs/var columns or obsm/varm. It differs on purpose: dtypes are kept, graphs become obsp/varp and global attributes uns["loom"], and genes are named by id (Accession) unless --var-names gene_symbols. The matrix is read a slab of cells at a time aligned to the file's chunks, into CSR, or dense with --dense. Strings are read and written as loompy does (XML references). Export writes spec 3.0.0 that loompy validates: X as /matrix, layers, attributes, graphs and uns scalars, dense by definition, a slab of cells at a time (a CSC source sorted to CSR on disk first). The summary gives the dense size, and a warning above 10 GB; nothing is refused. Checked on public velocyto and kallisto|bustools looms against anndata.read_loom, and a round trip that loompy's validator accepts. Fixes #26. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Test Results (py3.12)1 536 tests 1 536 ✅ 2m 12s ⏱️ Results for commit 9a39bd2. ♻️ This comment has been updated with latest results. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d69ac7ac61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Test Results (py3.13)1 536 tests 1 536 ✅ 3m 46s ⏱️ Results for commit 9a39bd2. ♻️ This comment has been updated with latest results. |
From review on #57: - --layer raw/X was checked against var; it now takes raw/var's genes, names and varm, and leaves out X and the layers, which follow var. - Gene and Accession are chosen separately, an explicit gene_ids column first; var_names were taken for ids whenever gene_symbols existed. - A graph's major coordinates are found per block, not materialised for every edge at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #26.
adata convertreads and writes loom (loompy, velocyto, kallisto|bustools; spec 2.0.1 and 3.0.0).Import follows
anndata.read_loom: X is thesplicedlayer when there is one (the main matrix kept aslayers["matrix"]), else the main matrix;--layerpicks another. Attributes with one value per cell/gene become obs/var columns, multi-column ones obsm/varm. Differences, on purpose:uns["loom"]["attrs"](anndata drops both);Accession, or kallisto|bustools'gene_id), symbols stay a column, unless--var-names gene_symbols;--obs-names-attr/--var-names-attrpick any attribute.The matrix is read a slab of cells at a time, a whole number of the file's chunks, into CSR (or dense with
--dense). Strings are read and written as loompy does (XML character references). SCope compound attributes are skipped with a pointer to #27.Export writes spec 3.0.0 that loompy's validator accepts: X (or
--layer) as/matrix, every layer, obs/var as attributes (CellID,Gene,Accession), 2-D obsm/varm, obsp/varp as graphs, uns scalars as global attributes. Matrices are dense by definition: the summary gives the size before compression and warns above 10 GB, but nothing is refused. A CSC source is sorted to CSR on disk first, so memory stays at one slab.Checked on public looms (Caltech data record 20030): velocyto and kallisto|bustools looms match
anndata.read_loom. On pbmc_10k (11,791 x 36,601, four layers): 26 s and a 159 MB Python-level peak, against 29 s and 9.9 GB forread_loom; exported back in 19 s, validated by loompy, read back identical.Also:
formats/table.write_text_column, a streaming text column writer shared with the CSV reader.🤖 Generated with Claude Code