Explore a standalone, dependency-free R package for the Data Viewer backend #1860

Motivation

The Data Viewer backend is currently implemented inside sess, which depends on external R packages such as jsonlite, processx, and later.
These dependencies can complicate installation and cause conflicts with packages managed by renv, pixi, or other environment managers.

PR #1856 improves installation isolation, but does not eliminate these dependencies.

I propose exploring a small, standalone R package for the Data Viewer backend, similar to how jgd provides a graphics device without external R package dependencies.

Unlike session communication, the enhanced Data Viewer is optional.
If its backend is unavailable, only the enhanced viewer needs to be disabled, without affecting the R session or the standard View() behavior.

The primary goal is dependency isolation and maintainability, not performance optimization.

Proposed direction

• Publish the backend as an independent CRAN package, installed into a vscode-R-managed library rather than bundled with the extension.
• Avoid external R package dependencies (Imports / LinkingTo), using base R and a small amount of C code where appropriate.
• Keep the backend independent of sess, and VS Code-specific communication protocols.
• Reuse existing R methods such as format() for displaying S3/S4 objects, including vector classes built with vctrs.
• Preserve native comparison and sorting semantics independently of formatted display values.
• Generate only the JSON structures required by the Data Viewer, potentially using an embedded C library such as cJSON.
• Keep the existing AG Grid frontend.

A general-purpose serializer such as jsonlite provides much more functionality than the Data Viewer needs.
By normalizing R values into a limited set of display types before JSON encoding, we may be able to avoid both a jsonlite dependency and vendoring its implementation.

Prior art

• jgd: A standalone R graphics device with embedded C code and no external R package dependencies.
• vctrs / pillar: Established conventions for separating vector formatting, type information, and comparison semantics.
• Julia VS Code Table Viewer: Converts language-specific values into a limited JSON representation for AG Grid, with lazy data retrieval.

Initial investigation

A prototype could focus on ordinary data frames and matrices, evaluating:

1. Whether the existing metadata, paging, sorting, and filtering behavior can be extracted into a standalone package.
2. Whether existing R formatting and comparison methods are sufficient for custom vector classes, including those based on vctrs.
3. Whether a minimal JSON encoder can preserve existing behavior for missing values, numeric precision, dates, integer64, and complex columns.
4. Whether the package can be installed and loaded from a managed library without modifying .libPaths().

This is an architectural investigation, not a proposal to immediately replace the current implementation.

PR #1804 (lazy Arrow/DBI viewing) should proceed independently.
Arrow IPC / nanoarrow are not required for the initial prototype.

Separating the Data Viewer also does not eliminate sess's own jsonlite dependency, which is a separate concern related to #1859.
However, an independent backend could eventually be reused through alternative transports such as arf.

Related: #1856, #1859, #1804.