Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions rust/contracts/rust-core-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,6 +551,7 @@ or removing a command fails until the table changes in the same commit.
| `processLifecycle.inventory` | Group a process snapshot of one Lithe instance into terminals, Agent tasks and background services |
| `processLifecycle.settle` | Turn confirmation answers into what ends, what is kept and the updated settings |
| `markdown.render` | Renders and sanitizes shared Markdown |
| `markdown.outline` | Outlines the headings of shared Markdown as a document-symbol tree |
| `markdown.resolveReference` | Decides what a link or an image of a rendered Markdown document points at |
| `keymap.parseContext` | Compile a bounded boolean shortcut context (from `lithe-core-rules`) |
| `keymap.evaluateContext` | Evaluate a precompiled shortcut context (from `lithe-core-rules`) |
Expand Down Expand Up @@ -5559,6 +5560,42 @@ load what this command refuses.
Shared examples: `rust/fixtures/editor/markdown-render-v1.json`,
`markdown-html-v1.json` and `markdown-rich-content-v1.json`.

### Markdown heading outline

`markdown.outline` accepts `{ "source": "<Markdown text>" }` and returns
`{ "symbols": [...] }`, the headings of the source as a tree in the normalized
document-symbol shape `lsp.request` returns for `textDocument/documentSymbol`.
Hosts show it where they show a language server's symbols, so a Markdown file
has a structure without a language server, and it lists the headings the
preview's outline lists: the source is parsed with the dialect of
`markdown.render`.

- Every heading Comrak parses is listed in document order, including ATX and
setext headings inside block quotes, lists and alerts. Front matter, code and
raw HTML headings such as `<h2>` are not headings.
- `name` is the heading's text as the preview shows it: text, inline code, math
literals and shortcode emoji, without images, footnote references and HTML
tags, with each run of whitespace collapsed to one space and the ends trimmed.
A heading whose name is empty is left out, and its section belongs to the
heading before it.
- A heading is a child of the nearest heading before it with a lower level, so a
skipped level nests one step.
- `kind` is `15` (`SymbolKind.String`, as editors mark Markdown headings);
`detail` and `containerName` are `null`, `tags` is empty and `deprecated` is
`false`.
- Positions are zero-based lines and UTF-16 columns (`{ line, utf16Column }`);
`\n`, `\r\n` and a lone `\r` end a line. `selectionRange` covers the heading
from its first character to just after its last, a setext heading's underline
included. `range` is its section: from the start of its first line to the start
of the line of the next heading of the same or a higher level, or to the end of
the source.

Hosts request the outline off their window thread, discard a response to an
older revision of the source, and bound the source size they send as they bound
the preview's.

Shared examples: `rust/fixtures/editor/markdown-outline-v1.json`.

### Structured logging of recoverable LSP maintenance warnings

`lsp.pollEvents` log observations may carry the existing `error` object even at
Expand Down
3 changes: 3 additions & 0 deletions rust/crates/core/src/protocol/command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ pub enum CoreCommand {
ProcessLifecycleSettle,
/// Renders and sanitizes shared Markdown (`markdown.render`).
MarkdownRender,
/// Outlines the headings of shared Markdown as document symbols (`markdown.outline`).
MarkdownOutline,
/// Decides what a link or an image of a rendered Markdown document points at
/// (`markdown.resolveReference`).
MarkdownResolveReference,
Expand Down Expand Up @@ -1187,6 +1189,7 @@ impl CoreCommand {
"processLifecycle.inventory" => Some(Self::ProcessLifecycleInventory),
"processLifecycle.settle" => Some(Self::ProcessLifecycleSettle),
"markdown.render" => Some(Self::MarkdownRender),
"markdown.outline" => Some(Self::MarkdownOutline),
"markdown.resolveReference" => Some(Self::MarkdownResolveReference),
"keymap.parseContext" => Some(Self::Keymap(KeymapCommand::ParseContext)),
"keymap.evaluateContext" => Some(Self::Keymap(KeymapCommand::EvaluateContext)),
Expand Down
22 changes: 20 additions & 2 deletions rust/crates/core/src/runtime/dispatcher.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
//! child module named after it; the remaining small families dispatch here.

use crate::project::{
self, DocumentLifecycleRequest, FileReadRequest, FileWriteRequest, MarkdownReferenceRequest,
MarkdownRenderRequest,
self, DocumentLifecycleRequest, FileReadRequest, FileWriteRequest, MarkdownOutlineRequest,
MarkdownReferenceRequest, MarkdownRenderRequest,
};
use crate::protocol::{CoreCommand, CoreError, CoreRequest, CoreResponse, ErrorCode, GitCommand};
use lithe_github::{NormalizeResponseRequest, ParseRemoteRequest, RequestPlanRequest};
Expand Down Expand Up @@ -456,6 +456,24 @@ fn execute_parsed(mut parsed: CoreRequest, document: Option<&str>) -> CoreRespon
Err(error) => CoreResponse::failure(id, error),
}
}
CoreCommand::MarkdownOutline => {
match serde_json::from_value::<MarkdownOutlineRequest>(parsed.payload).map_err(
|error| {
CoreError::new(
ErrorCode::InvalidRequest,
"Invalid Markdown outline request",
)
.with_details(error.to_string())
},
) {
Ok(request) => CoreResponse::success(
id,
serde_json::to_value(crate::project::outline(request))
.expect("Markdown outline response should encode"),
),
Err(error) => CoreResponse::failure(id, error),
}
}
CoreCommand::MarkdownResolveReference => {
match serde_json::from_value::<MarkdownReferenceRequest>(parsed.payload).map_err(
|error| {
Expand Down
1 change: 1 addition & 0 deletions rust/crates/core/src/runtime/request_trace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,7 @@ pub(crate) fn is_hot(method: &str) -> bool {
"document.classifyText"
| "document.lifecycle"
| "markdown.render"
| "markdown.outline"
| "markdown.resolveReference"
| "workspace.classifyVisibility"
| "workspace.visibilityDefaults"
Expand Down
33 changes: 33 additions & 0 deletions rust/crates/core/src/tests/project.rs
Original file line number Diff line number Diff line change
Expand Up @@ -926,6 +926,39 @@ fn markdown_render_command_returns_sanitized_html() {
assert!(!html.contains("alert(1)"));
}

#[test]
fn markdown_outline_command_answers_the_shared_fixture() {
let fixture: Value = serde_json::from_str(include_str!(
"../../../../fixtures/editor/markdown-outline-v1.json"
))
.expect("Markdown outline fixture should decode");
for case in fixture["cases"].as_array().expect("fixture cases") {
let request = serde_json::json!({
"id": "markdown-outline-1",
"command": "markdown.outline",
"payload": {"source": case["source"]}
});
let response: Value = serde_json::from_str(&execute_json(
&serde_json::to_string(&request).expect("outline request should encode"),
))
.expect("outline response should be JSON");

assert_eq!(response["ok"], true, "{}: {response}", case["name"]);
assert_eq!(response["data"], case["expected"], "{}", case["name"]);
}
}

#[test]
fn markdown_outline_command_rejects_a_payload_without_source() {
let response: Value = serde_json::from_str(&execute_json(
r#"{"id":"markdown-outline-2","command":"markdown.outline","payload":{}}"#,
))
.expect("outline response should be JSON");

assert_eq!(response["ok"], false);
assert_eq!(response["error"]["code"], "invalid_request");
}

#[test]
fn markdown_resolve_reference_command_answers_links_and_resources() {
let resolve = |reference: &str, purpose: &str| -> Value {
Expand Down
3 changes: 2 additions & 1 deletion rust/crates/project/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "lithe-project"
version = "0.1.0"
edition = "2021"
license = "Apache-2.0"
description = "Project services for Lithe: workspace files, indexing policy, search, local history and document rendering"
description = "Project services for Lithe: workspace files, indexing policy, search, local history, document rendering and Markdown outlines"

[features]
# Lets lithe-core's tests trace which directories a workspace scan enumerates.
Expand All @@ -19,6 +19,7 @@ lithe-diagnostics = { path = "../diagnostics" }
lithe-git = { path = "../git" }
lithe-protocol = { path = "../protocol" }
lithe-request-context = { path = "../request-context" }
lithe-text = { path = "../text" }
lithe-workspace = { path = "../workspace" }
regex = "1.11"
serde = { version = "1.0", features = ["derive"] }
Expand Down
4 changes: 3 additions & 1 deletion rust/crates/project/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
//! Project services: workspace files and visibility, indexing policy, search, local history,
//! document persistence decisions and Markdown rendering.
//! document persistence decisions, and Markdown rendering and heading outlines.
//!
//! Language-server sessions sit above this crate. Calls that change indexing or the search index
//! take an `update_providers` callback, which lithe-core uses to tell running sessions which
Expand All @@ -16,6 +16,7 @@ pub mod files;
mod history;
pub mod indexing;
mod markdown;
mod markdown_outline;
mod markdown_reference;
mod name_matching;
pub mod name_ranking;
Expand All @@ -27,6 +28,7 @@ pub use document_lifecycle::*;
pub use files::*;
pub use history::*;
pub use markdown::*;
pub use markdown_outline::*;
pub use markdown_reference::*;
pub use search_paging::{search_page, SearchPageRequest};

Expand Down
22 changes: 14 additions & 8 deletions rust/crates/project/src/markdown.rs
Original file line number Diff line number Diff line change
Expand Up @@ -195,13 +195,10 @@ pub struct MarkdownRenderResponse {
pub html: String,
}

/// Renders the shared Markdown dialect used by every Lithe frontend.
///
/// Raw HTML is deliberately enabled in Comrak so useful document elements such
/// as `<details>` survive parsing, then the complete fragment is passed through
/// Ammonia. This keeps formatting HTML while removing executable content,
/// event handlers, embeds, and unsafe URL schemes.
pub fn render(request: MarkdownRenderRequest) -> MarkdownRenderResponse {
/// The Comrak options of the shared dialect. Rendering and the heading outline
/// parse with the same options, so a heading the preview shows is a heading the
/// outline lists, on the same source line.
pub(crate) fn dialect() -> Options<'static> {
let mut options = Options::default();
options.extension.strikethrough = true;
// The final Ammonia pass is the security boundary. Leaving tagfilter off
Expand All @@ -226,8 +223,17 @@ pub fn render(request: MarkdownRenderRequest) -> MarkdownRenderResponse {
options.extension.header_id_prefix_in_href = true;
options.render.r#unsafe = true;
options.render.sourcepos = true;
options
}

let rendered = markdown_to_html(&request.source, &options);
/// Renders the shared Markdown dialect used by every Lithe frontend.
///
/// Raw HTML is deliberately enabled in Comrak so useful document elements such
/// as `<details>` survive parsing, then the complete fragment is passed through
/// Ammonia. This keeps formatting HTML while removing executable content,
/// event handlers, embeds, and unsafe URL schemes.
pub fn render(request: MarkdownRenderRequest) -> MarkdownRenderResponse {
let rendered = markdown_to_html(&request.source, &dialect());
let mut sanitizer = Builder::default();
sanitizer
.add_tags(&["section", "tfoot", "input", "audio", "video", "source"])
Expand Down
Loading
Loading