Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,13 @@ target/

.vscode/
.vs/
.claude/
# .claude/ is machine-local except the shared diagram skills and page templates
/.claude/*
!/.claude/skills/
/.claude/skills/*
!/.claude/skills/create-docs-diagram-figma/
!/.claude/skills/create-docs-diagram-html/
!/.claude/templates/

###############
# temp file #
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ Offline actions (`--plan`, `--self-test`, `--dump-orders`, `--baseline`, `--migr
| `--baseline` | Seed the status file from the existing translations: accept those that verify (after repair) and mark only the unrepairable ones untranslated (see [Baseline](#baseline)). | no |
| `--migrate-blobs` | Record `sourceBlob` for status entries that have only a `sourceHash`: the git blob of the English version whose content hashes to `sourceHash` (current file first, then history). Passthrough files are skipped. Prints per language how many were recorded, already valid and not found. Run it once after a `--baseline`; a run also finds the version on the fly when `sourceBlob` is missing. | no |
| `--merge-status OURS THEIRS` | Rebuild each status file after the translation branch `THEIRS` was merged into `OURS`: an entry the branch added, changed or removed since the merge base comes from the branch, every other entry from `OURS`. The workflow runs it after folding the translation branch in, because the text merge (`-X theirs`) resolves conflicting hunks of the JSON from the branch, so entries the branch only carries in an older state would overwrite newer ones from `main`. | no |
| `--apply-renames FROM TO` | Move each language's translation of an English file renamed between commits `FROM` and `TO` (git rename detection under `content/`) to the new path and rekey its `files` and `failures` entries. When a translation already exists at the new path, it's kept and the old translation is left for the next run to remove as an orphan. Without it, the next run deletes the translation at the old path as an orphan and translates the page again at the new path. | no |
| `--self-test` | Offline test suite: identity round-trip over the whole corpus, segmentation round-trip over every Markdown source, verifier/repair and segmentation fixtures, and fake-client end-to-end `--run`s in temporary mini repos (segment-level updates run in a temporary git repository). Exits 1 on any failure; the `pull_request` dry-run job runs it on PRs that touch the translation script or its config, and every `translate` run starts with it. | no |
| `--probe` | Fetch the account's service-type names and language list, print them, and check `sourceLocale`, every `translatedLocale` and the resolved service type against them (exit 1 with the closest matches on a mismatch). | yes |
| `--dump-orders DIR` | Build the exact `/translate/ai` request bodies through the same code path as `--run` and write them to `DIR/{lang}/{rel}.json` plus `DIR/summary.json` (counts, characters, environment). Nothing is sent and the status file is not modified. | no |
Expand Down
130 changes: 130 additions & 0 deletions build_scripts/translate-content.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
--baseline seed the status file from the existing translations (offline)
--migrate-blobs record sourceBlob for status entries that only have a sourceHash (offline)
--merge-status OURS THEIRS rebuild the status files entry by entry after merging a translation branch (offline)
--apply-renames FROM TO move translations of English files renamed between two commits (offline)
--self-test offline test suite (no key, no network)
--probe check service types and languages against the API
--dump-orders write the exact request bodies to a directory instead of sending
Expand Down Expand Up @@ -2029,6 +2030,61 @@ def cmd_merge_status(langs: list[str], ours_ref: str, theirs_ref: str, run: GitR
print(f"{lang}: {taken} status entries taken from {theirs_ref}, the others from {ours_ref}")


def content_renames(from_sha: str, to_sha: str, run: GitRunner = run_git) -> list[tuple[str, str]]:
"""(old, new) paths relative to content/ of every English file git detects as renamed between two commits."""
prefix = f"{CONTENT_DIR.as_posix()}/"
proc = run(["git", "diff", "--find-renames", "--name-status", "-z", from_sha, to_sha, "--", prefix])
if proc.returncode != 0:
raise SystemExit(
f"git diff {from_sha} {to_sha} failed: {proc.stderr.decode('utf-8', errors='replace').strip()}"
)
fields = proc.stdout.decode("utf-8").split("\0")
renames: list[tuple[str, str]] = []
i = 0
while i < len(fields) and fields[i]:
# Renames and copies carry two paths, every other status one.
if fields[i][0] == "R":
renames.append((fields[i + 1].removeprefix(prefix), fields[i + 2].removeprefix(prefix)))
i += 3 if fields[i][0] in "RC" else 2
return renames


def cmd_apply_renames(langs: list[str], from_ref: str, to_ref: str, run: GitRunner = run_git) -> None:
"""Move each language's translation of an English file renamed between FROM and TO to the new path and
rekey its status and failure entries. Without the move, the next run deletes the translation as an orphan
and translates the page again at its new path. A translation that already exists at the new path is kept
and the old one is left for the next run to remove."""
if not in_git_work_tree(run):
raise SystemExit("--apply-renames needs a git work tree.")
renames = content_renames(resolve_git_ref(from_ref, run), resolve_git_ref(to_ref, run), run)
for lang in langs:
status = load_status(lang)
moved, rekeyed = 0, 0
kept: list[str] = []
for old, new in renames:
source, target = target_file(lang, old), target_file(lang, new)
if target.exists():
if source.exists():
kept.append(f"{old} -> {new}")
continue
if source.exists():
target.parent.mkdir(parents=True, exist_ok=True)
source.replace(target)
moved += 1
print(f" {lang}: moved {old} -> {new}")
for section in ("files", "failures"):
if old in status[section]:
status[section][new] = status[section].pop(old)
rekeyed += 1
if moved or rekeyed:
save_status(lang, status)
print(
f"{lang}: {len(renames)} renamed English files, {moved} translations moved, {rekeyed} status entries rekeyed"
)
for pair in kept:
print(f" kept {pair}: a translation already exists at the new path")


# ---------------------------------------------------------------------------
# Commands: run / dump-orders
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -4444,6 +4500,69 @@ def entry(source_hash: str) -> dict[str, Any]:
t.check("failures are merged the same way", set(merged["failures"]) == {"b.md"}, str(merged["failures"]))


def selftest_apply_renames(t: SelfTest) -> None:
t.section("--apply-renames: translations follow renamed English files")
with git_mini_repo({"seg.md": SEG_MD}):
h = seg_harness()
seed_seg_translation(h)
seg_entry = load_status("es")["files"]["seg.md"]
status = load_status("es")
status["failures"]["guide.md"] = {"error": "x", "sourceHash": "g0"}
save_status("es", status)
write_text(target_file("es", "index.md"), "# Inicio\n")
write_text(target_file("es", "home.md"), "# Casa\n")
commit_all("Spanish")
base = git("rev-parse", "HEAD")
(CONTENT_DIR / "docs").mkdir()
git("mv", "content/seg.md", "content/docs/seg.md")
write_text(CONTENT_DIR / "docs" / "seg.md", SEG_MD.replace("**Ctrl+R**", "**Ctrl+Shift+R**"))
git("mv", "content/guide.md", "content/docs/guide.md")
git("mv", "content/index.md", "content/home.md")
write_text(CONTENT_DIR / "toc.yml", MINI_FILES["toc.yml"].replace("href: guide.md", "href: docs/guide.md"))
commit_all("English moves")
out = captured(lambda: cmd_apply_renames(["es"], base, "HEAD"))
status = load_status("es")
t.check(
"a renamed and edited file's translation moves with its status entry",
target_file("es", "docs/seg.md").exists()
and target_file("es", "docs/seg.md").read_text(encoding="utf-8") == SEG_MD_ES
and not target_file("es", "seg.md").exists()
and status["files"].get("docs/seg.md") == seg_entry
and "seg.md" not in status["files"],
out,
)
t.check(
"the moved translation is planned as outdated and updated from changed segments",
("docs/seg.md", "outdated") in h.plan().to_submit
and isinstance(
file_delta(h.config, "es", "docs/seg.md", "outdated", status["files"]["docs/seg.md"], PlanOptions()),
Delta,
),
str(h.plan().to_submit),
)
t.check(
"a failure entry is rekeyed when there is no translation to move",
"docs/guide.md" in status["failures"] and "guide.md" not in status["failures"],
str(status["failures"]),
)
t.check(
"an existing translation at the new path is kept",
target_file("es", "home.md").read_text(encoding="utf-8") == "# Casa\n"
and target_file("es", "index.md").exists()
and "kept index.md -> home.md" in out,
out,
)
t.check("an edited file that was not renamed is left alone", "toc.yml" not in out, out)
out = captured(lambda: cmd_apply_renames(["es"], "HEAD", "HEAD"))
t.check("no renames -> nothing moved", "0 renamed English files, 0 translations moved" in out, out)
with mini_repo():
t.expect_exit(
"--apply-renames outside a git work tree exits",
lambda: cmd_apply_renames(["es"], "HEAD~1", "HEAD"),
"needs a git work tree",
)


def cmd_self_test(config: TranslationConfig) -> int:
t = SelfTest()
selftest_corpus(t, config)
Expand All @@ -4455,6 +4574,7 @@ def cmd_self_test(config: TranslationConfig) -> int:
selftest_code_placeholders(t)
selftest_http_client(t)
selftest_merge_status(t)
selftest_apply_renames(t)
print(f"\nself-test: {t.passed} passed, {len(t.failures)} failed")
for name in t.failures:
print(f" FAILED: {name}")
Expand Down Expand Up @@ -4495,6 +4615,13 @@ def build_parser() -> argparse.ArgumentParser:
help="after merging translation branch THEIRS into OURS: rebuild each status file entry by entry, taking "
"THEIRS only where it changed an entry since the merge base (offline)",
)
actions.add_argument(
"--apply-renames",
nargs=2,
metavar=("FROM", "TO"),
help="move each translation of an English file renamed between commits FROM and TO to the new path and "
"rekey its status entries (offline)",
)
actions.add_argument("--self-test", action="store_true", help="offline self-test (no key, no network)")
actions.add_argument("--probe", action="store_true", help="check service types and languages against the API")
actions.add_argument(
Expand Down Expand Up @@ -4582,6 +4709,9 @@ def main() -> int:
if args.merge_status:
cmd_merge_status(langs, *args.merge_status)
return 0
if args.apply_renames:
cmd_apply_renames(langs, *args.apply_renames)
return 0
if args.baseline:
cmd_baseline(langs, sources, passthrough, args.baseline_ref, args.overwrite_baseline, args.repair_existing)
return 0
Expand Down
2 changes: 2 additions & 0 deletions content/features/Command-line-Options.md
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,8 @@ As of [version 2.8](https://github.com/TabularEditor/TabularEditor/releases/tag/

You can also annotate tables and columns if you want the schema check to treat them in a specific way. [More information here](xref:importing-tables-te2#ignoring-objects).

The Tabular Editor CLI runs the same check with [`te schemacheck`](xref:te-cli-commands#schemacheck), which also checks Power Query (M) and Direct Lake tables.

## Command Line output and Exit Codes
The command line provides various details, depending on the switches used and any events encountered during execution. Exit Codes were introduced in [version 2.7.4](https://github.com/TabularEditor/TabularEditor/releases/tag/2.7.4).

Expand Down
2 changes: 1 addition & 1 deletion content/features/te-cli/includes/te-cli-preview-notice.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
> [!IMPORTANT]
> The Tabular Editor CLI is in **Limited Public Preview**. It is offered for evaluation with a Tabular Editor account; no license is required during preview. Commands, flags, and outputs may change before general availability. **The preview build stops functioning after 2026-10-31.** We recommend against using the CLI in production CI/CD pipelines during preview.
> The Tabular Editor CLI is in **Limited Public Preview**. It is offered for evaluation with a Tabular Editor account; no license is required during preview. Commands, flags, and outputs may change before general availability. **The preview build stops functioning after 2026-12-31.** We recommend against using the CLI in production CI/CD pipelines during preview.
4 changes: 2 additions & 2 deletions content/features/te-cli/te-cli-auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
uid: te-cli-auth
title: Authentication and Connections
author: Peer Grønnerup
updated: 2026-09-11
updated: 2026-10-27
applies_to:
products:
- product: Tabular Editor 2
Expand Down Expand Up @@ -60,7 +60,7 @@ te auth login -u "$AZURE_CLIENT_ID" -t "$AZURE_TENANT_ID" --certificate ./sp.pfx
te auth login --identity # Alias: -I
```

After a successful service-principal login the CLI **caches the credentials** so every subsequent `te` command can acquire tokens silently - no need to re-pass `-u / -p / -t` or set the `AZURE_CLIENT_*` environment variables. Pass `--save=false` for a one-shot login that doesn't update the cache, or run `te auth logout` to clear it.
After a successful service-principal login the CLI **caches the credentials** so every subsequent `te` command can acquire tokens silently - no need to re-pass `-u / -p / -t` or set the `AZURE_CLIENT_*` environment variables. Run `te auth logout` to clear the cache. To use a service principal without storing its credentials on the machine, skip `te auth login`: set `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` and `AZURE_TENANT_ID` and pass `--auth env` to each command.

> [!WARNING]
> Passing secrets directly on the command line exposes them to process listings and shell history. Prefer the `AZURE_CLIENT_SECRET` environment variable, or pipe the secret via stdin with `-p -`.
Expand Down
6 changes: 3 additions & 3 deletions content/features/te-cli/te-cli-automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
uid: te-cli-automation
title: Automation and Scripting
author: Peer Grønnerup
updated: 2026-09-11
updated: 2026-10-27
applies_to:
products:
- product: Tabular Editor 2
Expand Down Expand Up @@ -36,7 +36,7 @@ te query -q "EVALUATE VALUES('Date'[Year])" --output-format csv
te bpa run --output-format json
```

Under `--output-format json`, `te validate`, `te bpa run`, `te test run`, and `te query` share one JSON document shape with a `summary`, a flat `findings[]` array, and `durationMs` - see @te-cli-findings for the shape to parse.
Under `--output-format json`, `te validate`, `te bpa run`, `te schemacheck`, `te test run`, and `te query` share one JSON document shape with a `summary`, a flat `findings[]` array, and `durationMs` - see @te-cli-findings for the shape to parse.

> [!NOTE]
> `--output-format` and `--error-format` are independent. Setting `--output-format json` does *not* switch stderr to JSON; pass `--error-format json` for that. There is no automatic format switching when stdout is redirected - the default is always `text` unless you ask otherwise.
Expand All @@ -59,7 +59,7 @@ Every `te` command exits with a predictable status code so callers can branch on
| Exit | Meaning |
| -- | -- |
| `0` | Success. |
| `1` | Generic failure - invalid arguments, command failed, validation errors, auth failure, BPA gate failed at severity >= error, a `te script` run in which a script called `Error(...)`, a `te deploy` the server accepted with object errors. For `te diff`: differences found (like the `diff`/`cmp` convention). |
| `1` | Generic failure - invalid arguments, command failed, validation errors, auth failure, BPA gate failed at severity >= error, a `te schemacheck` run that found an error (or any drift with `--fail-on warning`), a `te script` run in which a script called `Error(...)`, a `te deploy` the server accepted with object errors. For `te diff`: differences found (like the `diff`/`cmp` convention). |
| `2` | `te diff` only: an error occurred while comparing, so the difference status is unknown. |

Combine exit codes with `--ci <vsts\|github>` annotations and `--trx <file>` to surface rich failure information in CI - see @te-cli-cicd.
Expand Down
Loading