Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
push:
branches: ["master"]
pull_request:
branches: ["master"]
branches: ["master", "codex/**"]

permissions: read-all

Expand All @@ -15,11 +15,11 @@ jobs:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version: "1.27.1"
go-version: "1.27.4"
- name: Verify the pinned interpreter
run: test "$(go list -m -f '{{.Version}}' go.starlark.net)" = "v0.0.0-20260324133313-ffb3f39dd27a"
- name: Guard the compatible crypto dependency boundary
# v0.55.0 keeps Go 1.25 support. Its SSH/OpenPGP packages must not enter
# The reviewed crypto dependency requires Go 1.26. SSH/OpenPGP must not enter
# this application's build or tests; their advisories need a separate
# dependency and compatibility review before either package is used.
run: |
Expand All @@ -36,7 +36,7 @@ jobs:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version: "1.27.1"
go-version: "1.27.4"
- name: Build the runtime image
run: |
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -o starcli .
Expand All @@ -54,11 +54,11 @@ jobs:
# Star* module's pipeline. Pinned to a commit SHA for supply-chain safety;
# bump the pin when meta's workflow changes (Dependabot github-actions can
# automate this).
uses: 1set/meta/.github/workflows/go-ci.yml@23f088bcd6dce16c93e0056d7b70b4f4434208e8
uses: 1set/meta/.github/workflows/go-ci.yml@6b334e6a2762acaeefcea9fc7d3e86964e9246be
with:
# Match go.mod's compatibility floor; the shared workflow also tests
# the current stable Go release.
go-floor: "1.25"
go-floor: "1.26"
# Test-coverage ratchet (covcheck). The CLI-R0 characterization suite
# brought total statement coverage to ~69.5% (cli 68.9 / config 94.1 /
# util 92.7 / sys 77.8; main.go and web/ are the uncovered remainder).
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
branches: ["master"]
tags: ["v*"]
pull_request:
branches: ["master"]
branches: ["master", "codex/**"]

permissions:
contents: read
Expand All @@ -25,7 +25,7 @@ jobs:
persist-credentials: false
- uses: actions/setup-go@v6
with:
go-version: "1.27.1"
go-version: "1.27.4"
- name: Check source vulnerabilities
run: go run golang.org/x/vuln/cmd/govulncheck@v1.8.0 ./...
- name: Select release or snapshot build
Expand Down Expand Up @@ -79,7 +79,7 @@ jobs:
target: linux_arm64.tar.gz
- os: macos-15-intel
target: macOS_amd64.tar.gz
- os: macos-14
- os: macos-15
target: macOS_arm64.tar.gz
- os: windows-2022
target: windows_amd64.zip
Expand All @@ -91,7 +91,7 @@ jobs:
persist-credentials: false
- uses: actions/setup-go@v6
with:
go-version: "1.27.1"
go-version: "1.27.4"
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
with:
python-version: "3.13"
Expand Down
27 changes: 25 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ A command-line interface for executing [Starlark](https://github.com/google/star

StarCLI is a versatile tool that provides a convenient environment for running Starlark scripts from the command line. Starlark is a dialect of Python designed for configuration, extensibility, and embedding. StarCLI extends Starlark with additional modules and utilities to make it more powerful for various automation and scripting tasks.

This README describes this source tree. For an installed binary, check
`starcli --version` and its release's documentation. With Go installed,
`go version -m /path/to/starcli` shows the modules linked into that binary.
In a source module, `go list -m all` shows selected versions and replacements;
it describes that checkout, not an existing binary. Updating a checkout or an
upstream library does not update installed builds: select the dependency version
you need and rebuild.

Read the [security policy](SECURITY.md) before choosing capabilities. Scripts and
loaded source must be selected and reviewed by the host; capability tiers do not
provide a complete security boundary for arbitrary untrusted source.

## Features

- **Multiple Execution Modes**:
Expand Down Expand Up @@ -180,6 +192,14 @@ by a tier, not even `full`. For a one-flag "trust everything" run there is
command execution of any command in a single switch. Use it only with fully
trusted scripts.

SQLite's `register_function` is disabled unless the operator passes
`--dangerously-allow-all`. The `open`/`full` tiers and `--allow-cmd` do not enable
custom SQL callbacks. Ordinary SQL and built-in functions remain available.
Restricted connections also exclude callbacks registered earlier by a trusted
box in the same process. This intentionally tightens the previous default.
Custom callbacks run outside the Machine's step budget; use the trusted-script
flag only when that limitation is acceptable.

Set a stricter default for a whole deployment with the env var:

```bash
Expand Down Expand Up @@ -367,7 +387,9 @@ StarCLI is the **standard, fully-loaded** build. It is assembled from a small
reusable core — the [`kit`](kit) package — that you can use to build your **own**
CLI: a few-line Go shell that embeds your Starlark scripts and wires only the
modules you need. Your shell and the standard StarCLI construct their runtime
through the same path, so they behave identically.
through the same path. Matching behavior requires matching modules, options,
capability policy, and dependency versions. `kit.New` starts with an empty module
set and does not install the CLI's capability policy for you.

```go
//go:embed app.star
Expand Down Expand Up @@ -395,7 +417,8 @@ host_name: MyStarCLIServer

### Prerequisites

- Go 1.25.8 or later (prebuilt binaries do not require a Go installation)
- Go 1.26.0 or later for source builds and the kit API; production binaries use
Go 1.27.4 (prebuilt binaries do not require a Go installation).

### Building

Expand Down
6 changes: 6 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ do not provide process or memory isolation. The default `open` tier permits
network and filesystem access. `--allow-cmd` authorizes arbitrary host commands
and process environment access, including through `gum` and `runtime`.

SQLite custom functions require `--dangerously-allow-all`. Ordinary module and
command grants do not enable them. Normal connections exclude process-wide
callbacks, including ones registered earlier by a trusted box. Fully trusted
callbacks retain the existing separate-thread behavior and are not bounded by
`--max-steps`; this is not an isolated-worker or complete resource budget.

Use the narrowest capability grants and import directories that your script
needs. HTTP execution has request and cooperative execution limits, but these
do not preempt arbitrary Go builtins or impose a complete output-byte budget.
Expand Down
2 changes: 1 addition & 1 deletion cli/args.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ func ParseArgs() *Args {
flag.BoolVar(&args.AllowNet, "allow-net", false, "widen a restrictive tier with network modules (http, net, email, llm)")
flag.BoolVar(&args.AllowFS, "allow-fs", false, "widen a restrictive tier with filesystem modules (file, path)")
flag.BoolVar(&args.AllowCmd, "allow-cmd", false, "allow ANY host command, gum subprocesses, and runtime environment mutation (trusted scripts only)")
flag.BoolVar(&args.Dangerous, "dangerously-allow-all", false, "DANGER: open everything — network + filesystem + host command execution of ANY command. Use only with fully trusted scripts.")
flag.BoolVar(&args.Dangerous, "dangerously-allow-all", false, "DANGER: open network, filesystem, arbitrary host commands, and SQLite custom functions. Fully trusted scripts only.")
flag.BoolVar(&args.Check, "check", false, "syntax/resolve check the script (-c or file) without running it")
flag.StringVar(&args.LogFile, "log-file", "", "append the script's log module output to this file")
flag.StringVar(&args.LogFormat, "log-format", "console", "log file format: console (human) or json (machine)")
Expand Down
94 changes: 94 additions & 0 deletions cli/capability_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,15 @@ package cli
// - wired pure domain modules are classified pure and pass the safe gate
// - cmd execution gating: off by default, on with --allow-cmd /
// --dangerously-allow-all (the execCmd lever + end-to-end run())
// - SQLite custom functions require the fully trusted-script grant

import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"

"github.com/1set/starlet"
Expand Down Expand Up @@ -441,3 +443,95 @@ func TestProcessCapabilityBoundary(t *testing.T) {
}
}
}

// SQLite callbacks use their own Starlark threads and process-wide registry.
// Ordinary module/network/filesystem/command grants must not authorize them.
var sqlitePolicyFunctionID uint64

func TestSQLiteCustomFunctionPolicy(t *testing.T) {
cases := []struct {
name string
opts BoxOpts
allowed bool
}{
{"default open", BoxOpts{}, false},
{"explicit open", BoxOpts{caps: "open"}, false},
{"full tier", BoxOpts{caps: "full"}, false},
{"network and filesystem", BoxOpts{caps: "safe", allowNet: true, allowFS: true}, false},
{"command grant", BoxOpts{allowCmd: true}, false},
{"fully trusted", BoxOpts{caps: "safe", dangerous: true}, true},
}
for _, tc := range cases {
for _, eager := range []bool{false, true} {
t.Run(fmt.Sprintf("%s/eager=%t", tc.name, eager), func(t *testing.T) {
opts := tc.opts
checkSQLiteFunctionPolicy(t, &opts, eager, tc.allowed)
})
}
}
}

func checkSQLiteFunctionPolicy(t *testing.T, opts *BoxOpts, eager, allowed bool) {
t.Helper()
opts.scenario, opts.printerName = scenarioDirect, "none"
opts.moduleToLoad = []string{"sqlite"}
box, err := BuildBox(opts)
if err != nil {
t.Fatal(err)
}
fn := fmt.Sprintf("starcli_policy_fn_%d", atomic.AddUint64(&sqlitePolicyFunctionID, 1))
load, prefix := `load("sqlite", "connect", "register_function")`, ""
if eager {
load, prefix = "", "sqlite."
}
code := fmt.Sprintf(`
%s
%sregister_function(%q, lambda: 73, num_args=0)
db = %sconnect(":memory:")
def verify():
if db.query_one("SELECT %s() AS result")["result"] != 73:
fail("custom function returned an incorrect value")
verify()
db.close()
`, load, prefix, fn, prefix, fn)
_, err = box.Run(code)
if allowed {
if err != nil {
t.Fatal(err)
}
return
}
if err == nil || !strings.Contains(err.Error(), "custom functions are disabled by the host") {
t.Fatalf("registration must be rejected by host policy, got %v", err)
}
}

func TestSQLiteRestrictedConnectionIgnoresTrustedFunctions(t *testing.T) {
trusted, err := BuildBox(&BoxOpts{scenario: scenarioDirect, printerName: "none", moduleToLoad: []string{"sqlite"}, dangerous: true})
if err != nil {
t.Fatal(err)
}
fn := fmt.Sprintf("starcli_private_fn_%d", atomic.AddUint64(&sqlitePolicyFunctionID, 1))
if _, err = trusted.Run(fmt.Sprintf(`load("sqlite", "register_function"); register_function(%q, lambda: 41, num_args=0)`, fn)); err != nil {
t.Fatal(err)
}
restricted, err := BuildBox(&BoxOpts{scenario: scenarioDirect, printerName: "none", moduleToLoad: []string{"sqlite"}, caps: "full"})
if err != nil {
t.Fatal(err)
}
_, err = restricted.Run(fmt.Sprintf(`load("sqlite", "connect"); db = connect(":memory:"); row = db.query_one("SELECT %s() AS result")`, fn))
if err == nil || !strings.Contains(err.Error(), "no such function") {
t.Fatalf("restricted connection inherited a process-wide callback: %v", err)
}
if _, err = restricted.Run(`
load("sqlite", "connect")
db = connect(":memory:")
def verify():
if db.query_one("SELECT abs(-73) AS result")["result"] != 73:
fail("built-in SQLite functions must remain usable")
verify()
db.close()
`); err != nil {
t.Fatalf("ordinary SQL on a restricted connection: %v", err)
}
}
7 changes: 6 additions & 1 deletion cli/mods.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,12 @@ func loadCLIModuleByName(opts *BoxOpts, name string) (starlet.ModuleLoader, erro
}
return cmd.NewModule().LoadModule(), nil
case sqlite.ModuleName:
return sqlite.NewModule().LoadModule(), nil
// SQLite callbacks use separate Starlark threads and a process-wide
// registry. Only the fully trusted-script grant may enable them; ordinary
// SQL and built-in functions remain available under the normal grant.
return sqlite.NewModuleWithHostPolicy(sqlite.HostPolicy{
DisableCustomFunctions: !opts.dangerous,
}).LoadModule(), nil
case web.ModuleName:
return web.NewModule().LoadModule(), nil
case cache.ModuleName:
Expand Down
36 changes: 35 additions & 1 deletion e2e/e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
// *work* through the CLI, not merely that they load.
//
// Sections: local imports, golden scripts, domain modules, stdin, recordings,
// containers.
// containers, SQLite custom-function authorization.
package e2e

import (
Expand Down Expand Up @@ -378,3 +378,37 @@ func TestContainer(t *testing.T) {
}
})
}

// TestSQLiteCustomFunctionAuthorization exercises real argument parsing and the
// production module registry, including SQL calls after the explicit opt-in.
func TestSQLiteCustomFunctionAuthorization(t *testing.T) {
const code = `
load("sqlite", "connect", "register_function")
register_function("starcli_e2e_answer", lambda: 73, num_args=0)
db = connect(":memory:")
print(db.query_one("SELECT starcli_e2e_answer() AS result")["result"])
db.close()
`
for _, tc := range []struct {
name string
flags []string
allowed bool
}{
{"default", nil, false},
{"full tier", []string{"--caps", "full"}, false},
{"command grant", []string{"--allow-cmd"}, false},
{"fully trusted", []string{"--dangerously-allow-all"}, true},
} {
t.Run(tc.name, func(t *testing.T) {
args := append(append([]string{}, tc.flags...), "-o", "stdout", "-c", code)
stdout, stderr, exit := runCLI(t, "", args...)
if tc.allowed {
if exit != 0 || !strings.Contains(stdout, "73") {
t.Fatalf("trusted custom function: exit=%d stdout=%q stderr=%q", exit, stdout, stderr)
}
} else if exit == 0 || !strings.Contains(stderr, "custom functions are disabled by the host") {
t.Fatalf("custom function requires the trusted-script grant: exit=%d stdout=%q stderr=%q", exit, stdout, stderr)
}
})
}
}
20 changes: 13 additions & 7 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,19 @@
`starcli` is the **standard, fully-loaded** Star\* CLI. But it is built on a small
reusable core — the [`kit`](../kit) package — that you can use to assemble your
**own** CLI: a single Go shell that embeds your Starlark scripts and wires only
the modules you need. The standard `starcli` and your shell construct their
runtime through exactly the same path, so they behave the same.
the modules you need. The standard `starcli` and your shell share the runtime
construction path. Matching behavior requires matching modules, options,
capability policy, and dependency versions; `kit.New` does not install the CLI's
capability policy.

> **The idea:** Go is the shell, Starlark is the app. Your `main.go` stays a few
> lines and never changes; your logic lives in `.star` files you `//go:embed`.

This folder is a **separate Go module** on purpose: each demo depends on only the
runtime plus the modules it wires, so its `go.mod` is the honest, minimal
dependency tree a real shell would have.
This folder is a **separate Go module**, keeping the example mains out of the
standard CLI's coverage gate. Its `kit` dependency uses this source checkout via
`replace`. Run `go list -m all` here to inspect selected versions and replacements;
the graph can still include requirements of the parent CLI module. See the
[security policy](../SECURITY.md) before exposing scripts or loaders.

## Run the demos

Expand Down Expand Up @@ -49,8 +53,10 @@ func main() {
}
```

Its dependency tree is `starbox` + `qrcode` and nothing else — none of the other
starpkg modules the turnkey `starcli` carries.
It registers only the `qrcode` domain module, without wiring the other starpkg
modules that the standard `starcli` carries. The selected module graph can be
larger than what this shell links; `go version -m /path/to/qrcard` shows its linked
modules.

## How modules resolve

Expand Down
13 changes: 6 additions & 7 deletions examples/go.mod
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
// Separate module on purpose: these build-your-own demos each depend on only the
// runtime plus the modules they wire (not starcli's full module set), so this is
// the honest, minimal dependency tree a real shell would have. It also keeps the
// example mains out of the main module's coverage gate. CI builds it in its own
// non-gating job. The replace points at the in-repo kit until starcli is tagged.
// This separate module keeps example mains outside the standard CLI's coverage.
// CI builds it independently. The local kit replacement uses the parent CLI
// source, so selected module requirements can include the parent's graph.
// Each demo registers the runtime modules it chooses.
module github.com/1set/starcli/examples

go 1.25.8
go 1.26.0

require (
github.com/1set/starcli v0.0.0
Expand All @@ -29,7 +28,7 @@ require (
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
go.uber.org/zap v1.24.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/sys v0.49.0 // indirect
)

replace github.com/1set/starcli => ../
Loading
Loading