diff --git a/AGENTS.md b/AGENTS.md index b1f068a..d03b842 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,7 +14,7 @@ This file is for agents changing the gatecheck repository. To use the gatecheck ## Layout -- `cmd/`: Cobra commands. `cmd/gatecheck` is `main`. `newCommand` wires flags. Command behavior that is not flag parsing lives in `pkg/gatecheck`. +- `cmd/`: Cobra commands. `cmd/gatecheck` is `main`. `newCommand` builds the command and its flags. Command behavior that is not flag parsing lives in `pkg/gatecheck`. - `internal/fetch`: download and SHA-256 checks. `internal/bootstrap`: scanner install and passthrough. Pinned versions are the constants in `internal/bootstrap/versions.go`. - `pkg/`: report decoding, validation, bundles, EPSS, and KEV. - `docs/`: mdBook source for docs.gatecheck.dev. `docs/agent-guide.md` is the agent guide embedded by `docs/embed.go`. @@ -26,5 +26,7 @@ This file is for agents changing the gatecheck repository. To use the gatecheck - Check `gatecheck --help` before describing a flag. `docs/cli-reference.md` matches the CLI on main. - Report type is detected from file contents. A lowercase filename substring is the fallback when the contents are ambiguous. See `docs/supported-reports.md`. - Coverage thresholds and EPSS scores are ratios from 0 to 1. -- Exit codes from `cmd/gatecheck`: 0 success, 1 threshold failure (`gatecheck.ErrValidationFailure`), 255 any other error (`os.Exit(-1)`). `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, and `osv-scanner` return the scanner's status. A missing scanner still exits 255. +- Exit codes come from `cmd.ExitCode`: 0 success, 1 threshold failure (`gatecheck.ErrValidationFailure`), 3 EPSS or KEV data failure (`gatecheck.ErrExternalData`), 2 any other error. The scanner passthrough commands return the scanner's status. A missing scanner exits 2. +- `cmd.NewGatecheckCommand` builds a new command tree on each call. Keep command state in the command's flags, not package variables. Settings are read flag, then `GATECHECK_*` environment variable, then default, through the helpers in `cmd/settings.go`. +- Tests in `cmd/cli_test.go` run the real command tree. Add a case there for a new flag or exit path. - Do not add copyright notices. diff --git a/CHANGELOG.md b/CHANGELOG.md index 8d818e6..3c2918c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,14 +7,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [UNRELEASED] +This release reworks the CLI settings layer and changes exit codes. See Changed. Fixes #73. + ### Added +- OpenSSF Scorecard JSON (`scorecard --format json`) is a report type. The `scorecard` config section fails a report whose aggregate score, or a named check's score, is below a minimum. Inconclusive checks are skipped with a warning. +- trufflehog JSON (`--json`, one object per line) is a report type. `trufflehog.limitEnabled` fails on any finding, and `verifiedOnly` counts only verified secrets. `list` never prints secret values. +- `gatecheck bootstrap --tools scorecard,trufflehog,cosign` installs pinned OpenSSF Scorecard 5.6.0, trufflehog 3.99.8, and cosign 3.1.4, and `gatecheck scorecard`, `gatecheck trufflehog`, and `gatecheck cosign` run them. A failed `gatecheck cosign verify` fails the CI step with cosign's status. Fixes #105. + +- `--log-level debug|info|warn|error` and `--log-format text|json` on every command, with `GATECHECK_LOG_LEVEL` and `GATECHECK_LOG_FORMAT`. Text logs are colored only on a terminal, and `NO_COLOR` turns color off. +- `validate --format table|markdown|json` prints the result to stdout. `list` and `list-all` take `--format table|markdown|json`. `list-all --format json` is one array keyed by file. +- `validate --input-type`, and `validate` reads stdin when FILE is omitted and `--input-type` is set. +- `validate --epss-url` and `--kev-url`. `list-all --epss-filename` and `--epss-url`. +- `GATECHECK_KEV_FILENAME` sets `--kev-filename`. +- `validate` warns when there is no config, and when the config has keys gatecheck does not read. `config convert` warns about those keys too. + - `gatecheck bootstrap --tools gosec,hadolint,zizmor` installs pinned gosec 2.30.0, hadolint 2.15.1, and zizmor 1.32.0. `gatecheck gosec`, `gatecheck hadolint`, and `gatecheck zizmor` run them. Their SARIF output is gated with the `sarif` config section. They are not part of the default install. zizmor publishes no checksum file, so its asset digests are pinned in gatecheck. - `gatecheck bootstrap --list` shows every supported tool. - The `license` config section fails Syft JSON and CycloneDX SBOMs whose package licenses match `deny` patterns, fall outside a non-empty `allow` list, or are missing when `denyUnknown` is set. SPDX `AND`, `OR`, and `WITH` expressions are evaluated. `packageRiskAcceptance` skips packages by purl or name. Fixes #103. - `list` prints the packages and licenses in Syft JSON, and `validate` and bundles check Syft JSON with the license rules. Both used to return an error. - `validate --vex FILE`, `GATECHECK_VEX`, and the config's `vex.files` read OpenVEX and CycloneDX VEX documents. Grype, CycloneDX, Trivy, and osv-scanner findings marked `not_affected`, `fixed`, `false_positive`, or `resolved` are skipped before the KEV, EPSS, and severity rules. A statement that names packages matches by package URL. The latest statement wins. Fixes #102. +### Changed + +- Exit codes: 2 for usage errors, files that cannot be read or written, and reports, configs, or VEX files that cannot be decoded; 3 when EPSS or KEV data cannot be loaded. These used to be 255. A missing scanner for a passthrough command also exits 2. +- `--audit` only turns a threshold failure into exit 0. Unsupported reports, decode errors, and EPSS or KEV failures now fail in audit mode too. +- `--silent` keeps warnings. It used to show errors only. Use `--log-level error` for the old behavior. +- Coverage thresholds over 1 are read as percents, so `80` means `0.8`, with a warning. +- `config init` and `config convert` indent JSON and YAML by two spaces, and indent TOML tables. JSON config files write `metadata` for CVE entries instead of `Metadata`. Both spellings are read. +- `list-all` exits 2 when a file cannot be read or listed. It still lists the other files first. It loads EPSS data once per run instead of once per file. +- `bundle remove` of a label that is not in the bundle exits 2 and leaves the bundle unchanged. +- Each run builds a new command tree. configkit and Viper are gone. Settings are read flag, then environment variable, then default. + +### Fixed + +- `GATECHECK_VERBOSE`, `GATECHECK_SILENT`, and `GATECHECK_AUDIT` panicked on any value. They now take booleans, and a bad value is a usage error. +- `--audit=false` lost to `GATECHECK_AUDIT=true`. A flag now wins over its variable. +- The KEV file setting read `GATECHECK_EPSS_FILENAME`, so setting that variable made `validate` read the EPSS CSV as KEV. +- `bundle create` did not truncate an existing file, and `bundle add` and `bundle remove` could leave old bytes after a shorter bundle. +- The Grype KEV rule compared ids with case. It now ignores case and also checks a match's related CVE ids. +- EPSS CSV files with a `+0000` offset in the `score_date` header failed to parse. +- Config files are closed after reading. The EPSS and KEV files passed to `validate` and `list` are closed when the command ends. + ## [0.10.0] - 2026-10-30 ### Added diff --git a/README.md b/README.md index d9ee190..fca564e 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ gatecheck validate --config gatecheck.yaml grype-report.json `gatecheck agent-guide` (alias `llm`) prints a markdown guide for coding agents. The same text is `docs/agent-guide.md`. -`gatecheck bootstrap` installs pinned Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck semgrep`, `gatecheck osemgrep`, `gatecheck trivy`, and `gatecheck osv-scanner` run them. `gatecheck bootstrap --tools gosec,hadolint,zizmor` adds gosec, hadolint, and zizmor, whose SARIF output `validate` reads. +`gatecheck bootstrap` installs pinned Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck semgrep`, `gatecheck osemgrep`, `gatecheck trivy`, and `gatecheck osv-scanner` run them. `gatecheck bootstrap --tools gosec,hadolint,zizmor` adds gosec, hadolint, and zizmor, whose SARIF output `validate` reads. `--tools scorecard,trufflehog,cosign` adds OpenSSF Scorecard and trufflehog, whose JSON `validate` gates, and cosign for signature checks. ### Summarize Reports with List diff --git a/cmd/bootstrap.go b/cmd/bootstrap.go index 56abb88..bb31029 100644 --- a/cmd/bootstrap.go +++ b/cmd/bootstrap.go @@ -39,7 +39,7 @@ func newBootstrapCommand() *cobra.Command { }) }, } - cmd.Flags().String("tools", "", "comma-separated scanners to install (default grype,syft,gitleaks,semgrep,trivy,osv-scanner; also gosec,hadolint,zizmor)") + cmd.Flags().String("tools", "", "comma-separated scanners to install (default grype,syft,gitleaks,semgrep,trivy,osv-scanner; also gosec,hadolint,zizmor,scorecard,trufflehog,cosign)") cmd.Flags().Bool("force", false, "reinstall even when the pinned version is already present") cmd.Flags().Bool("list", false, "print pinned versions, installed versions, and paths") return cmd diff --git a/cmd/bundle.go b/cmd/bundle.go index e244593..a1adb7d 100644 --- a/cmd/bundle.go +++ b/cmd/bundle.go @@ -1,7 +1,7 @@ package cmd import ( - "log/slog" + "io" "os" "path" @@ -9,100 +9,90 @@ import ( "github.com/spf13/cobra" ) -var bundleCmd = &cobra.Command{ - Use: "bundle", - Short: "create and manage a gatecheck bundle", -} - -var bundleCreateCmd = &cobra.Command{ - Use: "create BUNDLE_FILE TARGET_FILE", - Short: "create a new bundle with a new file", - Aliases: []string{"init"}, - Args: cobra.ExactArgs(2), - PreRunE: func(cmd *cobra.Command, args []string) error { - bundleFilename := args[0] - targetFilename := args[1] - - bundleFile, err := os.OpenFile(bundleFilename, os.O_CREATE|os.O_WRONLY, 0o644) - if err != nil { - return err - } - targetFile, err := os.Open(targetFilename) - if err != nil { - return err - } +func newBundleCommand() *cobra.Command { + bundleCmd := &cobra.Command{ + Use: "bundle", + Short: "create and manage a gatecheck bundle", + } - RuntimeConfig.bundleFile = bundleFile - RuntimeConfig.targetFile = targetFile - RuntimeConfig.BundleTagValue = RuntimeConfig.BundleTag.Value().([]string) - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - targetFilename := args[1] - label := path.Base(targetFilename) - bf, tf := RuntimeConfig.bundleFile, RuntimeConfig.targetFile - tags := RuntimeConfig.BundleTagValue - return gatecheck.CreateBundle(bf, tf, label, tags) - }, -} + createCmd := &cobra.Command{ + Use: "create BUNDLE_FILE TARGET_FILE", + Short: "create a new bundle with one file, replacing BUNDLE_FILE if it exists", + Aliases: []string{"init"}, + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + target, err := os.Open(args[1]) + if err != nil { + return err + } + defer func() { _ = target.Close() }() + bundleFile, err := os.OpenFile(args[0], os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644) + if err != nil { + return err + } + err = gatecheck.CreateBundle(bundleFile, target, path.Base(args[1]), stringSliceSetting(cmd, "tag", "GATECHECK_BUNDLE_TAG")) + return closeAfter(bundleFile, err) + }, + } -var bundleAddCmd = &cobra.Command{ - Use: "add BUNDLE_FILE TARGET_FILE", - Short: "add a file to a bundle", - Args: cobra.ExactArgs(2), - PreRunE: func(cmd *cobra.Command, args []string) error { - bundleFilename := args[0] - targetFilename := args[1] + addCmd := &cobra.Command{ + Use: "add BUNDLE_FILE TARGET_FILE", + Short: "add a file to a bundle", + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + target, err := os.Open(args[1]) + if err != nil { + return err + } + defer func() { _ = target.Close() }() + bundleFile, err := os.OpenFile(args[0], os.O_RDWR, 0o644) + if err != nil { + return err + } + err = gatecheck.AppendToBundle(bundleFile, target, path.Base(args[1]), stringSliceSetting(cmd, "tag", "GATECHECK_BUNDLE_TAG")) + return closeAfter(bundleFile, truncateAtOffset(bundleFile, err)) + }, + } - bundleFile, err := os.OpenFile(bundleFilename, os.O_RDWR, 0o644) - if err != nil { - return err - } - targetFile, err := os.Open(targetFilename) - if err != nil { - return err - } + removeCmd := &cobra.Command{ + Use: "remove BUNDLE_FILE LABEL", + Short: "remove a file from a bundle by label", + Aliases: []string{"rm"}, + Args: cobra.ExactArgs(2), + RunE: func(cmd *cobra.Command, args []string) error { + bundleFile, err := os.OpenFile(args[0], os.O_RDWR, 0o644) + if err != nil { + return err + } + err = gatecheck.RemoveFromBundle(bundleFile, args[1]) + return closeAfter(bundleFile, truncateAtOffset(bundleFile, err)) + }, + } - RuntimeConfig.bundleFile = bundleFile - RuntimeConfig.targetFile = targetFile - RuntimeConfig.BundleTagValue = RuntimeConfig.BundleTag.Value().([]string) - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - targetFilename := args[1] - slog.Info("bundle tag", "environment", os.Getenv("GATECHECK_BUNDLE_TAG")) - label := path.Base(targetFilename) - bf, tf := RuntimeConfig.bundleFile, RuntimeConfig.targetFile - tags := RuntimeConfig.BundleTagValue - return gatecheck.AppendToBundle(bf, tf, label, tags) - }, + for _, c := range []*cobra.Command{createCmd, addCmd} { + c.Flags().StringSliceP("tag", "t", nil, "tags for the added file, repeatable or comma-separated (env GATECHECK_BUNDLE_TAG)") + } + bundleCmd.AddCommand(createCmd, addCmd, removeCmd) + return bundleCmd } -var bundleRemoveCmd = &cobra.Command{ - Use: "remove BUNDLE_FILE TARGET_FILE", - Short: "remove a file from a bundle by label", - Aliases: []string{"rm"}, - Args: cobra.ExactArgs(2), - PreRunE: func(cmd *cobra.Command, args []string) error { - bundleFilename := args[0] - - bundleFile, err := os.OpenFile(bundleFilename, os.O_RDWR, 0o644) - if err != nil { - return err - } - RuntimeConfig.bundleFile = bundleFile - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - label := args[1] - return gatecheck.RemoveFromBundle(RuntimeConfig.bundleFile, label) - }, +// truncateAtOffset drops bytes left over from a longer old bundle after it +// was rewritten in place. +func truncateAtOffset(f *os.File, err error) error { + if err != nil { + return err + } + offset, err := f.Seek(0, io.SeekCurrent) + if err != nil { + return err + } + return f.Truncate(offset) } -func newBundleCommand() *cobra.Command { - RuntimeConfig.BundleTag.SetupCobra(bundleCreateCmd) - RuntimeConfig.BundleTag.SetupCobra(bundleAddCmd) - - bundleCmd.AddCommand(bundleCreateCmd, bundleAddCmd, bundleRemoveCmd) - return bundleCmd +func closeAfter(f *os.File, err error) error { + closeErr := f.Close() + if err != nil { + return err + } + return closeErr } diff --git a/cmd/cli-config.go b/cmd/cli-config.go deleted file mode 100644 index 4848441..0000000 --- a/cmd/cli-config.go +++ /dev/null @@ -1,236 +0,0 @@ -package cmd - -import ( - "io" - "os" - "strings" - - "github.com/gatecheckdev/configkit" - "github.com/gatecheckdev/gatecheck/pkg/gatecheck" - "github.com/spf13/cobra" -) - -var ( - metadataFlagUsage = "flag_usage" - metadataFieldType = "field_type" - metadataActionInputName = "action_input_name" - // metadataRequired = "required" -) - -type metaConfig struct { - BundleTag configkit.MetaField - EPSSURL configkit.MetaField - KEVURL configkit.MetaField - EPSSFilename configkit.MetaField - EPSSCacheDir configkit.MetaField - EPSSNoCache configkit.MetaField - KEVFilename configkit.MetaField - Verbose configkit.MetaField - Silent configkit.MetaField - ConfigFilename configkit.MetaField - Audit configkit.MetaField - BundleTagValue []string - bundleFile *os.File - targetFile *os.File - epssFile *os.File - kevFile *os.File - listSrcReader io.Reader - listSrcName string - listFormat string - gatecheckConfig *gatecheck.Config - // listAll bool - // configOutputWriter io.Writer - // configOutputFormat string -} - -var RuntimeConfig = metaConfig{ - BundleTag: configkit.MetaField{ - FieldName: "BundleTag", - EnvKey: "GATECHECK_BUNDLE_TAG", - DefaultValue: []string{}, - FlagValueP: new([]string), - EnvToValueFunc: func(s string) any { - return strings.Split(s, ",") - }, - Metadata: map[string]string{ - metadataFlagUsage: "file properties for metadata", - metadataFieldType: "string", - metadataActionInputName: "bundle_tag", - }, - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*[]string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringSliceVarP(valueP, "tag", "t", []string{}, usage) - }, - }, - EPSSURL: configkit.MetaField{ - FieldName: "EPSSURL", - EnvKey: "GATECHECK_EPSS_URL", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringVar(valueP, "epss-url", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "The url for the FIRST.org EPSS API (\"\" will use FIRST.org official API)", - metadataFieldType: "string", - metadataActionInputName: "epss_url", - }, - }, - KEVURL: configkit.MetaField{ - FieldName: "KEVURL", - EnvKey: "GATECHECK_KEV_URL", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringVar(valueP, "kev-url", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "The url for the CISA KEV API (\"\" will use CISA Official API)", - metadataFieldType: "string", - metadataActionInputName: "kev_url", - }, - }, - EPSSFilename: configkit.MetaField{ - FieldName: "EPSSFilename", - EnvKey: "GATECHECK_EPSS_FILENAME", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringVar(valueP, "epss-filename", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "the filename for a FIRST.org EPSS csv file", - metadataFieldType: "string", - metadataActionInputName: "epss_filename", - }, - }, - EPSSCacheDir: configkit.MetaField{ - FieldName: "EPSSCacheDir", - EnvKey: "GATECHECK_EPSS_CACHE_DIR", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringVar(valueP, "epss-cache-dir", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "cache directory for EPSS downloads (default: user cache dir, env GATECHECK_EPSS_CACHE_DIR)", - metadataFieldType: "string", - metadataActionInputName: "epss_cache_dir", - }, - }, - EPSSNoCache: configkit.MetaField{ - FieldName: "EPSSNoCache", - DefaultValue: false, - FlagValueP: new(bool), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*bool) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().BoolVar(valueP, "epss-no-cache", false, usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "download EPSS on every run instead of using the local cache (env GATECHECK_EPSS_CACHE=off)", - metadataFieldType: "bool", - metadataActionInputName: "epss_no_cache", - }, - }, - KEVFilename: configkit.MetaField{ - FieldName: "KEVFilename", - EnvKey: "GATECHECK_EPSS_FILENAME", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.Flags().StringVar(valueP, "kev-filename", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "the filename for a FIRST.org KEV json file", - metadataFieldType: "string", - metadataActionInputName: "kev_filename", - }, - }, - Verbose: configkit.MetaField{ - FieldName: "Verbose", - EnvKey: "GATECHECK_VERBOSE", - DefaultValue: false, - FlagValueP: new(bool), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*bool) - usage := f.Metadata[metadataFlagUsage] - cmd.PersistentFlags().BoolVarP(valueP, "verbose", "v", false, usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "log level set to debug", - metadataFieldType: "bool", - metadataActionInputName: "verbose", - }, - }, - Silent: configkit.MetaField{ - FieldName: "Silent", - EnvKey: "GATECHECK_SILENT", - DefaultValue: false, - FlagValueP: new(bool), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*bool) - usage := f.Metadata[metadataFlagUsage] - cmd.PersistentFlags().BoolVar(valueP, "silent", false, usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "log level set to only warnings & errors", - metadataFieldType: "bool", - metadataActionInputName: "silent", - }, - }, - ConfigFilename: configkit.MetaField{ - FieldName: "ConfigFilename", - EnvKey: "GATECHECK_CONFIG_FILENAME", - DefaultValue: "", - FlagValueP: new(string), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*string) - usage := f.Metadata[metadataFlagUsage] - cmd.PersistentFlags().StringVarP(valueP, "config", "f", "", usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "a validation configuration file", - metadataFieldType: "string", - metadataActionInputName: "config_filename", - }, - }, - Audit: configkit.MetaField{ - FieldName: "Audit", - EnvKey: "GATECHECK_AUDIT", - DefaultValue: false, - FlagValueP: new(bool), - CobraSetupFunc: func(f configkit.MetaField, cmd *cobra.Command) { - valueP := f.FlagValueP.(*bool) - usage := f.Metadata[metadataFlagUsage] - cmd.PersistentFlags().BoolVarP(valueP, "audit", "a", false, usage) - }, - Metadata: map[string]string{ - metadataFlagUsage: "audit mode - will run all rules but wil always exit 0 for validation failures", - metadataFieldType: "bool", - metadataActionInputName: "audit", - }, - }, -} - -func epssCacheOptions() []gatecheck.Option { - var opts []gatecheck.Option - if p, ok := RuntimeConfig.EPSSNoCache.FlagValueP.(*bool); ok && p != nil && *p { - opts = append(opts, gatecheck.WithEPSSCache(false)) - } - if p, ok := RuntimeConfig.EPSSCacheDir.FlagValueP.(*string); ok && p != nil && strings.TrimSpace(*p) != "" { - opts = append(opts, gatecheck.WithEPSSCacheDir(*p)) - } - return opts -} diff --git a/cmd/cli_test.go b/cmd/cli_test.go new file mode 100644 index 0000000..c018ba0 --- /dev/null +++ b/cmd/cli_test.go @@ -0,0 +1,355 @@ +package cmd + +import ( + "bytes" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/gatecheckdev/gatecheck/pkg/archive" +) + +const ( + grypeReport = "../test/grype-report.json" + kevJSON = "../test/known_exploited_vulnerabilities.json" + epssCSV = "../test/epss_scores-2023-06-01.csv" +) + +type result struct { + stdout, stderr string + code int +} + +// run executes a fresh command tree, the same way main does. +func run(t *testing.T, stdin string, args ...string) result { + t.Helper() + var out, errb bytes.Buffer + root := NewGatecheckCommand() + root.SetOut(&out) + root.SetErr(&errb) + root.SetIn(strings.NewReader(stdin)) + root.SetArgs(args) + err := root.Execute() + return result{stdout: out.String(), stderr: errb.String(), code: ExitCode(err)} +} + +func writeFile(t *testing.T, name, body string) string { + t.Helper() + path := filepath.Join(t.TempDir(), name) + if err := os.WriteFile(path, []byte(body), 0o600); err != nil { + t.Fatal(err) + } + return path +} + +func clearEnv(t *testing.T) { + t.Helper() + for _, key := range []string{ + "GATECHECK_CONFIG_FILENAME", "GATECHECK_AUDIT", "GATECHECK_VERBOSE", "GATECHECK_SILENT", + "GATECHECK_LOG_LEVEL", "GATECHECK_LOG_FORMAT", "GATECHECK_EPSS_FILENAME", "GATECHECK_KEV_FILENAME", + "GATECHECK_EPSS_URL", "GATECHECK_KEV_URL", "GATECHECK_VEX", "GATECHECK_BUNDLE_TAG", + } { + t.Setenv(key, "") + } +} + +const criticalLimit = "version: \"1\"\ngrype:\n severityLimit:\n critical:\n enabled: true\n limit: 0\n" + +func TestFreshCommandTreePerCall(t *testing.T) { + a, b := NewGatecheckCommand(), NewGatecheckCommand() + if a == b { + t.Fatal("same root returned twice") + } + findA, _, _ := a.Find([]string{"validate"}) + findB, _, _ := b.Find([]string{"validate"}) + if findA == findB { + t.Fatal("validate command is shared between trees") + } + if err := findA.Flags().Set("audit", "true"); err != nil { + t.Fatal(err) + } + if findB.Flags().Lookup("audit").Value.String() != "false" { + t.Fatal("flag value leaked between trees") + } +} + +func TestValidateExitCodes(t *testing.T) { + clearEnv(t) + config := writeFile(t, "gatecheck.yaml", criticalLimit) + + if r := run(t, "", "validate", "-f", config, grypeReport); r.code != ExitValidation { + t.Fatalf("threshold failure: %+v", r) + } + if r := run(t, "", "validate", "--audit", "-f", config, grypeReport); r.code != ExitOK || !strings.Contains(r.stderr, "validation failure") { + t.Fatalf("audit should exit 0 and print the failure: %+v", r) + } + if r := run(t, "", "validate", "--audit", "-f", config, "missing.json"); r.code != ExitUsage { + t.Fatalf("audit must not hide a missing file: %+v", r) + } + if r := run(t, "", "validate", "--audit", "-f", "missing.yaml", grypeReport); r.code != ExitUsage { + t.Fatalf("missing config: %+v", r) + } + if r := run(t, "", "validate", "--bogus"); r.code != ExitUsage { + t.Fatalf("bad flag: %+v", r) + } + if r := run(t, "", "validate"); r.code != ExitUsage { + t.Fatalf("no file: %+v", r) + } + unsupported := writeFile(t, "notes.txt", "hello") + if r := run(t, "", "validate", "--audit", unsupported); r.code != ExitUsage { + t.Fatalf("audit must not hide an unsupported report: %+v", r) + } + + kev := writeFile(t, "gatecheck.yaml", "version: \"1\"\ngrype:\n kevLimitEnabled: true\n") + if r := run(t, "", "validate", "-f", kev, "--kev-url", "http://127.0.0.1:1/kev.json", grypeReport); r.code != ExitFetch { + t.Fatalf("kev fetch failure: %+v", r) + } + if r := run(t, "", "validate", "--audit", "-f", kev, "--kev-url", "http://127.0.0.1:1/kev.json", grypeReport); r.code != ExitFetch { + t.Fatalf("audit must not hide a fetch failure: %+v", r) + } + if r := run(t, "", "validate", "-f", kev, "--kev-filename", kevJSON, grypeReport); r.code == ExitFetch || r.code == ExitUsage { + t.Fatalf("local kev file: %+v", r) + } +} + +func TestEnvironmentSettings(t *testing.T) { + clearEnv(t) + config := writeFile(t, "gatecheck.yaml", criticalLimit) + + t.Setenv("GATECHECK_CONFIG_FILENAME", config) + if r := run(t, "", "validate", grypeReport); r.code != ExitValidation { + t.Fatalf("config from env: %+v", r) + } + t.Setenv("GATECHECK_AUDIT", "true") + if r := run(t, "", "validate", grypeReport); r.code != ExitOK { + t.Fatalf("audit from env: %+v", r) + } + if r := run(t, "", "validate", "--audit=false", grypeReport); r.code != ExitValidation { + t.Fatalf("--audit=false must beat GATECHECK_AUDIT=true: %+v", r) + } + t.Setenv("GATECHECK_AUDIT", "") + + for _, key := range []string{"GATECHECK_VERBOSE", "GATECHECK_SILENT", "GATECHECK_AUDIT"} { + t.Setenv(key, "true") + r := run(t, "", "validate", grypeReport) + if r.code == ExitUsage { + t.Fatalf("%s=true: %+v", key, r) + } + t.Setenv(key, "maybe") + r = run(t, "", "validate", grypeReport) + if r.code != ExitUsage || !strings.Contains(r.stderr, key) { + t.Fatalf("%s=maybe should be a usage error: %+v", key, r) + } + t.Setenv(key, "") + } + + // GATECHECK_KEV_FILENAME is its own setting, not the EPSS one. + t.Setenv("GATECHECK_CONFIG_FILENAME", writeFile(t, "kev.yaml", "version: \"1\"\ngrype:\n kevLimitEnabled: true\n")) + t.Setenv("GATECHECK_KEV_URL", "http://127.0.0.1:1/kev.json") + t.Setenv("GATECHECK_EPSS_FILENAME", epssCSV) + if r := run(t, "", "validate", grypeReport); r.code != ExitFetch { + t.Fatalf("EPSS file must not be read as KEV: %+v", r) + } + t.Setenv("GATECHECK_KEV_FILENAME", kevJSON) + if r := run(t, "", "validate", grypeReport); r.code == ExitFetch || r.code == ExitUsage { + t.Fatalf("GATECHECK_KEV_FILENAME: %+v", r) + } +} + +func TestLogging(t *testing.T) { + clearEnv(t) + r := run(t, "", "validate", grypeReport) + if !strings.Contains(r.stderr, "no config file") { + t.Fatalf("missing no-config warning: %q", r.stderr) + } + r = run(t, "", "--silent", "validate", grypeReport) + if !strings.Contains(r.stderr, "no config file") || strings.Contains(r.stderr, "INF") { + t.Fatalf("--silent should keep warnings and drop info: %q", r.stderr) + } + r = run(t, "", "--log-level", "error", "validate", grypeReport) + if strings.Contains(r.stderr, "no config file") { + t.Fatalf("--log-level error should drop warnings: %q", r.stderr) + } + r = run(t, "", "--log-format", "json", "validate", grypeReport) + lines := strings.Split(strings.TrimSpace(r.stderr), "\n") + var entry map[string]any + if err := json.Unmarshal([]byte(lines[0]), &entry); err != nil || entry["level"] == nil { + t.Fatalf("json log line %q: %v", lines[0], err) + } + if r := run(t, "", "--log-format", "xml", "version"); r.code != ExitUsage { + t.Fatalf("bad log format: %+v", r) + } + if r := run(t, "", "-v", "--silent", "version"); r.code != ExitUsage { + t.Fatalf("verbose and silent together: %+v", r) + } + t.Setenv("GATECHECK_LOG_LEVEL", "debug") + if r := run(t, "", "validate", grypeReport); !strings.Contains(r.stderr, "DBG") { + t.Fatalf("GATECHECK_LOG_LEVEL=debug: %q", r.stderr) + } +} + +func TestValidateConfigWarningsAndFormat(t *testing.T) { + clearEnv(t) + typo := writeFile(t, "gatecheck.yaml", criticalLimit+"grpye:\n kevLimitEnabled: true\n") + r := run(t, "", "validate", "-f", typo, "--format", "json", grypeReport) + if r.code != ExitValidation || !strings.Contains(r.stderr, "grpye") { + t.Fatalf("unknown key warning: %+v", r) + } + var got validateResult + if err := json.Unmarshal([]byte(r.stdout), &got); err != nil { + t.Fatalf("json result %q: %v", r.stdout, err) + } + if got.Passed || len(got.Failures) != 1 || !strings.Contains(got.Failures[0], "Grype") { + t.Fatalf("result %+v", got) + } + + r = run(t, "", "validate", "--format", "table", grypeReport) + if r.code != ExitOK || !strings.Contains(r.stdout, "pass") { + t.Fatalf("table pass: %+v", r) + } + r = run(t, "", "validate", "-f", typo, "--format", "markdown", grypeReport) + if !strings.Contains(r.stdout, "| fail") { + t.Fatalf("markdown: %q", r.stdout) + } + if r := run(t, "", "validate", "--format", "yaml", grypeReport); r.code != ExitUsage { + t.Fatalf("bad format: %+v", r) + } +} + +func TestValidateInputTypeStdin(t *testing.T) { + clearEnv(t) + body, err := os.ReadFile(grypeReport) + if err != nil { + t.Fatal(err) + } + config := writeFile(t, "gatecheck.yaml", criticalLimit) + if r := run(t, string(body), "validate", "-f", config, "--input-type", "grype"); r.code != ExitValidation { + t.Fatalf("stdin: %+v", r) + } + lcov := writeFile(t, "report.txt", "SF:a.go\nDA:1,1\nDA:2,0\nLF:2\nLH:1\nend_of_record\n") + cov := writeFile(t, "gatecheck.yaml", "version: \"1\"\ncoverage:\n lineThreshold: 80\n") + r := run(t, "", "validate", "-f", cov, "--input-type", "lcov", lcov) + if r.code != ExitValidation || !strings.Contains(r.stderr, "percent") { + t.Fatalf("lcov via --input-type with a percent threshold: %+v", r) + } + cov40 := writeFile(t, "gatecheck.yaml", "version: \"1\"\ncoverage:\n lineThreshold: 40\n") + if r := run(t, "", "validate", "-f", cov40, "-i", "lcov", lcov); r.code != ExitOK { + t.Fatalf("40 percent of a 50 percent report should pass: %+v", r) + } +} + +func TestListFormats(t *testing.T) { + clearEnv(t) + r := run(t, "", "list", "--format", "json", grypeReport) + var table struct { + Columns []string `json:"columns"` + Rows []map[string]string `json:"rows"` + } + if err := json.Unmarshal([]byte(r.stdout), &table); err != nil || len(table.Rows) == 0 { + t.Fatalf("list json %q: %v", r.stdout, err) + } + if _, ok := table.Rows[0][table.Columns[0]]; !ok { + t.Fatalf("rows are not keyed by column: %+v", table.Rows[0]) + } + if r := run(t, "", "list", "--markdown", grypeReport); !strings.HasPrefix(strings.TrimSpace(r.stdout), "|") { + t.Fatalf("markdown: %q", r.stdout) + } + if r := run(t, "", "list", "--markdown", "--format", "json", grypeReport); r.code != ExitUsage { + t.Fatalf("conflicting format flags: %+v", r) + } + + empty := writeFile(t, "gitleaks-report.json", "[]") + r = run(t, "", "list-all", "--format", "json", grypeReport, empty) + var all []struct { + File string `json:"file"` + Report json.RawMessage `json:"report"` + } + if err := json.Unmarshal([]byte(r.stdout), &all); err != nil || len(all) != 2 || all[0].File != grypeReport { + t.Fatalf("list-all json %q: %v", r.stdout, err) + } + if r := run(t, "", "list-all", grypeReport, "missing.json"); r.code != ExitUsage || !strings.Contains(r.stdout, "GRYPE") { + t.Fatalf("list-all with a missing file: %+v", r) + } + if r := run(t, "", "list-all", "--epss", "--epss-filename", epssCSV, grypeReport); r.code != ExitOK || !strings.Contains(r.stdout, "EPSS") { + t.Fatalf("list-all --epss-filename: %+v", r) + } +} + +func TestBundleCommands(t *testing.T) { + clearEnv(t) + dir := t.TempDir() + bundle := filepath.Join(dir, "gatecheck-bundle.tar.gz") + big := make([]byte, 200_000) + for i := range big { + big[i] = byte('a' + i%26) + } + if err := os.WriteFile(bundle, big, 0o600); err != nil { + t.Fatal(err) + } + if r := run(t, "", "bundle", "create", "-t", "ci,main", bundle, grypeReport); r.code != ExitOK { + t.Fatalf("create over a larger file: %+v", r) + } + readBundle := func() *archive.Bundle { + t.Helper() + f, err := os.Open(bundle) + if err != nil { + t.Fatal(err) + } + defer func() { _ = f.Close() }() + b := archive.NewBundle() + if err := archive.UntarGzipBundle(f, b); err != nil { + t.Fatalf("bundle is corrupt: %v", err) + } + return b + } + b := readBundle() + if tags := b.Manifest().Files["grype-report.json"].Tags; strings.Join(tags, ",") != "ci,main" { + t.Fatalf("tags %v", tags) + } + + if r := run(t, "", "bundle", "add", bundle, epssCSV); r.code != ExitOK { + t.Fatalf("add: %+v", r) + } + if r := run(t, "", "bundle", "remove", bundle, "epss_scores-2023-06-01.csv"); r.code != ExitOK { + t.Fatalf("remove: %+v", r) + } + info, err := os.Stat(bundle) + if err != nil { + t.Fatal(err) + } + if len(readBundle().Manifest().Files) != 1 { + t.Fatal("remove did not remove") + } + if r := run(t, "", "bundle", "remove", bundle, "not-there.json"); r.code != ExitUsage { + t.Fatalf("removing a missing label should fail: %+v", r) + } + after, err := os.Stat(bundle) + if err != nil { + t.Fatal(err) + } + if after.Size() != info.Size() { + t.Fatal("failed remove changed the bundle") + } +} + +func TestConfigCommands(t *testing.T) { + clearEnv(t) + r := run(t, "", "config", "init") + if r.code != ExitOK || !strings.Contains(r.stdout, "\n severityLimit:") { + t.Fatalf("config init should be indented by two spaces: %q", r.stdout) + } + r = run(t, "", "config", "init", "-o", "json") + if !strings.Contains(r.stdout, "\n \"version\"") { + t.Fatalf("json init: %q", r.stdout) + } + if r := run(t, "", "config", "init", "-o", "xml"); r.code != ExitUsage { + t.Fatalf("bad output: %+v", r) + } + src := writeFile(t, "gatecheck.yaml", criticalLimit) + r = run(t, "", "config", "convert", "-f", src, "-o", "toml") + if r.code != ExitOK || !strings.Contains(r.stdout, "[grype") { + t.Fatalf("convert: %+v", r) + } +} diff --git a/cmd/config.go b/cmd/config.go index ba12036..b0b70a7 100644 --- a/cmd/config.go +++ b/cmd/config.go @@ -2,76 +2,55 @@ package cmd import ( "errors" + "log/slog" + "strings" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" "github.com/spf13/cobra" ) -var configCmd = &cobra.Command{ - Use: "config", - Short: "manage the gatecheck configuration file", -} - -var configInitCmd = &cobra.Command{ - Use: "init", - Short: "output an example configuration file", - RunE: func(cmd *cobra.Command, args []string) error { - output, _ := cmd.Flags().GetString("output") - - switch output { - case "json", ".json": - output = ".json" - case "toml", ".toml": - output = ".toml" - case "yaml", "yml", ".yaml", ".yml": - output = ".yaml" - default: - return errors.New("invalid --output format, must be json,toml,yaml, or yml") - } - - return gatecheck.NewConfigEncoder(cmd.OutOrStdout(), output).Encode(gatecheck.NewDefaultConfig()) - }, -} - -var configConvertCmd = &cobra.Command{ - Use: "convert", - Short: "convert and existing configuration file into another format", - PreRunE: func(cmd *cobra.Command, args []string) error { - configFilename, _ := cmd.Flags().GetString("file") - RuntimeConfig.gatecheckConfig = &gatecheck.Config{} - err := gatecheck.NewConfigDecoder(configFilename).Decode(RuntimeConfig.gatecheckConfig) - if err != nil { - return err - } - - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - output, _ := cmd.Flags().GetString("output") - - switch output { - case "json", ".json": - output = ".json" - case "toml", ".toml": - output = ".toml" - case "yaml", "yml", ".yaml", ".yml": - output = ".yaml" - default: - return errors.New("invalid --output format, must be json, toml, yaml, or yml") - } - - return gatecheck.NewConfigEncoder(cmd.OutOrStdout(), output).Encode(RuntimeConfig.gatecheckConfig) - }, -} - func newConfigCommand() *cobra.Command { - configConvertCmd.Flags().StringP("file", "f", "gatecheck.yaml", "gatecheck validation config file") - configConvertCmd.Flags().StringP("output", "o", "yaml", "Format to convert into formats=[json yaml yml toml]") - configInitCmd.Flags().StringP("output", "o", "yaml", "Format to convert into formats=[json yaml yml toml]") - - _ = configConvertCmd.MarkFlagFilename("file", "json", "yaml", "yml", "toml") - _ = configInitCmd.MarkFlagFilename("file", "json", "yaml", "yml", "toml") - - configCmd.AddCommand(configInitCmd, configConvertCmd) + configCmd := &cobra.Command{ + Use: "config", + Short: "manage the gatecheck configuration file", + } + + initCmd := &cobra.Command{ + Use: "init", + Short: "print the default configuration file", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + output, _ := cmd.Flags().GetString("output") + return gatecheck.EncodeConfigTo(cmd.OutOrStdout(), gatecheck.NewDefaultConfig(), output) + }, + } + + convertCmd := &cobra.Command{ + Use: "convert", + Short: "convert an existing configuration file into another format", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + filename, _ := cmd.Flags().GetString("file") + output, _ := cmd.Flags().GetString("output") + if _, err := gatecheck.ConfigFormat(output); err != nil { + return err + } + config := &gatecheck.Config{} + err := gatecheck.NewConfigDecoder(filename).Decode(config) + if errors.Is(err, gatecheck.ErrUnknownConfigKeys) { + slog.Warn("config has keys gatecheck does not read; they are dropped from the output", "filename", filename, "keys", strings.TrimPrefix(err.Error(), gatecheck.ErrUnknownConfigKeys.Error()+": ")) + } else if err != nil { + return err + } + return gatecheck.EncodeConfigTo(cmd.OutOrStdout(), config, output) + }, + } + + convertCmd.Flags().StringP("file", "f", "gatecheck.yaml", "gatecheck validation config file") + convertCmd.Flags().StringP("output", "o", "yaml", "format to write: json, yaml, yml, or toml") + initCmd.Flags().StringP("output", "o", "yaml", "format to write: json, yaml, yml, or toml") + _ = convertCmd.MarkFlagFilename("file", "json", "yaml", "yml", "toml") + + configCmd.AddCommand(initCmd, convertCmd) return configCmd } diff --git a/cmd/download.go b/cmd/download.go index d8b900c..de41ff3 100644 --- a/cmd/download.go +++ b/cmd/download.go @@ -5,35 +5,33 @@ import ( "github.com/spf13/cobra" ) -var downloadCmd = &cobra.Command{ - Use: "download", - Short: "output data from supported APIs", -} +func newDownloadCommand() *cobra.Command { + downloadCmd := &cobra.Command{ + Use: "download", + Short: "output data from supported APIs", + } -var downloadEPSSCmd = &cobra.Command{ - Use: "epss", - Short: "download epss data from FIRST API as csv to STDOUT", - RunE: func(cmd *cobra.Command, args []string) error { - url := RuntimeConfig.EPSSURL.Value().(string) - opts := append([]gatecheck.Option{gatecheck.WithEPSSURL(url)}, epssCacheOptions()...) - return gatecheck.DownloadEPSS(cmd.OutOrStdout(), opts...) - }, -} + epssCmd := &cobra.Command{ + Use: "epss", + Short: "download epss data from FIRST API as csv to STDOUT", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + opts := append([]gatecheck.Option{gatecheck.WithEPSSURL(stringSetting(cmd, "epss-url", "GATECHECK_EPSS_URL"))}, epssCacheOptions(cmd)...) + return gatecheck.DownloadEPSS(cmd.OutOrStdout(), opts...) + }, + } + addEPSSFlags(epssCmd, false) -var downloadKEVCmd = &cobra.Command{ - Use: "kev", - Short: "download kev catalog from CISA as json to STDOUT", - RunE: func(cmd *cobra.Command, args []string) error { - url := RuntimeConfig.KEVURL.Value().(string) - return gatecheck.DownloadKEV(cmd.OutOrStdout(), gatecheck.WithKEVURL(url)) - }, -} + kevCmd := &cobra.Command{ + Use: "kev", + Short: "download kev catalog from CISA as json to STDOUT", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + return gatecheck.DownloadKEV(cmd.OutOrStdout(), gatecheck.WithKEVURL(stringSetting(cmd, "kev-url", "GATECHECK_KEV_URL"))) + }, + } + kevCmd.Flags().String("kev-url", "", `CISA KEV JSON URL; "" uses the official feed (env GATECHECK_KEV_URL)`) -func newDownloadCommand() *cobra.Command { - RuntimeConfig.EPSSURL.SetupCobra(downloadEPSSCmd) - RuntimeConfig.EPSSCacheDir.SetupCobra(downloadEPSSCmd) - RuntimeConfig.EPSSNoCache.SetupCobra(downloadEPSSCmd) - RuntimeConfig.KEVURL.SetupCobra(downloadKEVCmd) - downloadCmd.AddCommand(downloadEPSSCmd, downloadKEVCmd) + downloadCmd.AddCommand(epssCmd, kevCmd) return downloadCmd } diff --git a/cmd/download_test.go b/cmd/download_test.go index 1b296ad..67a2aaa 100644 --- a/cmd/download_test.go +++ b/cmd/download_test.go @@ -17,18 +17,6 @@ import ( func TestEPSSDownloadCommandCachesAndKeepsFlags(t *testing.T) { t.Setenv("GATECHECK_EPSS_CACHE", "") t.Setenv("GATECHECK_EPSS_CACHE_DIR", "") - t.Cleanup(func() { - if p, ok := RuntimeConfig.EPSSNoCache.FlagValueP.(*bool); ok && p != nil { - *p = false - } - if p, ok := RuntimeConfig.EPSSCacheDir.FlagValueP.(*string); ok && p != nil { - *p = "" - } - if p, ok := RuntimeConfig.EPSSURL.FlagValueP.(*string); ok && p != nil { - *p = "" - } - }) - var gets atomic.Int32 payload := gzipCSV(t, "#model_version:v2025.03.14,score_date:2026-10-22T00:00:00Z\ncve,epss,percentile\nCVE-2024-1234,0.42,0.84\n") srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { diff --git a/cmd/fetch.go b/cmd/fetch.go new file mode 100644 index 0000000..81f2aa4 --- /dev/null +++ b/cmd/fetch.go @@ -0,0 +1,78 @@ +package cmd + +import ( + "os" + "strings" + + "github.com/gatecheckdev/gatecheck/pkg/gatecheck" + "github.com/spf13/cobra" +) + +const inputTypes = "grype|cyclonedx|semgrep|gitleaks|trivy|osv|syft|sarif|scorecard|trufflehog|bundle|lcov|clover|cobertura" + +// addEPSSFlags adds the EPSS source and cache flags. withFile adds --epss-filename. +func addEPSSFlags(cmd *cobra.Command, withFile bool) { + flags := cmd.Flags() + flags.String("epss-url", "", `FIRST.org EPSS CSV URL; "" uses the official feed (env GATECHECK_EPSS_URL)`) + flags.String("epss-cache-dir", "", "cache directory for EPSS downloads (default: user cache dir, env GATECHECK_EPSS_CACHE_DIR)") + flags.Bool("epss-no-cache", false, "download EPSS on every run instead of using the local cache (env GATECHECK_EPSS_CACHE=off)") + if withFile { + flags.String("epss-filename", "", "local FIRST.org EPSS CSV to use instead of downloading (env GATECHECK_EPSS_FILENAME)") + } +} + +func addKEVFlags(cmd *cobra.Command) { + flags := cmd.Flags() + flags.String("kev-url", "", `CISA KEV JSON URL; "" uses the official feed (env GATECHECK_KEV_URL)`) + flags.String("kev-filename", "", "local CISA KEV JSON to use instead of downloading (env GATECHECK_KEV_FILENAME)") +} + +// epssCacheOptions reads --epss-no-cache and --epss-cache-dir. GATECHECK_EPSS_CACHE +// and GATECHECK_EPSS_CACHE_DIR are read by the epss package when these are unset. +func epssCacheOptions(cmd *cobra.Command) []gatecheck.Option { + var opts []gatecheck.Option + if noCache, _ := cmd.Flags().GetBool("epss-no-cache"); noCache { + opts = append(opts, gatecheck.WithEPSSCache(false)) + } + if dir, _ := cmd.Flags().GetString("epss-cache-dir"); strings.TrimSpace(dir) != "" { + opts = append(opts, gatecheck.WithEPSSCacheDir(dir)) + } + return opts +} + +// fetchOptions builds the EPSS and KEV options for validate. The returned +// function closes any files it opened. +func fetchOptions(cmd *cobra.Command) ([]gatecheck.Option, func(), error) { + var files []*os.File + closeAll := func() { + for _, f := range files { + _ = f.Close() + } + } + opts := []gatecheck.Option{ + gatecheck.WithEPSSURL(stringSetting(cmd, "epss-url", "GATECHECK_EPSS_URL")), + gatecheck.WithKEVURL(stringSetting(cmd, "kev-url", "GATECHECK_KEV_URL")), + } + opts = append(opts, epssCacheOptions(cmd)...) + + if name := stringSetting(cmd, "epss-filename", "GATECHECK_EPSS_FILENAME"); name != "" { + f, err := os.Open(name) + if err != nil { + return nil, closeAll, err + } + files = append(files, f) + opts = append(opts, gatecheck.WithEPSSFile(f)) + } + if cmd.Flags().Lookup("kev-filename") != nil { + if name := stringSetting(cmd, "kev-filename", "GATECHECK_KEV_FILENAME"); name != "" { + f, err := os.Open(name) + if err != nil { + closeAll() + return nil, func() {}, err + } + files = append(files, f) + opts = append(opts, gatecheck.WithKEVFile(f)) + } + } + return opts, closeAll, nil +} diff --git a/cmd/gatecheck/main.go b/cmd/gatecheck/main.go index 2885150..53ac985 100644 --- a/cmd/gatecheck/main.go +++ b/cmd/gatecheck/main.go @@ -2,27 +2,15 @@ package main import ( - "errors" "fmt" - "log/slog" "os" "runtime" - "time" - - "github.com/lmittmann/tint" "github.com/gatecheckdev/gatecheck/cmd" "github.com/gatecheckdev/gatecheck/pkg/epss" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" ) -const ( - exitSystemFail int = -1 - exitOk int = 0 - exitValidationFail int = 1 - exitFileAccessFail int = 2 -) - // GatecheckVersion see CHANGELOG.md const GatecheckVersion = "[Not Provided]" @@ -49,27 +37,9 @@ func run() int { Compiler: runtime.Compiler, } epss.SetVersion(cliVersion) - - // Colorized logging output for the CLI - logHandler := tint.NewTextHandler(os.Stderr, &tint.Options{Level: cmd.LogLeveler, TimeFormat: time.TimeOnly}) - slog.SetDefault(slog.New(logHandler)) - - command := cmd.NewGatecheckCommand() - - err := command.Execute() - return status(err) + return status(cmd.NewGatecheckCommand().Execute()) } func status(err error) int { - if err == nil { - return exitOk - } - var coder interface{ ExitCode() int } - if errors.As(err, &coder) { - return coder.ExitCode() - } - if errors.Is(err, gatecheck.ErrValidationFailure) { - return exitValidationFail - } - return exitSystemFail + return cmd.ExitCode(err) } diff --git a/cmd/gatecheck/main_test.go b/cmd/gatecheck/main_test.go index e713719..05af303 100644 --- a/cmd/gatecheck/main_test.go +++ b/cmd/gatecheck/main_test.go @@ -3,8 +3,10 @@ package main import ( "errors" "fmt" + "os" "testing" + "github.com/gatecheckdev/gatecheck/internal/bootstrap" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" ) @@ -14,16 +16,22 @@ func (e exitStub) Error() string { return fmt.Sprintf("exit status %d", int(e)) func (e exitStub) ExitCode() int { return int(e) } func TestStatus(t *testing.T) { - if status(nil) != exitOk { - t.Fatal("nil") + cases := []struct { + err error + want int + }{ + {nil, 0}, + {exitStub(4), 4}, + {fmt.Errorf("%w: semgrep", gatecheck.ErrValidationFailure), 1}, + {errors.New("boom"), 2}, + {os.ErrNotExist, 2}, + {bootstrap.ErrNotInstalled, 2}, + {fmt.Errorf("%w: kev", gatecheck.ErrExternalData), 3}, + {errors.Join(fmt.Errorf("%w: a", gatecheck.ErrValidationFailure), fmt.Errorf("%w: b", gatecheck.ErrExternalData)), 1}, } - if status(exitStub(4)) != 4 { - t.Fatal("passthrough") - } - if status(fmt.Errorf("%w: semgrep", gatecheck.ErrValidationFailure)) != exitValidationFail { - t.Fatal("validation") - } - if status(errors.New("boom")) != exitSystemFail { - t.Fatal("system") + for _, tc := range cases { + if got := status(tc.err); got != tc.want { + t.Errorf("status(%v) = %d, want %d", tc.err, got, tc.want) + } } } diff --git a/cmd/list.go b/cmd/list.go index 7f2b36f..c3dc201 100644 --- a/cmd/list.go +++ b/cmd/list.go @@ -2,171 +2,170 @@ package cmd import ( "bytes" + "encoding/json" "errors" "fmt" - "io" "log/slog" "os" + "strings" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" "github.com/spf13/cobra" ) -var listCmd = &cobra.Command{ - Use: "list", - Short: "print a table of the findings in a report or files in a gatecheck bundle", - Aliases: []string{"ls", "print"}, - PreRunE: func(cmd *cobra.Command, args []string) error { - inputType, _ := cmd.Flags().GetString("input-type") - if inputType == "" && len(args) == 0 { - return errors.New("need either input-type for STDIN or filename as argument") - } - - var err error = nil - - if len(args) == 0 { - RuntimeConfig.listSrcReader = cmd.InOrStdin() - RuntimeConfig.listSrcName = fmt.Sprintf("stdin:%s", inputType) - } else { - RuntimeConfig.listSrcReader, err = os.Open(args[0]) - RuntimeConfig.listSrcName = args[0] - } - - if err != nil { - return err - } - - RuntimeConfig.listFormat = "ascii" - - if markdownFlag, _ := cmd.Flags().GetBool("markdown"); markdownFlag { - RuntimeConfig.listFormat = "markdown" - } - - if epss, _ := cmd.Flags().GetBool("epss"); !epss { - return nil - } - - RuntimeConfig.epssFile = nil - - epssFilename := RuntimeConfig.EPSSFilename.Value().(string) - - if epssFilename == "" { - return nil - } - - RuntimeConfig.epssFile, err = os.Open(epssFilename) - if err != nil { - return err - } - - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - epss, _ := cmd.Flags().GetBool("epss") - - dst := cmd.OutOrStdout() - src := RuntimeConfig.listSrcReader - srcName := RuntimeConfig.listSrcName - displayOpt := gatecheck.WithDisplayFormat(RuntimeConfig.listFormat) +func addListFormatFlags(cmd *cobra.Command) { + cmd.Flags().String("format", "table", "output format: "+strings.Join(gatecheck.ListFormats, ", ")) + cmd.Flags().Bool("markdown", false, "print as a markdown table, the same as --format markdown") + cmd.Flags().Bool("epss", false, "add EPSS scores to Grype, CycloneDX, Trivy, and osv-scanner tables") + addEPSSFlags(cmd, true) +} - if !epss { - return gatecheck.List(dst, src, srcName, displayOpt) +func listFormat(cmd *cobra.Command) (string, error) { + listFmt, _ := cmd.Flags().GetString("format") + if markdown, _ := cmd.Flags().GetBool("markdown"); markdown { + if cmd.Flags().Changed("format") && listFmt != "markdown" { + return "", fmt.Errorf("--markdown conflicts with --format %s", listFmt) } + listFmt = "markdown" + } + if err := oneOf("--format", listFmt, gatecheck.ListFormats...); err != nil { + return "", err + } + return listFmt, nil +} - epssURL := RuntimeConfig.EPSSURL.Value().(string) - epssFile := RuntimeConfig.epssFile - - // if file is nil, API will be used - // if epssURL is empty, default API will be used - epssOpt, err := gatecheck.WithEPSS(epssFile, epssURL, epssCacheOptions()...) +// epssListOption loads EPSS data once for list and list-all. +func epssListOption(cmd *cobra.Command) (gatecheck.ListOptionFunc, error) { + var file *os.File + if name := stringSetting(cmd, "epss-filename", "GATECHECK_EPSS_FILENAME"); name != "" { + f, err := os.Open(name) if err != nil { - return err + return nil, err } - return gatecheck.List(dst, src, srcName, displayOpt, epssOpt) - }, + defer func() { _ = f.Close() }() + file = f + } + url := stringSetting(cmd, "epss-url", "GATECHECK_EPSS_URL") + return gatecheck.WithEPSS(file, url, epssCacheOptions(cmd)...) } -var listAllCmd = &cobra.Command{ - Use: "list-all [FILE...]", - Short: "list multiple report files", - RunE: func(cmd *cobra.Command, args []string) error { - epss, _ := cmd.Flags().GetBool("epss") - markdown, _ := cmd.Flags().GetBool("markdown") - slog.Debug("run list all", "epss", fmt.Sprintf("%v", epss), "markdown", fmt.Sprintf("%v", markdown)) - - for _, filename := range args { - cmd.Printf("%s\n", filename) - - src, err := os.Open(filename) - if err != nil { - if errors.Is(err, os.ErrNotExist) { - slog.Error("file not found, skip", "filename", filename) - continue - } - slog.Error("cannot open file, skip", "filename", filename, "error", err) - continue - } - content, err := io.ReadAll(src) - if closeErr := src.Close(); closeErr != nil { - slog.Debug("close file", "filename", filename, "error", closeErr) +func newListCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "list [FILE]", + Short: "print a table of the findings in a report or files in a gatecheck bundle", + Aliases: []string{"ls", "print"}, + Args: cobra.MaximumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + inputType, _ := cmd.Flags().GetString("input-type") + if inputType == "" && len(args) == 0 { + return errors.New("need either --input-type for stdin or a filename argument") } + listFmt, err := listFormat(cmd) if err != nil { - slog.Error("cannot read file, skip", "filename", filename, "error", err) - continue - } - - supported, withEPSS := gatecheck.ListAllAction(content, filename) - if !supported { - slog.Warn("file not supported, skip", "filename", filename) - continue + return err } - epssURL := RuntimeConfig.EPSSURL.Value().(string) - epssFile := RuntimeConfig.epssFile - - opts := []gatecheck.ListOptionFunc{} - displayOpt := gatecheck.WithDisplayFormat("ascii") - if markdown { - displayOpt = gatecheck.WithDisplayFormat("markdown") + src := cmd.InOrStdin() + srcName := "stdin:" + inputType + if len(args) == 1 { + f, err := os.Open(args[0]) + if err != nil { + return err + } + defer func() { _ = f.Close() }() + src = f + srcName = args[0] + if inputType != "" { + srcName = inputType + } } - opts = append(opts, displayOpt) - if epss && withEPSS { - epssOpt, err := gatecheck.WithEPSS(epssFile, epssURL, epssCacheOptions()...) + opts := []gatecheck.ListOptionFunc{gatecheck.WithDisplayFormat(listFmt)} + if epss, _ := cmd.Flags().GetBool("epss"); epss { + epssOpt, err := epssListOption(cmd) if err != nil { - slog.Error("epss fetch failure, skip", "filename", filename, "error", err) - continue + return err } opts = append(opts, epssOpt) } + return gatecheck.List(cmd.OutOrStdout(), src, srcName, opts...) + }, + } + cmd.Flags().StringP("input-type", "i", "", "report type when the contents are ambiguous, and the type of stdin when FILE is omitted ["+inputTypes+"]") + addListFormatFlags(cmd) + return cmd +} - dst := cmd.OutOrStdout() - err = gatecheck.List(dst, bytes.NewReader(content), filename, opts...) +func newListAllCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "list-all FILE...", + Short: "list multiple report files", + Args: cobra.MinimumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + listFmt, err := listFormat(cmd) if err != nil { - slog.Error("cannot list report, skip", "filename", filename, "error", err) - continue + return err } + withEPSS, _ := cmd.Flags().GetBool("epss") + var epssOpt gatecheck.ListOptionFunc - } - return nil - }, -} - -func newListAllCommand() *cobra.Command { - listAllCmd.Flags().Bool("markdown", false, "print as a markdown table") - listAllCmd.Flags().Bool("epss", false, "List with EPSS data") - RuntimeConfig.EPSSCacheDir.SetupCobra(listAllCmd) - RuntimeConfig.EPSSNoCache.SetupCobra(listAllCmd) - return listAllCmd -} + type listed struct { + File string `json:"file"` + Report json.RawMessage `json:"report"` + } + listedJSON := []listed{} -func newListCommand() *cobra.Command { - listCmd.Flags().StringP("input-type", "i", "", "stdin type word used when the report content is ambiguous [grype|cyclonedx|semgrep|gitleaks|trivy|osv|syft|sarif|bundle|lcov]") - listCmd.Flags().Bool("markdown", false, "print as a markdown table") - listCmd.Flags().Bool("epss", false, "List with EPSS data") - RuntimeConfig.EPSSURL.SetupCobra(listCmd) - RuntimeConfig.EPSSFilename.SetupCobra(listCmd) - RuntimeConfig.EPSSCacheDir.SetupCobra(listCmd) - RuntimeConfig.EPSSNoCache.SetupCobra(listCmd) - return listCmd + var failed int + for _, filename := range args { + content, err := os.ReadFile(filename) + if err != nil { + slog.Error("cannot read file, skip", "filename", filename, "error", err) + failed++ + continue + } + supported, epssCapable := gatecheck.ListAllAction(content, filename) + if !supported { + slog.Warn("file not supported, skip", "filename", filename) + continue + } + opts := []gatecheck.ListOptionFunc{gatecheck.WithDisplayFormat(listFmt)} + if withEPSS && epssCapable { + if epssOpt == nil { + if epssOpt, err = epssListOption(cmd); err != nil { + return err + } + } + opts = append(opts, epssOpt) + } + if listFmt == "json" { + var buf bytes.Buffer + if err := gatecheck.List(&buf, bytes.NewReader(content), filename, opts...); err != nil { + slog.Error("cannot list report, skip", "filename", filename, "error", err) + failed++ + continue + } + listedJSON = append(listedJSON, listed{File: filename, Report: bytes.TrimSpace(buf.Bytes())}) + continue + } + cmd.Printf("%s\n", filename) + if err := gatecheck.List(cmd.OutOrStdout(), bytes.NewReader(content), filename, opts...); err != nil { + slog.Error("cannot list report, skip", "filename", filename, "error", err) + failed++ + } + } + if listFmt == "json" { + enc := json.NewEncoder(cmd.OutOrStdout()) + enc.SetIndent("", " ") + if err := enc.Encode(listedJSON); err != nil { + return err + } + } + if failed > 0 { + return fmt.Errorf("%d of %d files could not be listed", failed, len(args)) + } + return nil + }, + } + addListFormatFlags(cmd) + return cmd } diff --git a/cmd/logging.go b/cmd/logging.go new file mode 100644 index 0000000..09e37cb --- /dev/null +++ b/cmd/logging.go @@ -0,0 +1,113 @@ +package cmd + +import ( + "fmt" + "io" + "log/slog" + "os" + "strings" + "time" + + "github.com/lmittmann/tint" + "github.com/spf13/cobra" +) + +var ( + logLevels = []string{"debug", "info", "warn", "error"} + logFormats = []string{"text", "json"} +) + +func addLoggingFlags(root *cobra.Command) { + flags := root.PersistentFlags() + flags.BoolP("verbose", "v", false, "debug logging, the same as --log-level debug (env GATECHECK_VERBOSE)") + flags.Bool("silent", false, "only warnings and errors, the same as --log-level warn (env GATECHECK_SILENT)") + flags.String("log-level", "info", "log level: "+strings.Join(logLevels, ", ")+" (env GATECHECK_LOG_LEVEL)") + flags.String("log-format", "text", "log format on stderr: "+strings.Join(logFormats, ", ")+" (env GATECHECK_LOG_FORMAT)") + root.MarkFlagsMutuallyExclusive("verbose", "silent", "log-level") +} + +// logLevel picks the level. Flags beat environment variables. Among flags or +// among variables, verbose and silent beat a log level. +func logLevel(cmd *cobra.Command) (slog.Level, error) { + flags := cmd.Flags() + switch { + case flags.Changed("verbose"): + return slog.LevelDebug, nil + case flags.Changed("silent"): + return slog.LevelWarn, nil + case flags.Changed("log-level"): + v, _ := flags.GetString("log-level") + return parseLevel(v, "--log-level") + } + verbose, err := boolSetting(cmd, "verbose", "GATECHECK_VERBOSE") + if err != nil { + return 0, err + } + silent, err := boolSetting(cmd, "silent", "GATECHECK_SILENT") + if err != nil { + return 0, err + } + switch { + case verbose && silent: + return 0, fmt.Errorf("GATECHECK_VERBOSE and GATECHECK_SILENT cannot both be true") + case verbose: + return slog.LevelDebug, nil + case silent: + return slog.LevelWarn, nil + } + return parseLevel(stringSetting(cmd, "log-level", "GATECHECK_LOG_LEVEL"), "log level") +} + +func parseLevel(v, name string) (slog.Level, error) { + switch strings.ToLower(strings.TrimSpace(v)) { + case "debug": + return slog.LevelDebug, nil + case "", "info": + return slog.LevelInfo, nil + case "warn", "warning": + return slog.LevelWarn, nil + case "error": + return slog.LevelError, nil + default: + return 0, fmt.Errorf("invalid %s %q, must be one of %s", name, v, strings.Join(logLevels, ", ")) + } +} + +// setupLogging installs the default slog logger on the command's stderr. +func setupLogging(cmd *cobra.Command) error { + level, err := logLevel(cmd) + if err != nil { + return err + } + logFormat := strings.ToLower(stringSetting(cmd, "log-format", "GATECHECK_LOG_FORMAT")) + if err := oneOf("log format", logFormat, logFormats...); err != nil { + return err + } + slog.SetDefault(slog.New(newLogHandler(cmd.ErrOrStderr(), logFormat, level))) + slog.Debug("logging configured", "level", level.String(), "format", logFormat) + return nil +} + +func newLogHandler(w io.Writer, format string, level slog.Level) slog.Handler { + if format == "json" { + return slog.NewJSONHandler(w, &slog.HandlerOptions{Level: level}) + } + return tint.NewTextHandler(w, &tint.Options{ + Level: level, + TimeFormat: time.TimeOnly, + NoColor: !colorOutput(w), + }) +} + +// colorOutput is true for a terminal when NO_COLOR is unset. +func colorOutput(w io.Writer) bool { + if _, ok := os.LookupEnv("NO_COLOR"); ok { + return false + } + f, ok := w.(*os.File) + if !ok { + return false + } + info, err := f.Stat() + return err == nil && info.Mode()&os.ModeCharDevice != 0 +} diff --git a/cmd/root.go b/cmd/root.go index cb3410c..eac561f 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -1,117 +1,120 @@ -// Package cmd contains the CLI code for Gatecheck +// Package cmd contains the CLI code for Gatecheck. // -// # Organization Methodology -// -// The goal for this package is readability, ease of maintainence, and -// seperation of concerns for easier testing and debugging. -// -// newCommand functions should only build the command structure -// to include flag, cli options, and viper bindings. -// sub commands can also be included here as determined by the complexity -// of the command. -// -// run functions are specific to cobra's runE functions -// it handles parsing arguments, opening files, and early returning errors. -// These commands eventually result in calls to functions in the package -// github.com/gatecheckdev/gatecheck/pkg/gatecheck - -// The root file contains common helper functions used by other commands. -// Major commands can be in seperate files for ease of readability. +// NewGatecheckCommand builds a fresh command tree on every call, so commands +// hold no state between runs. Each newCommand function wires flags and +// reads settings in the order flag, then environment variable, then default. +// Command behavior that is not flag parsing lives in +// github.com/gatecheckdev/gatecheck/pkg/gatecheck. package cmd import ( - "log/slog" - "sync" + "errors" + "fmt" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" "github.com/spf13/cobra" - "github.com/spf13/viper" ) -var ( - ApplicationMetadata gatecheck.ApplicationMetadata - LogLeveler *slog.LevelVar = &slog.LevelVar{} +// ApplicationMetadata is the build information printed by version. main sets it. +var ApplicationMetadata gatecheck.ApplicationMetadata + +// Exit codes. +const ( + ExitOK = 0 + // ExitValidation means a report failed a configured threshold. + ExitValidation = 1 + // ExitUsage means a bad command line, a file that could not be read or + // written, or a report or config that could not be decoded. + ExitUsage = 2 + // ExitFetch means EPSS or KEV data could not be downloaded or read. + ExitFetch = 3 ) -var gatecheckCmd = &cobra.Command{ - Use: "gatecheck", - Short: "Report validation tool", - PersistentPreRun: func(cmd *cobra.Command, args []string) { - verbose := RuntimeConfig.Verbose.Value().(bool) - silent := RuntimeConfig.Silent.Value().(bool) - - switch { - case verbose: - LogLeveler.Set(slog.LevelDebug) - slog.Debug("debug logging enabled") - case silent: - LogLeveler.Set(slog.LevelError) - slog.Debug("silent logging enabled") - } - }, - RunE: func(cmd *cobra.Command, args []string) error { - versionFlag, _ := cmd.Flags().GetBool("version") - if versionFlag { - return versionCmd.RunE(cmd, args) - } - return nil - }, -} - -var versionCmd = &cobra.Command{ - Use: "version", - Short: "print version and build information", - RunE: func(cmd *cobra.Command, args []string) error { - _, err := ApplicationMetadata.WriteTo(cmd.OutOrStdout()) - return err - }, +// ExitCode maps an error returned by the command tree to a process status. +// Scanner passthrough commands return the scanner's own status. +func ExitCode(err error) int { + if err == nil { + return ExitOK + } + var coder interface{ ExitCode() int } + if errors.As(err, &coder) { + return coder.ExitCode() + } + switch { + case errors.Is(err, gatecheck.ErrValidationFailure): + return ExitValidation + case errors.Is(err, gatecheck.ErrExternalData): + return ExitFetch + default: + return ExitUsage + } } -var setupOnce sync.Once - -// NewGatecheckCommand the root for all CLI commands +// NewGatecheckCommand returns a new root command and its subcommands. func NewGatecheckCommand() *cobra.Command { - setupOnce.Do(func() { - RuntimeConfig.Verbose.SetupCobra(gatecheckCmd) - RuntimeConfig.Silent.SetupCobra(gatecheckCmd) - - gatecheckCmd.MarkFlagsMutuallyExclusive("verbose", "silent") - gatecheckCmd.Flags().Bool("version", false, "print version and build information") - - _ = viper.BindEnv("cli.audit", "GATECHECK_CLI_AUDIT") - - _ = viper.BindEnv("cli.list.epss-file", "GATECHECK_EPSS_FILE") - _ = viper.BindEnv("cli.validate.epss-file", "GATECHECK_EPSS_FILE") - - _ = viper.BindEnv("cli.validate.kev-file", "GATECHECK_KEV_FILE") - - _ = viper.BindEnv("api.epss-url", "GATECHECK_EPSS_URL") - _ = viper.BindEnv("api.kev-url", "GATECHECK_KEV_URL") + root := &cobra.Command{ + Use: "gatecheck", + Short: "Report validation tool", + SilenceUsage: true, + SilenceErrors: false, + Args: cobra.NoArgs, + PersistentPreRunE: func(cmd *cobra.Command, _ []string) error { + return setupLogging(cmd) + }, + RunE: func(cmd *cobra.Command, _ []string) error { + if show, _ := cmd.Flags().GetBool("version"); show { + _, err := ApplicationMetadata.WriteTo(cmd.OutOrStdout()) + return err + } + return nil + }, + } + addLoggingFlags(root) + root.Flags().Bool("version", false, "print version and build information") + root.SetFlagErrorFunc(func(cmd *cobra.Command, err error) error { + return fmt.Errorf("%w\nRun '%s --help' for usage", err, cmd.CommandPath()) + }) - gatecheckCmd.SilenceUsage = true + root.AddCommand( + newVersionCommand(), + newConfigCommand(), + newListCommand(), + newListAllCommand(), + newBundleCommand(), + newValidateCommand(), + newDownloadCommand(), + newAgentGuideCommand(), + newBootstrapCommand(), + newUpdateCommand(), + ) + for _, tool := range []struct{ name, short string }{ + {"grype", "Run the managed grype scanner"}, + {"syft", "Run the managed syft scanner"}, + {"gitleaks", "Run the managed gitleaks scanner"}, + {"semgrep", "Run osemgrep from the managed semgrep wheel"}, + {"osemgrep", "Run osemgrep from the managed semgrep wheel"}, + {"trivy", "Run the managed trivy scanner"}, + {"osv-scanner", "Run the managed osv-scanner"}, + {"gosec", "Run the managed gosec scanner"}, + {"hadolint", "Run the managed hadolint linter"}, + {"zizmor", "Run the managed zizmor scanner"}, + {"scorecard", "Run the managed OpenSSF Scorecard"}, + {"trufflehog", "Run the managed trufflehog scanner"}, + {"cosign", "Run the managed cosign"}, + } { + root.AddCommand(newPassthroughCommand(tool.name, tool.short)) + } + return root +} - gatecheckCmd.AddCommand( - versionCmd, - newConfigCommand(), - newListCommand(), - newListAllCommand(), - newBundleCommand(), - newValidateCommand(), - newDownloadCommand(), - newAgentGuideCommand(), - newBootstrapCommand(), - newPassthroughCommand("grype", "Run the managed grype scanner"), - newPassthroughCommand("syft", "Run the managed syft scanner"), - newPassthroughCommand("gitleaks", "Run the managed gitleaks scanner"), - newPassthroughCommand("semgrep", "Run osemgrep from the managed semgrep wheel"), - newPassthroughCommand("osemgrep", "Run osemgrep from the managed semgrep wheel"), - newPassthroughCommand("trivy", "Run the managed trivy scanner"), - newPassthroughCommand("osv-scanner", "Run the managed osv-scanner"), - newPassthroughCommand("gosec", "Run the managed gosec scanner"), - newPassthroughCommand("hadolint", "Run the managed hadolint linter"), - newPassthroughCommand("zizmor", "Run the managed zizmor scanner"), - ) - addCommandOnce(gatecheckCmd, updateCmd) - }) - return gatecheckCmd +func newVersionCommand() *cobra.Command { + return &cobra.Command{ + Use: "version", + Short: "print version and build information", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + _, err := ApplicationMetadata.WriteTo(cmd.OutOrStdout()) + return err + }, + } } diff --git a/cmd/settings.go b/cmd/settings.go new file mode 100644 index 0000000..7bb9acb --- /dev/null +++ b/cmd/settings.go @@ -0,0 +1,87 @@ +package cmd + +import ( + "fmt" + "os" + "strconv" + "strings" + + "github.com/spf13/cobra" +) + +// Settings are read flag first, then environment variable, then the flag's +// default. An empty environment variable counts as unset. + +func envValue(env string) (string, bool) { + if env == "" { + return "", false + } + v, ok := os.LookupEnv(env) + if !ok || strings.TrimSpace(v) == "" { + return "", false + } + return strings.TrimSpace(v), true +} + +func stringSetting(cmd *cobra.Command, flag, env string) string { + f := cmd.Flags().Lookup(flag) + if f != nil && f.Changed { + return f.Value.String() + } + if v, ok := envValue(env); ok { + return v + } + if f != nil { + return f.Value.String() + } + return "" +} + +func boolSetting(cmd *cobra.Command, flag, env string) (bool, error) { + f := cmd.Flags().Lookup(flag) + if f != nil && f.Changed { + return strconv.ParseBool(f.Value.String()) + } + if v, ok := envValue(env); ok { + b, err := strconv.ParseBool(v) + if err != nil { + return false, fmt.Errorf("%s=%q is not a boolean (use true or false)", env, v) + } + return b, nil + } + if f != nil { + return strconv.ParseBool(f.Value.String()) + } + return false, nil +} + +func stringSliceSetting(cmd *cobra.Command, flag, env string) []string { + f := cmd.Flags().Lookup(flag) + if f != nil && f.Changed { + values, _ := cmd.Flags().GetStringSlice(flag) + return values + } + if v, ok := envValue(env); ok { + var out []string + for _, part := range strings.Split(v, ",") { + if part = strings.TrimSpace(part); part != "" { + out = append(out, part) + } + } + return out + } + if f != nil { + values, _ := cmd.Flags().GetStringSlice(flag) + return values + } + return nil +} + +func oneOf(name, value string, allowed ...string) error { + for _, a := range allowed { + if value == a { + return nil + } + } + return fmt.Errorf("invalid %s %q, must be one of %s", name, value, strings.Join(allowed, ", ")) +} diff --git a/cmd/update.go b/cmd/update.go index 227af19..0484e47 100644 --- a/cmd/update.go +++ b/cmd/update.go @@ -52,14 +52,3 @@ and latest versions and does not download.`, command.Flags().BoolVarP(&yes, "yes", "y", false, "do not ask for confirmation") return command } - -func addCommandOnce(root *cobra.Command, command *cobra.Command) { - for _, existing := range root.Commands() { - if existing.Name() == command.Name() { - return - } - } - root.AddCommand(command) -} - -var updateCmd = newUpdateCommand() diff --git a/cmd/update_test.go b/cmd/update_test.go index 7432947..f9d0485 100644 --- a/cmd/update_test.go +++ b/cmd/update_test.go @@ -15,6 +15,7 @@ import ( ) func TestUpdateCommandFlags(t *testing.T) { + updateCmd := newUpdateCommand() if updateCmd.Name() != "update" { t.Fatalf("name %s", updateCmd.Name()) } @@ -119,36 +120,21 @@ func resetUpdateCommand(t *testing.T) { t.Helper() updateAPIBase = "" updateExecutable = "" - if err := updateCmd.Flags().Set("check", "false"); err != nil { - t.Fatal(err) - } - if err := updateCmd.Flags().Set("version", ""); err != nil { - t.Fatal(err) - } - if err := updateCmd.Flags().Set("yes", "false"); err != nil { - t.Fatal(err) - } t.Cleanup(func() { updateAPIBase = "" updateExecutable = "" - _ = updateCmd.Flags().Set("check", "false") - _ = updateCmd.Flags().Set("version", "") - _ = updateCmd.Flags().Set("yes", "false") - updateCmd.SetArgs(nil) }) } func executeUpdate(t *testing.T, stdin string, args ...string) (string, string, error) { t.Helper() - addCommandOnce(gatecheckCmd, updateCmd) out := &bytes.Buffer{} errBuf := &bytes.Buffer{} - root := updateCmd.Root() + root := NewGatecheckCommand() root.SetOut(out) root.SetErr(errBuf) root.SetIn(strings.NewReader(stdin)) root.SetArgs(append([]string{"update"}, args...)) err := root.Execute() - t.Cleanup(func() { root.SetArgs([]string{}) }) return out.String(), errBuf.String(), err } diff --git a/cmd/validate.go b/cmd/validate.go index dbfd66c..2ede836 100644 --- a/cmd/validate.go +++ b/cmd/validate.go @@ -1,97 +1,185 @@ package cmd import ( + "encoding/json" + "errors" "fmt" + "io" "log/slog" "os" "strings" + "github.com/gatecheckdev/gatecheck/pkg/format" "github.com/gatecheckdev/gatecheck/pkg/gatecheck" "github.com/spf13/cobra" ) -var validateCmd = &cobra.Command{ - Use: "validate [FILE]", - Short: "compare vulnerabilities to configured thresholds", - Args: cobra.ExactArgs(1), - PreRunE: func(cmd *cobra.Command, args []string) error { - configFilename := RuntimeConfig.ConfigFilename.Value().(string) +var validateFormats = []string{"table", "markdown", "json"} - RuntimeConfig.gatecheckConfig = gatecheck.NewDefaultConfig() - if configFilename != "" { - err := gatecheck.NewConfigDecoder(configFilename).Decode(RuntimeConfig.gatecheckConfig) - if err != nil { - return err - } - } +func newValidateCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "validate [FILE]", + Short: "compare a report or bundle to configured thresholds", + Long: `Compare a report or bundle to the thresholds in a config file. - var err error +FILE is required unless --input-type is set, in which case the report is read +from stdin. Exit status: 0 pass, 1 threshold failure, 2 usage, file, or +decode error, 3 EPSS or KEV data could not be loaded. --audit turns a +threshold failure into exit 0. It does not hide any other error.`, + Args: cobra.MaximumNArgs(1), + RunE: runValidate, + } + flags := cmd.Flags() + flags.StringP("config", "f", "", "validation config file (env GATECHECK_CONFIG_FILENAME)") + flags.StringP("input-type", "i", "", "report type when the contents are ambiguous, and the type of stdin when FILE is omitted ["+inputTypes+"]") + flags.BoolP("audit", "a", false, "exit 0 when a threshold fails; other errors still fail (env GATECHECK_AUDIT)") + flags.String("format", "", "also print the result to stdout: "+strings.Join(validateFormats, ", ")) + flags.StringSlice("vex", nil, "OpenVEX or CycloneDX VEX file; findings it marks not_affected, fixed, false_positive, or resolved are skipped (repeatable, env GATECHECK_VEX)") + addEPSSFlags(cmd, true) + addKEVFlags(cmd) + _ = cmd.MarkFlagFilename("config", "json", "yaml", "yml", "toml") + return cmd +} - epssFilename := RuntimeConfig.EPSSFilename.Value().(string) - if epssFilename != "" { - RuntimeConfig.epssFile, err = os.Open(epssFilename) - } - if err != nil { +func runValidate(cmd *cobra.Command, args []string) error { + outputFormat, _ := cmd.Flags().GetString("format") + if outputFormat != "" { + if err := oneOf("--format", outputFormat, validateFormats...); err != nil { return err } + } + audit, err := boolSetting(cmd, "audit", "GATECHECK_AUDIT") + if err != nil { + return err + } + inputType, _ := cmd.Flags().GetString("input-type") + if len(args) == 0 && inputType == "" { + return errors.New("need a FILE argument, or --input-type to read the report from stdin") + } - kevFilename := RuntimeConfig.KEVFilename.Value().(string) - if kevFilename != "" { - RuntimeConfig.kevFile, err = os.Open(kevFilename) - } - if err != nil { - return err - } + config, err := loadValidateConfig(cmd) + if err != nil { + return err + } - targetFilename := args[0] - slog.Debug("open target file", "filename", targetFilename) - RuntimeConfig.targetFile, err = os.Open(targetFilename) + var src io.Reader + name := "stdin" + if len(args) == 1 { + name = args[0] + slog.Debug("open target file", "filename", name) + f, err := os.Open(name) if err != nil { return err } + defer func() { _ = f.Close() }() + src = f + } else { + src = cmd.InOrStdin() + } + kindHint := name + if inputType != "" { + kindHint = inputType + } - return nil - }, - RunE: func(cmd *cobra.Command, args []string) error { - opts := []gatecheck.Option{ - gatecheck.WithEPSSURL(RuntimeConfig.EPSSURL.Value().(string)), - gatecheck.WithKEVURL(RuntimeConfig.KEVURL.Value().(string)), - gatecheck.WithEPSSFile(RuntimeConfig.epssFile), // TODO: fix this - gatecheck.WithKEVFile(RuntimeConfig.kevFile), - } - opts = append(opts, epssCacheOptions()...) - vexFiles, _ := cmd.Flags().GetStringSlice("vex") - if env := strings.TrimSpace(os.Getenv("GATECHECK_VEX")); len(vexFiles) == 0 && env != "" { - vexFiles = strings.Split(env, ",") - } - opts = append(opts, gatecheck.WithVEXFiles(vexFiles...)) - err := gatecheck.Validate( - RuntimeConfig.gatecheckConfig, - RuntimeConfig.targetFile, - args[0], - opts..., - ) + opts, closeFiles, err := fetchOptions(cmd) + if err != nil { + return err + } + defer closeFiles() + opts = append(opts, gatecheck.WithVEXFiles(stringSliceSetting(cmd, "vex", "GATECHECK_VEX")...)) - audit := RuntimeConfig.Audit.Value().(bool) - if audit && err != nil { - slog.Error("validation failure in audit mode") - _, err = fmt.Fprintln(cmd.ErrOrStderr(), err) + validateErr := gatecheck.Validate(config, src, kindHint, opts...) + + if outputFormat != "" { + if err := writeValidateResult(cmd.OutOrStdout(), outputFormat, name, validateErr); err != nil { return err } + } + if audit && errors.Is(validateErr, gatecheck.ErrValidationFailure) { + slog.Warn("validation failure ignored in audit mode") + _, _ = fmt.Fprintln(cmd.ErrOrStderr(), validateErr) + return nil + } + return validateErr +} - return err - }, +func loadValidateConfig(cmd *cobra.Command) (*gatecheck.Config, error) { + config := gatecheck.NewDefaultConfig() + filename := stringSetting(cmd, "config", "GATECHECK_CONFIG_FILENAME") + if filename == "" { + slog.Warn("no config file; using the default config, which has every rule disabled") + return config, nil + } + err := gatecheck.NewConfigDecoder(filename).Decode(config) + if errors.Is(err, gatecheck.ErrUnknownConfigKeys) { + slog.Warn("config has keys gatecheck does not read; check them for typos", "filename", filename, "keys", strings.TrimPrefix(err.Error(), gatecheck.ErrUnknownConfigKeys.Error()+": ")) + return config, nil + } + if err != nil { + return nil, err + } + return config, nil } -func newValidateCommand() *cobra.Command { +type validateResult struct { + Report string `json:"report"` + Passed bool `json:"passed"` + Failures []string `json:"failures"` + Error string `json:"error,omitempty"` +} - RuntimeConfig.ConfigFilename.SetupCobra(validateCmd) - RuntimeConfig.EPSSFilename.SetupCobra(validateCmd) - RuntimeConfig.EPSSCacheDir.SetupCobra(validateCmd) - RuntimeConfig.EPSSNoCache.SetupCobra(validateCmd) - RuntimeConfig.KEVFilename.SetupCobra(validateCmd) - RuntimeConfig.Audit.SetupCobra(validateCmd) - validateCmd.Flags().StringSlice("vex", nil, "OpenVEX or CycloneDX VEX file; findings it marks not_affected, fixed, false_positive, or resolved are skipped (repeatable, env GATECHECK_VEX)") +func newValidateResult(name string, err error) validateResult { + result := validateResult{Report: name, Passed: err == nil, Failures: []string{}} + if err == nil { + return result + } + if !errors.Is(err, gatecheck.ErrValidationFailure) { + result.Error = err.Error() + return result + } + prefix := gatecheck.ErrValidationFailure.Error() + ": " + var walk func(error) + walk = func(e error) { + if joined, ok := e.(interface{ Unwrap() []error }); ok { + for _, child := range joined.Unwrap() { + walk(child) + } + return + } + if errors.Is(e, gatecheck.ErrValidationFailure) { + result.Failures = append(result.Failures, strings.TrimPrefix(e.Error(), prefix)) + return + } + result.Error = e.Error() + } + walk(err) + return result +} - return validateCmd +func writeValidateResult(w io.Writer, outputFormat, name string, err error) error { + result := newValidateResult(name, err) + if outputFormat == "json" { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(result) + } + table := format.NewTable(w) + if outputFormat == "markdown" { + format.SetMarkdown(table) + } + table.Header([]string{"Report", "Result", "Detail"}) + switch { + case result.Passed: + _ = table.Append([]string{name, "pass", "-"}) + case result.Error != "" && len(result.Failures) == 0: + _ = table.Append([]string{name, "error", result.Error}) + default: + for _, failure := range result.Failures { + _ = table.Append([]string{name, "fail", failure}) + } + if result.Error != "" { + _ = table.Append([]string{name, "error", result.Error}) + } + } + return table.Render() } diff --git a/docs/agent-guide.md b/docs/agent-guide.md index dd10da1..061f22a 100644 --- a/docs/agent-guide.md +++ b/docs/agent-guide.md @@ -1,6 +1,6 @@ # gatecheck -Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner, plus gosec, hadolint, and zizmor on request, and the matching subcommands run them. Coverage reports still come from the language toolchain. +Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner, plus gosec, hadolint, zizmor, OpenSSF Scorecard, trufflehog, and cosign on request, and the matching subcommands run them. Coverage reports still come from the language toolchain. `gatecheck` with no arguments prints nothing and exits 0. `gatecheck --help` lists commands. `gatecheck agent-guide` (alias `llm`) prints this file. @@ -14,12 +14,12 @@ Logs go to stderr. Tables, `config init`, downloads, and this guide go to stdout - `validate` compares one report, or one bundle, to a config. - `bundle create` (alias `init`), `bundle add`, and `bundle remove` (alias `rm`) manage a bundle. - `download epss` and `download kev` write upstream data to stdout. -- `bootstrap` installs Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner. `--tools gosec,hadolint,zizmor` installs those too. `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, and `zizmor` run those binaries. +- `bootstrap` installs Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner. `--tools gosec,hadolint,zizmor,scorecard,trufflehog,cosign` installs those too. `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, and `cosign` run those binaries. - `update` (aliases `self-update`, `upgrade`) replaces this binary with a GitHub release after checking `checksums.txt`. `update --check` only prints the current and latest versions. - `completion bash|fish|powershell|zsh` prints a shell completion script. - `help [command]` prints the same text as ` --help`. -Global flags, on every command: `-v` / `--verbose` (debug logs) and `--silent` (errors only). They cannot be set together. `-h` / `--help` is also global. The scanner commands pass every argument through, including `--help`, so `gatecheck grype --help` is Grype's help. +Global flags, on every command: `-v` / `--verbose` (debug logs), `--silent` (warnings and errors), `--log-level debug|info|warn|error` (default `info`), and `--log-format text|json` (default `text`). Only one of `--verbose`, `--silent`, and `--log-level` can be set. `-h` / `--help` is also global. The scanner commands pass every argument through, including `--help`, so `gatecheck grype --help` is Grype's help. ## Scanners @@ -31,6 +31,8 @@ osv-scanner comes from the `google/osv-scanner` GitHub release as a raw binary, gosec, hadolint, and zizmor are installed only when named in `--tools`. gosec comes from `securego/gosec` as `gosec___.tar.gz` (tar.gz on Windows too), checked against `gosec__checksums.txt`. hadolint comes from `hadolint/hadolint` as a raw binary such as `hadolint-linux-x86_64` or `hadolint-macos-arm64`, checked against `checksums.sha256`. zizmor comes from `zizmorcore/zizmor` as `zizmor--.tar.gz` (`.zip` on Windows). zizmor publishes no checksum file, so its digests are pinned in `internal/bootstrap/versions.go`. hadolint and zizmor have no Windows arm64 build. All three write SARIF, which `list` and `validate` read with the `sarif` config section. +scorecard (`ossf/scorecard`, `scorecard___.tar.gz`, checksums `scorecard_checksums.txt`) and trufflehog (`trufflesecurity/trufflehog`, `trufflehog___.tar.gz`, checksums `trufflehog__checksums.txt`) are also installed only when named, for amd64 and arm64 on macOS, Linux, and Windows. Their JSON is its own report type. cosign (`sigstore/cosign`) is a raw binary checked against `cosign_checksums.txt`, for macOS amd64 and arm64, Linux amd64, arm64, armv7, ppc64le, s390x, and riscv64, and Windows amd64. `gatecheck cosign verify` and `verify-blob` pass cosign's exit status through, so a bad signature fails the CI step. + The directory is `$GATECHECK_HOME/bin` when `GATECHECK_HOME` is set. Otherwise it is `$XDG_DATA_HOME/gatecheck/bin`, or `~/.local/share/gatecheck/bin` on Linux, `~/Library/Application Support/gatecheck/bin` on macOS, and `%LOCALAPPDATA%\gatecheck\bin` on Windows. A tool already at the pinned version is left in place. `--force` downloads it again. `--tools grype,syft` limits the set. `osemgrep` in `--tools` means semgrep. `--list` prints pinned versions, installed versions, and paths, and does not download. @@ -46,13 +48,15 @@ gatecheck bootstrap --tools gosec,hadolint,zizmor gatecheck gosec -fmt sarif -out gosec.sarif ./... gatecheck hadolint -f sarif Dockerfile > hadolint.sarif gatecheck zizmor --format sarif .github/workflows > zizmor.sarif +gatecheck trufflehog git file://. --json > trufflehog-report.json +GITHUB_AUTH_TOKEN=... gatecheck scorecard --repo github.com/example/app --format json > scorecard-report.json ``` `gatecheck semgrep scan` and `gatecheck osemgrep scan` insert `--experimental` after `scan` when that flag is absent. Other osemgrep commands need `--experimental` from the caller. `gatecheck semgrep --experimental --version` prints the osemgrep version. When the program that runs is the Python `semgrep` on `PATH`, the flag is left alone. `gatecheck osemgrep` looks for the managed binary, then `osemgrep` on `PATH`. -The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck trivy`, `gatecheck osv-scanner`, `gatecheck gosec`, `gatecheck hadolint`, and `gatecheck zizmor` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 255. A scanner's own exit status is returned as-is. +The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck trivy`, `gatecheck osv-scanner`, `gatecheck gosec`, `gatecheck hadolint`, `gatecheck zizmor`, `gatecheck scorecard`, `gatecheck trufflehog`, and `gatecheck cosign` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 2. A scanner's own exit status is returned as-is. -A platform with no release asset for a requested GitHub-hosted tool fails that tool and the command exits 255. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, and zizmor `1.32.0`, in `internal/bootstrap/versions.go`. +A platform with no release asset for a requested GitHub-hosted tool fails that tool and the command exits 2. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, zizmor `1.32.0`, scorecard `5.6.0`, trufflehog `3.99.8`, and cosign `3.1.4`, in `internal/bootstrap/versions.go`. Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). That BOM is listed and validated as CycloneDX. Native Syft JSON is recognized from `descriptor.name` or the schema URL. `list` prints its packages and licenses, and `validate` runs the license rules on it. CycloneDX is a report format, and coverage tools are left to the language toolchain. @@ -71,9 +75,11 @@ Content rules, in this order: 7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. 8. Trivy JSON. `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses are ignored. A CycloneDX BOM from Trivy is CycloneDX. 9. osv-scanner JSON. `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. `{"results":[]}` does not identify itself. -10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. -11. Clover XML. An XML document with `clover=` or a ` gatecheck.yaml` and set real thr ## Common mistakes - The contents do not identify a type and the filename has no lowercase type word (`scan.json`, `Grype.json`). A real bundle named `project.gcb` is still recognized from its contents. An empty document such as `{"matches":[]}` still needs a name such as `grype-report.json`, or `--input-type grype` on standard input. -- Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent, not `80` or `80.0`. +- Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent. A value over 1, such as `80`, is read as a percent with a warning. - `validate` takes the report as its positional argument. The config is `--config` / `-f`. `gatecheck validate gatecheck.yaml grype-report.json` tries to open the config as a report. `config convert` uses `--file` / `-f`, which is a different flag. -- `list` on a pipe needs `--input-type`. Clear contents still win over that flag. `validate` needs a filesystem path. -- `list` ignores every filename after the first. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. +- `list` and `validate` on a pipe need `--input-type`. Clear contents still win over that flag. +- `list` takes one filename, and a second one exits 2. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. - Native Syft JSON gets only the license rules. Use CycloneDX output when you also want vulnerabilities scored. That BOM is CycloneDX even when the filename contains `syft`. - `gatecheck semgrep --version` reaches osemgrep without `--experimental`, and osemgrep then tries to run `pysemgrep`. Pass the flag: `gatecheck semgrep --experimental --version`. -- `--audit` still exits 255 when the report, the config, or a named EPSS or KEV file cannot be opened. +- `--audit` only hides threshold failures. It still exits 2 when the report, the config, or a named EPSS, KEV, or VEX file cannot be opened, and 3 when EPSS or KEV data cannot be loaded. diff --git a/docs/bootstrap.md b/docs/bootstrap.md index 6178807..eeca2ef 100644 --- a/docs/bootstrap.md +++ b/docs/bootstrap.md @@ -6,12 +6,13 @@ gatecheck bootstrap gatecheck bootstrap --tools grype,syft gatecheck bootstrap --tools gosec,hadolint,zizmor +gatecheck bootstrap --tools scorecard,trufflehog,cosign gatecheck bootstrap --list ``` -Pinned versions are compiled into gatecheck. The current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, and zizmor `1.32.0`. They live in `internal/bootstrap/versions.go`. +Pinned versions are compiled into gatecheck. The current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, zizmor `1.32.0`, OpenSSF Scorecard `5.6.0`, trufflehog `3.99.8`, and cosign `3.1.4`. They live in `internal/bootstrap/versions.go`. -gosec, hadolint, and zizmor are not in the default install. Each one checks one kind of source (Go code, Dockerfiles, and GitHub Actions workflows), so you name them in `--tools` when you want them. +gosec, hadolint, zizmor, scorecard, trufflehog, and cosign are not in the default install. gosec, hadolint, and zizmor each check one kind of source (Go code, Dockerfiles, and GitHub Actions workflows). scorecard rates a repository's supply-chain practices, trufflehog is a second secrets scanner, and cosign verifies signatures. Name them in `--tools` when you want them. ## Directory @@ -38,17 +39,20 @@ osv-scanner 2.7.0 2.7.0 /home/ada/.local/share/gatecheck/bin/osv-scan gosec 2.30.0 - - hadolint 2.15.1 2.15.1 /home/ada/.local/share/gatecheck/bin/hadolint zizmor 1.32.0 - - +scorecard 5.6.0 - - +trufflehog 3.99.8 3.99.8 /home/ada/.local/share/gatecheck/bin/trufflehog +cosign 3.1.4 - - ``` ## Flags -- `--tools`: comma-separated names. The default is `grype,syft,gitleaks,semgrep,trivy,osv-scanner`. `gosec`, `hadolint`, and `zizmor` are also accepted. Names are lowercased. `osemgrep` means `semgrep`. An unknown name is an error: `unknown tool "not-a-scanner" (supported: grype, syft, gitleaks, semgrep, trivy, osv-scanner, gosec, hadolint, zizmor)`. +- `--tools`: comma-separated names. The default is `grype,syft,gitleaks,semgrep,trivy,osv-scanner`. `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, and `cosign` are also accepted. Names are lowercased. `osemgrep` means `semgrep`. An unknown name is an error: `unknown tool "not-a-scanner" (supported: grype, syft, gitleaks, semgrep, trivy, osv-scanner, gosec, hadolint, zizmor, scorecard, trufflehog, cosign)`. - `--force`: reinstall even when the pinned version is already present. - `--list`: print the table above. `bootstrap` takes no positional arguments. -## Grype, Syft, Gitleaks, Trivy, and gosec +## Grype, Syft, Gitleaks, Trivy, gosec, scorecard, and trufflehog Each tool is a GitHub release asset. gatecheck downloads `__checksums.txt`, looks up the asset's SHA-256, downloads the archive, and checks the digest before extracting the binary. The install is a write to a temporary file in the destination directory, then a rename. @@ -59,6 +63,8 @@ Each tool is a GitHub release asset. gatecheck downloads `__check | Gitleaks | `gitleaks/gitleaks` | `gitleaks___.tar.gz`, or `.zip` on Windows | | Trivy | `aquasecurity/trivy` | `trivy__-.tar.gz`, or `.zip` on Windows | | gosec | `securego/gosec` | `gosec___.tar.gz`, on Windows too | +| scorecard | `ossf/scorecard` | `scorecard___.tar.gz`, on Windows too. The checksum file is `scorecard_checksums.txt`. | +| trufflehog | `trufflesecurity/trufflehog` | `trufflehog___.tar.gz`, on Windows too | Gitleaks names `amd64` as `x64`, `386` as `x32`, and `arm` as `armv6` or `armv7`. @@ -71,8 +77,9 @@ Published assets: - Gitleaks: macOS x64 and arm64; Linux x64, x32, arm64, armv6, and armv7; Windows x64, x32, and arm64. - Trivy: macOS amd64 and arm64; Linux 386, amd64, armv7, arm64, ppc64le, and s390x; Windows amd64; FreeBSD amd64. - gosec: macOS amd64 and arm64; Linux amd64, arm64, ppc64le, and s390x; Windows amd64 and arm64. +- scorecard and trufflehog: amd64 and arm64 on macOS, Linux, and Windows. -A platform with no asset fails that tool. The message looks like `grype has no published release asset for windows/arm64`. Other requested tools still install. The command exits 255 when any requested tool failed this way, or when a download or checksum check failed. +A platform with no asset fails that tool. The message looks like `grype has no published release asset for windows/arm64`. Other requested tools still install. The command exits 2 when any requested tool failed this way, or when a download or checksum check failed. ## osv-scanner @@ -92,6 +99,20 @@ hadolint also publishes a raw binary. gatecheck downloads `checksums.sha256` fro Asset names: `hadolint-linux-x86_64`, `hadolint-linux-arm64`, `hadolint-macos-x86_64`, `hadolint-macos-arm64`, and `hadolint-windows-x86_64.exe`. Windows arm64 has no asset. +## cosign + +cosign publishes raw binaries. gatecheck downloads `cosign_checksums.txt` from the `sigstore/cosign` release, checks the asset, and installs it as `cosign`, or `cosign.exe` on Windows. Assets: `cosign-darwin-amd64`, `cosign-darwin-arm64`, `cosign-linux-amd64`, `cosign-linux-arm64`, `cosign-linux-arm` (armv7), `cosign-linux-ppc64le`, `cosign-linux-s390x`, `cosign-linux-riscv64`, and `cosign-windows-amd64.exe`. + +cosign is a gate on its own: `cosign verify` and `cosign verify-blob` exit non-zero when a signature does not check out, and `gatecheck cosign` passes that status through. Put it in the pipeline before `validate`: + +```shell +gatecheck cosign verify-blob --bundle app.tar.gz.sigstore.json \ + --certificate-identity-regexp 'https://github.com/example/app/' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com app.tar.gz +gatecheck cosign verify --certificate-identity-regexp 'https://github.com/example/app/' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com ghcr.io/example/app:1.2.3 +``` + ## zizmor zizmor publishes archives but no checksum file. The SHA-256 of each `zizmorcore/zizmor` asset for the pinned version is compiled into gatecheck next to the version in `internal/bootstrap/versions.go`. gatecheck checks the download against that digest before extracting `zizmor`, or `zizmor.exe` on Windows. Moving the zizmor pin means updating those digests too. @@ -134,15 +155,19 @@ gatecheck osv-scanner scan source --format json . > osv-scanner-report.json gatecheck gosec -fmt sarif -out gosec.sarif ./... gatecheck hadolint -f sarif Dockerfile > hadolint.sarif gatecheck zizmor --format sarif .github/workflows > zizmor.sarif +GITHUB_AUTH_TOKEN=... gatecheck scorecard --repo github.com/example/app --format json > scorecard-report.json +gatecheck trufflehog git file://. --json > trufflehog-report.json ``` +scorecard and trufflehog JSON are their own report types, gated with the `scorecard` and `trufflehog` config sections. See [Supported reports](./supported-reports.md). + gosec, hadolint, and zizmor reports are read as SARIF. Gate them with the `sarif` section of the config. See [Supported reports](./supported-reports.md). Arguments, standard input, standard output, standard error, and the scanner's exit status are passed through. `gatecheck grype --help` is Grype's help. gatecheck does not parse flags after these command names. -The managed binary is used when it is present. Otherwise gatecheck looks on `PATH`: `grype`, `syft`, `gitleaks`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, or `zizmor` for those commands; `semgrep` and then `osemgrep` for `gatecheck semgrep`; `osemgrep` only for `gatecheck osemgrep`. `--experimental` is inserted only when the program that will run is named `osemgrep`. A Python `semgrep` on `PATH` is left unchanged. +The managed binary is used when it is present. Otherwise gatecheck looks on `PATH`: `grype`, `syft`, `gitleaks`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, or `cosign` for those commands; `semgrep` and then `osemgrep` for `gatecheck semgrep`; `osemgrep` only for `gatecheck osemgrep`. `--experimental` is inserted only when the program that will run is named `osemgrep`. A Python `semgrep` on `PATH` is left unchanged. -When neither copy exists, the command prints one line and exits 255: +When neither copy exists, the command prints one line and exits 2: ``` gatecheck: grype is not installed; run `gatecheck bootstrap` diff --git a/docs/cli-reference.md b/docs/cli-reference.md index ae61ca4..8475a22 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -2,13 +2,25 @@ Global flags, accepted on every command: -- `-v`, `--verbose`: print debug logs. -- `--silent`: hide debug, info, and warning logs. Errors are still printed. +- `-v`, `--verbose`: print debug logs. The same as `--log-level debug`. `GATECHECK_VERBOSE=true` is the same setting. +- `--silent`: print only warnings and errors. The same as `--log-level warn`. `GATECHECK_SILENT=true` is the same setting. +- `--log-level`: `debug`, `info`, `warn`, or `error`. The default is `info`. `GATECHECK_LOG_LEVEL` is the same setting. +- `--log-format`: `text` or `json`. The default is `text`, colored only when standard error is a terminal and `NO_COLOR` is unset. `GATECHECK_LOG_FORMAT` is the same setting. - `-h`, `--help`: help for that command. -`--verbose` and `--silent` cannot be set together. +`--verbose`, `--silent`, and `--log-level` cannot be set together. A flag wins over any of the variables. Among the variables, `GATECHECK_VERBOSE` and `GATECHECK_SILENT` win over `GATECHECK_LOG_LEVEL`, and setting both of them to true is an error. -When a setting has both a flag and an environment variable, a non-empty flag wins. An empty flag falls through to the environment variable, then to the default. +Every setting is read in the same order: a flag on the command line, then its environment variable, then the default. An empty environment variable counts as unset. Boolean variables take `true`, `false`, `1`, `0`, `t`, or `f`. Any other value is a usage error, and the command exits 2. + +## Exit status + +- `0`: success. +- `1`: `validate` found a threshold failure. +- `2`: a usage error, a file that could not be read or written, or a report, config, or VEX file that could not be decoded. +- `3`: EPSS or KEV data could not be downloaded or read. +- The scanner commands (`grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, `cosign`) return the scanner's own status. + +`validate --audit` turns status 1 into 0. It does not change 2 or 3. ## gatecheck @@ -35,9 +47,9 @@ Prints the logo and the build information: CLI version, git commit, build date, gatecheck config init ``` -Prints a default config to standard output. Every rule is disabled and every coverage threshold is `0`. +Prints a default config to standard output. Every rule is disabled and every coverage threshold is `0`. JSON and YAML are indented by two spaces, and TOML tables are indented. -- `-o`, `--output`: `json`, `.json`, `toml`, `.toml`, `yaml`, `yml`, `.yaml`, or `.yml`. The default is `yaml`. Any other value is an error. The match is case sensitive. +- `-o`, `--output`: `json`, `toml`, `yaml`, or `yml`, with or without a leading dot, in any case. The default is `yaml`. Any other value is an error. ```shell gatecheck config init > gatecheck.yaml @@ -50,7 +62,7 @@ gatecheck config init --output json > gatecheck.json gatecheck config convert ``` -Reads a config file and prints it in another format. +Reads a config file and prints it in another format. Keys gatecheck does not read are named in a warning and left out of the output. - `-f`, `--file`: file to read. The default is `gatecheck.yaml`. The extension picks the decoder (`.yaml`, `.yml`, `.json`, `.toml`). - `-o`, `--output`: format to write. Same accepted values as `config init`. The default is `yaml`. @@ -67,38 +79,42 @@ gatecheck list [FILE] Aliases: `ls`, `print`. -Prints a table for one report. The first filename is the one that is opened. Further filenames are ignored. Use `list-all` for more than one file. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `--markdown` prints that same sentence. +Prints a table for one report. Use `list-all` for more than one file. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, osv-scanner, or Syft report with nothing in it prints one sentence and no table: `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, `No osv-scanner vulnerabilities`, or `No syft packages`. Markdown prints that same sentence. The report type comes from the file contents. See [Supported Reports](./supported-reports.md). With no filename, the command reads standard input and `--input-type` is required. -- `-i`, `--input-type`: type word used when reading standard input and the contents are ambiguous. It is matched with the same rules as a filename. `grype`, `cyclonedx`, `sarif`, `semgrep`, `gitleaks`, `trivy`, `osv`, `syft`, `bundle`, and the coverage words all work. Clear contents win over this flag. See [Supported Reports](./supported-reports.md). -- `--markdown`: print a markdown table. +- `-i`, `--input-type`: type word used when the contents are ambiguous, and the type of standard input. It is matched with the same rules as a filename. `grype`, `cyclonedx`, `sarif`, `semgrep`, `gitleaks`, `trivy`, `osv`, `syft`, `bundle`, and the coverage words `lcov`, `clover`, and `cobertura` all work. Clear contents win over this flag. See [Supported Reports](./supported-reports.md). +- `--format`: `table` (the default), `markdown`, or `json`. JSON is `{"columns": [...], "rows": [{column: value}], "footer": {...}}`. A report with nothing in it is `{"columns": [], "rows": [], "message": "No grype vulnerabilities"}`. A bundle is its manifest. +- `--markdown`: the same as `--format markdown`. It cannot be combined with another `--format`. - `--epss`: add EPSS score and percentile columns. This changes the table for Grype, CycloneDX, Trivy, and osv-scanner. Other report types still list, and the EPSS data is not shown. With no local file, gatecheck downloads the current EPSS CSV and caches it until the published date changes. Requests send a User-Agent with the gatecheck version and project URL. - `--epss-filename`: read that CSV instead of calling the API. `GATECHECK_EPSS_FILENAME` is the same setting. - `--epss-url`: EPSS API URL. An empty value uses `https://epss.cyentia.com`. `GATECHECK_EPSS_URL` is the same setting. - `--epss-cache-dir`: cache directory for that download. `GATECHECK_EPSS_CACHE_DIR` is the same setting. - `--epss-no-cache`: download on every run. `GATECHECK_EPSS_CACHE=off` does the same. +A failed EPSS download or an unreadable EPSS file exits 3. + ```shell gatecheck list grype-report.json -gatecheck list --markdown --epss grype-report.json +gatecheck list --format markdown --epss grype-report.json +gatecheck list --format json trivy-report.json grype bkimminich/juice-shop:latest -o json | gatecheck ls --input-type grype ``` ## gatecheck list-all ``` -gatecheck list-all [FILE...] +gatecheck list-all FILE... ``` -Prints each filename, then its table, or one sentence when that report has no findings. A file whose contents and name are not a supported `list-all` type is skipped with a warning. Coverage files are in that skipped set. A missing file is skipped. A list error is logged and the command continues. The command exits 0 after the last file. +Prints each filename, then its table, or one sentence when that report has nothing in it. A file whose contents and name are not a supported `list-all` type is skipped with a warning. Coverage files are in that skipped set. A file that cannot be read or listed is logged and the command goes on to the next one. When any file could not be read or listed, the command exits 2 after the last file. At least one file is required. -- `--markdown`: print markdown tables. -- `--epss`: download EPSS data and add it for Grype, CycloneDX, Trivy, and osv-scanner reports. The type comes from the file contents, then the filename. The download happens per file and is cached until the published date changes. There is no `--epss-filename` flag. `GATECHECK_EPSS_URL` overrides the API. `GATECHECK_EPSS_FILENAME` is not read by this command. Requests send a User-Agent with the gatecheck version and project URL. -- `--epss-cache-dir`: cache directory for that download. `GATECHECK_EPSS_CACHE_DIR` is the same setting. -- `--epss-no-cache`: download on every run. `GATECHECK_EPSS_CACHE=off` does the same. +- `--format`: `table`, `markdown`, or `json`. JSON is one array: `[{"file": "grype-report.json", "report": {...}}]`, where `report` is what `list --format json` prints. Skipped files are not in the array. +- `--markdown`: the same as `--format markdown`. +- `--epss`: add EPSS data for Grype, CycloneDX, Trivy, and osv-scanner reports. The type comes from the file contents, then the filename. EPSS data is loaded once, the first time a file needs it. +- `--epss-filename`, `--epss-url`, `--epss-cache-dir`, and `--epss-no-cache`: the same as on `list`, with the same environment variables. ## gatecheck validate @@ -106,21 +122,23 @@ Prints each filename, then its table, or one sentence when that report has no fi gatecheck validate [FILE] ``` -`FILE` is required. The config is a flag. +`FILE` is the report or bundle. With no `FILE`, `--input-type` is required, and the report is read from standard input. The config is a flag. -- `-f`, `--config`: config file. `GATECHECK_CONFIG_FILENAME` is the same setting. With neither, the built-in default config is used (every rule disabled, coverage thresholds `0`). +- `-f`, `--config`: config file. `GATECHECK_CONFIG_FILENAME` is the same setting. With neither, the built-in default config is used (every rule disabled, coverage thresholds `0`), and a warning says so. Keys in the config that gatecheck does not read are named in a warning, and the rest of the file is still used. +- `-i`, `--input-type`: report type word, the same as on `list`. It is used when the contents are ambiguous, and it is required when reading standard input. +- `-a`, `--audit`: print a threshold failure and exit 0. Any other error still fails: a missing report or config is 2, and an EPSS or KEV fetch failure is 3. `GATECHECK_AUDIT` is the same setting. +- `--format`: also print the result to standard output as `table`, `markdown`, or `json`. With no `--format`, standard output is empty. JSON is `{"report": "grype-report.json", "passed": false, "failures": ["Grype: Severity Limit Exceeded"]}`, plus `"error"` when the check stopped on something other than a threshold. +- `--vex`: OpenVEX or CycloneDX VEX file. Findings it marks not exploitable are skipped. Repeat the flag for more files. `GATECHECK_VEX` is a comma-separated list used when the flag is absent. Files in the config's `vex.files` are read too. A VEX file that cannot be read exits 2. See [VEX](./vex.md). - `--epss-filename`: EPSS CSV to use when an EPSS rule is enabled. `GATECHECK_EPSS_FILENAME` is the same setting. +- `--epss-url`: EPSS feed URL. `GATECHECK_EPSS_URL` is the same setting. - `--epss-cache-dir`: cache directory used when that CSV is downloaded. `GATECHECK_EPSS_CACHE_DIR` is the same setting. - `--epss-no-cache`: download on every run. `GATECHECK_EPSS_CACHE=off` does the same. -- `--kev-filename`: CISA KEV JSON to use when a KEV rule is enabled. -- `-a`, `--audit`: print the error from the check and exit 0. A missing report, a missing config, or a bad flag still exits 255. -- `--vex`: OpenVEX or CycloneDX VEX file. Findings it marks not exploitable are skipped. Repeat the flag for more files. `GATECHECK_VEX` is a comma-separated list used when the flag is absent. Files in the config's `vex.files` are read too. A VEX file that cannot be read exits 255. See [VEX](./vex.md). - -`validate` has no `--epss-url` or `--kev-url` flag. `GATECHECK_EPSS_URL` and `GATECHECK_KEV_URL` still override those APIs when the matching file flag is empty. The defaults are the same URLs `download` uses. +- `--kev-filename`: CISA KEV JSON to use when a KEV rule is enabled. `GATECHECK_KEV_FILENAME` is the same setting. +- `--kev-url`: KEV feed URL. `GATECHECK_KEV_URL` is the same setting. EPSS data is loaded only when an EPSS limit or EPSS risk acceptance rule is enabled. When `--epss-filename` is not set, gatecheck downloads the current EPSS file and caches it until the published date changes. Requests send a User-Agent with the gatecheck version and project URL. -KEV data is loaded only when `kevLimitEnabled` is true for Grype or CycloneDX. +KEV data is loaded only when `kevLimitEnabled` is true for Grype, CycloneDX, Trivy, or osv-scanner. ```shell gatecheck validate --config gatecheck.yaml grype-report.json @@ -137,7 +155,7 @@ gatecheck bundle create BUNDLE_FILE TARGET_FILE Alias: `init`. -Creates a bundle and stores the target file in it. The label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). +Creates a bundle and stores the target file in it. An existing `BUNDLE_FILE` is replaced. The label is the base name of the target file (`reports/grype-report.json` is stored as `grype-report.json`). - `-t`, `--tag`: tag stored on that file. Repeat the flag for more than one tag. `GATECHECK_BUNDLE_TAG` is a comma-separated list used when no tag flag is passed. @@ -161,7 +179,7 @@ gatecheck bundle remove BUNDLE_FILE LABEL Alias: `rm`. -Deletes the file with that label. The label is the base name used when the file was added, not a path that is opened. A label that is not in the bundle is logged and the command still exits 0. +Deletes the file with that label. The label is the base name used when the file was added, not a path that is opened. A label that is not in the bundle is an error: the bundle is left as it was and the command exits 2. ## gatecheck download epss @@ -177,6 +195,8 @@ EPSS scores are cached in the user cache directory, under `gatecheck/epss`. gate - `--epss-cache-dir`: cache directory. `GATECHECK_EPSS_CACHE_DIR` is the same setting. - `--epss-no-cache`: download on every run. `GATECHECK_EPSS_CACHE=off` does the same. +A failed download exits 3. + ```shell gatecheck download epss > epss.csv ``` @@ -191,6 +211,8 @@ Writes the CISA Known Exploited Vulnerabilities catalog as JSON to standard outp - `--kev-url`: API URL. An empty value uses `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json`. `GATECHECK_KEV_URL` is the same setting. +A failed download exits 3. + ```shell gatecheck download kev > kev.json ``` @@ -201,9 +223,9 @@ gatecheck download kev > kev.json gatecheck bootstrap ``` -Downloads Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner into the gatecheck data directory. gosec, hadolint, and zizmor are installed when named in `--tools`. Each GitHub asset is checked against the project's SHA-256 checksum file. zizmor publishes none, so its digests are compiled into gatecheck. The semgrep wheel is checked against the digest in the PyPI JSON. A tool already at the pinned version is skipped. See [Bootstrap](./bootstrap.md). +Downloads Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner into the gatecheck data directory. gosec, hadolint, zizmor, scorecard, trufflehog, and cosign are installed when named in `--tools`. Each GitHub asset is checked against the project's SHA-256 checksum file. zizmor publishes none, so its digests are compiled into gatecheck. The semgrep wheel is checked against the digest in the PyPI JSON. A tool already at the pinned version is skipped. See [Bootstrap](./bootstrap.md). -- `--tools`: comma-separated scanners. Default `grype,syft,gitleaks,semgrep,trivy,osv-scanner`. Also accepts `gosec`, `hadolint`, and `zizmor`. `osemgrep` selects semgrep. +- `--tools`: comma-separated scanners. Default `grype,syft,gitleaks,semgrep,trivy,osv-scanner`. Also accepts `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, and `cosign`. `osemgrep` selects semgrep. - `--force`: download again when the pinned version is already installed. - `--list`: print pinned version, installed version, and path for every supported tool, or for the `--tools` set. This does not download. @@ -214,7 +236,7 @@ gatecheck bootstrap gatecheck bootstrap --list ``` -## gatecheck grype, syft, gitleaks, semgrep, osemgrep, trivy, osv-scanner, gosec, hadolint, zizmor +## gatecheck grype, syft, gitleaks, semgrep, osemgrep, trivy, osv-scanner, gosec, hadolint, zizmor, scorecard, trufflehog, cosign ``` gatecheck grype [scanner arguments] @@ -227,6 +249,9 @@ gatecheck osv-scanner [scanner arguments] gatecheck gosec [scanner arguments] gatecheck hadolint [scanner arguments] gatecheck zizmor [scanner arguments] +gatecheck scorecard [scanner arguments] +gatecheck trufflehog [scanner arguments] +gatecheck cosign [scanner arguments] ``` Runs the managed binary. Arguments and streams are passed through, and the process exits with the scanner's status. Flags after the command name are not parsed by gatecheck, so `gatecheck grype --help` is Grype's help. @@ -235,7 +260,7 @@ The managed copy is preferred. If it is absent, the same tool name on `PATH` is `semgrep` and `osemgrep` run the osemgrep binary from the semgrep wheel. For `scan`, `--experimental` is added when you omitted it and the program that will run is named `osemgrep`. Other osemgrep commands need you to pass `--experimental`. `gatecheck semgrep --experimental --version` prints the osemgrep version. -If no binary is found, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 255. +If no binary is found, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 2. ## gatecheck agent-guide @@ -306,11 +331,19 @@ Prints help for a command. `gatecheck validate --help` does the same. | Variable | Commands | Same as | | --- | --- | --- | | `GATECHECK_CONFIG_FILENAME` | `validate` | `--config` | +| `GATECHECK_AUDIT` | `validate` | `--audit` | | `GATECHECK_VEX` | `validate` | `--vex`, comma-separated | -| `GATECHECK_EPSS_FILENAME` | `list`, `validate` | `--epss-filename` | -| `GATECHECK_EPSS_URL` | `download epss`, `list`, `list-all`, `validate` | `--epss-url` on `download epss` and `list` | +| `GATECHECK_EPSS_FILENAME` | `list`, `list-all`, `validate` | `--epss-filename` | +| `GATECHECK_EPSS_URL` | `download epss`, `list`, `list-all`, `validate` | `--epss-url` | | `GATECHECK_EPSS_CACHE_DIR` | `download epss`, `list`, `list-all`, `validate` | `--epss-cache-dir` | | `GATECHECK_EPSS_CACHE` | `download epss`, `list`, `list-all`, `validate` | `--epss-no-cache` when the value is `off` | -| `GATECHECK_KEV_URL` | `download kev`, `validate` | `--kev-url` on `download kev` | +| `GATECHECK_KEV_FILENAME` | `validate` | `--kev-filename` | +| `GATECHECK_KEV_URL` | `download kev`, `validate` | `--kev-url` | | `GATECHECK_BUNDLE_TAG` | `bundle create`, `bundle add` | `--tag`, as a comma-separated list | -| `GATECHECK_HOME` | `bootstrap`, `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor` | data directory; binaries are in `bin` under it | +| `GATECHECK_VERBOSE` | every command | `--verbose` | +| `GATECHECK_SILENT` | every command | `--silent` | +| `GATECHECK_LOG_LEVEL` | every command | `--log-level` | +| `GATECHECK_LOG_FORMAT` | every command | `--log-format` | +| `GATECHECK_HOME` | `bootstrap`, `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, `cosign` | data directory; binaries are in `bin` under it | + +A flag on the command line wins over its variable. An empty variable counts as unset. `GATECHECK_AUDIT`, `GATECHECK_VERBOSE`, and `GATECHECK_SILENT` take `true`, `false`, `1`, `0`, `t`, or `f`, and any other value exits 2. `GATECHECK_EPSS_CACHE` also disables the cache for `0`, `false`, `no`, `disabled`, `disable`, and `none`. diff --git a/docs/configuration.md b/docs/configuration.md index 54d2d52..4462c6c 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -14,7 +14,7 @@ gatecheck config convert --file gatecheck.yaml --output json `config convert` reads `--file` / `-f` (default `gatecheck.yaml`) and writes the other format to standard output. -A partial file is fine. Omitted fields keep the default. Keys the decoder does not know are ignored. +A partial file is fine. Omitted fields keep the default. Keys gatecheck does not read are named in a warning when `validate` or `config convert` loads the file, and the rest of the file is still used. `GATECHECK_CONFIG_FILENAME` is the same setting as `validate --config`. @@ -27,165 +27,172 @@ This is what `gatecheck config init` prints. ```yaml version: "1" metadata: - tags: [] + tags: [] grype: - severityLimit: - critical: - enabled: false - limit: 0 - high: - enabled: false - limit: 0 - medium: - enabled: false - limit: 0 - low: - enabled: false - limit: 0 - epssLimit: - enabled: false - score: 0 - kevLimitEnabled: false - cveLimit: - enabled: false - cves: [] - epssRiskAcceptance: - enabled: false - score: 0 - cveRiskAcceptance: - enabled: false - cves: [] + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + epssLimit: + enabled: false + score: 0 + kevLimitEnabled: false + cveLimit: + enabled: false + cves: [] + epssRiskAcceptance: + enabled: false + score: 0 + cveRiskAcceptance: + enabled: false + cves: [] cyclonedx: - severityLimit: - critical: - enabled: false - limit: 0 - high: - enabled: false - limit: 0 - medium: - enabled: false - limit: 0 - low: - enabled: false - limit: 0 - epssLimit: - enabled: false - score: 0 - kevLimitEnabled: false - cveLimit: - enabled: false - cves: [] - epssRiskAcceptance: - enabled: false - score: 0 - cveRiskAcceptance: - enabled: false - cves: [] + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + epssLimit: + enabled: false + score: 0 + kevLimitEnabled: false + cveLimit: + enabled: false + cves: [] + epssRiskAcceptance: + enabled: false + score: 0 + cveRiskAcceptance: + enabled: false + cves: [] trivy: - severityLimit: - critical: - enabled: false - limit: 0 - high: - enabled: false - limit: 0 - medium: - enabled: false - limit: 0 - low: - enabled: false - limit: 0 - epssLimit: - enabled: false - score: 0 - kevLimitEnabled: false - cveLimit: - enabled: false - cves: [] - epssRiskAcceptance: - enabled: false - score: 0 - cveRiskAcceptance: - enabled: false - cves: [] + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + epssLimit: + enabled: false + score: 0 + kevLimitEnabled: false + cveLimit: + enabled: false + cves: [] + epssRiskAcceptance: + enabled: false + score: 0 + cveRiskAcceptance: + enabled: false + cves: [] osv-scanner: - severityLimit: - critical: - enabled: false - limit: 0 - high: - enabled: false - limit: 0 - medium: - enabled: false - limit: 0 - low: - enabled: false - limit: 0 - epssLimit: - enabled: false - score: 0 - kevLimitEnabled: false - cveLimit: - enabled: false - cves: [] - epssRiskAcceptance: - enabled: false - score: 0 - cveRiskAcceptance: - enabled: false - cves: [] + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + epssLimit: + enabled: false + score: 0 + kevLimitEnabled: false + cveLimit: + enabled: false + cves: [] + epssRiskAcceptance: + enabled: false + score: 0 + cveRiskAcceptance: + enabled: false + cves: [] semgrep: - severityLimit: - error: - enabled: false - limit: 0 - warning: - enabled: false - limit: 0 - info: - enabled: false - limit: 0 - impactRiskAcceptance: - enabled: false - high: false - medium: false - low: false + severityLimit: + error: + enabled: false + limit: 0 + warning: + enabled: false + limit: 0 + info: + enabled: false + limit: 0 + impactRiskAcceptance: + enabled: false + high: false + medium: false + low: false gitleaks: - limitEnabled: false + limitEnabled: false sarif: - severityLimit: - critical: - enabled: false - limit: 0 - high: - enabled: false - limit: 0 - medium: - enabled: false - limit: 0 - low: - enabled: false - limit: 0 - ruleLimit: - enabled: false - rules: [] - ruleRiskAcceptance: - enabled: false - rules: [] + severityLimit: + critical: + enabled: false + limit: 0 + high: + enabled: false + limit: 0 + medium: + enabled: false + limit: 0 + low: + enabled: false + limit: 0 + ruleLimit: + enabled: false + rules: [] + ruleRiskAcceptance: + enabled: false + rules: [] coverage: - lineThreshold: 0 - functionThreshold: 0 - branchThreshold: 0 + lineThreshold: 0 + functionThreshold: 0 + branchThreshold: 0 vex: - files: [] + files: [] license: - enabled: false - deny: [] - allow: [] - denyUnknown: false - packageRiskAcceptance: [] + enabled: false + deny: [] + allow: [] + denyUnknown: false + packageRiskAcceptance: [] +scorecard: + enabled: false + minScore: 0 + checks: [] +trufflehog: + limitEnabled: false + verifiedOnly: false ``` ## version and metadata @@ -238,6 +245,35 @@ grype: `limitEnabled: true` fails validation when the report contains any finding. `false` skips the check. There is no count. +## trufflehog + +`trufflehog` gates `trufflehog --json` output, a second secrets scanner next to Gitleaks. + +```yaml +trufflehog: + limitEnabled: true + verifiedOnly: false +``` + +`limitEnabled: true` fails validation when the report has any finding. `verifiedOnly: true` counts only the secrets trufflehog verified as live, so unverified matches are logged and do not fail. Secret values in the report are never decoded, listed, or logged. + +## Scorecard + +`scorecard` gates an OpenSSF Scorecard report from `scorecard --format json`. + +```yaml +scorecard: + enabled: true + minScore: 6 + checks: + - name: Dangerous-Workflow + minScore: 10 + - name: Token-Permissions + minScore: 5 +``` + +Scores are 0 to 10. `minScore` fails the report when the aggregate score is lower. `0` lets any aggregate pass. Each `checks` entry fails the report when that check scores lower than its `minScore`. Check names are compared without regard to case. A check scored `-1` (inconclusive, such as `Signed-Releases` on a repo with no releases) is skipped with a warning, and so is a named check that is not in the report. A score equal to the minimum passes. + ## SARIF `sarif` is for any SARIF 2.1.0 report. Semgrep's own JSON still uses `semgrep`. `semgrep --sarif` uses `sarif`. @@ -306,7 +342,7 @@ license: - function coverage is covered functions divided by total functions - branch coverage is covered branches divided by total branches -The report fails when its ratio is less than the threshold. `0.8` is 80 percent. `80.0` fails a report whose real line coverage is 0.8, because 0.8 is less than 80. +The report fails when its ratio is less than the threshold. `0.8` is 80 percent. A threshold over 1 is read as a percent: `80` means `0.8`, and `validate` logs a warning. Write the ratio to avoid the warning. There is no `enabled` flag. The default `0` allows any coverage, because a ratio is never less than 0. diff --git a/docs/examples.md b/docs/examples.md index eb0f520..9d20778 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -34,7 +34,7 @@ gatecheck validate --config gatecheck.yaml gatecheck-bundle.tar.gz gatecheck validate --audit --config gatecheck.yaml grype-report.json ``` -A threshold failure is printed and the process exits 0. A missing report still exits 255. +A threshold failure is printed and the process exits 0. A missing report still exits 2, and an EPSS or KEV download failure still exits 3. ## Bootstrap a scanner diff --git a/docs/gatecheck-bundle.md b/docs/gatecheck-bundle.md index 97f20f5..5b1d6f2 100644 --- a/docs/gatecheck-bundle.md +++ b/docs/gatecheck-bundle.md @@ -27,7 +27,7 @@ The bundle file has to exist already. gatecheck bundle remove gatecheck-bundle.tar.gz semgrep-report.json ``` -`bundle rm` is the same command. The second argument is the label. A label that is not in the bundle is logged, and the command exits 0. +`bundle rm` is the same command. The second argument is the label. A label that is not in the bundle is an error. The bundle is left unchanged and the command exits 2. ## Inspect diff --git a/docs/installation.md b/docs/installation.md index 037c7b8..e5c14ab 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -64,7 +64,7 @@ gatecheck --help ## Install scanners -After gatecheck itself is installed, `gatecheck bootstrap` downloads pinned releases of Grype, Syft, Gitleaks, Trivy, and osv-scanner, and installs semgrep's experimental native CLI as `osemgrep`. `--tools gosec,hadolint,zizmor` adds those three. The files live under the gatecheck data directory (`GATECHECK_HOME`, or the platform default described in [Bootstrap](./bootstrap.md)). +After gatecheck itself is installed, `gatecheck bootstrap` downloads pinned releases of Grype, Syft, Gitleaks, Trivy, and osv-scanner, and installs semgrep's experimental native CLI as `osemgrep`. `--tools gosec,hadolint,zizmor,scorecard,trufflehog,cosign` adds those. The files live under the gatecheck data directory (`GATECHECK_HOME`, or the platform default described in [Bootstrap](./bootstrap.md)). ``` gatecheck bootstrap diff --git a/docs/library-usage.md b/docs/library-usage.md index f516491..66681bd 100644 --- a/docs/library-usage.md +++ b/docs/library-usage.md @@ -7,11 +7,12 @@ import "github.com/gatecheckdev/gatecheck/pkg/gatecheck" ``` - `NewDefaultConfig` returns the same defaults as `gatecheck config init`: rules disabled, coverage thresholds `0`. -- `NewConfigDecoder` reads `.yaml`, `.yml`, `.json`, or `.toml`. The extension selects the format. -- `NewConfigEncoder` writes a config. Pass `.yaml`, `.yml`, `.json`, or `.toml`, including the dot. +- `NewConfigDecoder` reads `.yaml`, `.yml`, `.json`, or `.toml`. The extension selects the format. When the file has keys gatecheck does not read, `Decode` still fills the config and returns an error wrapping `ErrUnknownConfigKeys`. Check it with `errors.Is` and treat it as a warning. `DecodeConfig` does the same for bytes you already have. +- `EncodeConfigTo` and `NewConfigEncoder` write a config as `json`, `yaml`, `yml`, or `toml`, with or without the dot. JSON and YAML are indented by two spaces. `ConfigFormat` checks a format name. - `Validate` takes the config, a reader, and the filename. The bytes select the report type. The filename is the fallback, with the same words as the CLI. -- `List` writes a table. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings writes one sentence instead, such as `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `WithDisplayFormat("markdown")` matches `--markdown` and writes that same sentence when the report is empty. `WithEPSS` matches `--epss` and takes an open CSV file or a URL. +- `List` writes a table. `WithDisplayFormat` takes `table`, `markdown`, or `json`. A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings writes one sentence instead, such as `No gitleaks findings`, `No trivy vulnerabilities`, or `No osv-scanner vulnerabilities`. `WithDisplayFormat("markdown")` matches `--markdown` and writes that same sentence when the report is empty. `WithEPSS` matches `--epss` and takes an open CSV file or a URL. - `CreateBundle`, `AppendToBundle`, and `RemoveFromBundle` are `bundle create`, `bundle add`, and `bundle remove`. -- `DownloadEPSS` and `DownloadKEV` write the API payloads. `WithEPSSURL`, `WithKEVURL`, `WithEPSSFile`, and `WithKEVFile` override the defaults. +- `DownloadEPSS` and `DownloadKEV` write the API payloads. `WithEPSSURL`, `WithKEVURL`, `WithEPSSFile`, and `WithKEVFile` override the defaults. `WithVEXFiles` adds VEX documents to `Validate`. +- Errors: `ErrValidationFailure` is a threshold failure. `ErrExternalData` means EPSS or KEV data could not be loaded. `ErrBundleLabelNotFound` comes from `RemoveFromBundle`. The CLI maps these to exit codes 1, 3, and 2 with `cmd.ExitCode`. Report types are described in [Supported Reports](./supported-reports.md). Config fields are described in [Configuration](./configuration.md). diff --git a/docs/list-reports.md b/docs/list-reports.md index 76166c5..6b5817d 100644 --- a/docs/list-reports.md +++ b/docs/list-reports.md @@ -22,7 +22,7 @@ gatecheck ls grype-scan-report.json The report type comes from the file contents. The filename, or `--input-type` on standard input, is used when the contents are ambiguous. `grype-scan-report.json` still works. See [Supported Reports](./supported-reports.md). -`--markdown` prints a markdown table instead of the plain text table. +`--format markdown` (or `--markdown`) prints a markdown table instead of the plain text table. `--format json` prints `{"columns": [...], "rows": [{column: value}]}`, plus a `footer` object for coverage. `--epss` adds the EPSS score and percentile for Grype, CycloneDX, Trivy, and osv-scanner. Other reports ignore that data. With no `--epss-filename`, gatecheck downloads the CSV from FIRST. `GATECHECK_EPSS_FILENAME` and `GATECHECK_EPSS_URL` are the same settings as those flags. @@ -41,11 +41,12 @@ Columns: - Trivy: severity, package, version, link. With `--epss`: CVE id, severity, EPSS score, EPSS percentile, package, version, link. Misconfigurations are omitted. - osv-scanner: severity, package, version, link. With `--epss`: id, severity, EPSS score, EPSS percentile, package, version, link. The id is the first CVE alias, or the OSV id when there is no CVE. Groups are listed once. - Coverage: lines, functions, and branches covered, with a percent in the footer. The percent is only for display. The config file uses a ratio. See [Configuration](./configuration.md). -- Bundle: label, SHA-256 digest, tags, size. +- Syft JSON: package, version, type, licenses. +- Bundle: label, SHA-256 digest, tags, size. With `--format json`, the bundle manifest. -A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, or osv-scanner report with no findings prints one sentence and no table. The lines are `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, and `No osv-scanner vulnerabilities`. `--markdown` prints that same line. +A Grype, CycloneDX, SARIF, Semgrep, Gitleaks, Trivy, osv-scanner, or Syft report with nothing in it prints one sentence and no table. The lines are `No grype vulnerabilities`, `No cyclonedx vulnerabilities`, `No sarif findings`, `No semgrep findings`, `No gitleaks findings`, `No trivy vulnerabilities`, `No osv-scanner vulnerabilities`, and `No syft packages`. Markdown prints that same line. JSON prints `{"columns": [], "rows": [], "message": "..."}`. -`list` opens the first filename only. `list-all` walks every filename, prints the name, then the table or that sentence, and skips files it does not support. Coverage files are in that skipped set. `--epss` on `list-all` adds EPSS columns for Grype, CycloneDX, Trivy, and osv-scanner, including a file whose name does not contain those words when the contents do. It always uses the API (`GATECHECK_EPSS_URL` to override it). +`list` takes one filename. A second one is a usage error. `list-all` walks every filename, prints the name, then the table or that sentence, and skips files it does not support. Coverage files are in that skipped set. `--epss` on `list-all` adds EPSS columns for Grype, CycloneDX, Trivy, and osv-scanner, including a file whose name does not contain those words when the contents do. The EPSS data is loaded once per run, from `--epss-filename` or the API. With `--format json`, `list-all` prints one array of `{"file": ..., "report": ...}`. When a file cannot be read or listed, the others are still listed and the command exits 2. ```shell gatecheck list-all grype-scan-report.json semgrep-report.json diff --git a/docs/supported-reports.md b/docs/supported-reports.md index ff90e39..a4d4035 100644 --- a/docs/supported-reports.md +++ b/docs/supported-reports.md @@ -13,9 +13,11 @@ Content rules, in this order: 7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. 8. Trivy JSON (`--format json`). `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses in the same file are ignored. 9. osv-scanner JSON (`osv-scanner scan source --format json`). `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. -10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. -11. Clover XML. An XML document with `clover=` or a `___.tar.gz layout that +// scorecard and trufflehog use on every platform, Windows included. +func goreleaserAsset(name, version, checksum string, p Platform) (githubAsset, error) { + switch p.GOOS { + case "darwin", "linux", "windows": + default: + return githubAsset{}, unsupported(name, p) + } + if p.GOARCH != "amd64" && p.GOARCH != "arm64" { + return githubAsset{}, unsupported(name, p) + } + return githubAsset{ + Asset: fmt.Sprintf("%s_%s_%s_%s.tar.gz", name, version, p.GOOS, p.GOARCH), + Checksum: checksum, + Binary: exeName(name, p.GOOS), + DestName: exeName(name, p.GOOS), + }, nil +} + +func scorecardAsset(version string, p Platform) (githubAsset, error) { + return goreleaserAsset("scorecard", version, "scorecard_checksums.txt", p) +} + +func trufflehogAsset(version string, p Platform) (githubAsset, error) { + return goreleaserAsset("trufflehog", version, fmt.Sprintf("trufflehog_%s_checksums.txt", version), p) +} + +func cosignAsset(_ string, p Platform) (githubAsset, error) { + ok := false + switch p.GOOS { + case "darwin": + ok = p.GOARCH == "amd64" || p.GOARCH == "arm64" + case "linux": + switch p.GOARCH { + case "amd64", "arm64", "ppc64le", "s390x", "riscv64": + ok = true + case "arm": + ok = p.GOARM == "" || p.GOARM == "7" + } + case "windows": + ok = p.GOARCH == "amd64" + } + if !ok { + return githubAsset{}, unsupported("cosign", p) + } + asset := fmt.Sprintf("cosign-%s-%s", p.GOOS, p.GOARCH) + if p.GOOS == "windows" { + asset += ".exe" + } + return githubAsset{ + Asset: asset, + Checksum: "cosign_checksums.txt", + Binary: asset, + DestName: exeName("cosign", p.GOOS), + Raw: true, + }, nil +} + func exeName(name, goos string) string { if goos == "windows" && !strings.HasSuffix(strings.ToLower(name), ".exe") { return name + ".exe" diff --git a/internal/bootstrap/assets_test.go b/internal/bootstrap/assets_test.go index ae50d25..d6754b5 100644 --- a/internal/bootstrap/assets_test.go +++ b/internal/bootstrap/assets_test.go @@ -301,3 +301,46 @@ func TestOptionalScannerAssets(t *testing.T) { t.Fatalf("zizmor checksum pins %d, platforms %d", len(zizmorChecksums), len(zizmorCases)) } } + +func TestSupplyChainToolAssets(t *testing.T) { + t.Parallel() + sc, err := scorecardAsset(ScorecardVersion, Platform{GOOS: "windows", GOARCH: "arm64"}) + if err != nil { + t.Fatal(err) + } + if sc.Asset != fmt.Sprintf("scorecard_%s_windows_arm64.tar.gz", ScorecardVersion) || sc.Checksum != "scorecard_checksums.txt" || sc.Binary != "scorecard.exe" { + t.Fatalf("scorecard %+v", sc) + } + th, err := trufflehogAsset(TrufflehogVersion, Platform{GOOS: "linux", GOARCH: "amd64"}) + if err != nil { + t.Fatal(err) + } + if th.Asset != fmt.Sprintf("trufflehog_%s_linux_amd64.tar.gz", TrufflehogVersion) || th.Checksum != fmt.Sprintf("trufflehog_%s_checksums.txt", TrufflehogVersion) { + t.Fatalf("trufflehog %+v", th) + } + if _, err := trufflehogAsset(TrufflehogVersion, Platform{GOOS: "linux", GOARCH: "386"}); err == nil { + t.Fatal("trufflehog linux/386 should fail") + } + cases := map[Platform]string{ + {GOOS: "linux", GOARCH: "amd64"}: "cosign-linux-amd64", + {GOOS: "linux", GOARCH: "arm"}: "cosign-linux-arm", + {GOOS: "linux", GOARCH: "s390x"}: "cosign-linux-s390x", + {GOOS: "darwin", GOARCH: "arm64"}: "cosign-darwin-arm64", + {GOOS: "windows", GOARCH: "amd64"}: "cosign-windows-amd64.exe", + } + for p, want := range cases { + got, err := cosignAsset(CosignVersion, p) + if err != nil { + t.Fatalf("cosign %s: %v", p, err) + } + if got.Asset != want || !got.Raw || got.Checksum != "cosign_checksums.txt" || got.DestName != exeName("cosign", p.GOOS) { + t.Fatalf("cosign %s %+v", p, got) + } + } + if _, err := cosignAsset(CosignVersion, Platform{GOOS: "windows", GOARCH: "arm64"}); err == nil { + t.Fatal("cosign windows/arm64 should fail") + } + if _, err := cosignAsset(CosignVersion, Platform{GOOS: "linux", GOARCH: "arm", GOARM: "6"}); err == nil { + t.Fatal("cosign armv6 should fail") + } +} diff --git a/internal/bootstrap/bootstrap.go b/internal/bootstrap/bootstrap.go index edcf90a..f88a484 100644 --- a/internal/bootstrap/bootstrap.go +++ b/internal/bootstrap/bootstrap.go @@ -134,7 +134,7 @@ var defaultTools = []string{"grype", "syft", "gitleaks", "semgrep", "trivy", "os // optionalTools are installed only when named in --tools. Each one scans a // single kind of source (Go code, Dockerfiles, GitHub workflows), so they are // left out of the default install. -var optionalTools = []string{"gosec", "hadolint", "zizmor"} +var optionalTools = []string{"gosec", "hadolint", "zizmor", "scorecard", "trufflehog", "cosign"} func supportedTools() []string { return append(append([]string(nil), defaultTools...), optionalTools...) @@ -226,6 +226,24 @@ func installTool(ctx context.Context, opts Options, tool string) error { return err } return installGitHub(ctx, opts, "zizmor", "zizmorcore/zizmor", ZizmorVersion, asset) + case "scorecard": + asset, err := scorecardAsset(ScorecardVersion, opts.platform()) + if err != nil { + return err + } + return installGitHub(ctx, opts, "scorecard", "ossf/scorecard", ScorecardVersion, asset) + case "trufflehog": + asset, err := trufflehogAsset(TrufflehogVersion, opts.platform()) + if err != nil { + return err + } + return installGitHub(ctx, opts, "trufflehog", "trufflesecurity/trufflehog", TrufflehogVersion, asset) + case "cosign": + asset, err := cosignAsset(CosignVersion, opts.platform()) + if err != nil { + return err + } + return installGitHub(ctx, opts, "cosign", "sigstore/cosign", CosignVersion, asset) default: return fmt.Errorf("unknown tool %q", tool) } @@ -496,6 +514,12 @@ func pinnedVersion(tool string) string { return HadolintVersion case "zizmor": return ZizmorVersion + case "scorecard": + return ScorecardVersion + case "trufflehog": + return TrufflehogVersion + case "cosign": + return CosignVersion default: return "-" } diff --git a/internal/bootstrap/bootstrap_test.go b/internal/bootstrap/bootstrap_test.go index c91cacf..ceea3d4 100644 --- a/internal/bootstrap/bootstrap_test.go +++ b/internal/bootstrap/bootstrap_test.go @@ -643,12 +643,12 @@ func TestListInstalled(t *testing.T) { if !strings.HasPrefix(text, "TOOL PINNED INSTALLED PATH\n") { t.Fatalf("header %q", text) } - for _, version := range []string{GrypeVersion, SyftVersion, GitleaksVersion, SemgrepVersion, TrivyVersion, OSVScannerVersion, GosecVersion, HadolintVersion, ZizmorVersion} { + for _, version := range []string{GrypeVersion, SyftVersion, GitleaksVersion, SemgrepVersion, TrivyVersion, OSVScannerVersion, GosecVersion, HadolintVersion, ZizmorVersion, ScorecardVersion, TrufflehogVersion, CosignVersion} { if !strings.Contains(text, version) { t.Fatalf("missing %s in %s", version, text) } } - if strings.Count(text, "\n") != 10 { + if strings.Count(text, "\n") != 13 { t.Fatalf("lines %q", text) } if !strings.Contains(text, "osv-scanner "+OSVScannerVersion) { diff --git a/internal/bootstrap/docs_test.go b/internal/bootstrap/docs_test.go index 024b27a..5463d08 100644 --- a/internal/bootstrap/docs_test.go +++ b/internal/bootstrap/docs_test.go @@ -21,6 +21,9 @@ func TestDocsMentionPins(t *testing.T) { GosecVersion, HadolintVersion, ZizmorVersion, + ScorecardVersion, + TrufflehogVersion, + CosignVersion, "GATECHECK_HOME", "osemgrep", "--experimental", diff --git a/internal/bootstrap/versions.go b/internal/bootstrap/versions.go index 357f787..211bfc1 100644 --- a/internal/bootstrap/versions.go +++ b/internal/bootstrap/versions.go @@ -23,6 +23,12 @@ const ( // ZizmorVersion is the zizmorcore/zizmor GitHub release. zizmor publishes // no checksum file, so its digests are pinned in zizmorChecksums. ZizmorVersion = "1.32.0" + // ScorecardVersion is the ossf/scorecard GitHub release. + ScorecardVersion = "5.6.0" + // TrufflehogVersion is the trufflesecurity/trufflehog GitHub release. + TrufflehogVersion = "3.99.8" + // CosignVersion is the sigstore/cosign GitHub release. + CosignVersion = "3.1.4" ) // zizmorChecksums are the SHA-256 digests of the ZizmorVersion release assets. diff --git a/pkg/artifacts/detect.go b/pkg/artifacts/detect.go index e9cd5f0..10eb22d 100644 --- a/pkg/artifacts/detect.go +++ b/pkg/artifacts/detect.go @@ -19,10 +19,13 @@ const ( KindTrivy ReportKind = "trivy" KindOSV ReportKind = "osv" KindSARIF ReportKind = "sarif" - KindBundle ReportKind = "bundle" - KindLCOV ReportKind = "lcov" - KindClover ReportKind = "clover" - KindCobertura ReportKind = "cobertura" + KindScorecard ReportKind = "scorecard" + // KindTrufflehog is newline-delimited trufflehog --json output. + KindTrufflehog ReportKind = "trufflehog" + KindBundle ReportKind = "bundle" + KindLCOV ReportKind = "lcov" + KindClover ReportKind = "clover" + KindCobertura ReportKind = "cobertura" // KindCoverage means the filename selected coverage, and the format is not known yet. KindCoverage ReportKind = "coverage" ) @@ -67,8 +70,16 @@ func detectJSON(content []byte) ReportKind { var head sniff if err := json.Unmarshal(content, &head); err != nil { + // More than one object: newline-delimited JSON, such as trufflehog. + first, _, _ := bytes.Cut(content, []byte("\n")) + if isTrufflehogLine(bytes.TrimSpace(first)) { + return KindTrufflehog + } return KindUnknown } + if isTrufflehogLine(content) { + return KindTrufflehog + } return classifyObject(head) } @@ -90,6 +101,10 @@ type sniff struct { Schema struct { URL string `json:"url"` } `json:"schema"` + Scorecard struct { + Version string `json:"version"` + } `json:"scorecard"` + Checks json.RawMessage `json:"checks"` } func classifyObject(s sniff) ReportKind { @@ -120,9 +135,16 @@ func classifyObject(s sniff) ReportKind { if isOSV(s) { return KindOSV } + if isScorecard(s) { + return KindScorecard + } return KindUnknown } +func isScorecard(s sniff) bool { + return strings.TrimSpace(s.Scorecard.Version) != "" && isJSONArray(s.Checks) +} + func isTrivy(s sniff) bool { if _, ok := s.SchemaVersion.(float64); !ok { return false diff --git a/pkg/artifacts/scorecard.go b/pkg/artifacts/scorecard.go new file mode 100644 index 0000000..8aebd71 --- /dev/null +++ b/pkg/artifacts/scorecard.go @@ -0,0 +1,28 @@ +package artifacts + +// ScorecardReport is the part of `scorecard --format json` that gatecheck reads. +type ScorecardReport struct { + Repo struct { + Name string `json:"name"` + Commit string `json:"commit"` + } `json:"repo"` + Scorecard struct { + Version string `json:"version"` + } `json:"scorecard"` + Score float64 `json:"score"` + Checks []ScorecardCheck `json:"checks"` +} + +// ScorecardCheck is one Scorecard check. Score is 0 to 10, or -1 when the +// check could not run or does not apply. +type ScorecardCheck struct { + Name string `json:"name"` + Score float64 `json:"score"` + Reason string `json:"reason"` + Documentation struct { + URL string `json:"url"` + } `json:"documentation"` +} + +// Inconclusive reports whether the check has no score. +func (c ScorecardCheck) Inconclusive() bool { return c.Score < 0 } diff --git a/pkg/artifacts/trufflehog.go b/pkg/artifacts/trufflehog.go new file mode 100644 index 0000000..5f81d8b --- /dev/null +++ b/pkg/artifacts/trufflehog.go @@ -0,0 +1,120 @@ +package artifacts + +import ( + "bufio" + "bytes" + "encoding/json" + "fmt" + "strings" +) + +// TrufflehogFinding is one line of `trufflehog --json` output. Raw secret +// values are not decoded, so they cannot end up in a table or a log. +type TrufflehogFinding struct { + SourceName string `json:"SourceName"` + DetectorName string `json:"DetectorName"` + DecoderName string `json:"DecoderName"` + Verified bool `json:"Verified"` + SourceMetadata json.RawMessage `json:"SourceMetadata"` +} + +// Location is file:line, or the best location the source metadata has. +func (f TrufflehogFinding) Location() string { + var meta struct { + Data map[string]struct { + File string `json:"file"` + Path string `json:"path"` + Line int `json:"line"` + Commit string `json:"commit"` + Link string `json:"link"` + Repository string `json:"repository"` + } `json:"Data"` + } + if err := json.Unmarshal(f.SourceMetadata, &meta); err != nil { + return "-" + } + for _, d := range meta.Data { + file := d.File + if file == "" { + file = d.Path + } + switch { + case file != "" && d.Line > 0: + return fmt.Sprintf("%s:%d", file, d.Line) + case file != "": + return file + case d.Link != "": + return d.Link + case d.Repository != "": + return d.Repository + } + } + return "-" +} + +// Commit is the git commit of the finding, or "-". +func (f TrufflehogFinding) Commit() string { + var meta struct { + Data map[string]struct { + Commit string `json:"commit"` + } `json:"Data"` + } + if err := json.Unmarshal(f.SourceMetadata, &meta); err != nil { + return "-" + } + for _, d := range meta.Data { + if d.Commit != "" { + if len(d.Commit) > 12 { + return d.Commit[:12] + } + return d.Commit + } + } + return "-" +} + +// DecodeTrufflehog reads newline-delimited trufflehog JSON. A JSON array of +// the same objects is also accepted. Lines that are not finding objects are +// skipped. +func DecodeTrufflehog(content []byte) ([]TrufflehogFinding, error) { + content = bytes.TrimSpace(content) + if len(content) == 0 { + return nil, nil + } + if content[0] == '[' { + var findings []TrufflehogFinding + if err := json.Unmarshal(content, &findings); err != nil { + return nil, err + } + return findings, nil + } + var findings []TrufflehogFinding + scanner := bufio.NewScanner(bytes.NewReader(content)) + scanner.Buffer(make([]byte, 0, 64*1024), 16<<20) + for scanner.Scan() { + line := bytes.TrimSpace(scanner.Bytes()) + if len(line) == 0 || line[0] != '{' { + continue + } + var f TrufflehogFinding + if err := json.Unmarshal(line, &f); err != nil { + return nil, fmt.Errorf("trufflehog line: %w", err) + } + if strings.TrimSpace(f.DetectorName) == "" { + continue + } + findings = append(findings, f) + } + return findings, scanner.Err() +} + +func isTrufflehogLine(line []byte) bool { + var probe struct { + DetectorName string `json:"DetectorName"` + SourceMetadata json.RawMessage `json:"SourceMetadata"` + } + if err := json.Unmarshal(line, &probe); err != nil { + return false + } + return probe.DetectorName != "" && len(probe.SourceMetadata) > 0 +} diff --git a/pkg/epss/epss.go b/pkg/epss/epss.go index 69e57d2..56925ea 100644 --- a/pkg/epss/epss.go +++ b/pkg/epss/epss.go @@ -179,7 +179,12 @@ func ParseEPSSDataCSV(r io.Reader, data *Data) error { slog.Warn("data model does not match supported model", "want", dataModel, "got", data.ModelVersion) } - scoreDate, err := time.Parse(modelDateLayout, strings.ReplaceAll(parts[1], "score_date:", "")) + rawDate := strings.ReplaceAll(parts[1], "score_date:", "") + scoreDate, err := time.Parse(modelDateLayout, rawDate) + if err != nil { + // Older FIRST files write the offset as +0000. + scoreDate, err = time.Parse("2006-01-02T15:04:05-0700", rawDate) + } if err != nil { return fmt.Errorf("failed to parse EPSS CSV, invalid date format in metadata header '%s'", scanner.Text()) } diff --git a/pkg/format/json.go b/pkg/format/json.go new file mode 100644 index 0000000..d6ab013 --- /dev/null +++ b/pkg/format/json.go @@ -0,0 +1,93 @@ +package format + +import ( + "encoding/json" + "fmt" + "io" + + "github.com/olekukonko/ll" + "github.com/olekukonko/tablewriter" + "github.com/olekukonko/tablewriter/tw" +) + +// TableJSON is the JSON form of a table. Each row is an object keyed by the +// column header. +type TableJSON struct { + Columns []string `json:"columns"` + Rows []map[string]string `json:"rows"` + Footer map[string]string `json:"footer,omitempty"` +} + +// NewJSONTable returns a table that writes TableJSON instead of drawing a grid. +// Cells are not wrapped or padded. +func NewJSONTable(w io.Writer) *tablewriter.Table { + return tablewriter.NewTable(w, + tablewriter.WithRenderer(&jsonRenderer{}), + tablewriter.WithRowAutoWrap(tw.WrapNone), + tablewriter.WithHeaderAutoFormat(tw.Off), + tablewriter.WithFooterAutoFormat(tw.Off), + tablewriter.WithPadding(tw.Padding{Overwrite: true}), + ) +} + +// WriteEmptyJSON writes a TableJSON with no rows and a message. +func WriteEmptyJSON(w io.Writer, message string) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(struct { + Columns []string `json:"columns"` + Rows []map[string]string `json:"rows"` + Message string `json:"message"` + }{Columns: []string{}, Rows: []map[string]string{}, Message: message}) +} + +type jsonRenderer struct { + w io.Writer + out TableJSON +} + +func (r *jsonRenderer) Start(w io.Writer) error { + r.w = w + r.out = TableJSON{Columns: []string{}, Rows: []map[string]string{}} + return nil +} + +func (r *jsonRenderer) Header(headers [][]string, _ tw.Formatting) { + if len(headers) > 0 { + r.out.Columns = append([]string(nil), headers[0]...) + } +} + +func (r *jsonRenderer) Row(row []string, _ tw.Formatting) { + r.out.Rows = append(r.out.Rows, r.keyed(row)) +} + +func (r *jsonRenderer) Footer(footers [][]string, _ tw.Formatting) { + if len(footers) > 0 { + r.out.Footer = r.keyed(footers[0]) + } +} + +func (r *jsonRenderer) keyed(cells []string) map[string]string { + m := make(map[string]string, len(cells)) + for i, cell := range cells { + key := "" + if i < len(r.out.Columns) { + key = r.out.Columns[i] + } + if key == "" { + key = fmt.Sprintf("column%d", i+1) + } + m[key] = cell + } + return m +} + +func (r *jsonRenderer) Line(tw.Formatting) {} +func (r *jsonRenderer) Config() tw.Rendition { return tw.Rendition{} } +func (r *jsonRenderer) Logger(*ll.Logger) {} +func (r *jsonRenderer) Close() error { + enc := json.NewEncoder(r.w) + enc.SetIndent("", " ") + return enc.Encode(r.out) +} diff --git a/pkg/gatecheck/bundle.go b/pkg/gatecheck/bundle.go index 35db33e..a44dbea 100644 --- a/pkg/gatecheck/bundle.go +++ b/pkg/gatecheck/bundle.go @@ -1,12 +1,17 @@ package gatecheck import ( + "errors" + "fmt" "io" "log/slog" "github.com/gatecheckdev/gatecheck/pkg/archive" ) +// ErrBundleLabelNotFound means RemoveFromBundle was asked for a label the bundle does not have. +var ErrBundleLabelNotFound = errors.New("bundle has no file with label") + // CreateBundle create a new bundle with a file // // If the bundle already exist, use CreateBundle. @@ -72,6 +77,9 @@ func RemoveFromBundle(bundleRWS io.ReadWriteSeeker, label string) error { if err := archive.UntarGzipBundle(bundleRWS, bundle); err != nil { return err } + if _, ok := bundle.Manifest().Files[label]; !ok { + return fmt.Errorf("%w: %q", ErrBundleLabelNotFound, label) + } bundle.Remove(label) // Seek errors are unlikely so just capture for edge cases _, seekErr := bundleRWS.Seek(0, io.SeekStart) diff --git a/pkg/gatecheck/config.go b/pkg/gatecheck/config.go index 2564b89..f82a987 100644 --- a/pkg/gatecheck/config.go +++ b/pkg/gatecheck/config.go @@ -9,11 +9,12 @@ import ( "log/slog" "os" "path" + "sort" + "strings" "github.com/gatecheckdev/gatecheck/pkg/format" "github.com/gatecheckdev/gatecheck/pkg/vex" "github.com/pelletier/go-toml/v2" - "github.com/spf13/viper" "gopkg.in/yaml.v3" ) @@ -35,6 +36,8 @@ type Config struct { Coverage configCoverageReport `json:"coverage" toml:"coverage" yaml:"coverage"` VEX configVEX `json:"vex" toml:"vex" yaml:"vex"` License configLicense `json:"license" toml:"license" yaml:"license"` + Scorecard configScorecard `json:"scorecard" toml:"scorecard" yaml:"scorecard"` + Trufflehog configTrufflehog `json:"trufflehog" toml:"trufflehog" yaml:"trufflehog"` vexDoc *vex.Document } @@ -42,17 +45,21 @@ type Config struct { func (c *Config) String() string { buf := new(bytes.Buffer) _ = json.NewEncoder(buf).Encode(c) - - v := viper.New() - v.SetConfigType("json") - _ = v.ReadConfig(buf) + var tree map[string]any + _ = json.Unmarshal(buf.Bytes(), &tree) + rows := map[string]string{} + flattenConfig("", tree, rows) + keys := make([]string, 0, len(rows)) + for key := range rows { + keys = append(keys, key) + } + sort.Strings(keys) contentBuf := new(bytes.Buffer) table := format.NewTable(contentBuf) table.Header([]string{"config key", "value"}) - - for _, key := range v.AllKeys() { - if err := table.Append([]string{key, fmt.Sprintf("%v", v.Get(key))}); err != nil { + for _, key := range keys { + if err := table.Append([]string{key, rows[key]}); err != nil { slog.Error("append config row", "error", err) return contentBuf.String() } @@ -63,6 +70,21 @@ func (c *Config) String() string { return contentBuf.String() } +func flattenConfig(prefix string, value any, out map[string]string) { + switch v := value.(type) { + case map[string]any: + for key, child := range v { + name := strings.ToLower(key) + if prefix != "" { + name = prefix + "." + name + } + flattenConfig(name, child, out) + } + default: + out[prefix] = fmt.Sprintf("%v", v) + } +} + type configCoverageReport struct { LineThreshold float32 `json:"lineThreshold" toml:"lineThreshold" yaml:"lineThreshold"` FunctionThreshold float32 `json:"functionThreshold" toml:"functionThreshold" yaml:"functionThreshold"` @@ -156,7 +178,7 @@ type configCVE struct { ID string `json:"id" toml:"id" yaml:"id"` Metadata struct { Tags []string `json:"tags" toml:"tags" yaml:"tags"` - } + } `json:"metadata" toml:"metadata" yaml:"metadata"` } type configLimit struct { @@ -250,105 +272,161 @@ func NewDefaultConfig() *Config { }, VEX: configVEX{Files: []string{}}, License: defaultConfigLicense(), + Scorecard: configScorecard{ + Checks: []configScorecardCheck{}, + }, } } +// WriteDefaultConfig writes the default config, tagged as generated. func WriteDefaultConfig(w io.Writer, format string) error { config := NewDefaultConfig() config.Metadata.Tags = append(config.Metadata.Tags, "auto generated from CLI") return EncodeConfigTo(w, config, format) } -func EncodeConfigTo(w io.Writer, config *Config, format string) error { - var encoder interface { - Encode(any) error +// ConfigFormat returns the canonical config format for a name or extension: +// "json", "yaml", or "toml". "yml" and a leading dot are accepted. +func ConfigFormat(name string) (string, error) { + switch strings.ToLower(strings.TrimPrefix(strings.TrimSpace(name), ".")) { + case "json": + return "json", nil + case "yaml", "yml": + return "yaml", nil + case "toml": + return "toml", nil + default: + return "", fmt.Errorf("unsupported config format %q, must be json, toml, yaml, or yml", name) } +} - slog.Debug("encode config file", "format", format) - switch format { +// EncodeConfigTo writes config in format (json, yaml, yml, or toml, with or +// without a leading dot). JSON and YAML are indented by two spaces. +func EncodeConfigTo(w io.Writer, config *Config, format string) error { + canonical, err := ConfigFormat(format) + if err != nil { + return err + } + slog.Debug("encode config file", "format", canonical) + switch canonical { case "json": enc := json.NewEncoder(w) enc.SetIndent("", " ") - encoder = enc - case "yaml", "yml": + return enc.Encode(config) + case "yaml": enc := yaml.NewEncoder(w) enc.SetIndent(2) - encoder = enc - case "toml": - encoder = toml.NewEncoder(w) + if err := enc.Encode(config); err != nil { + return err + } + return enc.Close() default: - return fmt.Errorf("unsupported format '%s'", format) + enc := toml.NewEncoder(w) + enc.SetIndentTables(true) + return enc.Encode(config) } - - return encoder.Encode(config) } +// ConfigEncoder writes a config in the format named by a file extension. type ConfigEncoder struct { writer io.Writer ext string } +// NewConfigEncoder takes ".json", ".yaml", ".yml", or ".toml". The dot is optional. func NewConfigEncoder(w io.Writer, ext string) *ConfigEncoder { - return &ConfigEncoder{ - writer: w, - ext: ext, - } + return &ConfigEncoder{writer: w, ext: ext} } func (e *ConfigEncoder) Encode(config *Config) error { - var encoder interface { - Encode(any) error - } - - switch e.ext { - case ".json": - encoder = json.NewEncoder(e.writer) - case ".toml": - encoder = toml.NewEncoder(e.writer) - case ".yaml", ".yml": - encoder = yaml.NewEncoder(e.writer) - default: - return errors.New("invalid file extension, only json, toml, yaml or yml supported") - } - - return encoder.Encode(config) - + return EncodeConfigTo(e.writer, config, e.ext) } +// ConfigDecoder reads a config file. The extension picks the format. type ConfigDecoder struct { filename string } func NewConfigDecoder(filename string) *ConfigDecoder { - return &ConfigDecoder{ - filename: filename, - } + return &ConfigDecoder{filename: filename} } -func (d *ConfigDecoder) Decode(config *Config) error { - ext := path.Ext(d.filename) +// ErrUnknownConfigKeys means a config file has keys gatecheck does not read. +var ErrUnknownConfigKeys = errors.New("config has unknown keys") - slog.Debug("decode", "filename", d.filename, "extension", ext) - f, err := os.Open(d.filename) +// Decode reads the file into config. Keys gatecheck does not know are +// skipped. When there are any, Decode fills config and then returns an error +// wrapping ErrUnknownConfigKeys, so the caller can warn and go on. +func (d *ConfigDecoder) Decode(config *Config) error { + slog.Debug("decode config", "filename", d.filename) + canonical, err := ConfigFormat(path.Ext(d.filename)) + if err != nil { + return fmt.Errorf("%s: %w", d.filename, err) + } + content, err := os.ReadFile(d.filename) if err != nil { return err } - defer func() { _ = f.Close() }() + return DecodeConfig(content, canonical, config) +} - var decoder interface { - Decode(any) error +// DecodeConfig reads content in format into config. See ConfigDecoder.Decode +// for how unknown keys are reported. +func DecodeConfig(content []byte, format string, config *Config) error { + canonical, err := ConfigFormat(format) + if err != nil { + return err } - - switch ext { - case ".json": - decoder = json.NewDecoder(f) - case ".toml": - decoder = toml.NewDecoder(f) - case ".yaml", ".yml": - decoder = yaml.NewDecoder(f) + var loose error + switch canonical { + case "json": + loose = json.Unmarshal(content, config) + case "yaml": + loose = yaml.Unmarshal(content, config) default: - return errors.New("invalid file extension, only json, toml, yaml or yml supported") + loose = toml.Unmarshal(content, config) + } + if loose != nil { + return loose } + if unknown := unknownConfigKeys(content, canonical); unknown != "" { + return fmt.Errorf("%w: %s", ErrUnknownConfigKeys, unknown) + } + return nil +} - return decoder.Decode(config) +// unknownConfigKeys decodes content strictly into a scratch config and returns +// the decoder's description of the unknown keys, or "". +func unknownConfigKeys(content []byte, format string) string { + scratch := &Config{} + var err error + switch format { + case "json": + dec := json.NewDecoder(bytes.NewReader(content)) + dec.DisallowUnknownFields() + err = dec.Decode(scratch) + case "yaml": + dec := yaml.NewDecoder(bytes.NewReader(content)) + dec.KnownFields(true) + err = dec.Decode(scratch) + if errors.Is(err, io.EOF) { + err = nil + } + default: + dec := toml.NewDecoder(bytes.NewReader(content)) + dec.DisallowUnknownFields() + err = dec.Decode(scratch) + var strict *toml.StrictMissingError + if errors.As(err, &strict) { + keys := make([]string, 0, len(strict.Errors)) + for _, e := range strict.Errors { + keys = append(keys, strings.Join(e.Key(), ".")) + } + return strings.Join(keys, ", ") + } + } + if err == nil { + return "" + } + return strings.TrimSpace(err.Error()) } diff --git a/pkg/gatecheck/config_test.go b/pkg/gatecheck/config_test.go new file mode 100644 index 0000000..53085f2 --- /dev/null +++ b/pkg/gatecheck/config_test.go @@ -0,0 +1,91 @@ +package gatecheck + +import ( + "bytes" + "errors" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestConfigRoundTripEveryFormat(t *testing.T) { + for _, format := range []string{"json", ".json", "yaml", "yml", ".yaml", "toml"} { + t.Run(format, func(t *testing.T) { + want := NewDefaultConfig() + want.Grype.SeverityLimit.Critical = configLimit{Enabled: true, Limit: 2} + want.License.Deny = []string{"GPL-*"} + cve := configCVE{ID: "CVE-2026-1"} + cve.Metadata.Tags = []string{"accepted"} + want.Grype.CVERiskAcceptance.CVEs = []configCVE{cve} + var buf bytes.Buffer + if err := EncodeConfigTo(&buf, want, format); err != nil { + t.Fatal(err) + } + canonical, _ := ConfigFormat(format) + got := &Config{} + if err := DecodeConfig(buf.Bytes(), canonical, got); err != nil { + t.Fatalf("decode: %v\n%s", err, buf.String()) + } + if got.String() != want.String() { + t.Fatalf("round trip changed the config:\n%s", buf.String()) + } + if canonical != "toml" && !strings.Contains(buf.String(), "\n ") { + t.Fatalf("%s output is not indented:\n%s", format, buf.String()) + } + }) + } + if _, err := ConfigFormat("xml"); err == nil { + t.Fatal("xml should be rejected") + } +} + +func TestConfigUnknownKeys(t *testing.T) { + cases := map[string]string{ + "gatecheck.yaml": "version: \"1\"\ngrype:\n severityLimit:\n critical:\n enabled: true\n severityLimt: {}\nbogus: 1\n", + "gatecheck.json": `{"version":"1","grype":{"severityLimit":{"critical":{"enabled":true}}},"bogus":1}`, + "gatecheck.toml": "version = \"1\"\nbogus = 1\n[grype.severityLimit.critical]\nenabled = true\n", + } + for name, body := range cases { + t.Run(name, func(t *testing.T) { + file := filepath.Join(t.TempDir(), name) + if err := os.WriteFile(file, []byte(body), 0o600); err != nil { + t.Fatal(err) + } + config := NewDefaultConfig() + err := NewConfigDecoder(file).Decode(config) + if !errors.Is(err, ErrUnknownConfigKeys) || !strings.Contains(err.Error(), "bogus") { + t.Fatalf("want unknown keys error naming bogus, got %v", err) + } + if !config.Grype.SeverityLimit.Critical.Enabled { + t.Fatal("known keys should still be decoded") + } + }) + } + clean := filepath.Join(t.TempDir(), "gatecheck.yml") + if err := os.WriteFile(clean, []byte("version: \"1\"\n"), 0o600); err != nil { + t.Fatal(err) + } + if err := NewConfigDecoder(clean).Decode(NewDefaultConfig()); err != nil { + t.Fatal(err) + } + empty := filepath.Join(t.TempDir(), "gatecheck.yaml") + if err := os.WriteFile(empty, nil, 0o600); err != nil { + t.Fatal(err) + } + if err := NewConfigDecoder(empty).Decode(NewDefaultConfig()); err != nil { + t.Fatalf("empty yaml: %v", err) + } +} + +func TestCoverageThresholdPercent(t *testing.T) { + if got := coverageThreshold("line", 80); got != 0.8 { + t.Fatalf("80 -> %v", got) + } + if got := coverageThreshold("line", 0.8); got != 0.8 { + t.Fatalf("0.8 -> %v", got) + } + if got := coverageThreshold("line", 1); got != 1 { + t.Fatalf("1 -> %v", got) + } +} diff --git a/pkg/gatecheck/download.go b/pkg/gatecheck/download.go index ce37dc0..588e784 100644 --- a/pkg/gatecheck/download.go +++ b/pkg/gatecheck/download.go @@ -1,6 +1,7 @@ package gatecheck import ( + "fmt" "io" "net/http" "os" @@ -131,7 +132,10 @@ func DownloadEPSS(w io.Writer, optionFuncs ...optionFunc) error { f(options) } - return epss.DownloadData(w, options.epssDownloadOptions()...) + if err := epss.DownloadData(w, options.epssDownloadOptions()...); err != nil { + return fmt.Errorf("%w: %w", ErrExternalData, err) + } + return nil } func DownloadKEV(w io.Writer, optionFuncs ...optionFunc) error { @@ -140,5 +144,8 @@ func DownloadKEV(w io.Writer, optionFuncs ...optionFunc) error { f(options) } - return kev.DownloadData(w, kev.WithClient(options.kevClient), kev.WithURL(options.kevURL)) + if err := kev.DownloadData(w, kev.WithClient(options.kevClient), kev.WithURL(options.kevURL)); err != nil { + return fmt.Errorf("%w: %w", ErrExternalData, err) + } + return nil } diff --git a/pkg/gatecheck/kind.go b/pkg/gatecheck/kind.go index f45b16b..7e1fbe5 100644 --- a/pkg/gatecheck/kind.go +++ b/pkg/gatecheck/kind.go @@ -61,6 +61,10 @@ func kindFromFilename(name string, listRules bool) artifacts.ReportKind { return artifacts.KindSyft case strings.Contains(name, "sarif"): return artifacts.KindSARIF + case strings.Contains(name, "scorecard"): + return artifacts.KindScorecard + case strings.Contains(name, "trufflehog"): + return artifacts.KindTrufflehog case strings.Contains(name, "bundle") || (listRules && strings.Contains(name, "gatecheck")): return artifacts.KindBundle case artifacts.IsCoverageReport(name): diff --git a/pkg/gatecheck/list.go b/pkg/gatecheck/list.go index 54b93a9..212be6e 100644 --- a/pkg/gatecheck/list.go +++ b/pkg/gatecheck/list.go @@ -55,22 +55,43 @@ func WithEPSS(epssFile *os.File, epssURL string, optionFuncs ...optionFunc) (fun } if epssFile == nil { - err := epss.FetchData(data, options.epssDownloadOptions()...) - return f, err + if err := epss.FetchData(data, options.epssDownloadOptions()...); err != nil { + return f, fmt.Errorf("%w: %w", ErrExternalData, err) + } + return f, nil + } + + if err := epss.ParseEPSSDataCSV(epssFile, data); err != nil { + return f, fmt.Errorf("%w: %w", ErrExternalData, err) } + return f, nil +} - err := epss.ParseEPSSDataCSV(epssFile, data) +// ListFormats are the values WithDisplayFormat accepts. "" and "ascii" are the table. +var ListFormats = []string{"table", "markdown", "json"} - return f, err +func normalizeDisplayFormat(name string) string { + switch strings.ToLower(strings.TrimSpace(name)) { + case "markdown", "md": + return "markdown" + case "json": + return "json" + default: + return "table" + } } func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOptionFunc) error { - table := format.NewTable(dst) var err error o := &listOptions{} for _, f := range options { f(o) } + displayFormat := normalizeDisplayFormat(o.displayFormat) + table := format.NewTable(dst) + if displayFormat == "json" { + table = format.NewJSONTable(dst) + } content, err := readReport(src) if err != nil { @@ -118,6 +139,12 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt case artifacts.KindSyft: err = listSyft(table, src) + case artifacts.KindScorecard: + err = listScorecard(table, src) + + case artifacts.KindTrufflehog: + err = listTrufflehog(table, src) + case artifacts.KindSARIF: err = ListSarif(table, src) @@ -126,6 +153,11 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt if err = archive.UntarGzipBundle(src, bundle); err != nil { return err } + if displayFormat == "json" { + enc := json.NewEncoder(dst) + enc.SetIndent("", " ") + return enc.Encode(bundle.Manifest()) + } _, err = fmt.Fprintln(dst, bundle.Content()) return err @@ -139,14 +171,16 @@ func List(dst io.Writer, src io.Reader, inputFilename string, options ...ListOpt if err != nil { var empty emptyReport if errors.As(err, &empty) { + if displayFormat == "json" { + return format.WriteEmptyJSON(dst, string(empty)) + } _, err = fmt.Fprintln(dst, string(empty)) return err } return err } - switch strings.ToLower(strings.TrimSpace(o.displayFormat)) { - case "markdown", "md": + if displayFormat == "markdown" { format.SetMarkdown(table) } diff --git a/pkg/gatecheck/osv.go b/pkg/gatecheck/osv.go index a544cd1..d55c249 100644 --- a/pkg/gatecheck/osv.go +++ b/pkg/gatecheck/osv.go @@ -3,6 +3,7 @@ package gatecheck import ( "encoding/json" "errors" + "fmt" "io" "log/slog" "slices" @@ -83,7 +84,7 @@ func validateOSVReportWithFetch(r io.Reader, config *Config, options *fetchOptio epssData := new(epss.Data) if err := LoadCatalogAndData(config, catalog, epssData, options); err != nil { slog.Error("validate osv-scanner report: load epss data from file or api", "error", err) - return errors.New("cannot run osv-scanner validation: Cannot load external validation data, see log for details") + return fmt.Errorf("%w: osv-scanner validation: %w", ErrExternalData, err) } return validateOSVFrom(r, config, catalog, epssData) } diff --git a/pkg/gatecheck/scorecard.go b/pkg/gatecheck/scorecard.go new file mode 100644 index 0000000..488c043 --- /dev/null +++ b/pkg/gatecheck/scorecard.go @@ -0,0 +1,177 @@ +package gatecheck + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "log/slog" + "sort" + "strings" + + "github.com/gatecheckdev/gatecheck/pkg/artifacts" + "github.com/olekukonko/tablewriter" +) + +// configScorecard gates an OpenSSF Scorecard JSON report. +type configScorecard struct { + Enabled bool `json:"enabled" toml:"enabled" yaml:"enabled"` + // MinScore fails the report when the aggregate score is lower. 0 to 10. + MinScore float64 `json:"minScore" toml:"minScore" yaml:"minScore"` + // Checks set a minimum for individual checks, by check name. + Checks []configScorecardCheck `json:"checks" toml:"checks" yaml:"checks"` +} + +type configScorecardCheck struct { + Name string `json:"name" toml:"name" yaml:"name"` + MinScore float64 `json:"minScore" toml:"minScore" yaml:"minScore"` +} + +// configTrufflehog gates trufflehog --json output. +type configTrufflehog struct { + // LimitEnabled fails the report when it has any finding. + LimitEnabled bool `json:"limitEnabled" toml:"limitEnabled" yaml:"limitEnabled"` + // VerifiedOnly counts only secrets trufflehog verified as live. + VerifiedOnly bool `json:"verifiedOnly" toml:"verifiedOnly" yaml:"verifiedOnly"` +} + +func decodeScorecard(src io.Reader) (*artifacts.ScorecardReport, error) { + report := &artifacts.ScorecardReport{} + if err := json.NewDecoder(src).Decode(report); err != nil { + return nil, err + } + return report, nil +} + +func listScorecard(table *tablewriter.Table, src io.Reader) error { + report, err := decodeScorecard(src) + if err != nil { + return err + } + if len(report.Checks) == 0 { + return emptyReport("No scorecard checks") + } + checks := append([]artifacts.ScorecardCheck(nil), report.Checks...) + sort.SliceStable(checks, func(i, j int) bool { return checks[i].Name < checks[j].Name }) + table.Header([]string{"Scorecard Check", "Score", "Reason"}) + for _, c := range checks { + if err := table.Append([]string{c.Name, scoreText(c.Score), c.Reason}); err != nil { + return err + } + } + table.Footer([]string{report.Repo.Name, scoreText(report.Score), "aggregate"}) + return nil +} + +func scoreText(score float64) string { + if score < 0 { + return "?" + } + return fmt.Sprintf("%g", score) +} + +func validateScorecardReport(src io.Reader, config *Config) error { + report, err := decodeScorecard(src) + if err != nil { + slog.Error("decode scorecard report for validation", "error", err) + return errors.New("cannot run Scorecard validation: Report decoding failed, See log for details") + } + return ruleScorecard(config, report) +} + +func ruleScorecard(config *Config, report *artifacts.ScorecardReport) error { + cfg := config.Scorecard + if !cfg.Enabled { + slog.Debug("scorecard limits not enabled", "score", report.Score) + return nil + } + var failures []string + switch { + case report.Score < 0: + slog.Warn("scorecard aggregate score is inconclusive", "repo", report.Repo.Name) + case report.Score < cfg.MinScore: + slog.Error("scorecard aggregate score below minimum", "repo", report.Repo.Name, "score", report.Score, "min_score", cfg.MinScore) + failures = append(failures, "aggregate") + default: + slog.Info("scorecard aggregate score valid", "repo", report.Repo.Name, "score", report.Score, "min_score", cfg.MinScore) + } + + byName := map[string]artifacts.ScorecardCheck{} + for _, c := range report.Checks { + byName[strings.ToLower(c.Name)] = c + } + for _, want := range cfg.Checks { + got, ok := byName[strings.ToLower(want.Name)] + switch { + case !ok: + slog.Warn("scorecard check not in report", "check", want.Name) + case got.Inconclusive(): + slog.Warn("scorecard check is inconclusive, not checked", "check", got.Name, "reason", got.Reason) + case got.Score < want.MinScore: + slog.Error("scorecard check below minimum", "check", got.Name, "score", got.Score, "min_score", want.MinScore, "reason", got.Reason) + failures = append(failures, got.Name) + default: + slog.Info("scorecard check valid", "check", got.Name, "score", got.Score, "min_score", want.MinScore) + } + } + if len(failures) > 0 { + return newValidationErr("Scorecard: score below minimum for " + strings.Join(failures, ", ")) + } + return nil +} + +func listTrufflehog(table *tablewriter.Table, src io.Reader) error { + content, err := io.ReadAll(src) + if err != nil { + return err + } + findings, err := artifacts.DecodeTrufflehog(content) + if err != nil { + return err + } + if len(findings) == 0 { + return emptyReport("No trufflehog findings") + } + table.Header([]string{"Trufflehog Detector", "Verified", "Location", "Commit"}) + for _, f := range findings { + verified := "no" + if f.Verified { + verified = "yes" + } + if err := table.Append([]string{f.DetectorName, verified, f.Location(), f.Commit()}); err != nil { + return err + } + } + return nil +} + +func validateTrufflehogReport(src io.Reader, config *Config) error { + content, err := io.ReadAll(src) + if err != nil { + return err + } + findings, err := artifacts.DecodeTrufflehog(content) + if err != nil { + slog.Error("decode trufflehog report for validation", "error", err) + return errors.New("cannot run Trufflehog validation: Report decoding failed, See log for details") + } + cfg := config.Trufflehog + if !cfg.LimitEnabled { + slog.Debug("secrets limit not enabled", "artifact", "trufflehog", "findings", len(findings)) + return nil + } + var counted int + for _, f := range findings { + if cfg.VerifiedOnly && !f.Verified { + continue + } + counted++ + slog.Warn("secret detected", "artifact", "trufflehog", "detector", f.DetectorName, "verified", f.Verified, "location", f.Location()) + } + if counted > 0 { + slog.Error("committed secrets violation", "artifact", "trufflehog", "secrets_detected", counted, "verified_only", cfg.VerifiedOnly) + return newValidationErr("Trufflehog: Secrets Detected") + } + slog.Info("secrets limit valid", "artifact", "trufflehog", "findings", len(findings), "verified_only", cfg.VerifiedOnly) + return nil +} diff --git a/pkg/gatecheck/scorecard_test.go b/pkg/gatecheck/scorecard_test.go new file mode 100644 index 0000000..4ea96fa --- /dev/null +++ b/pkg/gatecheck/scorecard_test.go @@ -0,0 +1,99 @@ +package gatecheck + +import ( + "bytes" + "errors" + "os" + "strings" + "testing" + + "github.com/gatecheckdev/gatecheck/pkg/artifacts" +) + +const ( + scorecardReport = "../../test/scorecard-report.json" + trufflehogReport = "../../test/trufflehog-report.json" +) + +func TestDetectScorecardAndTrufflehog(t *testing.T) { + for file, want := range map[string]artifacts.ReportKind{ + scorecardReport: artifacts.KindScorecard, + trufflehogReport: artifacts.KindTrufflehog, + } { + content, err := os.ReadFile(file) + if err != nil { + t.Fatal(err) + } + if got := artifacts.Detect(content); got != want { + t.Fatalf("%s detected as %q", file, got) + } + firstLine, _, _ := bytes.Cut(content, []byte("\n")) + if want == artifacts.KindTrufflehog && artifacts.Detect(firstLine) != want { + t.Fatal("a single trufflehog line should be detected") + } + } + if got := resolveKind(nil, "trufflehog-report.json", false); got != artifacts.KindTrufflehog { + t.Fatalf("empty trufflehog output by name: %q", got) + } +} + +func TestValidateScorecard(t *testing.T) { + if err := validateFile(t, NewDefaultConfig(), scorecardReport); err != nil { + t.Fatalf("rules off: %v", err) + } + config := NewDefaultConfig() + config.Scorecard.Enabled = true + config.Scorecard.MinScore = 6 + if err := validateFile(t, config, scorecardReport); err != nil { + t.Fatalf("aggregate 6.2 over 6: %v", err) + } + config.Scorecard.MinScore = 7 + err := validateFile(t, config, scorecardReport) + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "aggregate") { + t.Fatalf("aggregate under 7: %v", err) + } + + config.Scorecard.MinScore = 0 + config.Scorecard.Checks = []configScorecardCheck{ + {Name: "dangerous-workflow", MinScore: 10}, + {Name: "Signed-Releases", MinScore: 8}, + {Name: "Not-A-Check", MinScore: 5}, + } + if err := validateFile(t, config, scorecardReport); err != nil { + t.Fatalf("passing, inconclusive, and missing checks should pass: %v", err) + } + config.Scorecard.Checks = append(config.Scorecard.Checks, configScorecardCheck{Name: "Token-Permissions", MinScore: 1}) + err = validateFile(t, config, scorecardReport) + if !errors.Is(err, ErrValidationFailure) || !strings.Contains(err.Error(), "Token-Permissions") { + t.Fatalf("token permissions: %v", err) + } +} + +func TestValidateTrufflehog(t *testing.T) { + if err := validateFile(t, NewDefaultConfig(), trufflehogReport); err != nil { + t.Fatalf("rules off: %v", err) + } + config := NewDefaultConfig() + config.Trufflehog.LimitEnabled = true + if err := validateFile(t, config, trufflehogReport); !errors.Is(err, ErrValidationFailure) { + t.Fatalf("any secret: %v", err) + } + unverified := []byte(strings.Split(string(mustRead(t, trufflehogReport)), "\n")[0]) + config.Trufflehog.VerifiedOnly = true + if err := Validate(config, bytes.NewReader(unverified), "scan.json"); err != nil { + t.Fatalf("unverified only, verifiedOnly set: %v", err) + } + if err := validateFile(t, config, trufflehogReport); !errors.Is(err, ErrValidationFailure) { + t.Fatalf("verified secret: %v", err) + } + if err := Validate(config, bytes.NewReader(nil), "trufflehog-report.json"); err != nil { + t.Fatalf("empty trufflehog output: %v", err) + } + var out bytes.Buffer + if err := List(&out, bytes.NewReader(nil), "trufflehog-report.json"); err != nil || !strings.Contains(out.String(), "No trufflehog findings") { + t.Fatalf("empty list: %v %q", err, out.String()) + } + if strings.Contains(string(mustRead(t, trufflehogReport)), "hunter2") { + t.Fatal("fixture leaks a secret value") + } +} diff --git a/pkg/gatecheck/trivy.go b/pkg/gatecheck/trivy.go index bc2f5e4..46db4fe 100644 --- a/pkg/gatecheck/trivy.go +++ b/pkg/gatecheck/trivy.go @@ -3,6 +3,7 @@ package gatecheck import ( "encoding/json" "errors" + "fmt" "io" "log/slog" "slices" @@ -83,7 +84,7 @@ func validateTrivyReportWithFetch(r io.Reader, config *Config, options *fetchOpt epssData := new(epss.Data) if err := LoadCatalogAndData(config, catalog, epssData, options); err != nil { slog.Error("validate trivy report: load epss data from file or api", "error", err) - return errors.New("cannot run Trivy validation: Cannot load external validation data, see log for details") + return fmt.Errorf("%w: Trivy validation: %w", ErrExternalData, err) } return validateTrivyFrom(r, config, catalog, epssData) } diff --git a/pkg/gatecheck/validate.go b/pkg/gatecheck/validate.go index 2996ff0..553fe04 100644 --- a/pkg/gatecheck/validate.go +++ b/pkg/gatecheck/validate.go @@ -21,6 +21,9 @@ import ( var ErrValidationFailure = errors.New("validation failure") +// ErrExternalData means EPSS or KEV data could not be downloaded or read. +var ErrExternalData = errors.New("cannot load EPSS or KEV data") + func newValidationErr(details string) error { return fmt.Errorf("%w: %s", ErrValidationFailure, details) } @@ -71,6 +74,12 @@ func Validate(config *Config, reportSrc io.Reader, targetFilename string, option case artifacts.KindSyft: return validateSyftReport(reportSrc, config) + case artifacts.KindScorecard: + return validateScorecardReport(reportSrc, config) + + case artifacts.KindTrufflehog: + return validateTrufflehogReport(reportSrc, config) + case artifacts.KindSARIF: return validateSarifReport(reportSrc, config) @@ -274,7 +283,7 @@ func ruleGrypeKEVLimit(config *Config, report *artifacts.GrypeReportMin, catalog // Check if vulnerability is in the KEV Catalog for _, vulnerability := range report.Matches { inKEVCatalog := slices.ContainsFunc(catalog.Vulnerabilities, func(kevVul kev.Vulnerability) bool { - return kevVul.CveID == vulnerability.Vulnerability.ID + return slices.ContainsFunc(vulnerability.IDs(), func(id string) bool { return strings.EqualFold(kevVul.CveID, id) }) }) if inKEVCatalog { badCVEs = append(badCVEs, vulnerability.Vulnerability.ID) @@ -646,7 +655,7 @@ func validateGrypeReportWithFetch(r io.Reader, config *Config, options *fetchOpt if err := LoadCatalogAndData(config, catalog, epssData, options); err != nil { slog.Error("validate grype report: load epss data from file or api", "error", err) - return errors.New("cannot run Grype validation: Cannot load external validation data, see log for details") + return fmt.Errorf("%w: Grype validation: %w", ErrExternalData, err) } return validateGrypeFrom(r, config, catalog, epssData) @@ -671,7 +680,7 @@ func validateCyclonedxReportWithFetch(r io.Reader, config *Config, options *fetc if err := LoadCatalogAndData(config, catalog, epssData, options); err != nil { slog.Error("validate cyclonedx report: load epss data from file or api", "error", err) - return errors.New("cannot run Cyclonedx validation: Cannot load external validation data, See log for details") + return fmt.Errorf("%w: Cyclonedx validation: %w", ErrExternalData, err) } return validateCyclonedxFrom(r, config, catalog, epssData) } @@ -733,6 +742,7 @@ func validateCoverage(src io.Reader, kind artifacts.ReportKind, targetFilename s var errs error for _, check := range checks { + check.threshold = coverageThreshold(check.name, check.threshold) ratio, ok := coverageRatio(check.covered, check.total) if !ok { if check.threshold > 0 { @@ -751,6 +761,16 @@ func validateCoverage(src io.Reader, kind artifacts.ReportKind, targetFilename s return errs } +// coverageThreshold reads a threshold over 1 as a percent, so 80 means 0.8. +func coverageThreshold(name string, threshold float32) float32 { + if threshold <= 1 { + return threshold + } + ratio := threshold / 100 + slog.Warn("coverage threshold is over 1, reading it as a percent", "kind", name, "threshold", threshold, "ratio", ratio) + return ratio +} + // coverageRatio returns false when total is zero, so a report with no records // of a kind is not applicable rather than NaN. func coverageRatio(covered, total int) (float32, bool) { @@ -772,8 +792,8 @@ func validateBundle(r io.Reader, config *Config, options *fetchOptions) error { epssData := new(epss.Data) if err := LoadCatalogAndData(config, catalog, epssData, options); err != nil { - slog.Error("validate cyclonedx report: load epss data from file or api", "error", err) - return errors.New("cannot run Cyclonedx validation: Cannot load external validation data, See log for details") + slog.Error("validate bundle: load epss data from file or api", "error", err) + return fmt.Errorf("%w: bundle validation: %w", ErrExternalData, err) } var errs error @@ -806,6 +826,12 @@ func validateBundle(r io.Reader, config *Config, options *fetchOptions) error { case artifacts.KindSyft: err := validateSyftReport(bytes.NewBuffer(content), config) errs = errors.Join(errs, err) + case artifacts.KindScorecard: + err := validateScorecardReport(bytes.NewBuffer(content), config) + errs = errors.Join(errs, err) + case artifacts.KindTrufflehog: + err := validateTrufflehogReport(bytes.NewBuffer(content), config) + errs = errors.Join(errs, err) case artifacts.KindLCOV, artifacts.KindClover, artifacts.KindCobertura, artifacts.KindCoverage: err := validateCoverage(bytes.NewBuffer(content), kind, fileLabel, config) errs = errors.Join(errs, err) diff --git a/skills/gatecheck/SKILL.md b/skills/gatecheck/SKILL.md index fe64ebb..b32da8b 100644 --- a/skills/gatecheck/SKILL.md +++ b/skills/gatecheck/SKILL.md @@ -7,7 +7,7 @@ description: Use when running gatecheck to list, validate, or bundle scanner rep Run `gatecheck agent-guide` and follow that output. The alias is `gatecheck llm`. -`gatecheck bootstrap` can install Trivy and osv-scanner, and `--tools gosec,hadolint,zizmor` installs those SARIF scanners. Trivy JSON and osv-scanner JSON are recognized from their contents. A filename containing `trivy` or `osv` is the fallback when the contents are ambiguous. Severity limits and CVE allow and deny lists use the same fields as Grype. +`gatecheck bootstrap` can install Trivy and osv-scanner, `--tools gosec,hadolint,zizmor` installs those SARIF scanners, and `--tools scorecard,trufflehog,cosign` installs OpenSSF Scorecard, trufflehog, and cosign. Trivy JSON and osv-scanner JSON are recognized from their contents. A filename containing `trivy` or `osv` is the fallback when the contents are ambiguous. Severity limits and CVE allow and deny lists use the same fields as Grype. The command prints `docs/agent-guide.md`. That file is embedded in the binary, so the command and the file are the same text. Prefer the command when the `gatecheck` binary is installed. diff --git a/test/scorecard-report.json b/test/scorecard-report.json new file mode 100644 index 0000000..b121496 --- /dev/null +++ b/test/scorecard-report.json @@ -0,0 +1,85 @@ +{ + "date": "2026-11-04T12:00:00Z", + "repo": { + "name": "github.com/gatecheckdev/gatecheck", + "commit": "d554611" + }, + "scorecard": { + "version": "v5.6.0", + "commit": "unknown" + }, + "score": 6.2, + "checks": [ + { + "details": null, + "score": 10, + "reason": "no binaries found in the repo", + "name": "Binary-Artifacts", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#binary-artifacts", + "short": "Determines if the project has generated executable (binary) artifacts in the source repository." + } + }, + { + "details": null, + "score": 3, + "reason": "branch protection is not maximal on development and all release branches", + "name": "Branch-Protection", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection", + "short": "Determines if the default and release branches are protected with GitHub's branch protection settings." + } + }, + { + "details": null, + "score": 0, + "reason": "Found 0/30 approved changesets -- score normalized to 0", + "name": "Code-Review", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#code-review", + "short": "Determines if the project requires human code review before pull requests (aka merge requests) are merged." + } + }, + { + "details": null, + "score": 10, + "reason": "no dangerous workflow patterns detected", + "name": "Dangerous-Workflow", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#dangerous-workflow", + "short": "Determines if the project's GitHub Action workflows avoid dangerous patterns." + } + }, + { + "details": null, + "score": -1, + "reason": "no releases found", + "name": "Signed-Releases", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#signed-releases", + "short": "Determines if the project cryptographically signs release artifacts." + } + }, + { + "details": null, + "score": 4, + "reason": "dependency not pinned by hash detected -- score normalized to 4", + "name": "Pinned-Dependencies", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies", + "short": "Determines if the project has declared and pinned the dependencies of its build process." + } + }, + { + "details": null, + "score": 0, + "reason": "detected GitHub workflow tokens with excessive permissions", + "name": "Token-Permissions", + "documentation": { + "url": "https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions", + "short": "Determines if the project's workflows follow the principle of least privilege." + } + } + ], + "metadata": null +} \ No newline at end of file diff --git a/test/trufflehog-report.json b/test/trufflehog-report.json new file mode 100644 index 0000000..365ce6d --- /dev/null +++ b/test/trufflehog-report.json @@ -0,0 +1,2 @@ +{"SourceMetadata": {"Data": {"Filesystem": {"file": "th/creds.txt", "line": 1}}}, "SourceID": 1, "SourceType": 15, "SourceName": "trufflehog - filesystem", "DetectorType": 2, "DetectorName": "AWS", "DetectorDescription": "AWS (Amazon Web Services) is a comprehensive cloud computing platform offering a wide range of on-demand services like computing power, storage, databases. API keys for AWS can have varying amount of access to these services depending on the IAM policy attached.", "DecoderName": "PLAIN", "Verified": false, "VerificationFromCache": false, "Raw": "REDACTED", "RawV2": "REDACTED", "Redacted": "REDACTED", "ExtraData": {}, "StructuredData": null, "SecretParts": {"access_key_id": "REDACTED", "secret_access_key": "REDACTED"}} +{"SourceMetadata": {"Data": {"Git": {"commit": "3f2a9c1d8e7b6a5f4e3d2c1b0a9f8e7d6c5b4a39", "file": "deploy/config.env", "email": "dev ", "repository": "https://github.com/example/app.git", "timestamp": "2026-10-01 10:00:00 +0000", "line": 4}}}, "SourceID": 1, "SourceType": 16, "SourceName": "trufflehog - git", "DetectorType": 8, "DetectorName": "Github", "DetectorDescription": "AWS (Amazon Web Services) is a comprehensive cloud computing platform offering a wide range of on-demand services like computing power, storage, databases. API keys for AWS can have varying amount of access to these services depending on the IAM policy attached.", "DecoderName": "PLAIN", "Verified": true, "VerificationFromCache": false, "Raw": "REDACTED", "RawV2": "REDACTED", "Redacted": "REDACTED", "ExtraData": {}, "StructuredData": null, "SecretParts": {"token": "REDACTED"}} diff --git a/website/llms-full.txt b/website/llms-full.txt index dd10da1..061f22a 100644 --- a/website/llms-full.txt +++ b/website/llms-full.txt @@ -1,6 +1,6 @@ # gatecheck -Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner, plus gosec, hadolint, and zizmor on request, and the matching subcommands run them. Coverage reports still come from the language toolchain. +Gatecheck reads a scanner report, compares it to thresholds in a config file, and exits non-zero when a threshold fails. It also prints a table of a report, packs reports into one bundle, and downloads EPSS and KEV data. `gatecheck bootstrap` installs Grype, Syft, Gitleaks, semgrep's experimental osemgrep CLI, Trivy, and osv-scanner, plus gosec, hadolint, zizmor, OpenSSF Scorecard, trufflehog, and cosign on request, and the matching subcommands run them. Coverage reports still come from the language toolchain. `gatecheck` with no arguments prints nothing and exits 0. `gatecheck --help` lists commands. `gatecheck agent-guide` (alias `llm`) prints this file. @@ -14,12 +14,12 @@ Logs go to stderr. Tables, `config init`, downloads, and this guide go to stdout - `validate` compares one report, or one bundle, to a config. - `bundle create` (alias `init`), `bundle add`, and `bundle remove` (alias `rm`) manage a bundle. - `download epss` and `download kev` write upstream data to stdout. -- `bootstrap` installs Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner. `--tools gosec,hadolint,zizmor` installs those too. `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, and `zizmor` run those binaries. +- `bootstrap` installs Grype, Syft, Gitleaks, osemgrep, Trivy, and osv-scanner. `--tools gosec,hadolint,zizmor,scorecard,trufflehog,cosign` installs those too. `grype`, `syft`, `gitleaks`, `semgrep`, `osemgrep`, `trivy`, `osv-scanner`, `gosec`, `hadolint`, `zizmor`, `scorecard`, `trufflehog`, and `cosign` run those binaries. - `update` (aliases `self-update`, `upgrade`) replaces this binary with a GitHub release after checking `checksums.txt`. `update --check` only prints the current and latest versions. - `completion bash|fish|powershell|zsh` prints a shell completion script. - `help [command]` prints the same text as ` --help`. -Global flags, on every command: `-v` / `--verbose` (debug logs) and `--silent` (errors only). They cannot be set together. `-h` / `--help` is also global. The scanner commands pass every argument through, including `--help`, so `gatecheck grype --help` is Grype's help. +Global flags, on every command: `-v` / `--verbose` (debug logs), `--silent` (warnings and errors), `--log-level debug|info|warn|error` (default `info`), and `--log-format text|json` (default `text`). Only one of `--verbose`, `--silent`, and `--log-level` can be set. `-h` / `--help` is also global. The scanner commands pass every argument through, including `--help`, so `gatecheck grype --help` is Grype's help. ## Scanners @@ -31,6 +31,8 @@ osv-scanner comes from the `google/osv-scanner` GitHub release as a raw binary, gosec, hadolint, and zizmor are installed only when named in `--tools`. gosec comes from `securego/gosec` as `gosec___.tar.gz` (tar.gz on Windows too), checked against `gosec__checksums.txt`. hadolint comes from `hadolint/hadolint` as a raw binary such as `hadolint-linux-x86_64` or `hadolint-macos-arm64`, checked against `checksums.sha256`. zizmor comes from `zizmorcore/zizmor` as `zizmor--.tar.gz` (`.zip` on Windows). zizmor publishes no checksum file, so its digests are pinned in `internal/bootstrap/versions.go`. hadolint and zizmor have no Windows arm64 build. All three write SARIF, which `list` and `validate` read with the `sarif` config section. +scorecard (`ossf/scorecard`, `scorecard___.tar.gz`, checksums `scorecard_checksums.txt`) and trufflehog (`trufflesecurity/trufflehog`, `trufflehog___.tar.gz`, checksums `trufflehog__checksums.txt`) are also installed only when named, for amd64 and arm64 on macOS, Linux, and Windows. Their JSON is its own report type. cosign (`sigstore/cosign`) is a raw binary checked against `cosign_checksums.txt`, for macOS amd64 and arm64, Linux amd64, arm64, armv7, ppc64le, s390x, and riscv64, and Windows amd64. `gatecheck cosign verify` and `verify-blob` pass cosign's exit status through, so a bad signature fails the CI step. + The directory is `$GATECHECK_HOME/bin` when `GATECHECK_HOME` is set. Otherwise it is `$XDG_DATA_HOME/gatecheck/bin`, or `~/.local/share/gatecheck/bin` on Linux, `~/Library/Application Support/gatecheck/bin` on macOS, and `%LOCALAPPDATA%\gatecheck\bin` on Windows. A tool already at the pinned version is left in place. `--force` downloads it again. `--tools grype,syft` limits the set. `osemgrep` in `--tools` means semgrep. `--list` prints pinned versions, installed versions, and paths, and does not download. @@ -46,13 +48,15 @@ gatecheck bootstrap --tools gosec,hadolint,zizmor gatecheck gosec -fmt sarif -out gosec.sarif ./... gatecheck hadolint -f sarif Dockerfile > hadolint.sarif gatecheck zizmor --format sarif .github/workflows > zizmor.sarif +gatecheck trufflehog git file://. --json > trufflehog-report.json +GITHUB_AUTH_TOKEN=... gatecheck scorecard --repo github.com/example/app --format json > scorecard-report.json ``` `gatecheck semgrep scan` and `gatecheck osemgrep scan` insert `--experimental` after `scan` when that flag is absent. Other osemgrep commands need `--experimental` from the caller. `gatecheck semgrep --experimental --version` prints the osemgrep version. When the program that runs is the Python `semgrep` on `PATH`, the flag is left alone. `gatecheck osemgrep` looks for the managed binary, then `osemgrep` on `PATH`. -The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck trivy`, `gatecheck osv-scanner`, `gatecheck gosec`, `gatecheck hadolint`, and `gatecheck zizmor` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 255. A scanner's own exit status is returned as-is. +The managed binary wins when it is present. `gatecheck grype`, `gatecheck syft`, `gatecheck gitleaks`, `gatecheck trivy`, `gatecheck osv-scanner`, `gatecheck gosec`, `gatecheck hadolint`, `gatecheck zizmor`, `gatecheck scorecard`, `gatecheck trufflehog`, and `gatecheck cosign` otherwise run that name on `PATH`. `gatecheck semgrep` tries `semgrep` and then `osemgrep` on `PATH`. If none exist, the command prints `gatecheck: is not installed; run `gatecheck bootstrap`` and exits 2. A scanner's own exit status is returned as-is. -A platform with no release asset for a requested GitHub-hosted tool fails that tool and the command exits 255. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, and zizmor `1.32.0`, in `internal/bootstrap/versions.go`. +A platform with no release asset for a requested GitHub-hosted tool fails that tool and the command exits 2. A platform with no semgrep wheel prints a skip message on stderr and continues. Current pins are Grype `0.121.0`, Syft `1.55.0`, Gitleaks `8.30.1`, semgrep `1.182.1`, Trivy `0.76.1`, osv-scanner `2.7.0`, gosec `2.30.0`, hadolint `2.15.1`, zizmor `1.32.0`, scorecard `5.6.0`, trufflehog `3.99.8`, and cosign `3.1.4`, in `internal/bootstrap/versions.go`. Syft is installed so it can write CycloneDX (`syft -o cyclonedx-json`). That BOM is listed and validated as CycloneDX. Native Syft JSON is recognized from `descriptor.name` or the schema URL. `list` prints its packages and licenses, and `validate` runs the license rules on it. CycloneDX is a report format, and coverage tools are left to the language toolchain. @@ -71,9 +75,11 @@ Content rules, in this order: 7. Gitleaks JSON. A non-empty array of objects with `RuleID` and one of `File`, `Commit`, or `Fingerprint`. 8. Trivy JSON. `SchemaVersion` is a number, and `ArtifactType` is set or `Results` is an array. Vulnerability entries are listed and validated. Misconfigurations, secrets, and licenses are ignored. A CycloneDX BOM from Trivy is CycloneDX. 9. osv-scanner JSON. `results` is a non-empty array of objects with a `packages` array. One finding per vulnerability group. Aliases of that group are not counted again. License-only packages are skipped. Semgrep is checked first, so a `check_id` stays Semgrep. `{"results":[]}` does not identify itself. -10. LCOV text. A line starts with `SF:` and the file also has `DA:`, `LF:`, or `end_of_record`. -11. Clover XML. An XML document with `clover=` or a ` gatecheck.yaml` and set real thr ## Common mistakes - The contents do not identify a type and the filename has no lowercase type word (`scan.json`, `Grype.json`). A real bundle named `project.gcb` is still recognized from its contents. An empty document such as `{"matches":[]}` still needs a name such as `grype-report.json`, or `--input-type grype` on standard input. -- Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent, not `80` or `80.0`. +- Coverage thresholds are ratios. Set `coverage.lineThreshold` to `0.8` for 80 percent. A value over 1, such as `80`, is read as a percent with a warning. - `validate` takes the report as its positional argument. The config is `--config` / `-f`. `gatecheck validate gatecheck.yaml grype-report.json` tries to open the config as a report. `config convert` uses `--file` / `-f`, which is a different flag. -- `list` on a pipe needs `--input-type`. Clear contents still win over that flag. `validate` needs a filesystem path. -- `list` ignores every filename after the first. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. +- `list` and `validate` on a pipe need `--input-type`. Clear contents still win over that flag. +- `list` takes one filename, and a second one exits 2. Use `list-all` for several files. `list-all` skips coverage files such as `lcov.info`. List those with `gatecheck list lcov.info`. - Native Syft JSON gets only the license rules. Use CycloneDX output when you also want vulnerabilities scored. That BOM is CycloneDX even when the filename contains `syft`. - `gatecheck semgrep --version` reaches osemgrep without `--experimental`, and osemgrep then tries to run `pysemgrep`. Pass the flag: `gatecheck semgrep --experimental --version`. -- `--audit` still exits 255 when the report, the config, or a named EPSS or KEV file cannot be opened. +- `--audit` only hides threshold failures. It still exits 2 when the report, the config, or a named EPSS, KEV, or VEX file cannot be opened, and 3 when EPSS or KEV data cannot be loaded. diff --git a/website/llms.txt b/website/llms.txt index d77fceb..574285c 100644 --- a/website/llms.txt +++ b/website/llms.txt @@ -8,7 +8,7 @@ Report type comes from the file contents. The filename is the fallback when the - [Installation](https://docs.gatecheck.dev/installation.html): install a release binary or build from source - [Usage](https://docs.gatecheck.dev/usage.html): commands, logs, and exit status -- [Bootstrap](https://docs.gatecheck.dev/bootstrap.html): install Grype, Syft, Gitleaks, osemgrep, Trivy, osv-scanner, gosec, hadolint, and zizmor, then run them +- [Bootstrap](https://docs.gatecheck.dev/bootstrap.html): install Grype, Syft, Gitleaks, osemgrep, Trivy, osv-scanner, gosec, hadolint, zizmor, scorecard, trufflehog, and cosign, then run them - [CLI reference](https://docs.gatecheck.dev/cli-reference.html): flags, subcommands, and environment variables - [Supported reports](https://docs.gatecheck.dev/supported-reports.html): how file contents and filenames select the report type - [Configuration](https://docs.gatecheck.dev/configuration.html): config keys and threshold units