Skip to content

Parse grouped calculations and the new calculate functions - #90

Open
jtnelson wants to merge 4 commits into
developfrom
feature/GH-89-grouped-calculations
Open

jtnelson wants to merge 4 commits into
developfrom
feature/GH-89-grouped-calculations

Conversation

@jtnelson

@jtnelson jtnelson commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Summary

Concourse is adding calculations (cinchapi/concourse#1417), which are new
functions, grouped calculations, ranges of numbers or time, and several
calculations in one statement. CCL must parse each of them into a
CalculateSymbol that Concourse can turn into a command. A calculate statement
named only one of sum, avg, average, count, min or max, had no
grouping, and its symbol had no equals, hashCode or text.

This change adds the functions and a group by clause to the calculate
statement, and exposes them through a CalculationSymbol for each calculation
and a GroupSymbol for the group. The existing constructors and accessors of
CalculateSymbol keep their meaning, so a caller that reads function(),
key(), records() and timestamp() works as before. The page and the result
order of a grouped statement belong to its GroupSymbol, because they page and
order groups rather than records, and a grouped statement cannot order its
groups by a key.

Closes #89

Behavior

  • The new function names are countDistinct, median, percentile, stddev,
    stddevPopulation, variance and variancePopulation, in any letter case.
    percentile takes a fraction from 0 to 1 before its key, as in
    calculate percentile 0.99 latency.
  • CalculationSymbol#function() and CalculateSymbol#function() return a
    canonical name. It is camelCase for a new function, such as countDistinct,
    which is the name of the Concourse calculation it asks for. It is lower case
    for sum, avg, average, count, min and max, as before.
  • A group by clause comes after the scope (records or a where condition) and
    the timestamp. Its parts come in this order, and each part after the keys is
    optional.
    • One or more group keys, which may be navigation keys.
    • per <number> or per "<duration>" groups the values of the last key into
      number ranges of that width or time ranges of that period. A duration has
      the form of a within duration.
    • between <a> and <b> follows per and bounds the ranges. The bounds are
      numbers for number ranges and timestamps for time ranges.
    • having <function> <operator> <value> keeps the groups whose result
      satisfies a comparison.
    • order by <function> [asc|desc] orders the groups by a result, ascending by
      default.
    • limit and offset page the groups.
  • The function in having and order by names one of the statement's
    calculations in any letter case, and avg and average name the same
    function. A function that names none of the calculations, or more than one, is
    a syntax error. So order by <key> in a grouped statement is a syntax error.
  • A statement may list several calculations, separated by commas, only when it
    groups. calculations() returns them in the order the statement lists them,
    and function() and key() return those of the first.
  • GroupSymbol exposes keys(), width(), lowerBound() and upperBound()
    as numbers, period() as a java.time.Duration, start() and end() as
    TimestampSymbols, and having(), order() and page(). Each part the
    statement leaves out is null. A whole number is an Integer or a Long,
    and a decimal is a Double.
  • The calculation() of a HavingSymbol or a ResultOrderSymbol is the one of
    calculations() that its function names.
  • The condition of a calculate statement stays a child of the CommandTree. A
    grouped statement has no OrderTree or PageTree child.
  • An at right after a comparison belongs to the comparison, as it does for
    find. as of, or at after a parenthesized condition, sets the timestamp
    of the calculation.
  • CalculateSymbol, CalculationSymbol, GroupSymbol, HavingSymbol and
    ResultOrderSymbol implement equals and hashCode. Numbers in a group are
    equal when they have the same value, such as 10 and 10.0. The text of a
    CalculateSymbol is its statement without the condition, and that text parses
    to an equal symbol.
  • Equal functions have equal hash codes, so a symbol or tree that holds a key or
    value function, such as age | avg, keeps the equals and hashCode
    contract.
  • A CalculateSymbol or GroupSymbol built from its parts follows the rules of
    the grammar, and an invalid one throws IllegalArgumentException or, from
    GroupSymbol.Builder#build(), IllegalStateException.
  • Compiler#analyze reports the key of every calculation and every group key.
  • The text group by is reserved in any letter case, even when a longer word
    starts with by, as in group bypass. A value that contains it must be
    quoted. The words per and having are not reserved, so a key may have
    either name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parse grouped calculations and the new calculate functions

1 participant