Dev-iL · Nov 10, 2026 · Nov 10, 2026
diff --git a/CONTEXT.md b/CONTEXT.md
@@ -0,0 +1,35 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Apache Hamilton
+
+A Python library that builds a directed acyclic graph (DAG) of data transformations from plain Python functions, whose names are the nodes and whose parameters are the dependencies.
+
+This glossary records what words mean in this project where ordinary usage or generic tool knowledge would mislead. Add a term when the project settles one.
+
+## Language
+
+- **adapter** — three different things share this word. A *data adapter* is a `DataLoader`/`DataSaver` class registered by name in the adapter registry (`hamilton/registry.py`). A *legacy graph adapter* is a `HamiltonGraphAdapter` subclass (`hamilton/base.py`). A *lifecycle adapter* is a hook, method, or validator passed to `Builder.with_adapters` (`hamilton/lifecycle/`). Name which one you mean.
+- **function vs node** — a *function* is what the user writes in a module; a *node* is one step in the DAG. One function yields zero, one, or many nodes (function modifiers expand, rename, or drop them), so never assume one-to-one.
+- **function modifier** — Hamilton's name for its decorators (`@extract_columns`, `@parameterize`, `@config.when`, …). A function modifier changes how a function compiles into nodes when the graph is built; unlike a plain Python decorator it does not wrap the call. Modifiers resolve in a fixed order of phases, one per kind of modifier: resolvers, creators, injectors, expanders, transformers, and decorators (see [`writeups/adr/2611-03-function-modifiers-fixed-lifecycle.md`](writeups/adr/2611-03-function-modifiers-fixed-lifecycle.md)). `NodeDecorator` is the base class of the last of those phases, so "decorator" in the code can mean that phase rather than every modifier.
+- **config / inputs / overrides** — *config* shapes the DAG and is fixed when the driver is built; *inputs* are per-run values for nodes nothing in the DAG produces; *overrides* replace values the DAG would otherwise compute, skipping their upstream work. Putting a value in the wrong one changes the graph or silently skips computation.
+- **result builder** — combines the requested outputs into the return value of `execute()`. The default differs by entry point. The legacy `Driver(config, module)` uses `PandasDataFrameResult`. `driver.Builder` adds none, so `execute()` returns the outputs as a dict, as `DictResult` would. The same DAG can therefore return different types.
+- **hook / method / validator** — the three kinds of lifecycle adapter. *Hooks* observe or augment execution and stack freely; *methods* replace one step of execution, and only one adapter may implement each method (a sync and an async implementation count separately); *validators* check the graph or its nodes when it is built.
+- **materializer vs data loader/saver decorators** — *materializers* (`Driver.materialize` with `from_`/`to`) and `@load_from`/`@save_to` pick a registered data adapter by name and type. `@dataloader`/`@datasaver` instead wrap a user function that returns data plus metadata, with no registry involved. They are separate extension points.
+- **extension vs plugin module** — `hamilton/plugins/<lib>_extensions.py` modules are *extensions*: the registry autoloads them to register types and data adapters. `hamilton/plugins/h_<lib>.py` modules are *plugins* the user imports explicitly for adapters and helpers. Both live in `hamilton/plugins/`, so a new integration has to choose which kind it is.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -51,11 +51,9 @@ Additionally, the code should follow any stylistic and architectural guidelines
 prescribed by the project. For us here, this means you install a pre-commit hook and use
 the given style files. Basically, you should mimic the styles and patterns in the Apache Hamilton code-base.
 
-In terms of getting setup to develop, we invite you to read our [developer setup guide](developer_setup.md).
+In terms of getting set up to develop, we invite you to read our [developer setup guide](writeups/developer_setup.md).
 
-## Using circleci CLI to run tests locally
+## Finding your way around
 
-1. Install the [circleci CLI](https://circleci.com/docs/2.0/local-cli/).
-2. To run a circleci job locally, you should then, from the root of the repository, do
-   > circleci local execute --job=JOB_NAME
-3. This assumes you have docker installed, it will pull the images, but otherwise will run the circleci job.
+- [`CONTEXT.md`](CONTEXT.md) is a glossary of the words that mean something specific in Apache Hamilton, such as the three different things called an "adapter". Read it before naming something new.
+- [`writeups/adr/`](writeups/adr/README.md) holds the project's decision records: why its foundations were built the way they are. Read the one that covers what you are changing before you change it.
diff --git a/README.md b/README.md
@@ -236,7 +236,7 @@ Apache Hamilton is released under the Apache 2.0 License. See [LICENSE](https://
 ## 👨‍💻 Contributing
 We're very supportive of changes by new contributors, big or small! Make sure to discuss potential changes by creating an issue or commenting on an existing one before opening a pull request. Good first contributions include creating an example or an integration with your favorite Python library!
 
- To contribute, checkout our [contributing guidelines](https://github.com/apache/hamilton/blob/main/CONTRIBUTING.md), our [developer setup guide](https://github.com/apache/hamilton/blob/main/developer_setup.md), and our [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html).
+ To contribute, checkout our [contributing guidelines](https://github.com/apache/hamilton/blob/main/CONTRIBUTING.md), our [developer setup guide](https://github.com/apache/hamilton/blob/main/writeups/developer_setup.md), and our [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html).
 
 
 ## 😎 Used by

diff --git a/writeups/adr/2609-02-sql-metadata-keeps-1-0-0-filing.md b/writeups/adr/2609-02-sql-metadata-keeps-1-0-0-filing.md
@@ -52,7 +52,7 @@ dataset.
 - Existing consumers of `sql_metadata` see the same `query`/`table_name` values for writes and
   for two-argument calls. Default-mode lineage names are unchanged for every string tested.
 - `query` does not always hold SQL. A written name containing `SELECT` sits there, so lineage code
-  must use `operation` and not the key alone (see ADR 0003).
+  must use `operation` and not the key alone (see [2609-03](2609-03-resolving-sql-writes-and-statements.md)).
 
 ## Alternatives considered
 

diff --git a/writeups/adr/2609-03-resolving-sql-writes-and-statements.md b/writeups/adr/2609-03-resolving-sql-writes-and-statements.md
@@ -23,7 +23,7 @@ under the License.
 
 ## Context
 
-Under ADR 0002, a write's string may sit under `query` or `table_name`. It may be a table name
+Under [2609-02](2609-02-sql-metadata-keeps-1-0-0-filing.md), a write's string may sit under `query` or `table_name`. It may be a table name
 (`daily revenue`, `SELECT results`, `USER_SELECTIONS`) or, for custom savers, a statement
 (`INSERT INTO t SELECT ...`). `sql_datasets()` must name the tables written, and it must never
 invent a dataset named after SQL text or report a table name as the job's SQL.

diff --git a/writeups/adr/2611-02-functions-define-nodes.md b/writeups/adr/2611-02-functions-define-nodes.md
@@ -0,0 +1,54 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+# 2611-02. A function defines a node, and its parameter names are its dependencies
+
+**Status:** Reconstructed from codebase (decided 2019 at Stitch Fix: commit [f7c26ee8](https://github.com/apache/hamilton/commit/f7c26ee8c480abdeccda1cdc3f6fec4835a462ac) of 2019-07-25, the earliest in this history, already maps function names to nodes and parameters to dependencies; the other rules below arrived later, on the dates given with them)
+
+## Context
+
+Hamilton replaced a feature-engineering codebase in which a script built every column by mutating one shared dataframe. The origin post names the problems: "Column (i.e. feature) dependencies are hard to determine", "Documentation is hard", "Unit testing is hard", and "You need to execute and understand the entire script to get any columns (i.e. features)". The team "did not find any open-source tooling that would dramatically improve our capability to solve the aforementioned problems", and judged that "organizing code into functions is merely a tool, and does not provide adequate guard-rails".
+
+The 2020 README states the core choice: "Core to the design of hamilton is a 1:1 mapping of key (function name) to implementation. Rather than defining flexible, reusable transformations, Hamilton aims for DAG clarity, easy modifications, and documentation."
+
+This record was reconstructed from the code of commit [f7c26ee8](https://github.com/apache/hamilton/commit/f7c26ee8c480abdeccda1cdc3f6fec4835a462ac), the project's first README (commit [3b70736b](https://github.com/apache/hamilton/commit/3b70736ba3a40348581de9d113291d00a6814abe), 2020-08-21), the Stitch Fix origin post ("[Functions + DAGs: introducing Hamilton](https://multithreaded.stitchfix.com/blog/2021/10/14/functions-dags-hamilton/)", 2021-10-14, linked at [`README.md:245`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/README.md#L245)), [`docs/concepts/node.rst`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst), and [`writeups/basics.md`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/writeups/basics.md). The design is older than this repository's history, and no public discussion from before 2019-07-25 exists. [f7c26ee8](https://github.com/apache/hamilton/commit/f7c26ee8c480abdeccda1cdc3f6fec4835a462ac) , whose message lists "Refactored hamilton prototype to function on customizable modules", already builds one node per function, takes dependencies from parameter names, skips functions whose name starts with `_`, requires names and types to match, and refuses a function defined twice. The 2020 README is the earliest written reasoning; `node.rst` was written in 2024 ([#659](https://github.com/apache/hamilton/pull/659)), so its reasoning is the rationale as later documented.
+
+## Decision
+
+- Hamilton derives the directed acyclic graph (DAG) by inspecting function definitions. A user never declares the graph separately. The function name is the node name and the return annotation is the node type. The parameter names and annotations are the node's dependencies, and the docstring is its description ([`docs/concepts/node.rst:20`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L20) and the table under "Anatomy of a node", [`node.rst:76`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L76)). The docstring became the description in 2019-08 ([8c8f7843](https://github.com/apache/hamilton/commit/8c8f7843c87eb68ae125bc461c302a42a269350d): "We want to also easily display documentation relating to business logic that gets encoded").
+- A node name is a globally unique key. A DAG can have only one function named `sum_a_b` ([`writeups/basics.md:69-71`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/writeups/basics.md#L69-L71)). A name defined in two modules is an error unless the caller opts in with `Builder.allow_module_overrides()` ([#1134](https://github.com/apache/hamilton/pull/1134), 2024-09), in which case the later module wins ([`docs/concepts/builder.rst:78`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/builder.rst#L78)).
+- A function whose name starts with `_` is a helper and does not become a node ([`docs/concepts/node.rst:45`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L45)). The 2020 README gives the reason: "Hamilton tries to make use of every function in a module, so this allows us to easily indicate helper functions". Since 2024 the `@hamilton_exclude` decorator does the same for the exceptional case ([`configuration.py`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/hamilton/function_modifiers/configuration.py#L278-L283)).
+- A parameter that no function produces is an input. The caller must supply it, unless the parameter has a default. A default makes it optional ([`DependencyType.OPTIONAL`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/hamilton/node.py#L52-L60)), and when the caller does not supply a value, execution leaves the argument out so the function's default applies ([`hamilton/execution/graph_functions.py:211-223`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/hamilton/execution/graph_functions.py#L211-L223)). Optional inputs arrived in 2020-01 ([662dceef](https://github.com/apache/hamilton/commit/662dceefe2d8f4169c1faa1845612eaf471d6d20)) and were completed in [stitchfix/hamilton#96](https://github.com/stitchfix/hamilton/pull/96) (2022-03).
+- Type annotations are required. Hamilton uses them to check that each producer matches its consumers ([`writeups/basics.md:75`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/writeups/basics.md#L75)). Requiring them was a usability fix of 2019-08 ([11dec583](https://github.com/apache/hamilton/commit/11dec58393acf19c808755194b3143354fe988ef): "This is a usability improvement. I forgot to add a type and was scratching my head").
+
+## Consequences
+
+- A function's code shows its place in the dataflow. `node.rst` argues this under "Readability" ([line 119](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L119)), against frameworks that define steps first and wire them elsewhere. Under "Maintainability" ([line 123](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L123)) it argues that a change to a node definition reaches every dataflow that uses it.
+- Renaming a function or a parameter changes the graph. That rename is an application programming interface (API) change for any caller that requests the node or supplies it as an input.
+- Function names should be nouns that describe the result, not verbs, because the name is how the node is queried ([`docs/concepts/node.rst:67`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L67)).
+- Reusing one function body under several node names requires a function modifier (see 2611-03). [`writeups/basics.md:70`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/writeups/basics.md#L70) admits that the unique-name rule "is not optimal for functionality reuse". It accepts that cost because the rule "makes it extremely easy to learn exactly how a node in the DAG is generated".
+- A second function with the same name inside one module silently replaces the first, because Hamilton reads a module's members ([`hamilton/graph_utils.py:43`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/hamilton/graph_utils.py#L43)), where only the last definition survives. The refusal of duplicates applies across modules and across modifier outputs. This was known from the start ([36b18e54](https://github.com/apache/hamilton/commit/36b18e54f1163a653843df4d9d6c34e5e30c3394), 2019-09: "No two functions can have the same name (or they'll be overwritten)"), and a 2022 issue asking to catch it was closed because "we don't actually get returned multiple functions to know there are duplicates" ([stitchfix/hamilton#193](https://github.com/stitchfix/hamilton/issues/193)).
+
+## Alternatives considered
+
+- **Flexible, reusable transformations** rather than one implementation per name. Rejected in the 2020 README, for "DAG clarity, easy modifications, and documentation".
+- **Define steps, then wire them explicitly**, as in "popular orchestration / pipeline / workflow frameworks (Airflow, Kedro, Prefect, VertexAI, SageMaker, etc.)" ([`docs/concepts/node.rst:113`](https://github.com/apache/hamilton/blob/ef88b1e96140a01f3d72967316c2335f8b62a067/docs/concepts/node.rst#L113)). Rejected for readability and maintainability, in the 2024 documentation's words.
+- **Adopt existing open-source tooling.** The origin post says the team surveyed "a variety of offerings" and found none that solved the problems. Which tools it evaluated is not recorded.
+- **Plain functions with no framework.** The origin post rejected them as "merely a tool" without guard-rails.
+- **Dependencies selected by tag rather than by name.** Proposed in 2022 ([stitchfix/hamilton#226](https://github.com/stitchfix/hamilton/issues/226)). Elijah ben Izzy noted that it broke two of Hamilton's rules: "(a) that we can always determine the exact dependencies and (b) that functions have a fixed number of dependencies" ([comment](https://github.com/stitchfix/hamilton/issues/226#issuecomment-1301002225)), and Stefan Krawczyk's "bias here is for an API that's readable and understandable by people with little context" ([comment](https://github.com/stitchfix/hamilton/issues/226#issuecomment-1315930049)). It was not built.
+- **Redefining a node under the same name.** Proposed in 2023 ([stitchfix/hamilton#290](https://github.com/stitchfix/hamilton/issues/290)): it "makes implementation/onboarding easier, but doesn't help readability/maintainability of pipeline". Closed in favor of `@pipe`, which "gaurentees immutability, but allows chaining/redefinition" ([comment](https://github.com/apache/hamilton/issues/65#issuecomment-1769990243), on the migrated issue).