fix(synthetic_control): put the treated unit in every in-space placebo's donor pool

changelog.d/20261005-in-space-placebo-donor-pool.md

@@ -0,0 +1,23 @@
+### Fixed
+- **`SyntheticControl` in-space placebo is now an exact permutation test**:
+  `in_space_placebo()` now fits each placebo against all the other units, the
+  real treated unit included, as Abadie, Diamond & Hainmueller (2010, §3.4,
+  p. 501) do ("shifting California to the donor pool"). It used to drop the
+  treated unit from every placebo's donor pool (the `SCtools::generate.placebos`
+  construction); that reference set changes with the treated unit, so the
+  permutation p-value was not exact. Every placebo that puts weight on the
+  treated unit now gets a different RMSPE ratio, so `placebo_p_value` and the
+  placebo rows of `get_placebo_df()` change on every such fit, by whole ranks;
+  the fit itself (`att`, weights, the treated unit's `rmspe_ratio`) does not
+  change. On the 10-unit no-effect panel of issue #837, 5 of the 10
+  assignments move by one rank (3 up, 2 down) and the 10 p-values become
+  exactly 0.1, 0.2, ..., 1.0; before, 2 assignments gave `p = 0.1`. Over 300
+  simulated 10-unit no-effect panels, with each unit treated in turn, the old
+  construction gave `p <= 0.1` for 11.7% of the assignments (Monte Carlo SE 0.25
+  points); the new one gives exactly 10% on every panel.
+  `test_sharp_null()` and `confidence_set()` re-rank the new placebo gaps and
+  use the treated unit's null-imputed outcome `Y_1 - f` inside each placebo's
+  synthetic control (Firpo & Possebom 2018, fn 10), so their p-values and sets
+  change too. With exactly 2 donors each placebo is now fit against 2 units
+  (a real weight solve and `V` search instead of a forced `w = [1]`); fewer
+  than 2 donors still gives a NaN p-value.

diff_diff/diagnostic_report.py

@@ -2964,8 +2964,8 @@ def _scm_native(self, r: Any) -> Dict[str, Any]:
                     ),
                     "too_few_donors": (
                         "in_space_placebo() was run but fewer than 2 donors are "
-                        "available (each placebo is fit against the other donors); "
-                        "placebo_p_value is NaN."
+                        "available (with one, the treated unit and the donor are each "
+                        "other's synthetic control); placebo_p_value is NaN."
                     ),
                     "all_placebos_failed": (
                         "in_space_placebo() was run but every donor refit failed to "

diff_diff/guides/llms-full.txt

@@ -675,7 +675,7 @@ scm.fit(
 ) -> SyntheticControlResults
 ```
 
-**Inference:** NONE analytical — `se`/`t_stat`/`p_value`/`conf_int` are always NaN. `att` is the mean post-period gap. Significance via in-space placebo permutation inference: `results.in_space_placebo()` reassigns treatment to each donor, refits against the other J-1 donors (the real treated unit is excluded from every placebo pool), and sets `placebo_p_value = rank/(n_placebos+1)` from the post/pre RMSPE-ratio. The permutation `placebo_p_value` is a SEPARATE field from the (NaN) `p_value`; `is_significant` stays bound to `p_value`. **ADH-2015 §4 robustness (opt-in, analytical inference unchanged):** `results.leave_one_out()` drops each reportably-weighted donor (weight > 1e-6) and re-fits (per-drop ATT/`delta_att` — large `delta_att` ⇒ single-donor dependence); `results.in_time_placebo()` backdates the intervention and checks for a spurious pre-period gap (TRUNCATE windowing — predictor windows in the held-out region are dropped); `results.regression_weights()` computes the implied regression-counterfactual weights `W^reg = X0a'(X0a X0a')^{-1}X1a` (intercept-augmented, sums to 1 at full row rank) and flags donors outside `[0,1]` — the extrapolation an OLS counterfactual incurs but the simplex-constrained SC cannot (pure linear algebra, no refit; min-norm + `_regw_rank_deficient` when `k+1>J`); `results.sparse_synthetic_control(sizes=None, max_subsets=50000)` exhaustively searches `C(J,l)` size-`l` donor subsets holding `V` FIXED at the baseline (default sweep `[1,2,3]` skips over-cap sizes with a warning; an explicitly-requested oversize `l` raises), reporting how fit/ATT degrade as the synthetic is forced sparse (+ `get_sparse_synthetic_control_gaps()`). **Confidence sets by test inversion (Firpo-Possebom 2018 §4, opt-in, also non-analytical):** `results.test_sharp_null(effect, gamma=0.1)` tests `H_0: α_1t = f(t)` by re-ranking the in-space placebo gaps (no refits; `test_sharp_null(0)` == `placebo_p_value`), and `results.confidence_set(family="constant"|"linear", gamma=0.1)` inverts it into a confidence set for the effect path (constant-effect interval / linear-slope set, strict `p>gamma`), surfaced on `effect_confidence_set` / `get_confidence_set_df()` — the analytical `conf_int` still stays NaN. **Conformal inference (Chernozhukov-Wüthrich-Zhu 2021, opt-in, also non-analytical):** unlike the Firpo path (which re-ranks the cross-unit placebo gaps), the conformal layer fits its OWN time-permutation-invariant constrained-LS proxy (eqs 3–4, no V-matrix) under the null on ALL periods and permutes residuals over time for the single treated unit. `results.conformal_test(effect, q=1, scheme="moving_block")` gives a joint sharp-null p-value (eqs 1–2; `q∈{1,2,∞}`); `results.conformal_confidence_intervals(alpha=0.1)` gives pointwise per-period CIs (Algorithm 1 — each period `t` uses `Z=(pre, t)`, the other post-periods dropped); `results.conformal_average_effect(alpha=0.1)` gives an average-effect CI by collapsing into `T*`-blocks (Appendix A.1). All three accept `alternative={"two-sided","greater","less"}` (one-sided uses the SIGNED average-effect statistic per CWZ Remark 1, q fixed at 1; CIs become half-lines `[lower,+inf)`/`(-inf,upper]`) and `covariates=[...]` (pivoted-variable matching rows stacked RAW into the constrained-LS proxy per the note after eq 6; residuals/p-value stay outcome-only). `scheme="moving_block"` (default, serial-dependence-robust) or `"iid"` (finer). Surfaced on `conformal_inference` / `get_conformal_grid_df()`; `conf_int` stays NaN. Predictor periods must lie within the pre window; `post_periods` must be a contiguous suffix cross-checked against `D` (no anticipation).
+**Inference:** NONE analytical — `se`/`t_stat`/`p_value`/`conf_int` are always NaN. `att` is the mean post-period gap. Significance via in-space placebo permutation inference: `results.in_space_placebo()` reassigns treatment to each donor, refits it against all the other units (the real treated unit is in every placebo pool, ADH 2010 §3.4, so the permutation is exact), and sets `placebo_p_value = rank/(n_placebos+1)` from the post/pre RMSPE-ratio. The permutation `placebo_p_value` is a SEPARATE field from the (NaN) `p_value`; `is_significant` stays bound to `p_value`. **ADH-2015 §4 robustness (opt-in, analytical inference unchanged):** `results.leave_one_out()` drops each reportably-weighted donor (weight > 1e-6) and re-fits (per-drop ATT/`delta_att` — large `delta_att` ⇒ single-donor dependence); `results.in_time_placebo()` backdates the intervention and checks for a spurious pre-period gap (TRUNCATE windowing — predictor windows in the held-out region are dropped); `results.regression_weights()` computes the implied regression-counterfactual weights `W^reg = X0a'(X0a X0a')^{-1}X1a` (intercept-augmented, sums to 1 at full row rank) and flags donors outside `[0,1]` — the extrapolation an OLS counterfactual incurs but the simplex-constrained SC cannot (pure linear algebra, no refit; min-norm + `_regw_rank_deficient` when `k+1>J`); `results.sparse_synthetic_control(sizes=None, max_subsets=50000)` exhaustively searches `C(J,l)` size-`l` donor subsets holding `V` FIXED at the baseline (default sweep `[1,2,3]` skips over-cap sizes with a warning; an explicitly-requested oversize `l` raises), reporting how fit/ATT degrade as the synthetic is forced sparse (+ `get_sparse_synthetic_control_gaps()`). **Confidence sets by test inversion (Firpo-Possebom 2018 §4, opt-in, also non-analytical):** `results.test_sharp_null(effect, gamma=0.1)` tests `H_0: α_1t = f(t)` by re-ranking the in-space placebo gaps (no refits; `test_sharp_null(0)` == `placebo_p_value`), and `results.confidence_set(family="constant"|"linear", gamma=0.1)` inverts it into a confidence set for the effect path (constant-effect interval / linear-slope set, strict `p>gamma`), surfaced on `effect_confidence_set` / `get_confidence_set_df()` — the analytical `conf_int` still stays NaN. **Conformal inference (Chernozhukov-Wüthrich-Zhu 2021, opt-in, also non-analytical):** unlike the Firpo path (which re-ranks the cross-unit placebo gaps), the conformal layer fits its OWN time-permutation-invariant constrained-LS proxy (eqs 3–4, no V-matrix) under the null on ALL periods and permutes residuals over time for the single treated unit. `results.conformal_test(effect, q=1, scheme="moving_block")` gives a joint sharp-null p-value (eqs 1–2; `q∈{1,2,∞}`); `results.conformal_confidence_intervals(alpha=0.1)` gives pointwise per-period CIs (Algorithm 1 — each period `t` uses `Z=(pre, t)`, the other post-periods dropped); `results.conformal_average_effect(alpha=0.1)` gives an average-effect CI by collapsing into `T*`-blocks (Appendix A.1). All three accept `alternative={"two-sided","greater","less"}` (one-sided uses the SIGNED average-effect statistic per CWZ Remark 1, q fixed at 1; CIs become half-lines `[lower,+inf)`/`(-inf,upper]`) and `covariates=[...]` (pivoted-variable matching rows stacked RAW into the constrained-LS proxy per the note after eq 6; residuals/p-value stay outcome-only). `scheme="moving_block"` (default, serial-dependence-robust) or `"iid"` (finer). Surfaced on `conformal_inference` / `get_conformal_grid_df()`; `conf_int` stays NaN. Predictor periods must lie within the pre window; `post_periods` must be a contiguous suffix cross-checked against `D` (no anticipation).
 
 **Usage:**
 

diff_diff/synthetic_control.py

@@ -1861,6 +1861,8 @@ def _sharp_null_pvalue(
     f_post: np.ndarray,
     pre_denoms: Dict[Any, float],
     treated_id: Any,
+    *,
+    treated_weights: Optional[Dict[Any, float]] = None,
 ) -> Tuple[float, float, int]:
     """Permutation p-value for a common-effect sharp null (Firpo & Possebom Eq 13, φ=0, v=1).
 
@@ -1870,13 +1872,20 @@ def _sharp_null_pvalue(
     to its gap path; ``pre_denoms`` maps each unit (treated + those donors) to its floored
     pre-denominator. Ties counted via ``>=`` so the p-value is conservative, matching
     ``in_space_placebo``. Returns ``(p, rmspe_f_treated, n_ref)``.
+
+    ``treated_weights`` maps each placebo ``j`` to the weight ``w_j1`` its synthetic control
+    puts on the real treated unit, which is in every placebo's donor pool. Under the null the
+    treated unit's untreated outcome is ``Y_1 − f`` (Firpo & Possebom fn 10: the effect is known),
+    so placebo ``j``'s post gap moves by ``+w_j1·f`` and its residual is ``g_j − (1 − w_j1)·f``.
+    ``None`` means ``w_j1 = 0`` for every placebo.
     """
     f_post = np.asarray(f_post, dtype=float)
     r1 = _rmspe_f_ratio(treated_gap, post_periods, f_post, pre_denoms[treated_id])
     n_ref = 0
     n_ge = 0
     for j, g in placebo_gaps.items():
-        rj = _rmspe_f_ratio(g, post_periods, f_post, pre_denoms[j])
+        k = 1.0 if treated_weights is None else 1.0 - treated_weights[j]
+        rj = _rmspe_f_ratio(g, post_periods, k * f_post, pre_denoms[j])
         n_ref += 1
         if rj >= r1:
             n_ge += 1
@@ -1895,6 +1904,7 @@ def _invert_sharp_null(
     *,
     bounds: Optional[Tuple[float, float]] = None,
     n_grid: int = 200,
+    treated_weights: Optional[Dict[Any, float]] = None,
 ) -> Dict[str, Any]:
     """Invert the sharp-null RMSPE^f test over a one-parameter effect family.
 
@@ -1910,6 +1920,8 @@ def _invert_sharp_null(
     ``p > gamma`` is the set -- correctly handling accepted tails, disjoint components, and the
     empty / unbounded cases. With explicit ``bounds`` a fixed ``linspace(*bounds, n_grid)``
     grid is scanned instead (grid-limited membership). Each ``p`` is a pure O(J) re-ranking.
+    ``treated_weights`` is as in :func:`_sharp_null_pvalue`: placebo ``j``'s residual is
+    ``g_j − (1 − w_j1)·param·s_t``.
 
     Result dict: ``{lower, upper, contiguous, status, n_ref, point_estimate, grid}`` where
     ``status`` is ``"ran"`` / ``"empty"`` / ``"unbounded"``, ``point_estimate`` is the treated
@@ -1920,21 +1932,28 @@ def _invert_sharp_null(
     if family not in ("constant", "linear"):
         raise ValueError(f"family must be 'constant' or 'linear', got {family!r}")
     n_post = len(post_periods)
-    # Family shape s_t and per-unit moments of A_u(param) = mean_t((g_ut - param*s_t)**2)
-    # = S2*param**2 - 2*P_u*param + Q_u  (s_t = 1 constant / (t - T0) linear; S2 = mean(s**2)).
+    # Family shape s_t and per-unit moments of A_u(param) = mean_t((g_ut - k_u*param*s_t)**2)
+    # = k_u**2*S2*param**2 - 2*k_u*P_u*param + Q_u  (s_t = 1 constant / (t - T0) linear;
+    # S2 = mean(s**2); k_u = 1 for the treated unit, 1 - w_j1 for placebo j).
     s = (
         np.ones(n_post, dtype=float)
         if family == "constant"
         else np.arange(1, n_post + 1, dtype=float)
     )
     S2 = float(np.mean(s**2)) if n_post else 0.0
 
-    def moments(gap_path: Dict[Any, float], unit: Any) -> Tuple[float, float, float]:
+    def moments(
+        gap_path: Dict[Any, float], unit: Any, k: float = 1.0
+    ) -> Tuple[float, float, float, float]:
+        # (k*P_u, Q_u, D_u, k**2*S2): the k-scaled moments of A_u.
         g = np.array([gap_path[p] for p in post_periods], dtype=float)
-        return float(np.mean(g * s)), float(np.mean(g**2)), pre_denoms[unit]
+        return k * float(np.mean(g * s)), float(np.mean(g**2)), pre_denoms[unit], k * k * S2
 
-    P1, Q1, D1 = moments(treated_gap, treated_id)
-    placebo_mom = [moments(gp, j) for j, gp in placebo_gaps.items()]
+    P1, Q1, D1, _ = moments(treated_gap, treated_id)
+    placebo_mom = [
+        moments(gp, j, 1.0 if treated_weights is None else 1.0 - treated_weights[j])
+        for j, gp in placebo_gaps.items()
+    ]
     n_ref = len(placebo_mom)
     # The treated unit's own RMSPE-minimizing param: reported as the point estimate ONLY
     # (it is NOT assumed to maximize p -- see the exact-inversion comment below).
@@ -1946,7 +1965,9 @@ def pval(param: float) -> float:
         # _sharp_null_pvalue / test_sharp_null (the squared form just avoids the sqrt).
         r1 = ((S2 * param - 2.0 * P1) * param + Q1) / D1
         n_ge = sum(
-            1 for (Pj, Qj, Dj) in placebo_mom if ((S2 * param - 2.0 * Pj) * param + Qj) / Dj >= r1
+            1
+            for (Pj, Qj, Dj, S2j) in placebo_mom
+            if ((S2j * param - 2.0 * Pj) * param + Qj) / Dj >= r1
         )
         return (1 + n_ge) / (n_ref + 1)
 
@@ -2023,8 +2044,8 @@ def quad_roots(a: float, b: float, c: float) -> List[float]:
     # 1[g_j(param) >= 0] == 1[RMSPE^param_j >= RMSPE^param_1], and breakpoints (where an
     # indicator flips) are the real roots of each g_j.
     coeffs = [
-        (S2 * (D1 - Dj), -2.0 * (Pj * D1 - P1 * Dj), Qj * D1 - Q1 * Dj)
-        for (Pj, Qj, Dj) in placebo_mom
+        (S2j * D1 - S2 * Dj, -2.0 * (Pj * D1 - P1 * Dj), Qj * D1 - Q1 * Dj)
+        for (Pj, Qj, Dj, S2j) in placebo_mom
     ]
     raw_breaks: List[float] = []
     for a, b, c in coeffs:
@@ -2146,17 +2167,18 @@ def _placebo_fit_unit(
     unit: Any,
     donor_pool: List[Any],
     n_starts: int,
-) -> Tuple[Optional[Tuple[Dict[Any, float], float]], str]:
+) -> Tuple[Optional[Tuple[Dict[Any, float], float, Dict[Any, float]]], str]:
     """Refit a synthetic control for one (pseudo-)treated ``unit`` vs ``donor_pool``.
 
     Reuses the exact predictor build / standardization / weight solve / gap-path
     path as ``fit()`` — none of those helpers reads estimator (``self``) state, so
-    the refit is driven entirely by the snapshot. The real treated unit is never in
-    ``donor_pool`` (the caller passes the other ``J−1`` donors).
+    the refit is driven entirely by the snapshot. ``donor_pool`` may contain the real
+    treated unit: the in-space placebo passes every unit except ``unit`` (ADH 2010 §3.4).
 
     Returns ``(result, status)`` where ``status`` is one of:
 
-    - ``"ran"`` — ``result`` is ``(gap_path, rmspe_ratio)`` (a valid optimum).
+    - ``"ran"`` — ``result`` is ``(gap_path, rmspe_ratio, weights)`` (a valid optimum);
+      ``weights`` maps each ``donor_pool`` unit to its donor weight.
     - ``"infeasible"`` — ``result`` is ``None``; the refit is STRUCTURALLY
       unidentified (non-finite cells, or — under ``v_method="cv"`` — a re-aggregated
       window with no cross-donor variation for this pseudo-treated pool). Remedied by
@@ -2266,7 +2288,8 @@ def _placebo_fit_unit(
     pre_gaps = np.array([gap_path[p] for p in snap.pre_periods], dtype=float)
     post_gaps = np.array([gap_path[p] for p in snap.post_periods], dtype=float)
     scale = float(np.max(np.abs(Z1))) if Z1.size else 0.0
-    return (gap_path, _rmspe_ratio(pre_gaps, post_gaps, scale)), "ran"
+    weights = {d: float(w_d) for d, w_d in zip(donor_pool, w)}
+    return (gap_path, _rmspe_ratio(pre_gaps, post_gaps, scale), weights), "ran"
 
 
 # =============================================================================