Changelog
0.11.0
Weighting options
- The
treated_weightargument is removed. The ATT of a treated state in a period is weighted within its sub-aggregate ATT by that state's number of observations in that period underweighting = "both"and"att", and equally under"diff"and"none". This is what the defaulttreated_weight = "treated"did, so results obtained with the default are unchanged;treated_weight = "population"is no longer available. - Each value of
weightingnow has one meaning:"diff"weights the long differences of the control states,"att"weights the ATTs (within each sub-aggregate ATT and within the aggregate ATT),"both"does both and"none"neither.
Edge case standard errors under ccc = "int"
- With covariates, these standard errors are now conditional on the observed covariates: the variance of the ATT is the sum, over the four state-period cells involved, of the variance of each cell's fitted value at the covariate means of the treated state, with those means treated as fixed. 0.10.1 also included the sampling variability of the covariate means. Only standard errors computed with
edgecase = trueunderccc = "int"with covariates change; no ATT changes.
Jackknife standard errors
- Jackknife standard errors are now centered at the mean of the delete-one estimates (the CV3J estimator; Karim & Webb 2025, eq. 87) instead of at the full-sample estimate (CV3) as in 0.10.1. They are never larger than before, and all jackknife standard errors and p-values change.
Randomization inference p-values
- The p-values are kernel-smoothed when fewer than
ri_kernel_cutoffpermutations are used (default 99), as in Karim & Webb (2025). In 0.10.1 they were smoothed whenri_kernel_cutoffor fewer were used, so results change only when exactly 99 permutations are used (or exactly the value given).
didint_plot()
didint_plot()now returns aDiDIntPlotinstead of a DataFrame. One call prepares the data for every kind of plot, each in a table (a DataFrame) of theDiDIntPlot:conditional: the conditional parallel trends data as before, the covariate-adjusted mean outcome of every state and period (lambda) under everycccoption except"int";conditional2: for each treated state, the change inlambdaof that state and of its control states since the period before its treatment (delta);integrated: the data of the integrated parallel trends plot of Karim & Webb (2025, eqs. 36-38) underccc = "int": for each treated state (treated_state_covars), the change from one period to the next (delta) in the predicted outcome of that state and of its control states (state), all evaluated at that treated state's covariate means;integrated2: the same changes measured from the period before the treated state's treatment;event: the event study data.
In
conditional2andintegrated2, the change of a treated state minus the average change of its control states is the ATT thatdidint()estimates in each treated period, and a placebo estimate in each period before treatment.New
plot_typeargument, which replacesevent:"all"(the default) or any combination of"event","conditional","conditional2","integrated"and"integrated2", the kinds of plot data to prepare. Every kind except"conditional"needs the treated states (treated_statesorgvar); without them only the conditional data are prepared, with a warning.New
notyetargument: withnotyet = truethe control states of a treated state also include the states of later cohorts while they are not yet treated, as indidint().The event study now uses the first stage estimated on all states, as
didint()and the other plot data do; before, it was estimated on the treated states only. Its estimates change under thecccoptions"hom","time"and"add".The tables have a column
n, the number of observations behind each row.The
wrapperargument ofdidint_plot()is removed: the Stata and R wrappers receive theDiDIntPlotlike every other caller, and need updating to read its tables. Its dates are strings (formatted withdate_format), as the wrappers need.New: a
DiDIntPlotcan be drawn withplotonce a Makie backend such as CairoMakie is loaded (Makie is an optional dependency, loaded as a package extension). Itsplot_typeargument chooses among five plots:"event": the event study, one panel percccoption;"conditional": the conditional parallel trends, one panel percccoption with a line for every state or, withfacet = "state", one panel per treated state andcccoption that compares the treated state with its control states;"conditional2": the conditional parallel trends as changes since the pre-treatment period, one panel per treated state andcccoption (a row of panels per treated state);"integrated": the integrated parallel trends, one panel per treated state, as changes since the previous period;"integrated2": the same as changes since the pre-treatment period.
In the panels of a treated state, the control states are drawn as their average, weighted by their numbers of observations (
weights = falseweights them equally), or each on its own withcontrols = "separate".treated_statesandcccselect what is shown, andwindowlimits the periods.When
treated_states(orgvar) is given, it is now checked in the same way whichever kinds of plot data are asked for, as the plots use it to mark the treated states.
Warnings about collinear covariates and rank deficiency
- Under
ccc = "int", a warning is now given only when ATTs depend on a dropped covariate. It gives the number of ATTs affected and the covariates responsible. A cell's coefficients are evaluated at the covariate means of the treated state, and a covariate dropped from the cell's regression (because it does not vary in the cell, or is collinear with covariates listed before it) matters only when the cell's data cannot determine its fitted value at those means. In 0.10.1 every covariate dropped as collinear was reported, whether or not it affected an ATT. - Under
ccc = "add", the warning about rank deficiency beyond the split between states and periods is given only when an ATT depends on which least-squares solution of the first stage is used. Every direction in which the solution is undetermined is checked. In 0.10.1 the warning was given whenever the rank was below the expected rank, also when no ATT was affected. - Under
"hom","state"and"time"the warning is reworded; it is given in the same cases as before.
Bug fixes
- Under
ccc = "add", which columns of the first-stage regression count as linearly dependent is now decided by the same relative tolerance as the collinearity rule of the other options, after scaling every column to unit length. With a covariate that has a very large mean and is collinear with another, the rounding error left by demeaning could count as a covariate of its own, and the estimates then changed with the order of the rows of the data (and could differ between platforms). Estimates are otherwise unchanged.
Documentation
- The estimation procedure is described in five steps: the average of the ATTs within each sub-aggregate ATT and the average of the sub-aggregate ATTs are now separate steps.
- Shorter, rewritten sections on weighting, rank deficiency, the edge case standard errors of two-way DID-INT with covariates, how alternative assignments are drawn, and how randomization inference and the jackknife are computed quickly. The example of the aggregate ATT across aggregation options now uses the default weighting.
- The Details page describes the kinds of plot data of
didint_plot()with their formulas, and how to read the placebo estimates before treatment; the Examples page shows the tables of plot data; the README shows one plot of each kind. - A new Plotting Examples page shows every kind of plot and its options, the README shows three of them, and the Details page describes what the plots draw.
0.10.1
Naming
- The model with state-specific covariate coefficients is now called region-varying DID-INT (previously state-varying), and the model with coefficients specific to each state and period two-way DID-INT. The printed summary uses these names. The
cccvalues ("state","int") and every other argument and column name are unchanged. - DID-INT is spelled out as "integrated difference-in-differences" throughout.
Collinear covariates
Collinear covariates are dropped in the order of
covariatesunderccc = "hom","state","time"and"int": a covariate is dropped where it is a linear combination of the covariates listed before it, in the part of the data over which its coefficient is estimated (all data, a state, a period, or a state-period cell). The rule uses a relative tolerance and no column pivoting, so the estimates no longer depend on the units in which covariates are measured, and they can be changed only by reorderingcovariates.A warning names the covariates that were dropped as collinear and where. Covariates that do not vary within any state-period cell of that part of the data are still dropped silently, as they cannot affect the estimates.
Changes to estimates. Estimates change only where covariates are collinear:
- Under
"hom","state"and"time", no covariate was dropped in 0.10.0. Collinear covariates have infinitely many coefficient vectors that fit equally well, and 0.10.0 used the one with the smallest sum of squared coefficients (the minimum-norm least-squares solution). For example, withx2 = 2 * x1and a combined effect of 5 per unit ofx1, it gavex1a coefficient of 1 andx2a coefficient of 2. That choice changes when a covariate is rescaled (dollars to thousands of dollars), and so did the ATTs. 0.10.1 dropsx2and givesx1a coefficient of 5, whatever the units. - Under
"int", 0.10.0 already dropped collinear covariates within a cell, but chose which one by a column-pivoted QR decomposition, which favours the covariates with the most variation and therefore also depended on units. Estimates change in cells where it dropped a covariate other than the one listed last.
Estimates without collinear covariates, and all estimates under
"add", are unchanged.- Under
edgecase = trueno longer changes any ATT. Withedgecase = true, the cells involved in an edge case are estimated by a separate regression that also yields the covariance matrix needed for the standard error. In 0.10.0 that regression handled collinear covariates differently from the regression used for all other cells, so with collinear covariates the ATTs could depend on whetheredgecasewas on. Both now use the same covariates, determined once, and return identical ATTs.Fewer missing edge case standard errors. An edge case standard error was
missingin 0.10.0 when collinear covariates made that separate regression rank deficient. It is now computed.ccc = "add"warns about unusual rank deficiency. Its first-stage regression is always rank deficient in a way that does not affect the estimates. A warning is now given when it is rank deficient beyond that (for example, a covariate that does not vary within any cell of some state or period), as the ATTs may then depend on the least-squares solution used.
Edge case standard errors under ccc = "int" with covariates
- Bug fix: these standard errors were slightly too large. The variance of the ATT of a treated state in a period with a single control state has three terms, the third being a trace term for the product of two estimation errors. The first two terms are computed with estimates in place of unknown true values, which already makes each of them too large by that trace term on average. 0.10.0 added the trace term on top; it is now subtracted, which gives an estimate that is right on average. The difference is of the order of a few percent of the variance in cells with around ten observations and negligible in large cells. Only standard errors computed with
edgecase = trueunderccc = "int"with covariates change; no ATT changes. The Details page has the full derivation.
Jackknife warnings
- When removing a state leaves a sub-aggregate ATT (or a policy) without any treated state, its jackknife standard error is
missingand no warning is given. This reverses the warning introduced in 0.10.0. The jackknife standard error of the aggregate ATT (or the pooled ATT) is still reported. - The warning that the jackknife of the aggregate ATT averages over different sets of sub-aggregate ATTs is removed.
Documentation
- This changelog is now a page of the documentation site.
- The Details page has rewritten sections on weighting (as three successive averages), on the edge case standard errors of two-way DID-INT with covariates, and on how alternative assignments are drawn for randomization inference, and new sections on rank deficiency and on how randomization inference and the jackknife are computed quickly.
- The Examples page has new examples on cohort ATTs with and without policies and on the aggregate ATT across aggregation options, and explains when the parallel trends plots are valid.
- The
edgecasedocumentation states that edge case standard errors are generally biased with panel data.
0.10.0
This version changes how the estimates are computed and aggregated, so results differ from those of 0.9.6 in the cases listed under "Changes to estimates". The documentation site describes the current procedure in detail.
didint() returns a DiDIntResult
didint()now returns aDiDIntResultinstead of aDataFrame. Displaying it prints a summary of the specification and the estimates, and its fields hold the estimates as tables:agg(the aggregate ATT, or the ATT of each policy and the pooled ATT),sub(the sub-aggregate ATTs),att_st(the ATT of each treated state in each period) andspecs(the specification).coef,stderrorandcoefnamesreturn the estimates, their standard errors and their labels as vectors.- To update existing code:
DataFrame(didint(...))returns the table that earlier versions returned, with the same column names. - The Stata and R wrappers (
wrapper = "stata"or"r") still receive that table.
Changes to estimates
- ATT for each treated state and period first. An ATT is now estimated for every treated state in every period from its treatment time on, and every sub-aggregate ATT is a weighted mean of those. The aggregate ATT is a weighted mean of the sub-aggregate ATTs. Previously each sub-aggregate ATT came from a regression that pooled long differences.
agg = "sgt"estimates and standard errors are unchanged.agg = "simple"estimates are unchanged withtreated_weight = "population"; its standard errors change for groups with more than one treated state.agg = "cohort","state"and"time"estimates change in general.
- New
treated_weightargument, with default"treated": an ATT is weighted within its sub-aggregate by the treated state's observations in that period."population"gives the previous weighting (the observations of that period and of the period before treatment). - Integrated long differences (Karim & Webb 2025, eqs. 46-47). Under
ccc = "int"with covariates, long differences are evaluated at the treated state's covariate means, which changes the estimates. Estimates under the othercccoptions, and without covariates, are unaffected. - Common adoption is handled like staggered adoption, as the case of a single treatment cohort. The data is no longer collapsed into one pre-treatment and one post-treatment period: long differences are taken from the period immediately before treatment, an ATT is estimated for each post-treatment period, and every
aggoption is available. All common adoption results change. - Bug fix: with staggered adoption, the long differences that treated states contribute to other treatment cohorts were computed for the wrong periods. This affected estimates with
notyet = trueand all randomization inference p-values.
New features
agg = "calendar": sub-aggregate ATTs by period.- Policies. New arguments
pvarandtreated_policiesgive the policy of each treated state, and new aggregations"policy","policy_gt","policy_cohort","policy_calendar"and"policy_time"report an ATT for each policy (att_p).policy_aggregation = trueadds a pooled ATT across policies (att_pool), weighted as set bypolicy_weighting. - Exact randomization inference. When no more than
npermalternative treatment assignments exist, all of them are used. - Kernel-smoothed randomization inference p-values (Racine & MacKinnon 2007) when
ri_kernel_cutoffor fewer permutations are used (default 99). Setri_kernel_cutofftonothing,falseor0to turn this off. edgecasestandard errors now apply to any ATT of a treated state and period that has a single control state, whatever theaggoption (including"time"), and account for the covariate terms underccc = "int".
Changes to inference
- The randomization inference p-value is now (1 + number of placebo estimates at least as large in absolute value as the estimate) / (number of permutations + 1). Previously it was the share of placebo estimates strictly larger than the estimate. Ties count against rejecting, with a small tolerance for rounding.
- Permutations are drawn by a faster method and from a random number generator that only depends on
seed. Randomization inference p-values for a givenseedtherefore differ from earlier versions, and when aseedis givendidint()no longer changes the global random number generator. - Under
agg = "state"and"sgt", each treated state is represented by the same randomly drawn state in all of its periods within a permutation. - A jackknife standard error is reported for an estimate only if it involves at least two treated states and two control states.
- HC2, HC3 and HC4 standard errors of the ATT of a single treated state and period are
NaN(the treated state has leverage one).
Removed
agg = "none": everyaggoption reports the aggregate ATT inagg_att.- The
verboseargument: the randomization inference no longer prints its progress.
Other
- The jackknife and the randomization inference are computed for all deletions and permutations at once, and the long differences are built from lookups, which makes
didint()several times faster. - Bug fixes:
truejack = truewithagg = "time"no longer errors; string covariates with exactly two categories are handled correctly;edgecase = truewithccc = "add"is switched off with a warning.
Bug fixes that can change estimates
These affect particular kinds of data; results for other data are unchanged by them.
- Covariates measured at the state-period level. A covariate that is constant within every state-period cell should have no effect, but under
ccc = "hom","state","time"and"add"it could change the estimates substantially (rounding noise left by the within-cell demeaning was being fitted). Such covariates now have no effect. - Categorical covariates under
ccc = "int". When the reference category of a categorical covariate does not occur in a state-period cell, that cell's dummy variables add up to one. They were all kept, which made the estimates depend on the level of the outcome. One of them is now dropped in that cell. - Treated states with a treatment time outside the data. A state treated at or before the first date of the data (or of
start_date) was silently used as a control. Its observations are now dropped, with a warning. A state treated after the last date is used as a control, as before, now with a warning; previously a treatment time shortly after the last date could be counted as occurring in the last period. - Automatic period length. For data spaced several years apart (for example every five or ten years) the detected period could be one year too long. Specifying
freqandfreq_multiplierwas not affected. - Randomization inference with incomplete data. The randomization inference could run although a treated state was not observed in a period that it needs under alternative assignments. Its p-values are now
missingin that case, with a warning.
Other bug fixes
hc = "hc4"no longer errors underweighting = "both"or"diff".- Listing a treated state more than once, or a
gvarcolumn that is not constant within a state, is now an error instead of silently changing the estimates. gvarworks with aDatetime column, and with rows whose state is missing.- Numbers given as floats are accepted for
hc,freq_multiplier,start_dateandend_date;date_formatis not case sensitive and an unrecognized format is an error. didint_plot(event = true)returns its rows sorted bycccand by period relative to treatment.didint_plot()accepts a single treatment time for several treated states, a single treated state, and covariates given as symbols.- A jackknife deletion that removes every treated state of a sub-aggregate ATT now gives a warning.
- The kernel-smoothed p-value falls back to the unsmoothed one when the placebo estimates are equal up to rounding.