Examples
The examples below use the merit.csv dataset included in the package's test suite (test/data/merit.csv). It contains individual-level survey data on college attendance (coll), state identifiers (state), survey year (year), and demographic covariates (male, asian, black). The dataset covers U.S. states over the period 1989–2000 and is used to study the effect of merit scholarships on college attendance.
using DiDInt, CSV, DataFrames
merit = CSV.read(joinpath(@__DIR__, "../../test/data/merit.csv"), DataFrame);
first(merit, 5)| Row | coll | merit | male | black | asian | year | state | chst | styr | eta | alpha | stch | stchid | ind | stnchid | stid |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Float64 | Float64 | Float64 | Float64 | Float64 | Int64 | Int64 | Float64 | Float64 | Float64 | Float64 | Float64 | Float64? | Float64 | Float64? | Float64 | |
| 1 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1993 | 11 | 0.0 | 12993.0 | -0.383748 | 0.0337465 | 0.0 | missing | 52.0 | 11.0 | 1.0 |
| 2 | 1.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1997 | 11 | 0.0 | 12997.0 | 0.627288 | 0.0337465 | 0.0 | missing | 52.0 | 11.0 | 1.0 |
| 3 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1992 | 11 | 0.0 | 12992.0 | 0.547032 | 0.0337465 | 0.0 | missing | 52.0 | 11.0 | 1.0 |
| 4 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1989 | 11 | 0.0 | 12989.0 | -0.363045 | 0.0337465 | 0.0 | missing | 52.0 | 11.0 | 1.0 |
| 5 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1991 | 11 | 0.0 | 12991.0 | -0.372118 | 0.0337465 | 0.0 | missing | 52.0 | 11.0 | 1.0 |
Staggered Adoption
This example illustrates a staggered adoption setting in which ten states adopted merit-aid programmes at different points between 1991 and 2000.
Running didint()
# Note that treated_states & state column in the merit data could alternatively
# be string vectors, here they just happen to be numeric codes instead of state names
treated_states = [34, 57, 58, 59, 61, 64, 71, 72, 85, 88];
treated_times = [2000, 1998, 1993, 1997, 1999, 1996, 1991, 1998, 1997, 2000];
result = didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = "cohort",
nperm = 399
)DID-INT results
Outcome: coll
Model: Region-Varying DID-INT (ccc = "state")
Covariates: male, asian, black
Aggregation: "cohort" (by treatment time)
Weighting: "both"
Controls: never-treated states
Periods: 1 year, from 1989 to 2000
Standard errors: HC1 (se) and jackknife (jknife_se)
Permutations: 399 (seed 1234)
Aggregate ATT
estimate att se pval jknife_se jknife_pval ri_pval
────────────────────────────────────────────────────────────────────────────────────
Aggregate ATT 0.0462414 0.00769881 0.000959237 0.0123516 0.00046902 0.0975
Sub-aggregate ATTs by treatment time (7)
treatment_time att se pval jknife_se jknife_pval ri_pval weight
──────────────────────────────────────────────────────────────────────────────────────────────────────────
1991-01-01 0.0553354 0.0222024 0.0342914 missing missing 0.4575 0.201796
1993-01-01 0.0504489 0.0161905 0.0169391 missing missing 0.5175 0.191535
1996-01-01 0.0424116 0.0255707 0.172536 missing missing 0.5725 0.0756734
1997-01-01 0.0606879 0.0236147 0.037012 0.0328756 0.0708186 0.28 0.321077
1998-01-01 0.0398137 0.0406649 0.372511 0.0813398 0.626647 0.5825 0.108593
1999-01-01 0.00173622 0.00350574 0.707256 missing missing 0.985 0.0354853
2000-01-01 -0.0253317 0.0333203 0.586179 0.0971445 0.795346 0.755 0.0658401didint() returns a DiDIntResult. Displaying it, as above, prints a summary of the specification, the aggregate ATT, and the sub-aggregate ATTs (here, one per treatment time). Jackknife standard errors are missing for cohorts with a single treated state (see the Jackknife section).
Working with the results
The estimates are stored as tables in the fields of the result. The aggregate ATT with its standard errors and p-values:
result.agg| Row | estimate | att | se | pval | jknife_se | jknife_pval | ri_pval |
|---|---|---|---|---|---|---|---|
| String | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | |
| 1 | Aggregate ATT | 0.0462414 | 0.00769881 | 0.000959237 | 0.0123516 | 0.00046902 | 0.0975 |
The sub-aggregate ATTs, with the weight of each in the aggregate ATT:
result.sub| Row | treatment_time | att | se | pval | jknife_se | jknife_pval | ri_pval | weight |
|---|---|---|---|---|---|---|---|---|
| Date | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | |
| 1 | 1991-01-01 | 0.0553354 | 0.0222024 | 0.0342914 | missing | missing | 0.4575 | 0.201796 |
| 2 | 1993-01-01 | 0.0504489 | 0.0161905 | 0.0169391 | missing | missing | 0.5175 | 0.191535 |
| 3 | 1996-01-01 | 0.0424116 | 0.0255707 | 0.172536 | missing | missing | 0.5725 | 0.0756734 |
| 4 | 1997-01-01 | 0.0606879 | 0.0236147 | 0.037012 | 0.0328756 | 0.0708186 | 0.28 | 0.321077 |
| 5 | 1998-01-01 | 0.0398137 | 0.0406649 | 0.372511 | 0.0813398 | 0.626647 | 0.5825 | 0.108593 |
| 6 | 1999-01-01 | 0.00173622 | 0.00350574 | 0.707256 | missing | missing | 0.985 | 0.0354853 |
| 7 | 2000-01-01 | -0.0253317 | 0.0333203 | 0.586179 | 0.0971445 | 0.795346 | 0.755 | 0.0658401 |
The ATT for each treated state in each period from its treatment time on, which is what every other estimate is computed from:
first(result.att_st, 5)| Row | state | gvar | t | r1 | periods_post_treat | att | se | pval | n_t | n | n_controls |
|---|---|---|---|---|---|---|---|---|---|---|---|
| String | Date | Date | Date | Int64 | Float64? | Float64? | Float64? | Int64 | Int64 | Int64 | |
| 1 | 34 | 2000-01-01 | 2000-01-01 | 1999-01-01 | 0 | -0.0455338 | 0.0168927 | 0.0102302 | 127 | 257 | 41 |
| 2 | 57 | 1998-01-01 | 1998-01-01 | 1997-01-01 | 0 | 0.0557499 | 0.0172992 | 0.00252733 | 37 | 72 | 41 |
| 3 | 57 | 1998-01-01 | 1999-01-01 | 1997-01-01 | 1 | -0.0453683 | 0.0170296 | 0.0110738 | 34 | 69 | 41 |
| 4 | 57 | 1998-01-01 | 2000-01-01 | 1997-01-01 | 2 | -0.13436 | 0.0136416 | 2.98717e-12 | 25 | 60 | 41 |
| 5 | 58 | 1993-01-01 | 1993-01-01 | 1992-01-01 | 0 | 0.0311176 | 0.015669 | 0.0539236 | 58 | 103 | 41 |
The estimates and standard errors are also available as vectors:
coef(result), stderror(result), stderror(result; type = :jackknife)(Union{Missing, Float64}[0.0462413785365134], Union{Missing, Float64}[0.007698811001621377], Union{Missing, Float64}[0.012351552372901755])coefnames(result; level = :sub), coef(result; level = :sub)(["1991-01-01", "1993-01-01", "1996-01-01", "1997-01-01", "1998-01-01", "1999-01-01", "2000-01-01"], Union{Missing, Float64}[0.05533536915424712, 0.0504489015222826, 0.04241155528374173, 0.06068792505784653, 0.03981372957089521, 0.001736222501711654, -0.025331743559719733])Finally, every result can be collected in a single wide table with DataFrame(result) (the same table is passed to the Stata and R wrappers), and a result can be written straight to a file, for example with CSV.write("results.csv", result):
names(DataFrame(result))18-element Vector{String}:
"treatment_time"
"att_cohort"
"agg_att"
"se_agg_att"
"pval_agg_att"
"jknifese_agg_att"
"jknifepval_agg_att"
"ri_pval_agg_att"
"nperm"
"se_att_cohort"
"pval_att_cohort"
"jknifese_att_cohort"
"jknifepval_att_cohort"
"ri_pval_att_cohort"
"weights"
"period"
"start_date"
"end_date"Other aggregations
The agg argument controls how the ATTs for each treated state and period are grouped into sub-aggregate ATTs (see the Aggregation section). For example, agg = "time" groups them by the number of periods since treatment:
result_time = didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = "time",
nperm = 399
);
result_time.sub| Row | periods_post_treat | att | se | pval | jknife_se | jknife_pval | ri_pval | weight |
|---|---|---|---|---|---|---|---|---|
| Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | |
| 1 | 0.0 | 0.0370508 | 0.0276228 | 0.212682 | 0.0327185 | 0.262863 | 0.28 | 0.259513 |
| 2 | 1.0 | 0.0215869 | 0.0190731 | 0.294996 | 0.0231604 | 0.355784 | 0.5675 | 0.200513 |
| 3 | 2.0 | 0.047384 | 0.0237336 | 0.0928767 | 0.0266613 | 0.0816087 | 0.1875 | 0.178281 |
| 4 | 3.0 | 0.0708091 | 0.0290342 | 0.071299 | 0.0418609 | 0.0969594 | 0.13 | 0.138948 |
| 5 | 4.0 | 0.102901 | 0.0363787 | 0.105561 | 0.0482397 | 0.0378461 | 0.1175 | 0.0589996 |
| 6 | 5.0 | 0.110873 | 0.00448061 | 0.025713 | 0.0160936 | 8.94285e-9 | 0.195 | 0.0474562 |
| 7 | 6.0 | 0.0616531 | 0.0213678 | 0.212393 | 0.0345481 | 0.080401 | 0.425 | 0.0418982 |
| 8 | 7.0 | 0.0144139 | 0.075829 | 0.880416 | 0.114065 | 0.89995 | 0.86 | 0.0376229 |
| 9 | 8.0 | -0.0733538 | 0.015486 | 2.72849e-5 | missing | missing | 0.4575 | 0.0175289 |
| 10 | 9.0 | 0.0436031 | 0.016952 | 0.0139265 | missing | missing | 0.6475 | 0.019239 |
Running didint_plot()
didint_plot() prepares the data for parallel trends plots and event study plots, and returns them as a DiDIntPlot. One call prepares the data for every kind of plot:
plot_data = didint_plot(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
covariates = ["asian", "black", "male"]
)DID-INT plot data
Outcome: coll
Covariates: asian, black, male
ccc options: none, hom, time, state, add, int
Treated states: 34 (2000), 57 (1998), 58 (1993), 59 (1997), 61 (1999), 64 (1996),
71 (1991), 72 (1998), 85 (1997), 88 (2000)
Controls: never-treated states
Periods: 1 year, from 1989 to 2000
Tables of plot data, each drawn with plot(x; plot_type = ...) (see ?DiDIntPlot)
event 126 rows: ccc, time_since_treatment, y, se, ci_lower, ci_upper, ngroup
conditional 3060 rows: state, time, period, ccc, lambda, n
conditional2 23100 rows: state, time, period, ccc, delta, n, treated_state, base_period, treat_period
integrated 4620 rows: state, time, period, delta, n, treated_state_covars, treat_period
integrated2 4620 rows: state, time, period, delta, n, treated_state_covars, base_period, treat_periodWith a Makie backend loaded, plot(plot_data; plot_type = ...) draws each of these: the Plotting Examples page shows every plot, and the Plotting with didint_plot() section gives the formulas behind them. The data of each kind of plot are in the table of the same name. The plot_type argument of didint_plot() selects the kinds to prepare, and ccc the options.
Conditional parallel trends data
The conditional table has the mean outcome of each state and period, residualised by the covariates as each ccc option other than "int" specifies (lambda), and the number of observations behind it (n). "none" is the option without covariates.
plot_data.conditional[1:4, :]| Row | state | time | period | ccc | lambda | n |
|---|---|---|---|---|---|---|
| String | String | Int64 | String | Float64 | Int64 | |
| 1 | 11 | 1989 | 0 | hom | 0.467895 | 39 |
| 2 | 11 | 1990 | 1 | hom | 0.316912 | 35 |
| 3 | 11 | 1991 | 2 | hom | 0.445936 | 33 |
| 4 | 11 | 1992 | 3 | hom | 0.579997 | 32 |
How the series of a treated state changes over time relative to the series of its controls is exactly the comparison that didint() bases its estimates on under these options. The level of a series rests on a convention (the adjusted means are evaluated at covariates equal to zero), but another choice of covariate values would shift the series together and leave their parallelism unaffected.
The conditional2 table has those changes themselves: for each treated state (treated_state), the change in lambda of that state and of each of its control states (state) since the period before its treatment (base_period).
plot_data.conditional2[1:4, :]| Row | state | time | period | ccc | delta | n | treated_state | base_period | treat_period |
|---|---|---|---|---|---|---|---|---|---|
| String | String | Int64 | String | Float64 | Int64 | String | Int64 | Int64 | |
| 1 | 34 | 1989 | 0 | hom | 0.0209245 | 324 | 34 | 10 | 11 |
| 2 | 11 | 1989 | 0 | hom | 0.236957 | 84 | 34 | 10 | 11 |
| 3 | 12 | 1989 | 0 | hom | 0.0924239 | 65 | 34 | 10 | 11 |
| 4 | 13 | 1989 | 0 | hom | 0.0403898 | 60 | 34 | 10 | 11 |
Integrated parallel trends data
Under ccc = "int" another choice of covariate values would change each point of a series by its own amount, so the levels cannot be compared. The integrated table therefore has, for each treated state (treated_state_covars), the change in the predicted outcome since the previous period (delta) of that state and of each of its control states (state), with every state evaluated at the covariate means of treated_state_covars. treat_period is the period in which that treated state is first treated, and n the number of observations of state in the two periods.
plot_data.integrated[1:4, :]| Row | state | time | period | delta | n | treated_state_covars | treat_period |
|---|---|---|---|---|---|---|---|
| String | String | Int64 | Float64 | Int64 | String | Int64 | |
| 1 | 34 | 1990 | 1 | -0.0791523 | 375 | 34 | 11 |
| 2 | 11 | 1990 | 1 | -0.125941 | 74 | 34 | 11 |
| 3 | 12 | 1990 | 1 | 0.0223095 | 72 | 34 | 11 |
| 4 | 13 | 1990 | 1 | -0.160495 | 64 | 34 | 11 |
The integrated2 table has the same changes measured from the period before the treated state's treatment (base_period). In both tables the control states are the never-treated states; notyet = true adds the states of later cohorts while they are not yet treated.
Event study data
The event table has the treated-state means by period relative to treatment, with standard errors and confidence interval bounds.
plot_data.event[1:4, :]| Row | ccc | time_since_treatment | y | se | ci_lower | ci_upper | ngroup |
|---|---|---|---|---|---|---|---|
| String | Int64 | Float64 | Float64? | Float64? | Float64? | Int64 | |
| 1 | add | -11 | 0.51728 | 0.104552 | -0.81118 | 1.84574 | 2 |
| 2 | add | -10 | 0.439195 | 0.0549642 | 0.202703 | 0.675687 | 3 |
| 3 | add | -9 | 0.472126 | 0.0289763 | 0.391675 | 0.552577 | 5 |
| 4 | add | -8 | 0.4383 | 0.030128 | 0.364579 | 0.51202 | 7 |
The ccc column indicates which model was used ("hom", "time", "state", "add", "int", or "none" for no covariates), and time_since_treatment = 0 marks the treatment period; negative values are pre-treatment and positive values are post-treatment.
Separate ATTs by Policy
When treated states adopt different policies, the policy aggregation options report an ATT for each policy. The merit data does not record different types of merit-aid programmes, so purely for illustration the ten treated states are split here into two made-up policy types, "A" and "B", using the treated_policies argument (the pvar argument can be used instead when the policy is stored in a column of the data).
treated_policies = ["A", "A", "B", "A", "B", "A", "B", "B", "A", "B"];
result_policy = didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
treated_policies = treated_policies,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = "policy",
policy_aggregation = true,
nperm = 399
)DID-INT results
Outcome: coll
Model: Region-Varying DID-INT (ccc = "state")
Covariates: male, asian, black
Aggregation: "policy" (by policy)
Weighting: "both"
Controls: never-treated states
Periods: 1 year, from 1989 to 2000
Standard errors: HC1 (se) and jackknife (jknife_se)
Permutations: 399 (seed 1234)
Pooled ATT and ATT by policy
estimate att se pval jknife_se jknife_pval ri_pval policy_weight
────────────────────────────────────────────────────────────────────────────────────────────────
Pooled ATT 0.0461228 0.00749389 0.10254 0.0178092 0.0125462 0.135 missing
Policy A 0.0386289 0.0198199 0.0690503 0.0337412 0.257718 0.3025 0.5
Policy B 0.0536167 0.0124119 0.000253747 0.0102837 3.53862e-6 0.24 0.5Setting policy_aggregation = true also reports the pooled ATT across policies, here weighting the two policies equally (the policy_weight column). See the Policies section for the other policy aggregation options and for the weighting of policies in the pooled ATT.
Cohort ATTs With and Without Policies
agg = "cohort" reports one ATT per treatment time and ignores the policies, whereas agg = "policy_cohort" reports one ATT per treatment time within each policy. With the made-up policies above, the two treated states of the 1998 cohort have different policies, as do the two treated states of the 2000 cohort; the two states of the 1997 cohort both have policy "A", and every other cohort consists of a single state.
Under agg = "cohort" there are seven cohort ATTs, whether or not policies are given:
result_cohort = didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
treated_policies = treated_policies,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = "cohort",
nperm = 399
);
select(result_cohort.sub, :treatment_time, :att, :se, :jknife_se, :weight)| Row | treatment_time | att | se | jknife_se | weight |
|---|---|---|---|---|---|
| Date | Float64? | Float64? | Float64? | Float64? | |
| 1 | 1991-01-01 | 0.0553354 | 0.0222024 | missing | 0.201796 |
| 2 | 1993-01-01 | 0.0504489 | 0.0161905 | missing | 0.191535 |
| 3 | 1996-01-01 | 0.0424116 | 0.0255707 | missing | 0.0756734 |
| 4 | 1997-01-01 | 0.0606879 | 0.0236147 | 0.0328756 | 0.321077 |
| 5 | 1998-01-01 | 0.0398137 | 0.0406649 | 0.0813398 | 0.108593 |
| 6 | 1999-01-01 | 0.00173622 | 0.00350574 | missing | 0.0354853 |
| 7 | 2000-01-01 | -0.0253317 | 0.0333203 | 0.0971445 | 0.0658401 |
Under agg = "policy_cohort" the cohorts are formed within each policy:
result_policy_cohort = didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
treated_policies = treated_policies,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = "policy_cohort",
nperm = 399
)DID-INT results
Outcome: coll
Model: Region-Varying DID-INT (ccc = "state")
Covariates: male, asian, black
Aggregation: "policy_cohort" (by policy and treatment time)
Weighting: "both"
Controls: never-treated states
Periods: 1 year, from 1989 to 2000
Standard errors: HC1 (se) and jackknife (jknife_se)
Permutations: 399 (seed 1234)
ATT by policy
estimate att se pval jknife_se jknife_pval ri_pval
───────────────────────────────────────────────────────────────────────────────
Policy A 0.0386289 0.0208515 0.161013 0.0337412 0.257718 0.3025
Policy B 0.0536167 0.00606071 0.000901416 0.0102837 3.53862e-6 0.24
Sub-aggregate ATTs by policy and treatment time (9)
policy treatment_time att se pval jknife_se jknife_pval ri_pval weight
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
A 1996-01-01 0.0424116 0.0255707 0.172536 missing missing 0.6 0.153779
A 1997-01-01 0.0606879 0.0236147 0.037012 0.0328756 0.0708186 0.2575 0.652476
A 1998-01-01 -0.0295705 0.0527826 0.631703 missing missing 0.8575 0.0834057
A 2000-01-01 -0.0455338 0.0168927 0.0102302 missing missing 0.7075 0.110339
B 1991-01-01 0.0553354 0.0222024 0.0342914 missing missing 0.445 0.397306
B 1993-01-01 0.0504489 0.0161905 0.0169391 missing missing 0.5225 0.377104
B 1998-01-01 0.0819712 0.0533283 0.264087 missing missing 0.3325 0.132997
B 1999-01-01 0.00173622 0.00350574 0.707256 missing missing 0.9775 0.0698653
B 2000-01-01 0.0696929 0.0168927 0.000181788 missing missing 0.55 0.0227273Comparing the two sets of results:
- The 1998 and 2000 cohorts, which contain both policies, are each split in two: one ATT for the state with policy
"A"and one for the state with policy"B". Each of these is now based on a single treated state, so it has no jackknife standard error. The split can matter: the 2000 cohort has an ATT of about -0.025 as a whole, which is made up of about -0.046 under policy"A"and 0.070 under policy"B". - The cohorts that contain a single policy (1991, 1993, 1996, 1997 and 1999) appear once, under that policy, with the same ATT and standard errors as under
agg = "cohort". Their randomization inference p-values differ slightly, because for the policy aggregation options a state's policy is reassigned together with its treatment time. - The
weightcolumn now gives the weight of each cohort ATT within its policy, and the ATT of each policy (the first table of the output) is the weighted mean of the cohort ATTs of that policy. There is no single aggregate ATT; settingpolicy_aggregation = trueadds the pooled ATT across policies, as in the example above.
The Aggregate ATT Across Aggregation Options
With the default weighting (weighting = "both"), and also with weighting = "att", the aggregate ATT does not depend on the agg option (see the Aggregate ATT Equivalency section). Running the staggered adoption example above with every (non-policy) agg option and the default weighting:
aggs = ["cohort", "simple", "state", "sgt", "time", "calendar"]
results = [didint(
"coll", "state", "year", merit;
treatment_times = treated_times,
treated_states = treated_states,
seed = 1234,
ccc = "state",
covariates = [:male, :asian, :black],
agg = agg,
nperm = 399
) for agg in aggs]
# The aggregate ATT and its inference, one row per agg option
aggregate(result) = result.agg[1, :]
DataFrame(
agg = aggs,
agg_att = [aggregate(result).att for result in results],
se = [aggregate(result).se for result in results],
pval = [aggregate(result).pval for result in results],
jknife_se = [aggregate(result).jknife_se for result in results],
jknife_pval = [aggregate(result).jknife_pval for result in results],
ri_pval = [aggregate(result).ri_pval for result in results]
)| Row | agg | agg_att | se | pval | jknife_se | jknife_pval | ri_pval |
|---|---|---|---|---|---|---|---|
| String | Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | |
| 1 | cohort | 0.0462414 | 0.00769881 | 0.000959237 | 0.0123516 | 0.00046902 | 0.0975 |
| 2 | simple | 0.0462414 | 0.010071 | 6.48658e-5 | 0.0123516 | 0.00046902 | 0.0975 |
| 3 | state | 0.0462414 | 0.00955215 | 0.000919824 | 0.0123516 | 0.00046902 | 0.14 |
| 4 | sgt | 0.0462414 | 0.0114307 | 0.00023188 | 0.0123516 | 0.00046902 | 0.14 |
| 5 | time | 0.0462414 | 0.00873387 | 0.000497471 | 0.0123516 | 0.00046902 | 0.0975 |
| 6 | calendar | 0.0462414 | 0.00718411 | 0.000120006 | 0.0123516 | 0.00046902 | 0.0975 |
The aggregate ATT (agg_att) is identical in every row. Its HC standard error and p-value (se, pval) differ, as they are computed from the spread of the sub-aggregate ATTs of each option. The jackknife standard error and p-value do not differ: each removal of a state recomputes the same aggregate ATT, whatever the grouping. Neither does the randomization inference p-value, except under "state" and "sgt", whose randomization inference draws an additional stand-in state for each treated state (see the Randomization Inference section).
Common Adoption
When every treated state is treated at the same time, didint() is used in exactly the same way, and every aggregation option is available. This example keeps the two states that adopted merit-aid programmes in 1998 along with the never-treated states. A single treatment time applies to every treated state.
merit_common = filter(
row -> row.state ∈ [57, 72] || !(row.state ∈ treated_states),
merit
);
result_common = didint(
"coll", "state", "year", merit_common;
treated_states = [57, 72],
treatment_times = 1998,
seed = 1234,
ccc = "hom",
covariates = [:male, :asian, :black],
agg = "time",
nperm = 399
)DID-INT results
Outcome: coll
Model: Homogeneous DID-INT (ccc = "hom")
Covariates: male, asian, black
Aggregation: "time" (by periods since treatment)
Weighting: "both"
Controls: never-treated states
Periods: 1 year, from 1989 to 2000
Standard errors: HC1 (se) and jackknife (jknife_se)
Permutations: 399 (seed 1234)
Aggregate ATT
estimate att se pval jknife_se jknife_pval ri_pval
────────────────────────────────────────────────────────────────────────────────
Aggregate ATT 0.0362295 0.0134074 0.114003 0.0728969 0.621784 0.5775
Sub-aggregate ATTs by periods since treatment (3)
periods_post_treat att se pval jknife_se jknife_pval ri_pval weight
───────────────────────────────────────────────────────────────────────────────────────────────
0.0 0.0114218 0.035171 0.800098 0.0548238 0.835973 0.9025 0.358268
1.0 0.0517801 0.0677093 0.584371 0.102537 0.616206 0.5375 0.338583
2.0 0.0481794 0.118292 0.75377 0.19972 0.810547 0.5 0.30315Trying a different CCC assumption
The ccc argument controls the assumption made about how covariate effects vary across states and time. Swapping ccc = "hom" for any of "time", "state", "add", or "int" changes whether covariate effects are allowed to vary by time, by state, additively by both, or interactively by both, respectively. For example:
result_common_int = didint(
"coll", "state", "year", merit_common;
treated_states = [57, 72],
treatment_times = 1998,
seed = 1234,
ccc = "int",
covariates = [:male, :asian, :black],
agg = "time",
nperm = 399
);
result_common_int.sub| Row | periods_post_treat | att | se | pval | jknife_se | jknife_pval | ri_pval | weight |
|---|---|---|---|---|---|---|---|---|
| Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | Float64? | |
| 1 | 0.0 | 0.0415982 | 0.0503724 | 0.560552 | 0.077153 | 0.59262 | 0.63 | 0.358268 |
| 2 | 1.0 | 0.0370877 | 0.101872 | 0.777727 | 0.153767 | 0.810578 | 0.6675 | 0.338583 |
| 3 | 2.0 | 0.0481968 | 0.13202 | 0.777159 | 0.223535 | 0.830334 | 0.4975 | 0.30315 |
A Single Control State
This example compares a single treated state (state 71, treated in 1991) against a single untreated control state (state 73). With agg = "sgt", an ATT is reported for state 71 in each period from 1991 onwards. With only one control state, the usual standard error of each of these ATTs cannot be computed; setting edgecase = true recovers it from the first-stage estimates instead.
merit_pair = filter(row -> row.state ∈ [71, 73], merit);
result_pair = didint(
"coll", "state", "year", merit_pair;
treated_states = [71],
treatment_times = [1991],
seed = 1234,
ccc = "hom",
covariates = [:male, :asian, :black],
agg = "sgt",
edgecase = true
);
select(result_pair.sub, :state, :t, :att, :se)| Row | state | t | att | se |
|---|---|---|---|---|
| String | Date | Float64? | Float64? | |
| 1 | 71 | 1991-01-01 | -0.0449382 | 0.136587 |
| 2 | 71 | 1992-01-01 | -0.173757 | 0.133532 |
| 3 | 71 | 1993-01-01 | -0.0908704 | 0.130533 |
| 4 | 71 | 1994-01-01 | -0.133035 | 0.134481 |
| 5 | 71 | 1995-01-01 | 0.0521876 | 0.133253 |
| 6 | 71 | 1996-01-01 | -0.0268792 | 0.12889 |
| 7 | 71 | 1997-01-01 | 0.0876822 | 0.133043 |
| 8 | 71 | 1998-01-01 | 0.107933 | 0.14195 |
| 9 | 71 | 1999-01-01 | -0.115658 | 0.127827 |
| 10 | 71 | 2000-01-01 | -0.147419 | 0.137571 |
No p-value accompanies these standard errors, as the regression behind each ATT has no residual degrees of freedom. With one treated state and one control state there is also nothing for the jackknife to compute and only one alternative treatment assignment for the randomization inference, so didint() issues warnings to that effect. For more information on these standard errors, including a caveat for panel data, see the section Computation of Edge Case Standard Errors on the "Details" page of the documentation site.
See the Functions for function syntax and the Under the Hood page for more details.