EFIMM0120 Applied Quantitative Research in Accounting and Finance Lecture 5: Difference-in-Difference Design and Quasi-Natural Experiments Part 1: Endogeneity concerns in OLS regression Dr. Zilu Shan [email protected] Endogeneity Problem in Causal Inference § Causal inference is the leveraging of theory and deep knowledge of institutional details to estimate the impact of events and choices on a given outcome of interest. § In traditional OLS regression, there are three main source of bias –Omitted Variable –Simultaneously –Measurement error Endogeneity Source (1) Omitted variables § The possibility that an explanatory variable modeled as exogenous will in fact be endogenous because of omitted variable Assume that the structural equation is given by! = #! + #”%” + ##&” + ‘ § The OLS method provides the Best Linear Unbiased Estimate (BLUE) if all explanatory variables in this regression is exogenous § The bias is equal to E[ (#”] #” = ## $%&((!)*!),-.((!) We are only concerned with the omitted variable that impact both explanatory and explained variables Endogeneity Source (1) Omitted variables –example § Omitted variable issue are particularly severe in corporate finance, because the objects of study (Firms or CEOs, for example) are heterogeneous along many different dimensions § Some examples: –Executive compensation executives’ abilities –Corporate financial and investment politics financing frictions (i.e. information asymmetry and incentive conflicts) –Corporate decisions both public and unpublic information Endogeneity Source (2) Simultaneity Assume a two-equation structural model:!! = #” + #!%! + ##&! + ‘!%! = (” + (!!! + (#&! + ‘# If
written in reduced form%! = $!%$”&!!’$”&” + $”&#!’$”&”
&! + $#!’$”&” &# + $”!’$”&” ‘! + !!’$”&” ‘# If (! ≠ 0, then there exists endogeneity due to simultaneity § Simultaneity arises when one or more explanatory variables are jointly determined with the explained variable in an equilibrium. § If simultaneity exists, the structural error term is correlated with the explanatory variable § The causal relationship between an explained and an explanatory variable runs both ways Endogeneity Source (2) Simultaneity – example § In a regression of a value multiple (such as market-to-book) on an index of anti-takeover provisions, the usual result is a negative coefficient on the index. § However, it doesn’t mean that the presence of anti-takeover provisions leads to a loss of firm value. § Alternative explanation: Mangers of low value firms adopt anti- takeover provisions in order to entrench themselves. Endogeneity Source (3) Measurement errors § Measurement error comes from any discrepancy between the true variables and the proxy for unobservable or difficult to quantify variables. § It could happen for either dependent variables or independent variables. § Some examples: –Market value of debt: Most debt is privately held by banks and other financial institutions, so there is no observable market value –Executive compensation: Stock options often vest over time and valued using an approximation, such as Black-Scholes –Corporate governance: it’s a nebulous concept with a variety of different facets. Current use of anti-takeover provision index or the presence of large blockholders are unlikely to be sufficient EFIMM0120 Applied Quantitative Research in Accounting and Finance Lecture 5: Difference-in-Difference Design and Quasi-Natural Experiments Part 2: An introduction to DID method Dr. Zilu Shan [email protected] ControlledVs. (Quasi-) Natural Experiments (1) Controlled experiments are often used in other area of science § i.e. check if certain drug reduces cholesterol, researchers could randomly assign patients to treatment group (certain drug) and control group (placebo pills) § The difference in the average change in cholesterol between the two group of patients is average treatment effect (ATE), or in regression format, difference-in-difference estimator. § It’s very rare in social science that researcher can apply controlled experiments, because of the difficulty of imposing treatment. ControlledVs. (Quasi-) Natural Experiments (2) Natural experiments is a sharp change in one or more variables of interest that occurs for exogenous reasons. § It could either – by natural causes (e.g. natural disasters), – or by some kind of human action, such as changes in regulation, economic policy and political changes (generally referred to as “quasi-natural” experiments) § Key assumption to allow causal inference – the treatment assignment is random or at least “as good as random” – In other words, any other variable that is important to determine the outcome variable is uncorrelated with the treatment assignment § In literature, it often be called “the shock” Causal inference issue § We ideally would like the average treatment effect (ATE)* !” * !! § But we only observe an estimate of the na ve estimator:* !”|, = 1 * !!|, = 0 § If we assume * !”|, = 1 = * !!|, = 1 then we can get the ATT (average treatment on the treated) from observed data* !”|, = 1 * !!|, = 0 Single difference in cross-section Comparing the post-treatment values of the outcome variable between treated and untreated firms!! = #” + ##%! + &! § !! is the outcome variable § %! is a dummy variable indicates whether firm ‘ is in treatment group § If treatment is random, then %! is uncorrelated with error term &! § (## is therefore an unbiased estimate of the average treatment effect(ATE) § The approach is useful when the researcher does not have data on the values of outcome ! previously to the treatment § The potential problem: the average ! of treated and untreated firms were different ex-ante (i.e. before the treatment) Time-difference regressions Comparing post-treatment values to pre-treatment values for all firms!/,1 = #! + #”P1 + ‘/,1 § !/,1 is the outcome variable § 01 is a dummy variable which takes value 1 for the observations in the post-shock period, and 0 for the observations in the pre-shock period § No other event that affect the outcome !/,1 occurred between the pre-shock and post-shock period. § In other words, there is no omitted variable, correlated to 01 , that affects !/,1 Difference-in-Difference § Difference in difference model combines the cross-sectional and the time-series differences model into a single model § Intuition: compute the difference of – the change in outcome ! pre- versus post-treatment for the treated group and – the change in outcome ! pre- versus post-treatment for the control group § We would need a panel of treated and untreated firms, with observations before the shock and after it § A typical DID regression would be!”,$ = #% +#&P$ +#’&” +#(P$ &” +(“,$ DiD coefficient !”,$ = #% +#&P$ +#’&” +#(P$ &” +(“,$ § (#” captures the average change in !/,1 from the pre- to post- shock periods for the untreated group (,/ = 0) § (## captures the pre-shock difference in !/,1 between treated and untreated firms (01 = 0) § Our main coefficient (#2 (DiD coefficient) captures the effect of the shock, that is the average differential change in !/,1 from the pre- to post- treatment period for the treatment group relative to control group Further comments on Diff-in-Diff § We can add individual level covariates, including fixed effects, but they really aid only for getting more precise estimate. § Diff-in-diff logic with ‘ treated and ) not treated (control group), taking expectations to eliminate the * which we assume orthogonal to treatment+ !$%## |%$%# = 1 + !”|%$ = 1 + + !$”|%$ = 0 – + !$%#” |%$%# = 0 = 0 + 1 + 2$%# 0 + 2& +(0+ 2&) (0 +2$%#) =1 § If we assume + !”|% = 0 = + !”|% = 1 then we can get the ATT (average treatment on the treated) from observed data+ !#|% = 1 + !”|% = 1 Graphic illustration – DiD without trends § Pre- and post- shock average ! of treated and untreated observations are constant (no trend) § +#! captures the difference betweenaverage pre- and average post- shock outcome for untreated observations § +## captures the pre-shock differencein outcome between treated and untreated firms § +#( (DiD coefficient) capturesdifference between observed average post-shock ! and the average unobserved counterfactual ! after the shock (i.e. The hypothetical value of ! of treated observation absent the shock) Graphic illustration – DiD with trends § Pre- and post- shock average ! of treated and untreated observations increase at a constant trend § +#! captures the difference betweenaverage pre- and average post- shock outcome for untreated observations § +## captures the pre-shock differencein outcome between treated and untreated firms § +#( (DiD coefficient) capturesdifference between observed average post-shock ! and
the average unobserved counterfactual ! after the shock (i.e. The hypothetical value of ! of treated observation absent the shock) Graphic illustration – combined figure Parallel trending assumption § In truly random experiment (controlled experiment), the treatment and control group should be indistinguishable § Parallel trending assumption: – In order to meet the statistical assumption of causal inference with Did, even if treated and control firms present different averages for ! prior to the shock (thus not indistinguishable), their trends should be parallel – In the regression format, no correlation between the )$ &” and the error term Defining the pre- and post- period § If using too few periods (one or two) prior to the shock, it doesn’t allow checking for parallel trends assumption § If using too long periods for pre- or post- shock, the analysis can be subject to the occurrence of other events that also effects the outcome, causing confounding issue § In some cases, firms may be expected to respond to the shock gradually over time § If the data is not annual and the outcome presents a seasonal pattern, one should use the same calendar month or quarters in the pre- and post- period to avoid potential seasonality problems. § If the data of shock cannot be precisely defined (i.e. the pass date and effective date of law are usually different), researchers should consider exclude one or more period to avoid wrongly assigning a period to the pre- or post- shock Other DiD method issues § Control variables – The inclusion of other control variables that affect 3 may be useful to increase the fit of the regression and the precision of estimation by reducing their standard error – Avoid including control variables that are themselves affected by the treatment § Anticipatory effects – Trend could be affected by anticipation of treatment § SUTVA(Stable Unit Treatment Value Assumption) not holding: – treated could be influence the outcomes of the control group § Change in composition – Control group may change over time – suggests being explicit about its composition § Standard errors: – Clustered. Otherwise, the analysis may be underestimating them. EFIMM0120 Applied Quantitative Research for Accounting and Finance Lecture 5: Difference-in-Difference Design and Quasi-Natural Experiments Part 3: DID – an empirical application Dr. Zilu Shan [email protected] Bank risk and bailout probability Research question: Will the governmental guarantees to banks affect their risk Theoretical framework (1) The expected effect of governmental protection on bank risk is ambiguous § Charter value theory: guarantees cause protected banks to take less risk than unprotected banks, because this implicit protection allows them to fund at abnormally low costs, which is a source of value (charter value) that banks do not want to put at risk § Moral hazard hypothesis: the depositors of protected banks have less incentives to monitor the risk of the bank, thereby leading an increase in risk-taking by the banks The empirical evidence is also mixed. Theoretical framework (2) § Systematically important banks benefit from a perception of protection by depositors and investors in general § This perception became more pronounced during financial crisis § Therefore, systematically important banks enjoy implicit guarantees than other smaller groups do not, even in countries whose financial systems were not directly affected by the crisis. The shock: the 2008 financial crisis § Timeline of 2008 financial crisis – The liquidity crisis started in the US in the second half of 2017 – In September of 2008, the failure of Lehman Brothers signifies that the crisis actually spread internationally – In the US, government backs to systemically important financial institutions, restoring their deposits – October 2008, G-7 action plan was launched, causing the perception of an increased bailout probably of large banks that went beyond the boarders of G-7 countries § Use a sample of banks from countries whose financial systems were not directly affected by the crisis – the crisis changed the perception of governmental protection to large banks (treatment group) – The perception of a bailout probability didn’t change much for less protected banks (control group) Identification strategy:DiD model To test whether governmental protection affects bank risk, we estimate the following DiD model *+ ,_./012″,$ =#% + #&314545 + #’)1062/627″,$ + #( 314545 )1062/627 “,$ + (“,$ § ,_./012″,$ is a measure of bank risk, defined by the bank’s distance todefault in terms of the standard deviation of ROA § Time period for main regression 2005 – 2010 – Pre-crisis period : 2005 to 2007 (456767 =0), – Post-crisis period: 2008 to 2010 (456767=1) § Treatment: the probability of expected external support § Main coefficient of interest: 8#( – If positive, the result favors the charter value hypothesis – If negative, the result favors the moral hazard hypothesis Endogenous shock concern § When choosing the treatment (shock), it’s important to consider whether the shock is exogenous § One concern here is that the protected and unprotected banks might be differently exposed to toxic assets § The different exposure could be treated as an omitted variable – that explains bank risk(outcome variable) – and is correlated to our main regressor (protected bank) Solution: § We use a sample of banks located in OECD countries whose banks were not directly affected by the crisis Descriptive statistics § Protected and unprotected banks have roughly the same level of risk(L9(Z sccore)) prior to the crisis on average § Both protected and unprotected banks increase their risk during crisis period, but the average of protected banks increase more § Prior to the crisis, unprotected had a slightly larger ratio of liquid asset to short- term liabilities compared to protected banks on average, but the difference is statistically insignificant (Liquidity) § Protected banks are significantly larger (Asset) Parallel trends § The average &_345678 of protected (treatment) and unprotected (control) banks § In pre-crisis years, &_345678 are roughly constant (parallel trend assumption) § In the first year of crisis, we observe a dramatic decrease in &_345678 (an increase in bank risk), which is more pronounced for protected banks than for unprotected banks DiD Results D iD re su lt s – b as el in e re g re ss io n § Th e es tim at ed A !that th e Z- Sc or e de cr ea se d by 36 .8 % on av er ag e fo rt he co nt ro lg ro up (u np ro te ct ed ba nk s) fro m pr e- cr is is pe rio d to cr is is pe rio d § Th e es tim at ed A #isn ot st at is tic al ly si gn ifi ca nt ,i nd ic at es th at th e av er ag e pr e- cr is is ris k le ve lb et w ee n tre at ed an d co nt ro lg ro up s ar e no ts ta tis tic al ly di ffe re nt § Th e es tim at ed A ((ma in co ef fic ie nt of in te re st )i nd ic at es th at th e av er ag e Z- Sc or e of th e pr ot ec te d ba nk s de cr ea se s by 31 % m or e th an th at of th ei r un pr ot ec te d co un te rp ar ts do es DiD results – with additional controls § Column (2) add bank-level controls: size and liquidity – Result show that bank size and liquidity do not materially affect the bank risk – The coefficients interpretation doesn’t change § Column (3) add bank level controls and country-level controls – To address the possible concern that country features might be important in determining bank risk – Loss some observation because of missing data – None of the country-level coefficient is significant – The coefficients interpretation doesn’t change DiD Result – with fixed effects § Column (4) replace country-level controls by country fixed effects – It might capture other time-invariant country features that are not captured by macroeconomic controls (i.e. market microstructure, quality of regulation, enforcement of the law) – The sign and statistical significance of coefficient remain unchanged although the magnitude changes § Column (5) add bank fixed effect – It’s perfectly collinear with the protected dummy, thus A# is no longer identified § Column (6) add year fixed effect Falsification and robustness test § Check for the treatment reversals – If the treatment is subsequently reversed, we should expect the opposite effect of what was observed with treatment – In this case, we could check the bank risk after 2011, when the financial crisis is supposed over § Check the non-parametric version of DiD,- &_/0123),+ =#” + #!425656 + ##721830839),+ +:, ;( 425656 721830839 ),+ + ‘),+ – ‘$ captures the difference in outcome in each year – It’s useful when the treatment is not a sharp event § Restrict the sample to banks that have observations both before and after the crisis § Use a placebo timing of treatment