Videos and questions for Chapter 1d of the course "Market Analysis with Econometrics and Machine Learning" at Ulm University (taught by Sebastian Kranz)

Hypotheses Tests and t-test

Take a look at the t-value: \[t_k = \frac {\hat\beta_k} {\hat sd(\hat\beta_k)}\]

Assume H0: \(\beta_k = 0\) and all assumptions (A1)-(A4) of the linear regression model hold true. Which statement is then true for our test statistic (t-value) \(t_k\) if we see it as a random variable?

The t-statistic essentially normalizes the estimated coefficient \(\hat \beta_l\) by its estimated standard deviation.

Assume you have any random variable \(x\). What is the standard deviation of the normalized random variable \(z=\frac x {sd(x)}\) that divides \(x\) by its standard deviation?

p-value

What more strongly suggests that a null hypothesis H0 is false?


Quiz questions about p-values

Assume the null hypothesis H0 is rejected at a significance level \(\alpha = 0.1\%\) (p = 0.001). Is this strong evidence that H0 is false?

Assume the null hypothesis H0 is not rejected and we find a p-value of p = 0.999. Is this strong evidence that H0 is true?

Assume a null hypothesis H0 is rejected at a significance level \(\alpha = 5\%\) (p = 0.05). Does it mean that the probability that H0 is true is equal to or smaller than 5%?

If you understand the following comic, you probably also understand the answer to the question above:

http://xkcd.com/1132/


Multiple Testing

Take a look at the following xkcd comic

https://xkcd.com/882/

Replication Crisis, Publication Bias and Multiple Testing

Measures for more robust scientific insights

Misrepresentation in business and other domains

Controlling the false discovery rate


Diagnostic tests

While t-tests as discussed above are typically use to make discoveries, diagnostic tests are mainly used to check whether some assumptions of an econometric model are likely to be violated.

We will look at 3 diagnostic tests for instrumental variable estimation:

  • Weak instruments test
  • Wu-Hausman test
  • Sargan test

Weak Instruments and Wu-Hausman Tests for Instrumental Variable Estimation

Is the weak instrument test a test about the relevance condition or about the exogeniety condition of our instrumental variable \(c\)?

What do you think is the null hypothesis of the weak instruments test we have run?

What is the null hypothesis of the Wu-Hausman test?

In the code example above, we found the diagnostic test results:

Diagnostic tests:
                 df1 df2 statistic p-value    
Weak instruments   1 497 3.174e+02  <2e-16 ***
Wu-Hausman         1 496 8.015e+29  <2e-16 ***

Do the results suggest that an OLS estimator of the demand function would be consistent or inconsistent?

In our simulation we had the following code to generate the prices:

if (runif(1)<0.5) {
  # Profit maximizing prices
  p = (beta0+eps+beta2*s) / (-2*beta1) + c/2
} else {
  # Random markup above costs
  p = c*runif(T, 1, 1.1)
}

So we draw a random variable by calling runif if it is below 0.5, we choose profit maximizing prices and otherwise prices are just a markup above costs.

What does the result of the Wu-Hausman test suggest of how prices were actually drawn in our simulation?

Another Wu-Hausman Test example

Look at the following alternative simulation code:

T = 500
beta0 = 100
beta1 = -1
beta2 = 30
eps = rnorm(T, 0, 7)
c = runif(T,10,30) + eps
s = sample(0:1, T, replace=TRUE)
p = runif(T, 20,40)
q = beta0 + beta1*p + beta2*s + eps

What holds true for our simulated ice cream demand above?

When we run the IV regression we get the following output

summary(ivreg(q~p+s | c+s), diagnostics=TRUE)
Call:
ivreg(formula = q ~ p + s | c + s)

Residuals:
     Min       1Q   Median       3Q      Max 
-190.637  -97.356    5.399  101.379  188.391 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)   
(Intercept)  -450.75     513.90  -0.877   0.3809   
p              17.34      17.10   1.014   0.3111   
s              34.35      10.76   3.192   0.0015 **

Diagnostic tests:
                 df1 df2 statistic p-value    
Weak instruments   1 497     1.156   0.283    
Wu-Hausman         1 496   629.108  <2e-16 ***
Sargan             0  NA        NA      NA    

Even though the price is exogenous, our Wu-Hausman test rejects the null hypothesis. That is because the instrument is endogenous. One can therefore think of the Wu-Hausman test testing the joint null hypothesis that all explanatory variables and all instruments are exogenous.

Note that in this example the OLS estimator would be consistent while the IV estimator is inconsistent.


The Sargan Test for Instrumental Variable Estimation

The third diagnostic test is the Sargan tests which (sometimes) can detect endogeneity problems of the instruments.

What is the null hypothesis of the Sargan test? Make an educated guess.

What do you think is more likely for our simulation above.

Examples where the Sargan test works and where it does not work

That's it for Chapter 1d. If you have not started yet with the corresponding RTutor problem set, now would be a good time!