This is a good discussion topic, but I found this post confusing. Why would the test not control Type I error under different point nulls? Property 2 depends on what your effect size of interest is. Another benefit of confidence intervals via test inversion is that you actually get p values that can be adjusted for multiple comparisons. Not sure you can do that if you do testing via confidence intervals.
Thanks Jacob. 'Why would the test not control Type I error under different point nulls?' One reason you can lose control of type I error is bias. Suppose you have an estimator with bias jθ– θ, j>1. Then it's only unbiased under a 0 null.
I see, I’ve been so used to using unbiased estimators that I forgot that sometimes biased estimators can be preferable! I would be very interested in seeing how this plays out with a concrete example. Does the CI width change depending on your point estimate for this estimator?
You can adapt most multiple comparison methods to confidence intervals (most directly: anything adjusting alpha can similarly alter CI width. There's also FDR for CIs)
This is a good discussion topic, but I found this post confusing. Why would the test not control Type I error under different point nulls? Property 2 depends on what your effect size of interest is. Another benefit of confidence intervals via test inversion is that you actually get p values that can be adjusted for multiple comparisons. Not sure you can do that if you do testing via confidence intervals.
Thanks Jacob. 'Why would the test not control Type I error under different point nulls?' One reason you can lose control of type I error is bias. Suppose you have an estimator with bias jθ– θ, j>1. Then it's only unbiased under a 0 null.
I see, I’ve been so used to using unbiased estimators that I forgot that sometimes biased estimators can be preferable! I would be very interested in seeing how this plays out with a concrete example. Does the CI width change depending on your point estimate for this estimator?
You can adapt most multiple comparison methods to confidence intervals (most directly: anything adjusting alpha can similarly alter CI width. There's also FDR for CIs)