Для яких моделей зміщення MLE падає швидше, ніж дисперсія?

$\hat\theta$$\theta^*$$n$$\lVert\hat\theta-\theta^*\rVert$$O(1/\sqrt n)$$\lVert \mathbb E\hat\theta - \theta^*\rVert$$\lVert \mathbb E\hat\theta - \hat\theta\rVert$$O(1/\sqrt{n})$

Мене цікавлять моделі, які мають ухил, який скорочується швидше, ніж , але де помилка не зменшується з такою швидкістю, оскільки відхилення все ще скорочується як . Зокрема, я хотів би знати достатні умови, щоб ухил моделі зменшився зі швидкістю .$O(1/\sqrt n)$$O(1/\sqrt n)$$O(1/n)$

In general, you need models where the MLE is not asymptotically normal but converges to some other distribution (and it does so at a faster rate). This usually happens when the parameter under estimation is at the boundary of the parameter space. Intuitively, this means that the MLE will approach the parameter "only from the one side", so it "improves on convergence speed" since it is not "distracted" by going "back and forth" around the parameter.

A standard example, is the MLE for $\theta$ in an i.i.d. sample of $U(0,\theta)$ uniform r.v.'s The MLE here is the maximum order statistic,

$$\hat \theta_n = u_{(n)}$$

Its finite sample distribution is

$$F_{\hat \theta_n} = \frac {(\hat \theta_n)^n}{\theta ^n},\;\;\; f_{\hat \theta}=n\frac {(\hat \theta_n)^{n-1}}{\theta ^n}$$

$$\mathbb E(\hat \theta_n) = \frac {n}{n+1}\theta \implies B(\hat \theta) = -\frac {1}{n+1}\theta$$

So $B(\hat \theta_n) = O(1/n)$. But the same increased rate will hold also for the variance.

One can also verify that to obtain a limiting distribution, we need to look at the variable $n(\theta - \hat \theta_n)$,(i.e we need to scale by $n$) since

$$P[n(\theta - \hat \theta_n)\leq z] = 1-P[\hat \theta_n\leq \theta - (z/n)]$$

$$=1-\frac 1 {\theta^n}\cdot \left(\theta + \frac{-z}{n}\right)^n = 1-\frac {\theta^n} {\theta^n}\cdot \left(1 + \frac{-z/\theta}{n}\right)^n$$

$$\to 1- e^{-z/\theta}$$

which is the CDF of the Exponential distribution.

I hope this provides some direction.

Following comments in my other answer (and looking again at the title of the OP's question!), here is an not very rigorous theoretical exploration of the issue.

We want to determine whether Bias $B(\hat \theta_n) = E(\hat \theta_n) - \theta$ may have different convergence rate than the square root of the Variance,

$$B(\hat \theta_n) = O(1/n^{\delta}),\;\;\; \sqrt {\text{Var}(\hat \theta_n)} = O(1/n^{\gamma}), \;\;\gamma \neq \delta \;???$$

We have

$$B(\hat \theta_n) = O(1/n^{\delta}) \implies \lim n^{\delta}\mathbb E(\hat \theta_n) < K \implies \lim n^{2\delta}[\mathbb E(\hat \theta_n)]^2 < K'$$

$$\implies [\mathbb E(\hat \theta_n)]^2 = O(1/n^{2\delta}) \tag{1}$$

while

$$\sqrt {\text{Var}(\hat \theta_n)} = O(1/n^{\gamma}) \implies \lim n^{\gamma}\sqrt{\mathbb E (\hat \theta_n^2) - [\mathbb E(\hat \theta_n)]^2 }<M$$

$$\implies \lim \sqrt{n^{2\gamma}\mathbb E (\hat \theta_n^2) - n^{2\gamma}[\mathbb E(\hat \theta_n)]^2 }<M$$

$$\implies \lim n^{2\gamma}\mathbb E (\hat \theta_n^2) - \lim n^{2\gamma}[\mathbb E(\hat \theta_n)]^2 < M' \tag{2}$$

We see that $(2)$ may hold happen if

A) both components are $O(1/n^{2\gamma})$, in which case we can only have $\gamma = \delta$.

B) But it may also hold if

$$\lim n^{2\gamma}[\mathbb E(\hat \theta_n)]^2 \to 0 \implies [\mathbb E(\hat \theta_n)]^2 = o(1/n^{2\gamma}) \tag{3}$$

For $(3)$ to be compatible with $(1)$, we must have

$$n^{2\gamma} < n^{2\delta} \implies \delta > \gamma\tag {4}$$

So it appears that in principle it is possible to have the Bias converging at a faster rate than the square root of the variance. But we cannot have the square root of the variance converging at a faster rate than the Bias.