Vwwxrf

Why is the Maximum Likelihood Estimator Normally distributed? I can't figure out why it is true for large n in general. My attempt (for single parameter)

Let $L(theta)$ be the maximum likelihood function for the distribution $f(x;theta)$

Then after taking sample of size n

$$L(theta)=f(x_1;theta)cdot f(x_2theta)...f(x_n;theta)$$

And we want to find $theta_max$ such that $L(theta)$ is maximized and $theta_max$ is our estimate (once a sample has actually been selected)

Since $theta_max$ maximizes $L(theta)$ it also maximizes $ln(L(theta))$

where

$$ln(L(theta))=ln(f(x_1;theta))+ln(f(x_2;theta))...+ln(f(x_n;theta))$$

Taking the derivative with respect to $theta$

$$fracf'(x_1;theta)f(x_1;theta)+fracf'(x_2;theta)f(x_2;theta)...+fracf'(x_n;theta)f(x_n;theta)$$

$theta_max$ would be the solution of the above when set to 0 (after selecting values for all $x_1,x_2...x_n$) but why is it normally distributed and how do I show that it's true for large n?

MLE requires $$fracpartial ln L(theta)partial theta = sum_i=1^n frac f'(x_i;theta)f(x_i;theta),$$
where $f'(x_i;theta)$ could denote a gradient (allowing for the multivariate case, but still sticking to your notation). Define a new function $g(x;theta)=frac f'(x;theta)f(x;theta).$ Then $g(x_i;theta)_i=1^n$ is a new iid sequence of random variables, with $Eg(x_1;theta)=0$. If $Eg(x_1;theta)g(x_1;theta)'<infty$, CLT implies,
$$sqrtn(barg_n(theta)-Eg(x_1;theta))=sqrtnbarg_n(theta) rightarrow_D N(0,E(g(x;theta)g(x;theta)'),$$
where $barg_n(theta)=frac1n sum_i=1^n g(x_i;theta).$ The ML estimator solves the equation
$$barg_n(theta)=0.$$
It follows that the ML estimator is given by
$$hattheta=barg_n^-1(0).$$
So long as the set of discontinuity points of $barg_n^-1(z)$, i.e. the set of all values of $z$ such that $barg_n^-1(z)$ is not continuous, occur with probability zero, the continuous mapping theorem gives us asymptotic normality of $theta$.