The maximum likelihood estimations for a type of left ellipsoidal distribution

Abstract

In this paper, we prove that if the distribution of an $n\times mp$ random matrix Y is a left ellipsoidal distribution with parameter ${\mu }^{\ast }=\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)$, ${\mathrm{\Sigma }}^{\ast }=m\mathrm{\Sigma }$, and $Y_1,Y_2,\dots ,Y_m$ are independent and identical distributions, the maximum likelihood estimations of μ, Σ are $\overline{Y}$, $S^2$ if and only if $Y\sim N_{n\times mp}({\mu }^{\ast },{\mathrm{\Sigma }}^{\ast })$. If $Y_1,Y_2,\dots ,Y_m$ are not independent and identical distributions, then Y may be not a normal distribution.

1 Introduction

Let $X=(x_1,x_2,\dots ,x_n)$, $x_1,x_2,\dots ,x_n$ be independent standard normal distributions. A random matrix $Y_{n\times p}$ is said to be a left stochastic ellipsoid matrix if it satisfies (see [1])

$$Y_{n\times p}=A_{n\times n}{X}_{n\times p}+{\mu }_{n\times p},{\mathrm{\Gamma }}_{n\times n}{X}_{n\times p}\stackrel{d}{=}{X}_{n\times p},EX{X}^{\prime }=I_n,$$

and its distribution, which is denoted by $E_{n\times p}(\mu ,\mathrm{\Sigma })$, is called a left ellipsoidal distribution, where $\mathrm{\Sigma }=A{A}^{\prime }$ is reversible, and E is ellipsoid. Besides, d denotes identical distribution, S denotes $X{X}^{\prime }$, and all ${\mathrm{\Gamma }}_{n\times n}\in o(n)$ are orthogonal matrices with order n. Then we have conclusions:

$$E{S}^{k}X=0,E{S}^k={\alpha }^k{I}_n,$$

where k is an arbitrary integer and k is a constant (see [2]).

If X has the distribution density, the distribution density has the form $f(X^{\prime }X)$, then the distribution density is said to be spherical distribution. If the distribution density of Y is

$${|\mathrm{\Sigma }|}^{-\frac{p}{2}}f({(Y-\mu )}^{\prime }{\mathrm{\Sigma }}^{-1}(Y-\mu )),$$

then the distribution density is called left ellipsoidal distribution contour, and the contour surface is

$${(Y-\mu )}^{\prime }{\mathrm{\Sigma }}^{-1}(Y-\mu )=D.$$

When $p=1$, it is an ellipsoidal surface. The spectral decomposition of Σ is $\mathrm{\Gamma }\mathrm{\Lambda }{\mathrm{\Gamma }}^{\prime }$, where Γ is an orthogonal matrix, $\mathrm{\Lambda }=diag({\lambda }_1,\dots ,{\lambda }_n)$, ${\lambda }_1>\cdots >{\lambda }_n>0$.

If the distribution density of Y is

$${|\mathrm{\Sigma }|}^{-\frac{p}{2}}f[{(Y-\mu )}^{\prime }{\mathrm{\Sigma }}^{-1}(Y-\mu )]={\left(\sqrt{\frac{p}{2\pi }}\right)}^{np}{|\mathrm{\Sigma }|}^{-\frac{p}{2}}exp\{-\frac{p}{2}tr{(Y-\mu )}^{\prime }{\mathrm{\Sigma }}^{-1}(Y-\mu )\},$$

the elongated vector of Y, which is denoted by $\overrightarrow{Y}$, is normal $N_{np}(\overrightarrow{\mu },\frac{1}{p}{\mathrm{I}}_p\otimes \mathrm{\Sigma })$ distribution. For the sake of consistency of symbols, we denote the normal distribution by $N_{n\times p}(\mu ,\mathrm{\Sigma })$.

Classical statistical analysis is built on the basis of normal distribution. However, it is still an important problem whether these graceful properties can also be satisfied without the condition that it is a normal distribution.

Since the left ellipsoidal distribution family has much more members than the normal distribution family (in fact, it almost includes all common distributions), a large amount of scholars’ fruitful research shows: on the one hand, the left ellipsoidal distribution as a multivariate normal distribution promotion is ideal; on the other hand, based on the research of the left ellipsoidal distribution, we can get many statistics used as solid properties of hypothesis test (see [3]). It is a trivial idea to extend the properties of the normal distribution family to the left ellipsoidal distribution family. Nevertheless, that is not always true. In this article, we prove that the maximum likelihood property of normal distribution cannot be extended to the left ellipsoidal distribution.

Let $Y_1,Y_2,\dots ,Y_m$ be the samples of independent identical distributions, and in the condition of normal distribution,

$$\overline{Y}=\frac{1}{m}\sum _{i=1}^m{Y}_i,S^2=\frac{1}{m}\sum _{i=1}^m(Y_i-\overline{Y}){(Y_i-\overline{Y})}^{\prime }$$

are the maximum likelihood estimations of μ, Σ, respectively, if m is big enough and $S^2$ is positive definite. When $p=1$, $S^2$ is positive definite with probability 1 if $m>n$ (see [4]). Additionally, the rank of $S^2$ does not decrease with the increasing of the columns in Y so that $S^2$ is positive definite when $m>n$. Now we discuss them in the case of $m>max(n,2)$. When one focuses on the maximum likelihood estimation, the likelihood equations need to be deduced by the matrix differential method. Therefore, we add a trivial condition that the distribution density of $E_{n\times p}(\mu ,\mathrm{\Sigma })$ is differentiable (see [5–8]).

We draw the following conclusions.

Theorem 1.1 Let $Y_1,Y_2,\dots ,Y_m$ be independent identically distributed, and $Y_i\sim E_{n\times p}(\mu ,\mathrm{\Sigma })$. The maximum likelihood estimation of μ, Σ is $\overline{Y}$, $S^2$ if and only if $Y_i\sim N_{n\times p}(\mu ,\mathrm{\Sigma })$.

Note that for the proof of Theorem 1.1 one can refer to [9].

When the variables of ${\mu }^{\ast }$ are not independent, we have the solution as follows.

Theorem 1.2 Let $Y\sim E_{n\times mp}({\mu }^{\ast },{\mathrm{\Sigma }}^{\ast })$, ${\mu }^{\ast }=\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)$, and ${\mathrm{\Sigma }}^{\ast }=m\mathrm{\Sigma }$. Besides, $Y=(Y_1,Y_2,\dots ,Y_m)$, where $Y_i$ is an $n\times p$ matrix. Then if $Y_1,Y_2,\dots ,Y_m$ are mutually independent identically distributed, the maximum likelihood estimations of μ, Σ are

$$\overline{Y}=\frac{1}{m}Y({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),S^2=\frac{1}{m}Y(I-P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p)Y^{\prime }$$

if and only if $Y\sim N_{n\times mp}({\mu }^{\ast },{\mathrm{\Sigma }}^{\ast })$, where $P_{{\mathrm{II}}_m}=\frac{1}{m}{\mathrm{II}}_m{\mathrm{II}}_m^{\prime }$, we note it $P=P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p$ in the proof below.

Theorem 1.3 Under the conditions of Theorem  1.2, if $Y_1,Y_2,\dots ,Y_m$ are not independent identically distributed, and the maximum likelihood estimations of μ, Σ are

$$\overline{Y}=\frac{1}{m}Y({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),S^2=\frac{1}{m}Y(I-P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p)Y^{\prime },$$

then Y may be not a normal distribution.

2 The proof of Theorem 1.2

Let $Y_1,Y_2,\dots ,Y_m$ be mutually identically independent distributions, and $Y_i\sim E_{n\times p}(\mu ,\mathrm{\Sigma })$. Besides, we assume that $Y=Y_1,Y_2,\dots ,Y_m$, and $Y_i=A{X}_i+\mu$, then

$$\begin{array}{c}Y=mA\left(\frac{1}{m}X\right)+\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),\hfill \\ {\mathrm{\Gamma }}_{n\times n}\left(\frac{1}{m}X\right)\stackrel{d}{=}\frac{1}{m}X,\hfill \\ E\left(\frac{1}{m}X\right){\left(\frac{1}{m}X\right)}^{\prime }=I,\hfill \end{array}$$

where Γ is a random orthogonal matrix with order n. As a result, $Y\sim E_{n\times mp}(\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$.

Now the above proposition can be transformed to a research of the form of the left stochastic ellipsoid distribution with a degraded mean.

Notice that

$$\begin{array}{c}Y=\frac{1}{m}\sum _{i=1}^m{Y}_i=\frac{1}{m}Y({\mathrm{II}}_m\otimes {\mathrm{I}}_p),\hfill \\ S^2=\frac{1}{m}\sum _{i=1}^m(Y_i-\mu ){(Y_i-\mu )}^{\prime }\hfill \\ \phantom{S^2}=\frac{1}{m}\sum _{i=1}^m(Y_i-Y){(Y_i-Y)}^{\prime }=\frac{1}{m}(YP-Y){(YP-Y)}^{\prime }=\frac{1}{m}Y(I-P)Y^{\prime },\hfill \end{array}$$

where $\mu =Y$, $P=P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p$. Therefore, utilizing the single mean degraded left stochastic ellipsoid to express Theorem 1.1, we can obtain Theorem 1.2.

3 The proof of Theorem 1.3

Based on the above discussion, we get the question: If the condition of mutually identically independent distribution is discarded, can one obtain the same solution with Theorem 1.1 from $Y\sim E_{n\times mp}(\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$? In other words, can the proposition be proved or not?

Let $Y\sim E_{n\times mp}(\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$, and then the maximum likelihood estimations of μ, Σ are

$$\overline{Y}=\frac{1}{m}Y({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),S^2=\frac{1}{m}Y(I-P)Y^{\prime }.$$

Here, $P=P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p$ if and only if $Y\sim N_{n\times mp}(\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$.

The proposition is proved not to be true through the above discussion, and we can give the paradoxical instance. Now, we firstly deduce the necessary and sufficient conditions that the maximum likelihood estimation of the parameters μ, Σ in the left stochastic ellipsoid $E_{n\times mp}(\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$ is $\overline{Y}$, $S^2$.

We assume that Y has the distribution density, namely the likelihood function is

$$L(Y,\mu ,\mathrm{\Sigma })={|\mathrm{\Sigma }|}^{-\frac{mp}{2}}f\left({(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }{(m\mathrm{\Sigma })}^{-1}(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))\right).$$

Computing the logarithm difference of both sides, we can get

$$\begin{array}{rcl}dlnL(Y,\mu ,\mathrm{\Sigma })& =& -2trG{(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }{\mathrm{\Sigma }}^{-1}\cdot d\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)\\ -tr({\mathrm{\Sigma }}^{-1}{(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }G(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)){\mathrm{\Sigma }}^{-1}+\frac{mp}{2}{\mathrm{\Sigma }}^{-1})d\mathrm{\Sigma },\end{array}$$

where $A={(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }{(m\mathrm{\Sigma })}^{-1}(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))={(a_{ij})}_{n\times n}$ and $A^{\prime }=A$.

$$\begin{array}{rcl}dlnf(A)& =& \sum _{i<j}\frac{\partial lnf(A)}{\partial a_{ij}}d{a}_{ij}\\ =& \sum _{i\cdot j}lnf{(A)}_{ij}d{a}_{ij}\\ =& trG(A)dA.\end{array}$$

We define

$$G{(A)}_{ii}=\frac{\partial lnf(A)}{\partial a_{ij}},G{(A)}_{ij}=\frac{1}{2}\frac{\partial lnf(A)}{\partial a_{ij}},G=G(A).$$

Let $P=P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p$, and then the maximum likelihood estimation of μ, Σ, which is denoted by $\hat{\mu }$, $\hat{\mathrm{\Sigma }}$, satisfies

$$\begin{array}{c}({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)G\cdot P\cdot Y^{\prime }=({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)G\cdot Y^{\prime },\hfill \\ Y((I-P)G(I-P)+\frac{P}{2}{\mathrm{I}}_p)Y^{\prime }=0.\hfill \end{array}$$

Namely,

$$\begin{array}{c}PGP=PG,\hfill \\ (I-P)G(I-P)+\frac{P}{2}(I-P)=0.\hfill \end{array}$$

Then we can obtain

$$\begin{array}{c}PGP=PG,\hfill \\ (G+\frac{P}{2})(I-P)=0.\hfill \end{array}$$

As a result, we can conclude that

$$G=-\frac{P}{2}I+CP,$$

where C is a random $mp\times mp$ matrix.

Since $G^{\prime }=G$, $CP=P{C}^{\prime }$. Besides, the 2-order differential of a likelihood function needs to satisfy: $\mathrm{\forall }d\mu ,d\mathrm{\Sigma },d^{2}lnL(Y,\mu ,\mathrm{\Sigma })<0$.

$$\begin{array}{c}\because L={|\mathrm{\Sigma }|}^{-\frac{mp}{2}}f(A),\hfill \\ \therefore dlnL=-\frac{mp}{2}tr{\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }+dlnf(A)=-\frac{mp}{2}tr{\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }+trGd(A).\hfill \end{array}$$

Additionally,

$$\begin{array}{rcl}{d}^{2}lnL& =& \frac{mp}{2}tr{\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }{\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }+trG\cdot d^{2}A+tr(dG)dA\\ =& tr(\frac{mp}{2}+{\mathrm{\Sigma }}^{-1}\left((Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)){(Y-\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }\right)){\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }{\mathrm{\Sigma }}^{-1}d\mathrm{\Sigma }\\ +trG({\mathrm{II}}_m\otimes {\mathrm{I}}_p)d{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)+tr(dG)dA\\ =& trG({\mathrm{II}}_m\otimes {\mathrm{I}}_p)d{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)+tr(dG)dA\\ =& -\frac{mp}{2}trd{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu +trC\cdot P({\mathrm{II}}_m\otimes {\mathrm{I}}_p)d{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)\\ +tr(dC)\cdot PdA\\ =& -\frac{mp}{2}trd{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu +tr({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)\cdot C\cdot ({\mathrm{II}}_m\otimes {\mathrm{I}}_p)\cdot d{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu \\ +tr(dC)\cdot PdA.\end{array}$$

Consequently, we have the conclusion as follows.

Let $Y\sim E_{n\times mp}(\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$, and then the maximum likelihood estimations of μ, Σ are $\overline{Y}$, $S^2$ if and only if

• (1) $G=-\frac{P}{2}I+CP$, $CP=P{C}^{\prime }$.

• (2) $tr({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)\cdot C\cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)\cdot d{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu +tr(dC)\cdot PdA<\frac{mp}{2}trd{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu$ $\mathrm{\forall }d\mu ,d\mathrm{\Sigma }$.

Since $Y\sim E_{n\times mp}(\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$ if and only if $C=0$, Y may not be a normal random matrix in general.

For instance, let $C=cI$, $c=-\frac{p}{2}trPA$, $P=P_{{\mathrm{II}}_m}\otimes {\mathrm{I}}_p$, and $A={(Y-\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))}^{\prime }{(m\mathrm{\Sigma })}^{-1}(Y-\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p))$.

Then the distribution density of Y is

$$L(Y,\mu ,\mathrm{\Sigma })=C{|\mathrm{\Sigma }|}^{-\frac{mp}{2}}exp\{-\frac{p}{2}(trA+{tr}^{2}PA)\},$$

where $C={(\int {|\mathrm{\Sigma }|}^{-\frac{mp}{2}}exp\{-\frac{p}{2}(trA+{tr}^{2}PA)\}dY)}^{-1}$.

Since $G=-\frac{P}{2}I+CP$ and $CP=P{C}^{\prime }$, and $(mtrd{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu )(-\frac{p}{2}trPA)+(trPdA)(-\frac{p}{2}trPdA)<0<\frac{mp}{2}trd{\mu }^{\prime }{(m\mathrm{\Sigma })}^{-1}d\mu$ so that the maximum likelihood estimations of μ, Σ are $\overline{Y}$, $S^2$, then the distribution of Y is not a normal distribution.

4 Conclusion

In this paper, we proved that if the distribution of an $n\times mp$ random matrix Y is a left ellipsoidal distribution with parameter ${\mu }^{\ast }=\mu ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p)$, ${\mathrm{\Sigma }}^{\ast }=m\mathrm{\Sigma }$, and $Y_1,Y_2,\dots ,Y_m$ are independent and identical distributions, and $Y\sim N_{n\times mp}({\mu }^{\ast },{\mathrm{\Sigma }}^{\ast })$, then the maximum likelihood estimations of μ, Σ are $\overline{Y}$, $S^2$. If $Y_1,Y_2,\dots ,Y_m$ are not independent and identical distributions, then the maximum likelihood estimations of μ, Σ are $\overline{Y}$, $S^2$. We used the matrix differential method to deduce that only and only if $Y\sim E_{n\times mp}(\mu \cdot ({\mathrm{II}}_m^{\prime }\otimes {\mathrm{I}}_p),m\mathrm{\Sigma })$, which needs to satisfy two conditions.