# Statistical analysis of mixture distribution - Smith A.F.M

ISBN 0-470-90763-4

**Download**(direct link)

**:**

**81**> 82 83 84 85 86 87 .. 103 >> Next

However, in the case of multiparameter finite mixture models, complications arise in applying recursions (6.4.1) or (6.4.5) in the computation and inversion of I(tf,*). Numerical integration is often necessary and the fact that we arc dealing with incomplete data will add to the complications. Suppose, with reference to

(6.4.1), that we write

k,=[«w>r

f

Then the following alternatives to V~ 1 suggest themselves as more easily calculated approximations.

(a) nl((f/'), where if/' is fixed at an initial parameter estimate, or is only updated infrequently, rather than at each iteration.

n

(b) ? J,{t/'*), where Jj( ) denotes the sample information matrix from the ith

i= i

observation.

(c) t W?)-

i= 1 n

Id) I

i= 1

Suggestion (a) corresponds to a familiar modification to the method of scoring for obtaining maximum likelihood estimates. Suggestion (b) is similar to Newton’s method for the same purpose. Suggestions (c) and (d) would be very useful in providing recursive calculation of {Vn If (c) is used, for example, we obtain

Vn-'=Vn\\ + /(*•). (6A6)

Recursion (6.4.1), with exactly this modification, was used by Walker and Duncan ( 1967) in the recursive estimation of parameters in a linear logistic model

for quantal response.

Theoretical and practical investigation of all these modifications would be of interest, but we shall just concentrate on the following modification of (6.4.1), which is suggested by considering the link between complete and incomplete data

problems:

= ft,+ [»',( fc)r‘s(wfc). n = 0-1.......... ,6A7)

where 1C(<J/) denotes the Fisher Information matrix corresponding to a

observation.

The modification of (6.4.7) analogous to (6.4.5) is given by

*.?, = + [(» +1 W?J]''«<*.? f " = 0''........ ,6A8’

Although such recursions do not lead to full asymptotic efficiency, it is pi s; ih't in some cases to guarantee ^»-consistency and asymptotic norma ity

iQA Statistical analysis offinite mixture distributions

following theorem, stated in its univariate version, is taken from Sacks (1958) and Fabian (1968).

Theorem 6.4.1

Given conditions corresponding to those above and provided 2/(^0)/c(^0)“1 > l,

JV{0.7c(^o)-1 - 1]}

in distribution, as n-*oo.

As will become clear later, it does not always happen that 2I(t//0) > Ic(i//0).

Suppose

0<J!< 2Wo)/IM < 1

and we consider the recursion

.f + n -<1 ’ ¹2/<(.?„)-1 S(x„ , „ i?„), « = 0,1.... (6.4.9)

Then, according to Fabian (1968),

n'/2($„ - iAo) - N{0, /c(^0)~ 2/(«Ao)/[2/(«/'oUc('Ao)'1 - /*]}

in distribution, as n -*? oo.

Thus, provided there is some information in the incomplete data (/(tp0) > 0), a modified version of (6.4.7) leads to a consistent, asymptotically normal estimator.

Multidimensional versions of these results are, of course, required for many applications of interest, but details will not be given here: see Sacks (1958) and Fabian (1968).

The important practical advantage of recursions (6.4.7), (6.4.8), and (6.4.9) is that lc(ij/) will usually be much easier to evaluate and, in the case of an information matrix, much easier to invert than I(tJ/).

In the following, we derive versions of some of these recursions for two examples involving finite mixtures. We denote by xlt x2,... a sequence of incomplete observations and by yu y2,... corresponding ‘complete’ versions. Thus, given x, y belongs to a subset ^(x) of the overall sample space x and, if g(y\tj/) denotes the p.d.f. of y, then

g{y\'l')dy

4{x)

(see Dempster, Laird, and Rubin, 1977).

6.4.3 Illustrations of the general recursion

Example 6.4.1 Estimation of mixing weights

We consider first the case of a mixture of k known densities fj{’),j= 1 ,---,k:

Sequential problems and procedures 209

where the n1,...,nk are all non-zero probabilities. Then

S/x | n) = - A(x)]/p(x|/r), j = 1 k- 1,

Djr(x| it) = - [fj{x) -A(x)] [/r(x) -/*(x)]/[p(x|/r)]2,

j 1 , . . . , /c 1, r — 1 ft — 1,

and

/,>(*) =

[//*) -/*(*)] [/,(*)-AM]p(x |ff) 1 dx, j, r = 1 k - 1.

Verification of the regularity conditions is subsumed in Kazakos (1977) and Smith and Makov (1978).

For the special case of k = 2, with n, = tc, we obtain, for (6.4.1), as in Kazakos (1977),

7T*+l = 7T* + [n/(7r*)]“1[/1(Xn+1)-/2(Xn+1)]/p(Xn+||7T*), 71 = 1,2,...,

with /(71) =

We concentrate our discussion on the case k = 2. Here the incompleteness is caused by ignorance of the source of an observed x; is it component 1 or component 2? We may write

y = (x, z),

where zT = (1,0) or (0,1) according to the source. Thus

log0(y|7r) = ZTU(7r) + ZTv(7T)

where UT(7t) = [log 71, log (1 7r)]

and

vT(7r) = [log/!(x),log/2(x)].

Thus (cf. Section 3.2), /c(tt) = 1/tt(1 - n) and (6.4.7) becomes

7in+ ! = 7In + M_ 17Tn( l — 7tJ[/,(xn+1) -fl(xn+\)]IP(Xn + 1 1^")-

(6.4.10)

n T 1 n rp - ----

Asymptotically, if I(n) > \1M Theorem 6.4.1 holds. Otherwise, strong consistency can still be guaranteed (see Makov and Smith, *977, Smit an Makov, 1978) and recursions like (6.4.9) may also be used (cf. Section

Example 6.4.2 Mixture of two univariate normals Let

**81**> 82 83 84 85 86 87 .. 103 >> Next