# An introduction to ergodic theory - Walters P.

**Download**(direct link)

**:**

**35**> 36 37 38 39 40 41 .. 99 >> Next

7j w(Cj) им m(Dk) >n(Dk)

v , n ч m(-A‘ n D*) i m(A> n = £ m(Dk) —77ГТ log‘

i.A

m{Dk) m(Dk

or < - H(,pJ/9). Therefore, H(s4j9) < H(s*№).

(vi) Put = Jf in (v).

(vii) Use (i) and (v).

(viii) Set 9 = .Sf in (vii).

(ix), (x) Clear from definitions. □ The following also fits in with our intuitive ideas

Theorem 4.4. Let sJ, 42 be finite sub-algebras of 38. Then

(i) Il(s//V) = 0 (i.e. H(s/ vtf) = H(<#)) iff.9/ с: <.

(ii) НЫ/У) = H(s/) (i.e. H(.mfvV) = H(s/) + H(V)) iffsf and (G are independent (i.e. m(A n C) = m(A) ■ m(C) whenever A e sd, С e rS).

Proof. Let i&jrf) = {Au ..., Ak}, £(<£') = {C1;... ,CP}. Without loss of generality we can assume all these sets have non-zero measure.

(i) si с '£ means for each i and each j either т(Л, n Cj) = m(C;) or m(A( n Cj) = 0. Clearly this implies H(&//<6) = 0.

Suppose H(s//<9) = 0. Then

$4 j CoEuiinoiijI Umropy

83

and since

\se must have

for each i,j. Нгпсе either »i(A, n Cj) = m(Cj) or /п(А; n Cj) = 0. Therefore

(ii) If .с/ and <6 are independent we quickly see from the definition of H(.c/fY,) that H(.«/,"<?) = НЫ). To prove the converse suppose H(.&//%') = Ну.с/) Then

If we fix i and apply Theorem 4.2. wilh a_, = m(Cj) and Xj = m{Al n С;)/ш(С,) we get

with equality only when w(/4; Cj)/m(Cj) does not depend on j. If л, denotes this constant value then by summing the equations m(Ai n Cy) = u^nlC) o\er j we have aj = hiM,)- Hence equality holds only when ш(Л; n Cj) = ш(С,)1?|(Л,). However equation (*) says equality holds in (**) for each i and so m(A, n Cj) = m(Cj)m(Aj) for all i, j. Therefore m(A n C) = m(C)m(A) whenever A e л/, С e . □

Theorem 4.5. Let V denote the space of all finite sub-algebras of У6 where two finite algebras s/,(6 are identified if sJ = (6. Then d(.v/,W) = H(.s/j'fi) + H(’6j.rJ) is и metric on V.

(A corresponding statement about the space of finite partitions can be mudc).

Pkook We have d(.p//<?) > 0 and equality holds iff.с/ = ((! (Theorem 4.4). Also HWcJ) < H^w'C/S) = H(V/$) + H(s//V v Qi) < H(V/S) + IH.dM) and similarly H(&/.s/) < Therefore d(s/,&) <

We can also define conditional entropy H(sJ/.У') when sJ is a finite sub-cr-algebra of .ift and & is an arbitrary sub-cr-algebra of Л. To do this we use the conditional expectation map Ll(X,.jtf,m)-+ Ll(X,&,m).

I['# is a finite sub-cr-algebra of.^ with = {Clt , Cp} then E{f/(£)(x) = B=1 /c^KlM^j)) icjfdm. If s/ is also finite and q(s?) = [Au ... ,Ak}

(**)

d[j/,<e) -r dw.m).

□

>4

•J tnlropy

;nen

n С

HI-/ Ч\ = -4 ■*:(.-!. С йог- -Г "'-'С,

= - Z I / -I l*JS F[/a ?. )(/»!

= — I Z £i/.,/?f)Sos£f/4i This leacb to the following (Minilion.

Definition 4.8. Lei (Abe a probability space. If.с/ is a finite sub-<7-algebra of.^ with <;(•»') = ,/5*1, and is an arbitrary sub-c-algebra

of 16 the cniropy of .с/ given '? is the number

| V E{/AJ^:)\o"E(xAJ.^)dm.

‘ i 1

Rtmark. Since E( /fF) is a positive linear operator and Y)=t %л, = 1 wc have 0 < E(yAi ‘./"Kv) < 1 a.e.. and therefore к

Е(гл,1^)М\о^Е(1А>1у7^к)<к max (-rlogf) = ke.

I 16(0.1]

Hence //(cZ/.tF) is finite.

One can show thai ihc properties listed in Theorem 4.3 are satisfied by this more general conditional entropy. However, they can also be deduced from Theorem 4.3 (in the case when (X,3d,in) has a countable basis) by using a limit theorem that wc shall use for another purpose. To prepare for the proof we give the following lemma. If {}'[ is a family of sub-<7-algebras of 41 we let \J'"l ! denote the smallest sub-c-algebra containing nil the .F„.

Lemma 4.6. Let (A\jS,m) he a probability space and let {&„}'* he an increasing sequence of suh-a-ulgebras of ■£. Denote \Д1, by &. For each f e L2(X,.!/!, in) we have

||£(//’^„) - £(//^)||2 - 0.

Proof. Recall that E(-j.¥„) is the orthogonal projection of L2(X,.tf.ni) onto L2(X,.rn,m). Lei йе/. Choose B„ e JF„ with m(Bn Д B) -* 0. ’Since £i/лЛЛ.) >s that member of L2(X,^'„,m) closest to y_0 we have

II~ 7.u\\\ ^ \\Zb„ ~ 7.в\\1 = т(Вя Д В) - 0.

Since finite linear combinations of characteristic functions arc dense in L2(X,.?,m) wc have |j£(/i/^„) —/i||2-»0 for all he L2(X,.F,m). Hence

$4 3 Condiiional Entropy , 85

M*

if / e L‘ m), —

ilE(fi^n) - E(f/.W)IU -» 0 because £(£(//&)/&„) = E(f/rsn). □

Remarks

(1) The same result holdB for a decreasing sequence of sub-rr-algebras wuh Г],'= , //'„ = p.

(2) If f e L'(X,.rt,m) and {■>„Jff increase to .T7 then Doob’s martingale theorem implies E(f/.•?„) -» E(f/S') a.e. and in Ll(X.jtf,m). The corresponding statement holds for a decreasing sequence of (7-algebras Ucc Pat'ihasarathy [2], p. 230).

Theorem 4.7. Let be a probability space. Let s/J be a finite sub-

algebra of 38 and let {&„}i be an increasing sequence of sub-a-ulgebras of .'Ji with \/*= ] &n = &. Then Н(.я//&„) -* H(sJ/^').

Proof. Let q]{s/) = {А1л. .. ,AK). From Lemma 4.6. we know that for each i

**35**> 36 37 38 39 40 41 .. 99 >> Next