From: Further advances on Bayesian Ying-Yang harmony learning
Year | BYY harmony learning formulation | |
2004 | (a) | H(p||q) in Equation 9 with R={Y,θ} is proposed in Sect.II(B) of Xu (2004b), not only integrating the thread of data smoothing regularisation via 1997(b) and 1999(b) and of normalisation regularisation via 2000(c) into a specific formulation of a priori; but also covering a usual priori q(θ) as a component. |
(b) | Subsequent elaborations are referred to Sect.3.4 in Xu (2007a), Sect.3.4 in Xu (2007b), Sect.2 in Xu (2008), Eq. (8) in Xu (2009), especially to Sect.4 in Xu (2010a) and Sect.4 in Xu (2012a) for recent surveys. | |
2007 | Beyond 2001(a), efforts on designing the structure of p(R|X) based on q(X|R) and q(R) progress from the early concept of bi-directional architecture further towards. | |
(a) | Either a preservation principle p(Y|θ,X _{ N })=q(Y|θ,X _{ N }), e.g. by Eq. (40) in Xu (2001a), Eq. (24), and Eq. (27) in Xu (2001c); | |
(b) | Or that p(Y|θ,X _{ N }) preserves certain statistics of q(Y|θ,X _{ N }), e.g. equal covariance by Eqs. (72)(73) in Xu (2007a), which are elaborated under the name of uncertainty conversation or variety preservation between Ying and Yang, see pp69-72 in Xu (2009), with details referred to Sect.4.2 in Xu (2010a) and Sect.3.2.2 in Xu (2012a). | |
2008 | Learning tasks are summarised into three levels of inverse problems and integrated into a unified representation of BYY system, see Xu (2008, 2009), and an introduction in Sect.1 of Xu (2010a). | |
(a) | Radon-Nikodym derivative based formulation of Ying-Yang harmony information was proposed, with degenerated cases covering Shannon information and Kullback Leibler information. Details are referred to Sect.4.1 in Xu (2010a) and an overview in Figure five of Xu (2012a). | |
(b) | Hierarchical temporal BYY harmony learning was developed in Sect. 5 of Xu (2010a), see Figures twelve and fourteen in Xu (2010a) and Figure eleven in Xu (2012a). | |
(c) | BYY system provides an all-in-one formulation for unsupervised, supervised and semi- supervised learning, see Sect.4.4 in Xu (2010a) and Table two in Xu (2012a). | |
2011 | Co-dim matrix pair formulation and a hierarchy of co-dim matrix pairs for BYY harmony learning have been proposed, with details referred to Sect.2.2, Sect.4 and Figure three in Xu (2011). Its special cases cover not only several typical learning models but also de-noised Gaussian mixture (see Algorithm 13), manifold learning as previously discussed about Equation 80, and the dual formulation as previously introduced in Equations 55 and 56. | |
Type | BYY system design | |
3-A | Started from the very beginning in 1995 Xu (1995), BYY system was classified into three architectures (3-A), i.e. forward architecture with q(X|R) in a free structure, backward architecture with p(R|X) in a free structure, and bi-directional architecture with both q(X|R) and p(R|X) in parametric structures, rather thoroughly examined before the mid of 2000th (Xu 2000c,2001a,2001e,2002, 2003a,2004a,2004c). | |
3-P | Focuses are turned to three principles (3-P) for designing the structures of each component in a BYY system, i.e. the principle of least redundancy for q(Y), the principle of divid-and-conquer for q(X|R), and the principle of uncertainty conversation or variety preservation for p(Y|X), as stated above by the item 2007(a) and (b). An overview is referred to Figure three in Xu (2012a). |