# 基于模型的机器学习 Model Based Machine Learning - PRML大神、微软剑桥研究院院长

(think) #1

MBML的目标是“提供支持创建各种定制模型的单一开发框架”。 这个框架来自三个关键思想的重要汇合：

1. 采用贝叶斯观点，
2. 使用因子图（一种概率图模型），和
3. 快速，确定，有效和近似推理算法的应用。

MBML的关键思想

1. 贝叶斯推理 (Bayesian Inference)

The first key idea enabling this different framework for machine learning is Bayesian inference/learning. In MBML, latent/hidden parameters are expressed as random variables with probability distributions. This allows for a coherent and principled manner of quantification of uncertainty in the model parameters. Once the observed variables in the model are fixed to their observed values, initially assumed probability distributions (i.e. priors) are updated using the Bayes’ theorem.

This is in contrast to the traditional/classical machine learning framework where model parameters are assigned average values that are determined by optimizing an objective function. Bayesian inference on large models over millions of variables is similarly implemented using the Bayes’ theorem but in a more complex manner. This is because Bayes’ theorem is an exact inference technique that is intractable over large datasets. In the past decade, the increase in processing power of computers has enabled research and development of fast and efficient inference algorithms that can scale to large data like Belief Propagation (BP), Expectation Propagation (EP), and Variational Bayes (VB).

2. Factor Graphs

The second cornerstone to MBML is the use of Probabilistic Graphical Models (PGM), particularly factor graphs. A PGM is a diagrammatic representation of the joint probability distribution over all random variables in a model expressed as a graph. Factor graphs are a type of PGM that consist of circular nodes representing random variables, square nodes for the conditional probability distributions (factors), and vertices for conditional dependencies between nodes (Figure 1). They provide a general framework for modeling the joint distribution of a set of random variables.

The joint probability P ( μ , X ) over the whole model in Figure 1 is factorized as:

P ( μ , X )= P ( μ )* P ( X | μ )

Where μ is the model parameter and X are the set of observed variables.

Figure 1: A Factor Graph

In factor graphs, we treat the latent parameters as random variables and learn their probability distributions using Bayesian inference algorithms along the graph. Inference/learning is simply the product of factors over a subset of variables in the graph. This allows for easy implementation of local message passing algorithms.

3. Probabilistic Programming (PP)

There’s a revolution in Computer Science called Probabilistic programming (PP) where programming languages are now built to compute with uncertainity in addition to computing with logic. This means that existing programming languages can now support random variables, constraints on variables and inference packages. Using a PP language, you can now describe a model of your problem in a compact form with a few lines of code. Then an inference engine is called to automatically generate inference routines (and even source code) to solve that problem. Some notable examples of PP languages include Infer.Net, Stan, BUGS, church, Figarro and PyMC. In this blog post, we will access Stan algorithms through the R interface.

Stages of MBML

There are 3 steps to model based machine learning, namely:

1. Describe the Model : Describe the process that generated the data using factor graphs.
2. Condition on Observed Data : Condition the observed variables to their known quantities.
3. Perform Inference : Perform backward reasoning to update the prior distribution over the latent variables or parameters. In other words, calculate the posterior probability distributions of latent variables conditioned on observed variables.

(think) #2

PRML大神、微软剑桥研究院院长Chris Bishop与John Winn的机器学习新书。最入门级别的机器学习图书， 全书从实际案例开始讲，数学公式很少，非常适合当做读PRML之前的入门。这个专集记录这本书的读书笔记，可能会断更，但是希望能把这本书读完。这本书目前放出来前6章，后面还会有两章陆续放出。

# How can machine learning solve my problem?

“我对机器学习方法和技术的选择感到不知所措。 有太多东西需要学习！“

“我不知道使用哪种算法，或者为什么我的问题会比另一种更好。”

“我的问题似乎不符合任何标准算法。”

image.png

### How to read this book

model-based machine learning ：一种机器学习方法，其中关于问题域的所有假设都以模型的形式明确。 然后，该模型用于创建特定于模型的算法，以学习或推理域。 该过程的算法创建部分可以自动化。
model ：关于问题域的一组假设，以精确的数学形式表示，用于创建机器学习解决方案。
Infer.NET ：Microsoft Research Cambridge开发的软件框架，可以根据模型定义自动执行基于模型的机器学习。 可从以下网址下载 the Infer.NET website.

##### References
1. [Minka et al., 2014] Minka, T., Winn, J., Guiver, J., Webster, S., Zaykov, Y., Yangel, B., Spengler, A., and Bronskill, J. (2014). Infer.NET 2.6. Microsoft Research Cambridge. http://research.microsoft.com/infernet.
2. [Lunn et al., 2000] Lunn, D., Thomas, A., Best, N., and Spiegelhalter, D. (2000). WinBUGS – a Bayesian modelling framework. Statistics and Computing, 10:325–337. MRC Biostatistics Unit. http://www.mrc-bsu.cam.ac.uk/software/bugs.
3. [Goodman et al., 2008] Goodman, N., Mansinghka, V. K., Roy, D. M., Bonawitz, K., and Tenenbaum, J. B. (2008). Church. MIT. http://projects.csail.mit.edu/church/wiki/Church.
4. [Stan Development Team, 2014] Stan Development Team (2014). Stan: A C++ Library for Probability andSampling, Version 2.5.0.
5. [Bishop, 2006] Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.

(think) #3