Categorical and Limited Dependent Variable Models
類別與受限依變數模型

Maximum Likelihood Principle
最大似然原理

• Estimation methods for ML
  最大似然估計方法
• Properties of ML methods
  最大似然方法的性質
• Generalized linear models as a mix of OLS and ML methods.
  廣義線性模型作為 OLS 與 ML 方法的混合體。

ML Estimation: Likelihood function
最大似然估計：似然函數

More comments更多評論

MLE definition最大似然估計定義

Steps in Maximum Likelihood Estimation
最大似然估計的步驟

1. Select a conditional probability distribution for the data, $f(y\mid \theta )$$f(y\mid \theta )$f(y∣theta)f(y \mid \theta).
   為數據選擇一個條件概率分佈， $f(y\mid \theta )$$f(y\mid \theta )$f(y∣theta)f(y \mid \theta) 。
2. Construct the (log-) likelihood function, $\log (f(y\mid \theta ))$$\log (f(y\mid \theta ))$log(f(y∣theta))\log (f(y \mid \theta)).
   構建（對數）似然函數， $\log (f(y\mid \theta ))$$\log (f(y\mid \theta ))$log(f(y∣theta))\log (f(y \mid \theta)) 。
3. Find the parameters that maximize the likelihood function.
   找到使似然函數最大化的參數。

Numerical optimization數值優化

• Goal is to compute $\theta$$\theta$theta\theta that maximizes $L(\theta \mid y)=\prod _{i=1}^{n}f(y\mid \theta )$$L(\theta \mid y)=\prod _{i=1}^n f(y\mid \theta )$L(theta∣y)=prod_(i=1)^(n)f(y∣theta)L(\theta \mid y)=\prod_{i=1}^{n} f(y \mid \theta).
  目標是計算出使 $L(\theta \mid y)=\prod _{i=1}^{n}f(y\mid \theta )$$L(\theta \mid y)=\prod _{i=1}^n f(y\mid \theta )$L(theta∣y)=prod_(i=1)^(n)f(y∣theta)L(\theta \mid y)=\prod_{i=1}^{n} f(y \mid \theta) 最大化的 $\theta$$\theta$theta\theta 。
• The MLE is the $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} that maximizes this function. It is then the value where
  最大似然估計(MLE)就是使此函數最大化的 $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} 。因此，它是在該值處

• Define $\ell (\theta \mid y)=\log (L(\theta \mid y))$$\ell (\theta \mid y)=\log (L(\theta \mid y))$ℓ(theta∣y)=log(L(theta∣y))\ell(\theta \mid y)=\log (L(\theta \mid y)) as the log-likelihood function.
  將 $\ell (\theta \mid y)=\log (L(\theta \mid y))$$\ell (\theta \mid y)=\log (L(\theta \mid y))$ℓ(theta∣y)=log(L(theta∣y))\ell(\theta \mid y)=\log (L(\theta \mid y)) 定義為對數似然函數。
• The score function, or first derivative of $\ell$$\ell$ℓ\ell is
  得分函數，即 $\ell$$\ell$ℓ\ell 的一階導數是

• Setting this to zero will produce the optimum of the MLE.
  將其設為零將產生最大似然估計的最優解。

How is this done in the computer?
這在計算機中是如何實現的？

• Computers do not do calculus well.
  電腦並不擅長處理微積分。
• Instead, the computer tries candidate values ${\theta }^j$${\theta }^j$theta^(j)\theta^{j} :
  相反地，電腦會嘗試候選值 ${\theta }^j$${\theta }^j$theta^(j)\theta^{j} ：

• Continue these iterations until either ${\theta }^{j+1}-{\theta }^j<ϵ$${\theta }^{j+1}-{\theta }^j<ϵ$theta^(j+1)-theta^(j) < epsilon\theta^{j+1}-\theta^{j}<\epsilon or $\ell ({\theta }^{j+1}\mid y)-\ell ({\theta }^j\mid y)<ϵ$$\ell \left({\theta }^{j+1}\mid y\right)-\ell \left({\theta }^j\mid y\right)<ϵ$ℓ(theta^(j+1)∣y)-ℓ(theta^(j)∣y) < epsilon\ell\left(\theta^{j+1} \mid y\right)-\ell\left(\theta^{j} \mid y\right)<\epsilon.
  持續這些迭代，直到滿足 ${\theta }^{j+1}-{\theta }^j<ϵ$${\theta }^{j+1}-{\theta }^j<ϵ$theta^(j+1)-theta^(j) < epsilon\theta^{j+1}-\theta^{j}<\epsilon 或 $\ell ({\theta }^{j+1}\mid y)-\ell ({\theta }^j\mid y)<ϵ$$\ell \left({\theta }^{j+1}\mid y\right)-\ell \left({\theta }^j\mid y\right)<ϵ$ℓ(theta^(j+1)∣y)-ℓ(theta^(j)∣y) < epsilon\ell\left(\theta^{j+1} \mid y\right)-\ell\left(\theta^{j} \mid y\right)<\epsilon 的條件。
• This, and its variants, are versions of Newton-Raphson optimization.
  這個方法及其變體，都是牛頓-拉弗森優化法的版本。

MLE Properties最大似然估計的性質

• Normality: Estimated parameters $(\hat{\theta })$$(\widehat{\theta })$( hat(theta))(\hat{\theta}) are normal: $\hat{\theta }\sim N(\theta ,V[\theta ])$$\widehat{\theta }\sim N(\theta ,V[\theta ])$hat(theta)∼N(theta,V[theta])\hat{\theta} \sim N(\theta, V[\theta]).
  常態性：估計參數 $(\hat{\theta })$$(\widehat{\theta })$( hat(theta))(\hat{\theta}) 服從常態分佈： $\hat{\theta }\sim N(\theta ,V[\theta ])$$\widehat{\theta }\sim N(\theta ,V[\theta ])$hat(theta)∼N(theta,V[theta])\hat{\theta} \sim N(\theta, V[\theta]) 。
• Efficiency / Minimum Variance: The estimated parameters have minimum variance mong admmissible estimators, the Cramer-Rao lower bound for the variance.
  效率性/最小變異性：估計參數在所有可容許估計量中具有最小變異數，達到 Cramer-Rao 方差下界。
• Consistency: As the number of observations tends to infinity, the MLE $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} converges in probability to the population values. That is, $\mathrm{plim}\hat{\theta }\to \theta$$\mathrm{plim}\widehat{\theta }\to \theta$plim hat(theta)rarr theta\operatorname{plim} \hat{\theta} \rightarrow \theta.
  一致性：當觀察次數趨近於無限時，最大似然估計量 $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} 會以概率收斂至總體參數值。也就是說， $\mathrm{plim}\hat{\theta }\to \theta$$\mathrm{plim}\widehat{\theta }\to \theta$plim hat(theta)rarr theta\operatorname{plim} \hat{\theta} \rightarrow \theta 。
• Sufficiency: The MLE completely characterizes the observed data. That is, there is no other parameter $\gamma$$\gamma$gamma\gamma that should be included in the model to characterize the distribution of the data.
  充分性：最大似然估計量能完整刻畫觀察數據。也就是說，不存在其他參數 $\gamma$$\gamma$gamma\gamma 需要納入模型以刻畫數據的分佈。
• Asymptotic Unbiasedness: The probability that the expected values of the estimated parameters equal the population values is one as the sample size goes to infinity. That is, plim $E[\hat{\theta }]\to \theta$$E[\widehat{\theta }]\to \theta$E[ hat(theta)]rarr thetaE[\hat{\theta}] \rightarrow \theta.
  漸近無偏性：當樣本量趨近於無限時，估計參數的期望值等於總體參數值的概率為一。即 plim $E[\hat{\theta }]\to \theta$$E[\widehat{\theta }]\to \theta$E[ hat(theta)]rarr thetaE[\hat{\theta}] \rightarrow \theta 。

THESE PROPERTIES ONLY HOLD IF THE MODEL IS SPECIFIED CORRECTLY!
上述性質僅在模型設定正確時成立！

More properties更多性質

• For the ML estimators, these properties are all asymptotic. This means that they only hold for large samples. In fact, in most cases the MLE estimators are biased and inefficient in small samples.
  對於最大似然估計量而言，這些屬性都是漸近的。這意味著它們僅適用於大樣本。事實上，在多數情況下，最大似然估計量在小樣本中是有偏且低效的。
• These properties hold for “correctly specified models.” In many cases if your model is incorrect some or all of these properties will be invalidated. See Gourieoux, Monfort and Trognon (1984a and 1984b) and White (1994) for when the model is misspecified, estimator can often be referred to as a quasi- or pseudo-MLE which may have some desirable properties.
  這些屬性適用於「正確設定的模型」。在許多情況下，如果模型設定不當，部分或全部這些屬性將失效。關於模型設定錯誤時的情況，可參見 Gourieoux、Monfort 和 Trognon（1984a 與 1984b）以及 White（1994）的研究，此時估計量常被稱為擬或偽最大似然估計量，可能仍具有某些理想屬性。

An important non-asymptotic result
一個重要的非漸近結果

Computing coefficients and their variances
計算係數及其變異數

Log Likelihood defined對數概似定義

Log likeihood optimization
對數概似優化

Information matrix信息矩陣

GLMs廣義線性模型

• For many maximum likelihood problems, the solution that maximizes the likelihood can be found by a repeated application of generalized least squares.
  對於許多最大概似問題，可以通過反覆應用廣義最小平方法來找到使概似最大化的解。
• This is a generalized linear model and works in many cases where the dependent variable follow exponential family distributions (other than a normal).
  這是一個廣義線性模型，適用於許多情況下依變量遵循指數族分佈（非正態分佈）的情形。
• This set of results was established by Nelder and Wedderburn (1972) with important generalizations later by Jorgensen (1987)
  這組結果由 Nelder 和 Wedderburn（1972 年）確立，後由 Jorgensen（1987 年）進行了重要推廣。

Basic Notation基本符號

Canonical form標準形式

• Canonical form is a 1-1 transformation that reduces the complexity of the model.
  標準形式是一種一對一變換，能降低模型的複雜度。
• If $t(z)=z$$t(z)=z$t(z)=zt(z)=z, the model is in canonical form for $z$$z$zz. If not, redefine $y=t(z)$$y=t(z)$y=t(z)y=t(z) to do so.
  若 $t(z)=z$$t(z)=z$t(z)=zt(z)=z ，則該模型對於 $z$$z$zz 呈現標準形式。若非如此，則重新定義 $y=t(z)$$y=t(z)$y=t(z)y=t(z) 以達成之。
• If $u(\zeta )=\zeta$$u(\zeta )=\zeta$u(zeta)=zetau(\zeta)=\zeta, the model is in canonical form for $\zeta$$\zeta$zeta\zeta. If not, force the canonical form by defining $\theta =u(\zeta )$$\theta =u(\zeta )$theta=u(zeta)\theta=u(\zeta).
  若 $u(\zeta )=\zeta$$u(\zeta )=\zeta$u(zeta)=zetau(\zeta)=\zeta ，則該模型對於 $\zeta$$\zeta$zeta\zeta 呈現標準形式。若非如此，則透過定義 $\theta =u(\zeta )$$\theta =u(\zeta )$theta=u(zeta)\theta=u(\zeta) 強制轉換為標準形式。
• The final canonical form will be $f(y\mid \theta )=\exp (y\theta -b(\theta )+c(y))$$f(y\mid \theta )=\exp (y\theta -b(\theta )+c(y))$f(y∣theta)=exp(y theta-b(theta)+c(y))f(y \mid \theta)=\exp (y \theta-b(\theta)+c(y))
  最終的標準形式將是 $f(y\mid \theta )=\exp (y\theta -b(\theta )+c(y))$$f(y\mid \theta )=\exp (y\theta -b(\theta )+c(y))$f(y∣theta)=exp(y theta-b(theta)+c(y))f(y \mid \theta)=\exp (y \theta-b(\theta)+c(y))

Why do this?為何要這麼做？

• Defines a family of models that can be estimated by the same methods.
  定義了一組可以用相同方法估計的模型家族。
• Simplifies a class of ML models based on pdfs and pmfs in the exponential family.
  簡化了基於指數族概率密度函數(pdfs)和概率質量函數(pmfs)的一類機器學習模型。
• This is common group of models: normal, Poisson, negative binomial, logit, probit, gamma, etc.
  這是常見的模型群組：常態分佈、泊松分佈、負二項分佈、邏輯迴歸、概率單位迴歸、伽瑪分佈等。
• Score functions for the MLE of GLMs have a simple form:
  廣義線性模型(GLM)的最大似然估計(MLE)的得分函數具有簡單形式：

• Standard ML properties apply to the estimates.
  標準最大似然估計性質適用於這些估計值。

Link functions for GLMs
廣義線性模型（GLM）的連結函數

• Start with a standard linear model
  從標準線性模型開始

• What we want is some mapping from $V$$V$VV to the mean of the outcome of interest, or $E[y]=\mu :$$E[y]=\mu :$E[y]=mu:E[y]=\mu:
  我們需要的是某種從 $V$$V$VV 映射到感興趣結果的平均值，即 $E[y]=\mu :$$E[y]=\mu :$E[y]=mu:E[y]=\mu: 的關係

• The link function $g()$$g()$g()g() connects the structure of the explanatory variables and their coefficients to the linear predictor $g(\mu )=\theta$$g(\mu )=\theta$g(mu)=thetag(\mu)=\theta.
  連結函數 $g()$$g()$g()g() 將解釋變量及其係數的結構與線性預測變量 $g(\mu )=\theta$$g(\mu )=\theta$g(mu)=thetag(\mu)=\theta 連接起來

What the link does
連結函數的作用

• Gill (2007: 542) `'The link function connects the stochastic component that describes some response variable from a wide variety of forms to all the standard normal theory supporting the systematic component through the mean function"
  吉爾（2007: 542）所述：「連結函數通過均值函數，將描述某種反應變量的隨機成分與支持系統性成分的所有標準正態理論連接起來」
• Huh?什麼？

• Ah! So it ties together the non-normal response to a linear regression model.
  啊！原來它是將非正態響應與線性回歸模型聯繫起來的。

Components of GLMs廣義線性模型的組成部分

1. Stochastic component: $y$$y$yy is the random of stochastic component that is iid with mean $\mu$$\mu$mu\mu.
   隨機成分： $y$$y$yy 是獨立同分布且均值為 $\mu$$\mu$mu\mu 的隨機或隨機成分。
2. Systematic component: $\theta =X\beta$$\theta =X\beta$theta=X beta\theta=X \beta with Gauss-Markov basis.
   系統性成分：以高斯-馬可夫基為基礎的 $\theta =X\beta$$\theta =X\beta$theta=X beta\theta=X \beta 。
3. Link function: stochastic and systematic parts of the model are linked by a function of $\theta$$\theta$theta\theta, which is the canonical link that lets us “trick” a linear model.
   連結函數：模型的隨機部分與系統性部分通過 $\theta$$\theta$theta\theta 的函數相連結，這是讓我們能夠「欺騙」線性模型的典型連結。
4. Residuals: can construct deviance or weighted residuals to evaluate the model.
   殘差：可以構建偏差或加權殘差來評估模型。

Common link functions常見的連結函數

Estimation, IWLS, and Properties of Estimation of GLMs
估計、迭代加權最小平方法與廣義線性模型的估計特性

• Estimation of GLMs is by iteratively weighted least squares (IWLS).
  廣義線性模型的估計採用迭代加權最小平方法（IWLS）。
• Uses a repeated weighted least squares computation to adjust the values of $X\beta =g(\mu )$$X\beta =g(\mu )$X beta=g(mu)X \beta=g(\mu) until $X{\beta }^j-X{\beta }^{j+1}<ϵ$$X{\beta }^j-X{\beta }^{j+1}<ϵ$Xbeta^(j)-Xbeta^(j+1) < epsilonX \beta^{j}-X \beta^{j+1}<\epsilon.
  使用重複加權最小平方法計算來調整 $X\beta =g(\mu )$$X\beta =g(\mu )$X beta=g(mu)X \beta=g(\mu) 的值，直到 $X{\beta }^j-X{\beta }^{j+1}<ϵ$$X{\beta }^j-X{\beta }^{j+1}<ϵ$Xbeta^(j)-Xbeta^(j+1) < epsilonX \beta^{j}-X \beta^{j+1}<\epsilon 。
• See Gill 2007: 549-551 for details.
  詳情請參閱 Gill 2007 年著作第 549 至 551 頁。

Properties of GLMs廣義線性模型的特性

• For cases where the canonical link functions are used, GLMs are MLEs.
  在使用典型連結函數的情況下，廣義線性模型即為最大似然估計。
• So in these cases, we can say that the results / properties of MLE hold.
  因此，在這些情況下，我們可以說最大似然估計的結果/特性成立。
• In other cases, namely where the full parametric assumptions of GLM or MLE are not met, the estimates are quasi-maximum likelihood estimates.
  在其他情況下，即當廣義線性模型(GLM)或最大似然估計(MLE)的完整參數假設未被滿足時，這些估計值即為擬最大似然估計。
• Quasi-MLEs are consistent and asymptotically normal, but not efficient (see McCullagh 1983, McCullagh and Nelder 1989, Gourieroux, Monfort and Trognon, 1984a,b).
  擬最大似然估計具有一致性與漸近常態性，但不具有效性（參見 McCullagh 1983 年文獻、McCullagh 與 Nelder 1989 年著作，以及 Gourieroux、Monfort 與 Trognon 1984 年 a、b 兩篇論文）。

Interpreting GLM and MLE Coefficients
解讀廣義線性模型與最大似然估計係數

• Quantities of interest: predicted means, probabilities, odds, etc.
  關注的量值：預測均值、概率、勝算比等。
• Hypothesis tests: Wald, likelihood ratio, and Lagrangean multiplier (score) tests.
  假設檢定：沃爾德檢定、概似比檢定及拉格朗日乘數（得分）檢定。
• Want to be able to compute these and report them in a consistent manner across different models.
  希望能夠計算這些量值，並在不同模型間以一致的方式進行報告。

The "Intuitive Version" of tests
「直觀版本」的檢定

• Since inference in the ML models is based on asymptotic properties, we need to derive new test statistics for these models based on these properties and assumptions.
  由於 ML 模型的推論基於漸近性質，我們需要根據這些性質和假設為這些模型推導新的檢定統計量。
• The nice thing about ML estimators though is that we only have to derive the testing framework once.
  不過，ML 估計量的一個優點是我們只需推導一次檢定框架。
• Since all the MLEs share the same properties, hypothesis testing is done the same way in each of the models.
  由於所有 MLE 都具有相同的性質，因此在每個模型中進行假設檢定的方式都是相同的。
• Tests are based on measuring scaled distances around the likelihood surface (function).
  檢驗基於對概似函數曲面周圍的尺度距離進行測量。

Three tests三種檢驗

• There are three general hypothesis tests used with ML models.
  最大概似模型中使用三種通用的假設檢驗。
• These are the Wald (W), likelihood ratio (LR) and Lagrangean multiplier (LM) tests.
  這些分別是沃爾德檢驗（W）、概似比檢驗（LR）和拉格朗日乘數檢驗（LM）。
• Each test has a ${\chi }^2$${\chi }^2$chi^(2)\chi^{2} distribution. The three tests are asymptotically equivalent for the same null hypothesis.
  每個檢定都有一個 ${\chi }^2$${\chi }^2$chi^(2)\chi^{2} 分佈。對於相同的虛無假設，這三個檢定是漸進等價的。

Test Construction測驗建構

• Assume that we have ML estimates such that
  假設我們有最大似然估計量使得

• Suppose that our hypothesis conjectures that $\theta ={\theta }_0$$\theta ={\theta }_0$theta=theta_(0)\theta=\theta_{0} - the ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} values being the ones under the null hypothesis.
  假設我們的假設推測 $\theta ={\theta }_0$$\theta ={\theta }_0$theta=theta_(0)\theta=\theta_{0} ——這些 ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} 值是在虛無假設下的數值。
• What does this mean in terms of a log likelihood function? That is, what so the likelihoods of $L(\theta \mid Y)$$L(\theta \mid Y)$L(theta∣Y)L(\theta \mid Y) and $L({\theta }_0\mid Y)$$L\left({\theta }_0\mid Y\right)$L(theta_(0)∣Y)L\left(\theta_{0} \mid Y\right) look like?
  這對數似然函數意味著什麼？也就是說， $L(\theta \mid Y)$$L(\theta \mid Y)$L(theta∣Y)L(\theta \mid Y) 和 $L({\theta }_0\mid Y)$$L\left({\theta }_0\mid Y\right)$L(theta_(0)∣Y)L\left(\theta_{0} \mid Y\right) 的似然函數看起來是什麼樣子？

Likelihood tests似然比檢驗

What can we compare?
我們可以比較什麼？

1. the value of the likelihood for $\theta$$\theta$theta\theta and ${\theta }_0$${\theta }_0$theta_(0)\theta_{0}
   對於 $\theta$$\theta$theta\theta 和 ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} 的概似值
2. the curvature of the likelihood
   概似函數的曲率
3. so we need a way to combine these two pieces of information
   因此我們需要一種方法來結合這兩部分信息

Formal derivations形式推導

1. Look at the heights of the likelihood functions evaluated at $\theta$$\theta$theta\theta and ${\theta }_0$${\theta }_0$theta_(0)\theta_{0}, weighted by the likelihood function’s curvature. This is the likelihood ratio.
   觀察在 $\theta$$\theta$theta\theta 和 ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} 處評估的似然函數高度，並以似然函數的曲率加權。這就是似然比。
2. Look at the distance between $\theta$$\theta$theta\theta and ${\theta }_0$${\theta }_0$theta_(0)\theta_{0}, weighted by the likelihood function’s curvature. This is a Wald test.
   觀察 $\theta$$\theta$theta\theta 和 ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} 之間的距離，並以似然函數的曲率加權。這便是沃爾德檢驗。
3. Look at the slope of the likelihood function at ${\theta }_0$${\theta }_0$theta_(0)\theta_{0}, denoted $S\left({\theta }_0\right)$$S\left({\theta }_0\right)$S(theta_(0))S\left(\theta_{0}\right) and see how far it is from zero, again weighted by the curvature of the likelihood function at this point. This is a score or Lagrangean test.
   觀察似然函數在 ${\theta }_0$${\theta }_0$theta_(0)\theta_{0} 處的斜率，記為 $S\left({\theta }_0\right)$$S\left({\theta }_0\right)$S(theta_(0))S\left(\theta_{0}\right) ，看看它與零的距離有多遠，再次以該點似然函數的曲率加權。這是一個分數或拉格朗日檢驗。

How do we measure "curvature"?
我們如何衡量「曲率」？

• The degree of curvature of a function is determined by the function’s second derivatives.
  函數的曲率程度由函數的二階導數決定。
• For a multivariate problem, this is the same as the Hessian - the matrix of second derivatives with respect to all the parameters for the likelihood function.
  對於多變量問題，這與 Hessian 矩陣相同——即關於似然函數所有參數的二階導數矩陣。

Recall回顧

• The negative of the expected value of Hessian matrix, is known as the information matrix:
  海森矩陣的期望值之負值，被稱為信息矩陣：

• The inverse of the information matrix gives the variance-covariance matrix for the ML parameters:
  信息矩陣的逆矩陣給出了最大似然參數的方差-協方差矩陣：

• Thus, we will be able to use the inverse of the variance-covariance of the maximum likelihood parameters to “weight” the distances we are measuring in these tests.
  因此，我們將能夠利用最大似然參數的方差-協方差矩陣的逆矩陣來「加權」這些測試中所測量的距離。

Notation符號表示

• Log-likelihood function對數似然函數

• First derivative $k\times 1$$k\times 1$k xx1k \times 1 vector of $\ell$$\ell$ℓ\ell, scores:
  一階導數 $k\times 1$$k\times 1$k xx1k \times 1 向量 $\ell$$\ell$ℓ\ell ，得分：

• Second derivative $k\times k$$k\times k$k xx kk \times k matrix of $\ell$$\ell$ℓ\ell, Hessian:
  二階導數 $k\times k$$k\times k$k xx kk \times k 矩陣的 $\ell$$\ell$ℓ\ell ，海森矩陣：

• Information matrix equality, evaluated at $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} :
  信息矩陣等式，在 $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} 處評估：

Likelihood ratio test概似比檢定

Wald test沃爾德檢定

Lagrangean multiplier or score test
拉格朗日乘數檢定或得分檢定

Comparison of tests檢驗比較

• In small samples it is generally the case that the three tests can give different results. Typically this means that the test statistics are ordered: $W\ge LR\ge LM$$W\ge LR\ge LM$W >= LR >= LMW \geq L R \geq L M.
  在小樣本情況下，通常會出現三種檢驗方法得出不同結果的情形。這通常意味著檢驗統計量存在排序關係： $W\ge LR\ge LM$$W\ge LR\ge LM$W >= LR >= LMW \geq L R \geq L M 。
• For a further, very intuitive discussion of likelihood based tests, see A. Buse. 1982. “'Likelihood ratio, Wald, and Lagrange Multiplier Tests: An Expository Note.”
  關於基於概似比檢驗的進一步且非常直觀的討論，可參閱 A. Buse 於 1982 年發表的《概似比檢驗、沃爾德檢驗與拉格朗日乘數檢驗：一篇說明性註記》。
  American Statistician 36(3): 153-157.
  美國統計學家 36(3): 153-157.

Equivalences and choice of the tests
等效性與檢驗方法的選擇

Things you can and cannot do
可行與不可行之事

• LR, LM, and $W$$W$WW tests must be computed for the same data.
  LR、LM 及 $W$$W$WW 檢定必須基於相同數據計算。
• Tests must be nested: linear or non-linear parametric restrictions of some larger parameter space.
  檢定必須具有嵌套性：是對更大參數空間的線性或非線性參數限制。
• Results are asymptotic! Tests are equal only as samples are large.
  結果為漸近性質！僅當樣本量趨大時檢定結果才會相等。

Computing quantities of interest
計算感興趣的數量

• What do we want to compute / report?
  我們想要計算/報告什麼？
• What might be these be?
  這些可能是什麼？

1. Hypothesis tests.假設檢驗
2. Comparative statics about how changes in the data or model parameters affect the dependent variable or its probability.
   關於數據或模型參數變化如何影響依變量或其概率的比較靜態分析。
3. Measures of our uncertainty about the parameters of the model and associated comparative statics.
   關於模型參數及其相關比較靜態分析的不確定性度量。

More specifically更具體地說

• Predicted values: means. variances, probabilities, odds.
  預測值：均值、變異數、概率、勝算比。
• Conditional predictions: based on specific values of the independent variables that are “representative” observations.
  條件預測：基於自變量的特定值，這些值是「代表性」的觀察結果。
• Marginal effects, or the changes in the dependent variable that are expected for changes in the independent variable.
  邊際效應，或預期因自變量變化而導致的因變量變化。
• First differences: effect of a unit change in an independent variable on the dependent variable.
  首次差異：自變量單位變化對因變量的影響。

A running example一個運行的範例

• To illustrate, use a logit model. In the logit model, we analyze the probability of observing a value of 1 on the dependent variable:
  舉例來說，使用一個 logit 模型。在 logit 模型中，我們分析觀察到依變量值為 1 的概率：

• Suppose we have available the MLE, $\hat{\beta }$$\widehat{\beta }$widehat(beta)\widehat{\beta}. The next sections demonstrate the different methods that can be used to interpret how changes in $X$$X$XX are then related to changes in $\mathrm{Pr}(Y=1\mid X)$$\mathrm{Pr}(Y=1\mid X)$Pr(Y=1∣X)\operatorname{Pr}(Y=1 \mid X).
  假設我們有可用的最大似然估計量 $\hat{\beta }$$\widehat{\beta }$widehat(beta)\widehat{\beta} 。接下來的章節將展示可以用來解釋 $X$$X$XX 的變化如何與 $\mathrm{Pr}(Y=1\mid X)$$\mathrm{Pr}(Y=1\mid X)$Pr(Y=1∣X)\operatorname{Pr}(Y=1 \mid X) 的變化相關聯的不同方法。
• Note that this generalizes using the link functions $g()$$g()$g()g() for any GLM.
  請注意，這推廣了使用連結函數 $g()$$g()$g()g() 於任何廣義線性模型(GLM)的做法。

Expected values, predicted outcomes
期望值，預測結果

Choosing $\tilde{X}$$\tilde{X}$tilde(X)\tilde{X}選擇 $\tilde{X}$$\tilde{X}$tilde(X)\tilde{X}

• Use the mean values of the X’s.
  使用 X 的平均值。
• Use the range of the X’s.
  使用 X 的範圍。
• Use the extrema of the X’s.
  使用 X 的極值。
• Select X’s based on in-sample cases of interest (as generated by hypotheses).
  根據假設生成的樣本內感興趣案例選擇 X。
• Choose relevant percentile points of the X’s: 25%, 50%, 75%-iles, etc.
  選擇 X 的相關百分位點：25%、50%、75%等。

Partial changes or First derivatives
部分變更或一階導數

• A second method is based on the instantaneous change in the probability for a change in $X$$X$XX, which is the first derivative of the quantity of interest.
  第二種方法基於概率對 $X$$X$XX 變化的瞬時變化率，即所關注量的一階導數。
• For the logit model this would be
  對於 Logit 模型而言，這將是

• Notice that we still have to choose a vector of X’s at which to evaluate this derivative.
  請注意，我們仍需選擇一個 X 向量來評估此導數。
• The reason why is that $E[f(x)]\ne f(E[x])$$E[f(x)]\ne f(E[x])$E[f(x)]!=f(E[x])E[f(x)] \neq f(E[x]). We typically are going to want the former, not the latter. So just plugging in the average X 's for the derivative is generally the wrong thing to do.
  原因在於 $E[f(x)]\ne f(E[x])$$E[f(x)]\ne f(E[x])$E[f(x)]!=f(E[x])E[f(x)] \neq f(E[x]) 。通常我們會想要前者，而非後者。因此，直接將平均 X 值代入導數計算通常是錯誤的做法。

Relative Marginal Effects
相對邊際效應

• Notice that this leads to a more general computation that allows you to assess the relative marginal effect of two variables:
  請注意，這導向了一種更通用的計算方法，讓您能夠評估兩個變量的相對邊際效應：

• This quantity allows you to assess the relative impact of each variable on the change in probability. If the number is greater (less) than 1 , the $k^{\prime }th\left({\ell }^{\prime }th\right)$$k^{\prime }th\left({\ell }^{\prime }th\right)$k^(')th(ℓ^(')th)k^{\prime} t h\left(\ell^{\prime} t h\right) variable has a bigger impact on the probability $\mathrm{Pr}(Y=1)$$\mathrm{Pr}(Y=1)$Pr(Y=1)\operatorname{Pr}(Y=1) than the ${\ell }^{\prime }th\left(k^{\prime }th\right)$${\ell }^{\prime }th\left(k^{\prime }th\right)$ℓ^(')th(k^(')th)\ell^{\prime} t h\left(k^{\prime} t h\right) variable.
  這個數值讓你能評估每個變量對概率變化的相對影響。如果該數值大於（小於）1，則變量 $k^{\prime }th\left({\ell }^{\prime }th\right)$$k^{\prime }th\left({\ell }^{\prime }th\right)$k^(')th(ℓ^(')th)k^{\prime} t h\left(\ell^{\prime} t h\right) 對概率 $\mathrm{Pr}(Y=1)$$\mathrm{Pr}(Y=1)$Pr(Y=1)\operatorname{Pr}(Y=1) 的影響大於（小於）變量 ${\ell }^{\prime }th\left(k^{\prime }th\right)$${\ell }^{\prime }th\left(k^{\prime }th\right)$ℓ^(')th(k^(')th)\ell^{\prime} t h\left(k^{\prime} t h\right) 。

First Differences首差法

• This then measures the effect of changing a particular value of $X$$X$XX, say $x_k$$x_k$x_(k)x_{k} from $x_k$$x_k$x_(k)x_{k} to $x_k+\delta$$x_k+\delta$x_(k)+deltax_{k}+\delta.
  這便衡量了將特定值 $X$$X$XX （例如從 $x_k$$x_k$x_(k)x_{k} 變為 $x_k+\delta$$x_k+\delta$x_(k)+deltax_{k}+\delta ）改變時所產生的效應。
• This computation can be done for the logit model as follows:
  此計算可以針對邏輯模型進行如下操作：

• Notice this computation depends on the starting values for $x_k$$x_k$x_(k)x_{k}.
  請注意，此計算取決於 $x_k$$x_k$x_(k)x_{k} 的起始值。

Choosing the change, $\delta$$\delta$delta\delta
選擇變更， $\delta$$\delta$delta\delta

• A unit change: $\delta =1$$\delta =1$delta=1\delta=1.
  一個單位的變化： $\delta =1$$\delta =1$delta=1\delta=1 。
• A centered discrete change: $\delta ={\overline{x}}_k\pm 0.5$$\delta ={\overline{x}}_k\pm 0.5$delta= bar(x)_(k)+-0.5\delta=\bar{x}_{k} \pm 0.5.
  一個中心化的離散變化： $\delta ={\overline{x}}_k\pm 0.5$$\delta ={\overline{x}}_k\pm 0.5$delta= bar(x)_(k)+-0.5\delta=\bar{x}_{k} \pm 0.5 。
• A change from zero to one (for dummy variables).
  從零到一的變更（針對虛擬變數）。
• A one standard deviation change: $\delta ={\sigma }_{x_k}$$\delta ={\sigma }_{x_k}$delta=sigma_(x_(k))\delta=\sigma_{x_{k}} (for continuous variables).
  一個標準差的變更： $\delta ={\sigma }_{x_k}$$\delta ={\sigma }_{x_k}$delta=sigma_(x_(k))\delta=\sigma_{x_{k}} （針對連續變數）。

Odds Ratios勝算比

• A final interpretation method that can be used with any likelihood model where the mean or location parameter is of the form $\lambda =\exp (X\beta )$$\lambda =\exp (X\beta )$lambda=exp(X beta)\lambda=\exp (X \beta) is the odds-ratio.
  最後一種解釋方法可用於任何似然模型，其中均值或位置參數的形式為 $\lambda =\exp (X\beta )$$\lambda =\exp (X\beta )$lambda=exp(X beta)\lambda=\exp (X \beta) ，即勝算比。
• This means this method can be used with Poisson and other event count regressions and most of the event history models.
  這意味著該方法可應用於泊松回歸及其他事件計數回歸模型，以及大多數事件史模型。
• Recall that the odd are defined from a set of probabilities, so for an event with probability $p$$p$pp, the odds are $\frac{p}{1-p}$$\frac{p}{1-p}$(p)/(1-p)\frac{p}{1-p}.
  回想一下，勝算是由一組概率定義的，因此對於概率為 $p$$p$pp 的事件，其勝算為 $\frac{p}{1-p}$$\frac{p}{1-p}$(p)/(1-p)\frac{p}{1-p} 。

Logit Odds對數勝算

• For the logit example, odds of the event give $X$$X$XX is defined as,
  以 logit 模型為例，當事件勝算為 $X$$X$XX 時，其定義如下：

• This implies that $\log (\mathrm{\Omega }(X))=X\beta$$\log (\mathrm{\Omega }(X))=X\beta$log(Omega(X))=X beta\log (\Omega(X))=X \beta. So the change in the log odds for a change in $x_k$$x_k$x_(k)x_{k} can be defined as
  這意味著 $\log (\mathrm{\Omega }(X))=X\beta$$\log (\mathrm{\Omega }(X))=X\beta$log(Omega(X))=X beta\log (\Omega(X))=X \beta 。因此， $x_k$$x_k$x_(k)x_{k} 變化所導致的對數勝算變化可定義為

Issues with (log) odds
（對數）勝算的問題

• Further, the change in the odds depends on the baseline point at which this change is measured.
  此外，勝算的變化取決於測量此變化的基準點。
• Since we abstract away from this with the measure, we need to have a clear idea of the scale of the data and the baseline point of comparison.
  由於我們透過該測量方式抽象化這一點，因此需要清楚了解數據的規模及比較的基準點。

Mayhew Example: Poisson regression
梅休範例：卜瓦松迴歸

• The basic model is as follows:
  基本模型如下：

• The log-likelihood function for this model is
  此模型的對數似然函數為

Mayhew data梅休數據

Stata outputStata 輸出結果

Marginal effects邊際效應

Details詳情

Deficit effects赤字效應

Results結果

The following shows how this is done:
以下展示了如何進行此操作：

More更多

• We see that at the time of the greatest surplus, Congress passes on average 12.48 laws, all else constant. At the times of greatest deficit, they pass about 10.79 major laws, all else constant. This difference of nearly 2 laws is a better statement of the difference between the two different regimes.
  我們可以看到，在盈餘最大的時期，國會平均通過 12.48 項法律，其他條件保持不變。而在赤字最大的時期，他們通過約 10.79 項主要法律，其他條件同樣不變。這近 2 項法律的差異，更能說明兩種不同制度之間的區別。
• Note however that it may not make sense to leave the other variables at their means when doing this computation.
  但請注意，在進行此計算時，將其他變量保持在平均值可能並不合理。
• Since all other variables are discrete, it makes more sense to put them at specific values. We can do this with more modifications to the at option of the mfx command.
  由於所有其他變量都是離散的，更合理的做法是將它們設定為特定值。我們可以通過進一步修改 mfx 命令的 at 選項來實現這一點。
• Consider the difference in legislation between the unified government periods of the Great Society and the rest of the era when there if a Republican president. The first condition is when Unified = 1 and Mood=1. In the second case both are zero.
  考量「大社會」時期統一政府與共和黨總統執政其他時期在立法上的差異。第一種情況是當「統一」=1 且「民意」=1 時；第二種情況則是兩者皆為零。

Conclusion結論

Reporting uncertainty about Quantities
報告數量不確定性

• Uncertainty about the ML estimates: The estimates are generated conditional on the data, subject to a probability model. This introduces uncertainty since the parameters of the probability model mean that the likelihood depends on the uncertainty of the likelihood function.
  關於最大似然估計的不確定性：這些估計是在數據條件下生成的，並受概率模型影響。這引入了不確定性，因為概率模型的參數意味著似然函數依賴於其自身的不確定性。
• Uncertainty about Data Generating Process: There is a fundamental uncertainty in any data generation process. In ML models, we assume a particular distribution for the data – which generates a measure of the uncertainty of the estimates based on the parameters and the variance of the data under the DGP.
  數據生成過程的不確定性：任何數據生成過程都存在根本性的不確定性。在最大似然模型中，我們假設數據服從特定分佈——這會基於參數和數據在 DGP 下的方差產生估計的不確定性度量。
• Uncertainty about the Data Itself: The data used to compute the likelihood may not be fixed, inducing more uncertainty.
  數據本身的不確定性：用於計算似然函數的數據可能並非固定，從而引發更多不確定性。

Computing tips: How to do this right
計算技巧：如何正確操作

• margins in Stata: Clearly, this is the workhorse command if you are working with ML and linear models in Stata. It is worth spending some time reading about how this command works.
  Stata 中的 margins 指令：顯然，這是你在 Stata 中使用機器學習和線性模型時的主力指令。值得花些時間閱讀這個指令的運作方式。
• marginaleffects in R: There is a whole package for doing this (right).
  R 中的 marginaleffects 套件：有一個完整的套件專門用來做這件事（沒錯）。
• Brute Force Methods: The computations for the effects interpretations are not so hard to do by hand if you have the fitted models and the summary statistics on the regressors.
  暴力計算方法：如果你有擬合好的模型和回歸變量的摘要統計量，那麼手工計算效應解釋的運算並不那麼困難。
• Clarify: King et al. have exploited a well known property of ML % models to develop software that can be used to do statistical % simulations of quantities of interest for ML models. The basic % idea is as follows: Suppose we have an ML model that produces % estimates $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta}. Since these are normally distributed, % we can simulate values of them and use these values to compute % whatever quantity of interest we want based on them.
  Clarify：King 等人利用了一個機器學習模型的著名特性來開發軟體，該軟體可用於對機器學習模型感興趣的數量進行統計模擬。基本想法如下：假設我們有一個機器學習模型產生估計值 $\hat{\theta }$$\widehat{\theta }$hat(theta)\hat{\theta} 。由於這些估計值呈正態分佈，我們可以模擬這些值，並用這些值來基於它們計算我們想要的任何感興趣的數量。

Examples of LDV models
有限應變數模型的例子

• Poisson and count regressions (2)
  卜瓦松與計數迴歸（2）
• Logit model for binary outcomes
  二元結果的 Logit 模型

Poisson model: POTUS Vetoes
泊松模型：美國總統否決權

• Number of public laws passed
  通過的公共法律數量
• Seat percentage in Congress (measure of partisan conflict)
  國會席位百分比（衡量黨派衝突的指標）
• Annual unemployment年度失業率
• Presidential approval總統支持率
• Succession in office職位繼任
• War戰爭
• Year in term任期中的年份
• Term in office (first or second)
  任期（第一或第二任）

Basic Count Model Estimation
基本計數模型估計

Basic Poisson count model output
基本泊松計數模型輸出

Basic negative binomial count model output
基本負二項計數模型輸出

link \(=\log )\)

Logit Model: Survey Vote Choice
羅吉特模型：調查投票選擇

Variables變量

• PID1: Three point self reported partisan identification scale in 1990; -1 = GOP, $0=-$$0=-$0=-0=- Independent, 1 = Democrat.
  PID1：1990 年三點自報黨派認同量表；-1 = 共和黨，0 = 獨立人士，1 = 民主黨。
• PVOTE88: Presidential vote in 1988. 0 = Bush, 1 = Dukakis, 2 = No vote.
  PVOTE88：1988 年總統選舉投票。0 = 布希，1 = 杜卡基斯，2 = 未投票。
• IDEO71: Seven point ideology scale, self reported in 1990. 0 = Liberal, $6=$$6=$6=6= Conservative.
  IDEO71：七點意識形態量表，1990 年自我報告。0 = 自由派， $6=$$6=$6=6= 保守派。
• INCOME: Income in 1990, self reported. 0 = lowest category, 23 = Highest.
  INCOME：1990 年自我報告的收入。0 = 最低類別，23 = 最高類別。
• SEX: 0 = Male, 1 = Female.
  SEX：0 = 男性，1 = 女性。
• RACE: 0 = White, 1 = Non-white.
  RACE：0 = 白人，1 = 非白人。
• PB1: Percentage of the respondent’s congressional district that is black in 1990.
  PB1：1990 年受訪者所屬國會選區中非裔人口百分比。
• SSOUTH: 1=Lives in the Solid South, 0 = Otherwise.
  SSOUTH：1=居住於南方保守州，0=其他情況。

Setup Code設定代碼

Logit Estimation邏輯回歸估計

---

Plot 1 (basic single term plot)
圖 1（基礎單項圖）

Plot 2 (Fancier termplot)
圖 2（進階版項圖）

Margins plot 1邊際效應圖 1

Margins plot 2邊際效應圖 2

Margins plot 3邊際效應圖 3

Margins all at once
一次顯示所有邊際效應

Mayhew Important Legislation Count Time Series
梅休重要立法計數時間序列

Data Setup and Model
數據設置與模型

DG ModelsDG 模型

Some comparative statics
一些比較靜態分析

terms

terms術語

Some comparisons with marginaleffects
與 marginaleffects 的一些比較

Marginal effects plots邊際效應圖

Another one另一個