Ed230B/C

Linear Statistical Models: Regression

The Matrix

Some Definitions

  • Scalar
  • Vector
  • Matrix

    Some Matrix Operations

  • Multiplication by a scalar
  • Matrix addition (subtraction)
  • Transpose of a matrix
  • Matrix multiplication
  • Inverse of a matrix

    Multiplication by a scalar

    Matrix Addition (& subtraction)

    Transpose of a matrix

    Note:

    (A')' = A

    (A*B)' = B' * A'

    Matrix Multiplication

    Note:

    In general A*B does not equal B*A (commutative law)

    (A*B)*C = A*(B*C) (associative law)

    A*(B+C) = A*B + A*C

    (B+C)*A = B*A + C*A (distributive law)

    Identity Matrix

    Note:

    A * I = A

    I * A = A

    Inverse of a Matrix

  • Mathematical notation: B = A-1

    Note:

    A-1 * A = A * A-1 = I

    In scalar terms:

    a-1 = 1/a

    a-1 * a = 1

    The Seven Basic Matrices
    Raw Score Matrix (Data Matrix)[vertical]
    Raw Score SSCP[square]
    Deviation Score Matrix[vertical]
    Deviation SSCP[square]
    Covariance Matrix[square]
    Standard Score Matrix[vertical]
    Correlation Matrix[square]

    Raw Score Matrix (Data Matrix)
    Vertical Matrix

    Raw Score SSCP
    Square Symmetric Matrix

    Deviation Score Matrix
    Vertical Matrix

    Deviation Score SSCP
    Square Symmetric Matrix

    Covariance Matrix
    Square Symmetric Matrix

    Standard Score Matrix
    Vertical Matrix

    Correlation Matrix
    Square Symmetric Matrix

    Matrix Formulae for Regression Coefficients

    b = (X´X)-1X´Y

    b = (Zx´Zx)-1Zx´Zy

    The b Vector

    Regression Equations in Matrix Form

    Y = Xb + e

    Zy = Zxb + Ze

    Matrices in Stata

    Stata has a complete set of matrix commands available to users. In this unit we will present a simple example to show how the commands work. The example will be intentionally primitive and will not make use of all of the features that Stata has for doing estimations based upon the matrix language. Out goal is not to create a finished professional program but to show the matrix commands in action.

    Example Program

    
program define matreg
  version 8.0
 
    syntax varlist(min=2 numeric) [if] [in] [, Level(integer $S_level)]
    marksample touse                       /* mark cases in the sample */
    tokenize "`varlist'"

    quietly matrix accum sscp = `varlist' if `touse'
    local nobs = r(N)
    local df = `nobs' - (rowsof(sscp) - 1) /* df residual */

    matrix XX = sscp[2...,2...]            /* X'X */
    matrix Xy = sscp[1,2...]               /* X'y */

    matrix b = Xy * syminv(XX)             /* (X'X)-1X'y */
    local k = colsof(b)                    /* number of coefs */
    matrix hat = Xy * b'
    matrix V = syminv(XX) * (sscp[1,1] - hat[1,1])/`df'
    matrix C = corr(V)
    matrix seb = vecdiag(V)
    matrix seb = seb[1, 1...]
    matrix t = J(1,`k',0)
    matrix p = t

    local i = 1
    while `i' <= `k' {
      matrix seb[1,`i'] = sqrt(seb[1,`i'])
      matrix t[1,`i'] = b[1,`i']/seb[1,`i']
      matrix p[1,`i'] = tprob(`df',t[1,`i'])
      local i = `i' + 1
    }

  display
  display "Dependent variable: `1'"
  display
  display "Regression coefficients"
  matrix list b
  display
  display "Standard error of coefficients"
  matrix list seb
  display
  display "Values of t"
  matrix list t
  display
  display "P values for t"
  matrix list p

  matrix drop sscp XX Xy b hat V seb t p C 
end

    Testing the Program

    The program listed above needs to be copied and saved into a file named matreg.ado. This file can be place into your default data directory but it is probably better if it is place into the ado/Personal directory.

    use http://www.philender/courses/data/hsbdemo

    matreg write read female

    regress write read female

    Linear Statistical Models Course

    Phil Ender, 15Jan98