Quadratic forms二次形式にじけいしき and positive definite matrices正定値行列せいていちぎょうれつ

1Introduction

The important point of this lecture is that a quadratic form二次形式にじけいしき is a quadratic expression determined by a matrix行列ぎょうれつ, and that the sign of the form can be read from the eigenvalues固有値こゆうち of a symmetric matrix対称行列たいしょうぎょうれつ.

Eigenvalues are not only tools for diagonalization. In quadratic forms they control shape, convexity, and how values grow in different directions.

2Terms and definitions

A quadratic form二次形式にじけいしき associated with a real square matrix $A$A is

In the complex case, a Hermitian formエルミート形式けいしき is

where $A$A is Hermitian. This condition ensures $x^{*}Ax$x^*Ax is real, so positivity and negativity make sense.

A real symmetric matrix $A$A is positive definite正定値せいていち if

for every $x\ne 0$x\ne0. It is positive semidefinite if $x^{T}Ax\text{[PARSE ERROR: Undefined("Command("ge")")]}0$x^TAx\ge0, negative definite if $x^{T}Ax<0$x^TAx<0 for all $x\ne 0$x\ne0, and indefinite if the expression takes both positive and negative values.

3Plan

Use orthogonal diagonalization. If $A=QD{Q}^T$A=QDQ^T and $y=Q^{T}x$y=Q^Tx, then

Thus the sign of the quadratic form is reduced to the signs of the eigenvalues.

4Intuitive explanation

A quadratic form measures how a value grows in each direction. If it is positive definite, moving away from the origin in any nonzero direction gives a positive value, like the bottom of a bowl. If it is indefinite, some directions give positive values and others give negative values, producing saddle behavior.

5Precise explanation

5.11. The skew-symmetric part does not contribute

For any real square matrix $A$A,

Let $S=(A+A^T)/2$S=(A+A^T)/2 be symmetric and $K=(A-A^T)/2$K=(A-A^T)/2 be skew-symmetric. Since

we have

Thus only the symmetric part matters for a real quadratic form.

5.22. Positive definiteness and eigenvalues

For a real symmetric matrix $A$A,

Indeed, write $A=QD{Q}^T$A=QDQ^T and $y=Q^{T}x$y=Q^Tx. Then

If all ${\lambda }_i>0$\lambda_i>0, then $x\ne 0$x\ne0 implies $y\ne 0$y\ne0, so the sum is positive. Conversely, if some ${\lambda }_i<0$\lambda_i<0, the corresponding eigenvector gives a negative value. If some ${\lambda }_i=0$\lambda_i=0, the corresponding nonzero eigenvector gives value $0$0, so $A$A is not positive definite. Thus positive semidefinite is not enough; strict positivity of all eigenvalues is required.

5.33. Sylvester's criterion

For a real symmetric matrix $A$A, positivity of all leading principal minors is equivalent to positive definiteness:

This is useful computationally, while the eigenvalue criterion explains the underlying geometry.

5.44. Sylvester's law of inertia

Under an invertible change of variables $x=Py$x=Py,

This is a congruence transformation. Even if the quadratic form is rewritten as

the numbers of positive, negative, and zero square terms are invariant. This is Sylvester's law of inertia and shows that the sign structure does not depend on the chosen coordinates.

6Concrete example

Let

Then

The eigenvalues are $3$3 and $1$1, both positive, so $A$A is positive definite. In orthogonally diagonalized coordinates,

The cross term disappears after changing to principal axes.

7Hessian test for extrema

Quadratic forms appear in second derivatives of multivariable functions. If $a$a is a critical point, meaning $\nabla f\left(a\right)=0$\nabla f(a)=0, and the Hessian matrix $H_f\left(a\right)$H_f(a) is positive definite, then $a$a is a strict local minimum. If the Hessian is negative definite, $a$a is a strict local maximum; if it is indefinite, $a$a is a saddle point. Without the critical-point assumption, positive definiteness of the Hessian alone does not imply a local extremum.

8Criteria

• For a quadratic form, only the symmetric part of the matrix contributes.
• A real symmetric matrix is positive definite exactly when all eigenvalues are positive.
• It is positive semidefinite exactly when all eigenvalues are nonnegative.
• It is indefinite when positive and negative eigenvalues both occur.
• At a critical point, a positive definite Hessian is a sufficient condition for a strict local minimum.

9Scope and limitations

Eigenvalue tests for definiteness assume a symmetric or Hermitian matrix. For a nonsymmetric real matrix, the quadratic form only sees the symmetric part. In the complex case, use $x^{*}Ax$x^*Ax, not $x^{T}Ax$x^TAx. If $A$A is not Hermitian, $x^{*}Ax$x^*Ax may be complex, so positivity is not directly meaningful.

10Final forms

11In one sentence

A quadratic form二次形式にじけいしき is a quadratic expression whose sign and shape are determined by the eigenvalues of its symmetric or Hermitian matrix.

12Theorem: equivalent conditions for positive definiteness

Let $A$A be a real symmetric matrix. The following are equivalent:

• $A$A is positive definite正定値せいていち.
• All eigenvalues of $A$A are positive.
• In an orthogonal diagonalization $A=QD{Q}^T$A=QDQ^T, every diagonal entry of $D$D is positive.
• All leading principal minors are positive, by Sylvester's criterion.

The core proof uses orthogonal diagonalization. If $A=QD{Q}^T$A=QDQ^T and $y=Q^{T}x$y=Q^Tx, then $Q$Q preserves nonzero vectors and

If all ${\lambda }_i>0$\lambda_i>0, this is positive for every $y\ne 0$y\ne0. Conversely, if some ${\lambda }_j\text{[PARSE ERROR: Undefined("Command("le")")]}0$\lambda_j\le0, choose $y=e_j$y=e_j to get $x^{T}Ax\text{[PARSE ERROR: Undefined("Command("le")")]}0$x^TAx\le0, so $A$A is not positive definite.

13Boundary cases: positive semidefinite versus positive definite

For

we have

But for $x=(0,1{)}^T\ne 0$x=(0,1)^T\ne0, $x^{T}Ax=0$x^TAx=0. Thus $A$A is positive semidefinite, not positive definite. Positive definiteness does not allow a nonzero flat direction.

For

we have $x^{T}Bx=x_{1^2}-x_{2^2}$x^TBx=x_1^2-x_2^2, which can be positive or negative depending on direction. Hence $B$B is indefinite.

14Practical $2\times 2$2\times2 test

For a real symmetric matrix

Sylvester's criterion gives

Computationally this is convenient. Geometrically it is another way of saying that all principal-axis stretch factors, namely the eigenvalues, are positive.

15Optimization warning

In the second-derivative test, the assumption $\nabla f\left(a\right)=0$\nabla f(a)=0 is essential. A positive definite Hessian means that the quadratic approximation opens upward near the point, but if the gradient is nonzero, a linear term remains and the point itself need not be a local minimum.

16Exercise link

17Related links