SVD and pseudoinverse擬似逆行列ぎじぎゃくぎょうれつ - Basic Exercises

date2026-07-14document_iddoc_95cbd6a79eba1d697066743883354eabdescription特異値分解と擬似逆行列を、潰れない方向、零特異値、最小二乗解、最小ノルム解から確認する演習である。prerequisites特異値分解の入口 / 擬似逆行列の基本 / 複素内積とユニタリ行列type問題演習content_typeexercisestatusactive

mathlinear-algebraexercisesvdpseudoinverse

data/lecture/math/linear-algebra/introduction-to-singular-value-decomposition.lecture.n.md data/lecture/math/linear-algebra/pseudoinverse-basics.lecture.n.md data/lecture/math/linear-algebra/complex-inner-products-and-unitary-matrices.lecture.n.md

1Exercise plan

Singular value decomposition特異値分解とくいちぶんかい decomposes a matrix行列ぎょうれつ into orthogonal input directions, nonnegative stretch factors, and orthogonal output directions. The pseudoinverse擬似逆行列ぎじぎゃくぎょうれつ inverts only the nonzero singular values特異値とくいち and leaves zero singular values at 0.

2Problem 1

Fill in the blanks.

"Singular value decomposition特異値分解とくいちぶんかい decomposes a rectangular matrix into an orthogonal basis on the input side, nonnegative stretch factors, and an orthogonal basis on the output side. The pseudoinverse擬似逆行列ぎじぎゃくぎょうれつ ____ the nonzero singular values特異値とくいち and ____ the zero singular values, thereby describing least-squares solutions最小二乗解さいしょうにじょうかい and minimum-norm solutions最小ノルム解."

2.1Answer example

"takes reciprocals of" and "keeps them as $0$ ."

2.2Explanation

Taking the reciprocal of a zero singular value特異値とくいち would require division by 0. The pseudoinverse擬似逆行列ぎじぎゃくぎょうれつ does not force recovery of directions that have been collapsed.

3Problem 2

Find one compact SVD[コンパクトSVD] of

A = (\begin{matrix} 3 & 0 \\ 0 & 0 \\ 0 & 4 \end{matrix}) .

3.1Answer example

A^{T} A = (\begin{matrix} 9 & 0 \\ 0 & 16 \end{matrix}),

so the singular values特異値とくいち are $4, 3$ . Choose right singular vectors特異とくいベクトル

v_{1} = (\begin{matrix} 0 \\ 1 \end{matrix}), v_{2} = (\begin{matrix} 1 \\ 0 \end{matrix}) .

Then

u_{1} = \frac{A v_{1}}{4} = (\begin{matrix} 0 \\ 0 \\ 1 \end{matrix}), u_{2} = \frac{A v_{2}}{3} = (\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}) .

Thus $A = U_{r} Σ_{r} V_{r^{T}}$ , where

U_{r} = (\begin{matrix} 0 & 1 \\ 0 & 0 \\ 1 & 0 \end{matrix}), Σ_{r} = (\begin{matrix} 4 & 0 \\ 0 & 3 \end{matrix}), V_{r} = (\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix}) .

3.2Explanation

A compact SVD[コンパクトSVD] keeps only the directions corresponding to nonzero singular values. This form is convenient for reading the rank階数かいすう and the column space列空間れつくうかん.

4Problem 3

For

A = (\begin{matrix} 3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{matrix}),

find the singular values特異値とくいち, the kernel核かく, and the image像ぞう. Explain what is preserved and what is discarded by the rank階数かいすう 1 approximation that keeps only the largest singular value.

4.1Answer example

Since $A^{T} A$ is

(\begin{matrix} 9 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{matrix}),

the singular values are $3, 1, 0$ . Also,

\ker A = span (e_{3}), Im A = span (e_{1}, e_{2}) .

The rank 1 approximation keeping only the largest singular value is

A_{1} = (\begin{matrix} 3 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix}) .

4.2Explanation

$A_{1}$ preserves the strongest surviving direction, the $e_{1}$ direction, and discards the weaker $e_{2}$ component together with the kernel核かく direction $e_{3}$ . SVD-based approximation measures importance by singular values.

5Problem 4

For

B = (\begin{matrix} 1 & 1 \\ 2 & 2 \\ 0 & 0 \end{matrix}),

use the nonzero singular value特異値とくいち to find $B^{+}$ , and explain what $B B^{+}$ and $B^{+} B$ project onto.

5.1Answer example

Write $B = u v^{T}$ , where

u = (\begin{matrix} 1 \\ 2 \\ 0 \end{matrix}), v = (\begin{matrix} 1 \\ 1 \end{matrix}) .

Since $∥ u ∥ = \sqrt{5}$ and $∥ v ∥ = \sqrt{2}$ , the nonzero singular value is $\sqrt{10}$ . Therefore

B^{+} = \frac{1}{10} v u^{T} = \frac{1}{10} (\begin{matrix} 1 & 2 & 0 \\ 1 & 2 & 0 \end{matrix}) .

$B B^{+}$ is the orthogonal projection直交射影ちょっこうしゃえい onto $span (u)$ , and $B^{+} B$ is the orthogonal projection onto $span (v)$ .

5.2Explanation

The pseudoinverse擬似逆行列ぎじぎゃくぎょうれつ reverses only directions that were not collapsed. $B B^{+}$ and $B^{+} B$ are not identity transformations; they are projections射影しゃえい onto the spaces where information remains.

6Problem 5

For

A = (\begin{matrix} 1 \\ 1 \end{matrix}), b = (\begin{matrix} 2 \\ 0 \end{matrix}),

use $A^{+}$ to find the least-squares solution最小二乗解さいしょうにじょうかい.

6.1Answer example

A^{+} = \frac{1}{2} (\begin{matrix} 1 & 1 \end{matrix}),

so $x = A^{+} b = 1$ . The approximating vector is $A x = (1, 1)^{T}$ , and the residual残差ざんさ is

r = (\begin{matrix} 1 \\ - 1 \end{matrix}) .

Since $A^{T} r = 0$ , the residual is orthogonal直交ちょっこう to the column space列空間れつくうかん.

6.2Explanation

In an overdetermined system過剰決定系かじょうけっていけい, $A x = b$ may have no exact solution. The pseudoinverse returns the least-squares solution corresponding to projection onto the column space.

7Problem 6

For

A = (\begin{matrix} 1 & 1 \end{matrix}), b = 2,

confirm that $A x = b$ has infinitely many solutions解かい, and explain why $A^{+} b$ is the minimum-norm solution最小ノルム解.

7.1Answer example

A^{+} = \frac{1}{2} (\begin{matrix} 1 \\ 1 \end{matrix}),

so

A^{+} b = (\begin{matrix} 1 \\ 1 \end{matrix}) .

The equation $A x = b$ is $x_{1} + x_{2} = 2$ , so

x = (\begin{matrix} 2 - t \\ t \end{matrix}) (t \in R) .

The squared norm is

∥ x ∥^{2} = (2 - t)^{2} + t^{2} = 2 (t - 1)^{2} + 2,

which is minimized at $t = 1$ .

7.2Explanation

In an underdetermined system劣決定系れつけっていけい, there may be infinitely many solutions. The pseudoinverse chooses the one of minimum length.

8Problem 7

For

C = (\begin{matrix} 1 & i \\ 0 & 0 \end{matrix}),

compute $C^{*} C$ and find the singular values特異値とくいち.

8.1Answer example

C^{*} = (\begin{matrix} 1 & 0 \\ - i & 0 \end{matrix}),

so

C^{*} C = (\begin{matrix} 1 & i \\ - i & 1 \end{matrix}) .

The eigenvalues固有値こゆうち of this matrix are $2, 0$ . Therefore the singular values of $C$ are

\sqrt{2}, 0.

8.2Explanation

For a complex matrix複素行列ふくそぎょうれつ, use $A^{*} A$ rather than $A^{T} A$ . The conjugate transpose共役転置きょうやくてんち makes $A^{*} A$ Hermitianエルミート行列ぎょうれつ and positive semidefinite非負定値ひふていち.

9Related exercises

data/exercise/math/linear-algebra/eigenvalues-diagonalization-and-extensions.exercise.n.md data/exercise/math/linear-algebra/quadratic-forms-minimal-polynomials-and-jordan-form.exercise.n.md data/exercise/math/linear-algebra/complex-inner-products-and-unitary-matrices.exercise.n.md

10Related lectures

data/lecture/math/linear-algebra/introduction-to-singular-value-decomposition.lecture.n.md data/lecture/math/linear-algebra/pseudoinverse-basics.lecture.n.md data/lecture/math/linear-algebra/complex-inner-products-and-unitary-matrices.lecture.n.md

11Supplementary proof and application problems

11.1Problem 8

Let $A = U_{r} Σ_{r} V_{r^{T}}$ be a compact SVD with positive diagonal entries $σ_{1}, \dots, σ_{r}$ . Define

A^{+} = V_{r} Σ_{r^{- 1}} U_{r^{T}} .

Show that $A A^{+}$ is the orthogonal projection直交射影ちょっこうしゃえい onto $Im A$ and that $A^{+} A$ is the orthogonal projection onto $(\ker A)^{⊥}$ .

11.2Answer example

Using $U_{r^{T}} U_{r} = I$ and $V_{r^{T}} V_{r} = I$ ,

A A^{+} = U_{r} Σ_{r} V_{r^{T}} V_{r} Σ_{r^{- 1}} U_{r^{T}} = U_{r} U_{r^{T}} .

The columns of $U_{r}$ form an orthonormal basis of $Im A$ , so $U_{r} U_{r^{T}}$ is the orthogonal projection onto $Im A$ . Similarly,

A^{+} A = V_{r} Σ_{r^{- 1}} U_{r^{T}} U_{r} Σ_{r} V_{r^{T}} = V_{r} V_{r^{T}} .

The columns of $V_{r}$ span the input directions with nonzero singular values, namely $(\ker A)^{⊥}$ . Hence $A^{+} A$ is the orthogonal projection onto $(\ker A)^{⊥}$ .

11.3Explanation

This theorem explains why the pseudoinverse is not a two-sided inverse in general. It becomes an inverse only on directions that survive and becomes a projection on the whole space.

11.4Problem 9

For

A = (\begin{matrix} 2 & 0 \\ 0 & 0 \end{matrix}), b = (\begin{matrix} 4 \\ 3 \end{matrix}),

find $A^{+} b$ , describe all least-squares solutions, and identify the minimum-norm one.

11.5Answer example

Here

A^{+} = (\begin{matrix} \frac{1}{2} & 0 \\ 0 & 0 \end{matrix}),

so

A^{+} b = (\begin{matrix} 2 \\ 0 \end{matrix}) .

The vector $A x = (2 x_{1}, 0)^{T}$ is closest to $b = (4, 3)^{T}$ when $2 x_{1} = 4$ , so $x_{1} = 2$ . The second variable does not affect $A x$ , so all least-squares solutions are

x = (\begin{matrix} 2 \\ t \end{matrix}) (t \in R) .

The norm $∥ x ∥^{2} = 4 + t^{2}$ is minimized at $t = 0$ , so $A^{+} b = (2, 0)^{T}$ is the minimum-norm least-squares solution.

11.6Explanation

The column-space error in the second coordinate cannot be removed. The pseudoinverse fixes the part that can be fitted and sets the kernel component to 0 to minimize the norm.

11.7Problem 10

For the matrix in Problem 9, verify the four Penrose equations

A A^{+} A = A, A^{+} A A^{+} = A^{+}, (A A^{+})^{T} = A A^{+}, (A^{+} A)^{T} = A^{+} A .

11.8Answer example

Since

A A^{+} = A^{+} A = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix}),

we have

A A^{+} A = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix}) (\begin{matrix} 2 & 0 \\ 0 & 0 \end{matrix}) = A,

and

A^{+} A A^{+} = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix}) (\begin{matrix} \frac{1}{2} & 0 \\ 0 & 0 \end{matrix}) = A^{+} .

The matrix $(\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ is symmetric, so both transpose conditions also hold.

11.9Explanation

The Penrose equations are a compact way to characterize the Moore-Penrose pseudoinverse. In this diagonal example, they reduce to the statement that surviving directions are kept and collapsed directions remain zero.