Basics of orthogonalization直交化ちょっこうか

date2026-07-14document_iddoc_c21549e9d8e64a63809a93395cc5464adescription直交化を、射影を順に除去して直交基底を構成するGram-Schmidt processとして説明し、最小二乗法への接続まで整理する講義である。prerequisites内積空間の基本 / ベクトル空間と基底 / 一次独立type講義content_typelecturestatusactive

mathlinear-algebraundergraduatelecture

1Introduction

The main point of this lecture is that orthogonalization直交化ちょっこうか replaces a given basis基底きてい by an orthogonal basis直交基底ちょっこうきてい spanning the same subspace部分空間ぶぶんくうかん. The subspace is unchanged; only the basis is made easier to compute with.

Even when a basis is linearly independent, lengths, coefficients, and projections may be awkward to calculate. If the basis vectors are mutually orthogonal直交ちょっこう, the calculations become transparent.

2Terms and definitions

An orthogonal basis直交基底ちょっこうきてい is a basis whose distinct basis vectors are orthogonal to each other.

An orthonormal basis正規直交基底せいきちょっこうきてい is an orthogonal basis in which every vector has length $1$ .

The Gram-Schmidt processグラム・シュミット法ほう constructs an orthogonal basis spanning the same subspace from a linearly independent list of vectors.

3Plan

Start with vectors $v_{1}, v_{2}, \dots$ . At each step, remove the components in the directions already constructed. What remains is the new direction not already accounted for.

4Intuitive explanation

If $v_{2}$ contains a component in the same direction as $v_{1}$ , subtract that component. The remainder is orthogonal to $v_{1}$ . Repeating this idea gives the Gram-Schmidt process.

5Precise explanation

5.10. Removing projections

For $u \neq 0$ , the projection射影しゃえい of $v$ onto the direction of $u$ is

{proj}_{u} (v) = \frac{⟨ v, u ⟩}{⟨ u, u ⟩} u .

This extracts the component of $v$ in the direction of $u$ . Orthogonalization removes projections onto previously constructed orthogonal directions.

In a complex inner product space複素内積空間ふくそないせきくうかん, the projection coefficient depends on the inner-product convention. In this series the first variable is linear, so the coefficient is $⟨ v, u ⟩ / ⟨ u, u ⟩$ .

data/lecture/math/linear-algebra/complex-inner-products-and-unitary-matrices.lecture.n.md

5.21. The case of two vectors

If projection or inner product is still unclear, review the inner product lecture first.

data/lecture/math/linear-algebra/inner-product-space-basics.lecture.n.md

Start with linearly independent $v_{1}, v_{2}$ . Set

u_{1} = v_{1.}

Then remove the component of $v_{2}$ in the direction of $u_{1}$ :

u_{2} = v_{2} - \frac{⟨ v_{2}, u_{1} ⟩}{⟨ u_{1}, u_{1} ⟩} u_{1.}

Now $u_{1}$ and $u_{2}$ are orthogonal, because

\begin{aligned} \langle u_2,u_1\rangle &= \left\langle v_2-\frac{\langle v_2,u_1\rangle}{\langle u_1,u_1\rangle}u_1,\, u_1 \right\rangle\\ &= \langle v_2,u_1\rangle - \frac{\langle v_2,u_1\rangle}{\langle u_1,u_1\rangle}\langle u_1,u_1\rangle\\ &=0. \end{aligned}\begin{aligned} \langle u_2,u_1\rangle &= \left\langle v_2-\frac{\langle v_2,u_1\rangle}{\langle u_1,u_1\rangle}u_1,\, u_1 \right\rangle\\ &= \langle v_2,u_1\rangle - \frac{\langle v_2,u_1\rangle}{\langle u_1,u_1\rangle}\langle u_1,u_1\rangle\\ &=0. \end{aligned}

5.32. General form

From $v_{1}, \dots, v_{n}$ , define

u_{1} = v_{1}

and

u_{k} = v_{k} - \sum_{j = 1}^{k - 1} \frac{⟨ v_{k}, u_{j} ⟩}{⟨ u_{j}, u_{j} ⟩} u_{j} .

Then $u_{1}, \dots, u_{n}$ are mutually orthogonal and span the same subspace as the original vectors.

The key point is that $u_{k}$ is obtained from $v_{k}$ by subtracting a linear combination of $u_{1}, \dots, u_{k - 1}$ , so

u_{k} \in span (v_{1}, \dots, v_{k}) .

Conversely,

v_{k} = u_{k} + \sum_{j = 1}^{k - 1} \frac{⟨ v_{k}, u_{j} ⟩}{⟨ u_{j}, u_{j} ⟩} u_{j},

so $v_{k} \in span (u_{1}, \dots, u_{k})$ . Therefore the spans agree at every stage.

5.43. Normalization

To obtain an orthonormal basis正規直交基底せいきちょっこうきてい, set

e_{k} = \frac{u_{k}}{∥ u_{k} ∥} .

5.54. Connection with least squares

Orthogonalization is one foundation of the least squares method最小二乗法さいしょうにじょうほう. When $A x = b$ cannot be solved exactly, one projects $b$ onto the column space列空間れつくうかん of $A$ . The residual

r = b - A x

is orthogonal to the column space, leading in the real case to the normal equations

A^{T} (b - A x) = 0.

Thus Gram-Schmidt and least squares share the same geometric structure: remove orthogonal components and keep the best projection.

6Criteria

Use orthogonalization直交化ちょっこうか when a basis spans the right subspace but is not orthogonal.
Projection, least squares, and Fourier expansion all rely on decomposing into orthogonal components.
The subspace is preserved; only the basis is changed.

7Scope and limitations

The procedure assumes an inner product and linearly independent input vectors. If some $u_{k} = 0$ appears, the original list was linearly dependent. In complex inner product spaces, the positions of conjugates must match the chosen convention.

8Final forms

[PARSE ERROR: Undefined("Command(\"boxed\")")] u_{k} = v_{k} - \sum_{j = 1}^{k - 1} \frac{⟨ v_{k}, u_{j} ⟩}{⟨ u_{j}, u_{j} ⟩} u_{j}

[PARSE ERROR: Undefined("Command(\"boxed\")")] e_{k} = \frac{u_{k}}{∥ u_{k} ∥}

[PARSE ERROR: Undefined("Command(\"boxed\")")] {proj}_{u} (v) = \frac{⟨ v, u ⟩}{⟨ u, u ⟩} u

9In one sentence

Orthogonalization直交化ちょっこうか rebuilds a basis of the same subspace as an orthogonal basis that is easier to compute with.

10Theorem and proof: Gram-Schmidt preserves spans

If Gram-Schmidt is applied to linearly independent vectors $v_{1}, \dots, v_{k}$ and produces $q_{1}, \dots, q_{k}$ , then for every $r$ ,

span (v_{1}, \dots, v_{r}) = span (q_{1}, \dots, q_{r}) .

The proof is by induction on $r$ . For $r = 1$ , $q_{1}$ is a nonzero scalar multiple of $v_{1}$ , so the spans agree.

At step $r$ , the vector before normalization is

w_{r} = v_{r} - \sum_{i = 1}^{r - 1} ⟨ v_{r}, q_{i} ⟩ q_{i} .

Thus $w_{r}$ lies in $span (v_{1}, \dots, v_{r})$ . Conversely,

v_{r} = w_{r} + \sum_{i = 1}^{r - 1} ⟨ v_{r}, q_{i} ⟩ q_{i},

so $v_{r}$ lies in $span (q_{1}, \dots, q_{r})$ . Hence the spans are equal.

11Exercise links

data/exercise/math/linear-algebra/inner-products-orthogonality-and-projections.exercise.n.md data/exercise/math/linear-algebra/complex-inner-products-and-unitary-matrices.exercise.n.md