Basics of inner product spaces内積空間ないせきくうかん

1Introduction

The central point of this lecture is that introducing an inner product内積ないせき adds length長さ and angle角度かくど to an ordinary vector spaceベクトル空間くうかん. Once this structure構造こうぞう is present, we can discuss orthogonality直交ちょっこう, shortest distance, orthogonal bases直交基底ちょっこうきてい, and the least squares method最小二乗法さいしょうにじょうほう.

A vector space by itself only has addition加法かほう and scalar multiplicationスカラー倍. An inner product lets us measure how much two vectors point in the same direction and how long a vector is.

2Terms and definitions

An inner product space内積空間ないせきくうかん is a vector spaceベクトル空間くうかん equipped with an inner product内積ないせき.

A real inner product実内積じつないせき assigns a real number実数じっすう $⟨u,v⟩$\langle u,v\rangle to two vectors and satisfies bilinearity双線型性そうせんけいせい, symmetry対称性たいしょうせい, and positive definiteness正定値性せいていちせい.

A complex inner product複素内積ふくそないせき on a complex複素ふくそ vector space uses conjugate symmetry共役対称性きょうやくたいしょうせい and sesquilinearity半線型性はんせんけいせい. In this series, the first variable変数へんすう is linear線型せんけい and the second variable is conjugate-linear共役線型きょうやくせんけい.

A Hermitian inner productエルミート内積ないせき is another name for the complex inner product used with conjugate transpose, adjoints, and unitary matrices.

The normノルム induced by an inner product is

3Plan

We first extract the properties of the ordinary dot product点積てんせき that support length and orthogonality. Then we compare the real実じつ and complex cases, especially where complex conjugation複素共役ふくそきょうやく enters.

4Intuitive explanation

The dot product $u·v$u\cdot v can be read as "the product of lengths times the similarity of direction." It is large and positive when vectors point in the same direction, zero at a right angle, and negative in opposite directions. An inner product space内積空間ないせきくうかん abstracts the properties needed for this interpretation.

5Precise explanation

5.11. Axioms of an inner product内積ないせき

For a real inner product space, for all vectors $u,v,w$u,v,w and real scalar $a$a,

and

Linearity線型性せんけいせい makes algebraic calculation possible, symmetry measures the relation between $u$u and $v$v independently of order, and positive definiteness lets $⟨v,v⟩$\langle v,v\rangle behave as squared length長さの二乗.

5.22. Complex inner product spaces

In a complex vector space, ordinary symmetry is not compatible with positivity. For example, a length squared must not become negative when a vector is multiplied by $i$i. Therefore the definition uses complex conjugation.

With the convention in this series,

and

The real case is the special case in which conjugation has no visible effect.

5.33. Length and orthogonality直交ちょっこう

An inner product defines length by

Positive definiteness ensures the expression under the square root is nonnegative and is positive for $v\ne 0$v\ne0.

Vectors $u$u and $v$v are orthogonal直交ちょっこう if

This abstracts the condition $\cos \theta =0$\cos\theta=0 from the familiar formula $u·v=\left|u\right|\left|v\right|\cos \theta$u\cdot v=|u||v|\cos\theta.

5.44. Relation to the high-school dot product

For $u=(u_1,u_2)$u=(u_1,u_2) and $v=(v_1,v_2)$v=(v_1,v_2),

is exactly the usual dot product in the plane. Inner product spaces generalize this geometric intuition.

5.55. Cauchy-Schwarz inequality

Every inner product satisfies the Cauchy-Schwarz inequalityコーシー・シュワルツの不等式ふとうしき

In a real inner product space, the proof uses the fact that

for all real $t$t. Expanding gives a quadratic in $t$t whose discriminant is nonpositive, hence

In a complex inner product space, one may take

for $v\ne 0$v\ne0 under the convention that the first variable is linear. Then $u-\alpha v$u-\alpha v is orthogonal to $v$v, and the orthogonal decomposition gives the same inequality. The case $v=0$v=0 is immediate.

For real inner product spaces, Cauchy-Schwarz justifies defining

for nonzero $u,v$u,v, because the right-hand side lies between $-1$-1 and $1$1. In complex inner product spaces, $⟨u,v⟩$\langle u,v\rangle is generally complex, so one does not use that quotient directly as $\cos \theta$\cos\theta without further conventions.

6Scope and limitations

The angle formula in this lecture is primarily for real inner product spaces. In complex inner product spaces, projection coefficients and orthogonal expansions depend on which variable is chosen to be linear. It is important to distinguish the real and complex definitions at the start.

7Criteria

• If length, angle, orthogonality直交ちょっこう, or projection appears, work in an inner product space.
• If a basis is required to be orthogonal or orthonormal, ordinary linear independence is not enough; inner product structure is being used.
• Least squares and Fourier expansions both rely on inner products.

8Final forms

9In one sentence

An inner product space内積空間ないせきくうかん is the framework that adds length, angle, and orthogonality to a vector space.

An inner product内積ないせき is a rule that turns algebraic vectors into geometric objects with length, angle, and orthogonality直交ちょっこう. The important point is not only the formula but the axioms: positivity prevents nonzero vectors from having zero length, linearity controls calculation, and conjugate symmetry is essential over complex scalars.

Cauchy-Schwarz is the bridge from the axioms to geometry. It proves the estimate $|⟨x,y⟩|\text{[PARSE ERROR: Undefined("Command("le")")]}\parallel x\parallel \parallel y\parallel$|\langle x,y\rangle| \le \|x\|\|y\|, leads to the triangle inequality for the induced norm developed in the next norm lecture, and identifies equality as the case where the two vectors are linearly dependent. These equality and boundary cases are part of the theorem, not optional details.

10Theorem and proof: Cauchy-Schwarz inequality

For all $u,v$u,v in an inner product space,

If $v=0$v=0, both sides are $0$0. Assume $v\ne 0$v\ne0. With the convention used here, take

Since $⟨v,v⟩=\parallel v{\parallel }^2>0$\langle v,v\rangle=\|v\|^2>0, this is valid. From

and expansion, we obtain

Multiplying by $\parallel v{\parallel }^2>0$\|v\|^2>0 gives

Taking square roots gives the conclusion.

11Exercise link

12Related links