Inequalities

Cauchy–Schwarz inequality

From Wikipedia, the free encyclopedia

In mathematics, the Cauchy–Schwarz inequality (also known as the Bunyakovsky inequality, the Schwarz inequality, or theCauchy–Bunyakovsky–Schwarz inequality), is a useful inequality encountered in many different settings, such as linear algebra,analysis, in probability theory, and other areas. It is a specific case of Hölder’s inequality.

The inequality for sums was published by Augustin-Louis Cauchy (1821), while the corresponding inequality for integrals was first stated by Viktor Bunyakovsky (1859) and rediscovered by Hermann Amandus Schwarz (1888) (often misspelled “Schwartz”).

Statement of the inequality

The Cauchy–Schwarz inequality states that for all vectors x and y of an inner product space,

| \langle x,y\rangle|^2 \leq \langle x,x\rangle \cdot \langle y,y\rangle,

where \langle\cdot,\cdot\rangle is the inner product. Equivalently, by taking the square root of both sides, and referring to the norms of the vectors, the inequality is written as

 |\langle x,y\rangle| \leq \|x\| \cdot \|y\|.\,

Moreover, the two sides are equal if and only if x and y are linearly dependent (or, in a geometrical sense, they are parallel or one of the vectors is equal to zero).

If x_1,\ldots, x_n\in\mathbb C and y_1,\ldots, y_n\in\mathbb C are any complex numbers and the inner product is the standard inner product then the inequality may be restated in a more explicit way as follows:

|x_1 \bar{y_1} + \cdots + x_n \bar{y_n}|^2 \leq (|x_1|^2 + \cdots + |x_n|^2) (|y_1|^2 + \cdots + |y_n|^2).

When viewed in this way the numbers x1, …, xn, and y1, …, yn are the components of x and y with respect to an orthonormal basis of V.

Even more compactly written:

\left|\sum_{i=1}^n x_i \bar{y_i}\right|^2 \leq \sum_{j=1}^n |x_j|^2 \sum_{k=1}^n |y_k|^2 .

Equality holds if and only if x and y are linearly dependent, that is, one is a scalar multiple of the other (which includes the case when one or both are zero).

The finite-dimensional case of this inequality for real vectors was proved by Cauchy in 1821, and in 1859 Cauchy’s studentBunyakovsky noted that by taking limits one can obtain an integral form of Cauchy’s inequality. The general result for an inner product space was obtained by Schwarz in 1885.

Proof

Let uv be arbitrary vectors in a vector space V over F with an inner product, where F is the field of real or complex numbers. We prove the inequality

 \big| \langle u,v \rangle \big| \leq \left\|u\right\| \left\|v\right\|. \,

This inequality is trivial in the case v = 0, so we assume that <vv> is nonzero. Let δ be any number in the field F. Then,

 0 \leq \left\| u-\delta v \right\|^2 = \langle u-\delta v,u-\delta v \rangle = \langle u,u \rangle - \bar{\delta} \langle u,v \rangle - \delta \langle v,u \rangle + |\delta|^2 \langle v,v\rangle. \,

Choose the value of δ that minimizes this quadratic form, namely

 \delta = \langle u,v \rangle \cdot \langle v,v \rangle^{-1}. \,

(A quick way to remember this value of δ is to imagine F to be the reals, so that the quadratic form is a quadratic polynomial in the real variable δ, and the polynomial can easily be minimized by setting its derivative equal to zero.)

We obtain

 0 \leq \langle u,u \rangle - |\langle u,v \rangle|^2 \cdot \langle v,v \rangle^{-1} \,

which is true if and only if

 |\langle u,v \rangle|^2 \leq \langle u,u \rangle \cdot \langle v,v \rangle, \,

or equivalently:

 \big| \langle u,v \rangle \big| \leq \left\|u\right\| \left\|v\right\|, \,

which completes the proof.

Notable special cases

Rn

In Euclidean space Rn with the standard inner product, the Cauchy–Schwarz inequality is

\left(\sum_{i=1}^n x_i y_i\right)^2\leq \left(\sum_{i=1}^n x_i^2\right) \left(\sum_{i=1}^n y_i^2\right).

To prove this form of the inequality, consider the following quadratic polynomial in z.

(x_1 z + y_1)^2 + \cdots + (x_n z + y_n)^2.

Since it is nonnegative it has at most one real root in z, whence its discriminant is less than or equal to zero, that is,

\left(\sum ( x_i \cdot y_i ) \right)^2 - \sum {x_i^2} \cdot \sum {y_i^2} \le 0,

which yields the Cauchy–Schwarz inequality.

An equivalent proof for Rn starts with the summation below.

Expanding the brackets we have:

 \sum_{i=1}^n \sum_{j=1}^n \left( x_i y_j - x_j y_i \right)^2   = \sum_{i=1}^n x_i^2 \sum_{j=1}^n y_j^2 + \sum_{j=1}^n x_j^2 \sum_{i=1}^n y_i^2  - 2 \sum_{i=1}^n x_i y_i \sum_{j=1}^n x_j y_j ,

collecting together identical terms (albeit with different summation indices) we find:

 \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n \left( x_i y_j - x_j y_i \right)^2   = \sum_{i=1}^n x_i^2 \sum_{i=1}^n y_i^2 - \left( \sum_{i=1}^n x_i y_i \right)^2 .

Because the left-hand side of the equation is a sum of the squares of real numbers it is greater than or equal to zero, thus:

 \sum_{i=1}^n x_i^2 \sum_{i=1}^n y_i^2 - \left( \sum_{i=1}^n x_i y_i \right)^2 \geq 0.

This form is used usually when solving school math problems.

Yet another approach when n ≥ 2 (n = 1 is trivial) is to consider the plane containing x and y. More precisely, recoordinatize Rn with any orthonormal basis whose first two vectors span a subspace containing x and y. In this basis only x_1,~x_2,~y_1 and y_2~ are nonzero, and the inequality reduces to the algebra of dot product in the plane, which is related to the angle between two vectors, from which we obtain the inequality:

|x \cdot y| = \|x\| \|y\| | \cos \theta | \le \|x\| \|y\|.

When n = 3 the Cauchy–Schwarz inequality can also be deduced from Lagrange’s identity, which takes the form

\langle x,x\rangle \cdot \langle y,y\rangle = |\langle x,y\rangle|^2 + |x \times y|^2

from which readily follows the Cauchy–Schwarz inequality.

L2

For the inner product space of square-integrable complex-valued functions, one has

\left|\int f(x) g(x)\,dx\right|^2\leq\int \left|f(x)\right|^2\,dx \cdot \int\left|g(x)\right|^2\,dx.

A generalization of this is the Hölder inequality.

Use

The triangle inequality for the inner product is often shown as a consequence of the Cauchy–Schwarz inequality, as follows: given vectors x and y:

 \begin{align} \|x + y\|^2 & = \langle x + y, x + y \rangle \\ & = \|x\|^2 + \langle x, y \rangle + \langle y, x \rangle + \|y\|^2 \\ & \le \|x\|^2 + 2|\langle x, y \rangle| + \|y\|^2 \\ & \le \|x\|^2 + 2\|x\|\|y\| + \|y\|^2 \\ & = \left(\|x\| + \|y\|\right)^2. \end{align}

Taking square roots gives the triangle inequality.

The Cauchy–Schwarz inequality allows one to extend the notion of “angle between two vectors” to any real inner product space, by defining:

 \cos\theta_{xy}=\frac{\langle x,y\rangle}{\|x\| \|y\|}.

The Cauchy–Schwarz inequality proves that this definition is sensible, by showing that the right hand side lies in the interval [−1, 1], and justifies the notion that (real) Hilbert spaces are simply generalizations of the Euclidean space.

It can also be used to define an angle in complex inner product spaces, by taking the absolute value of the right hand side, as is done when extracting a metric from quantum fidelity.

The Cauchy–Schwarz is used to prove that the inner product is a continuous function with respect to the topology induced by the inner product itself.

The Cauchy–Schwarz inequality is usually used to show Bessel’s inequality.

Other proofs

If either \left|x\right> or \left|y\right> are the zero vector, the statement holds trivially, so assume that both are nonzero.

For any nonzero vector \left|V\right>, \left<V|V\right> > 0. (NOTE: merits own proof)

\displaystyle \left< \alpha X + Y| \alpha X + Y \right> \geq 0

If the inner product is symmetric. Let \alpha be a real scalar.

\displaystyle \alpha^2\left< X |X\right>+\alpha(\left< X |Y\right>+\left< Y |X\right>)+ \left<Y|Y \right> \geq 0

The last expression is a quadratic polynomial that is non-negative for any \alpha.  The quadratic has either two complex roots,or  a single  real root. Intuitively, the polynomial is either ‘floating above’ the horizontal axis, if it has two complex roots, or tangent to it if it has one real root, since it can’t have two real roots because the graph of the function would have to ‘pass under’ the horizontal axis and take some negative values.

The roots are given by the quadratic formula

math

In particular, the term math must either be negative, yielding two complex roots, or zero, yielding a single real root. Thus

math
math
math
math

Substituting the values of mathmath and math into the last of these inequalities, it can be seen that

\displaystyle (\left< X |Y\right>+\left< Y |X\right>)^2 \leq 4\left< X |X\right> \left<Y|Y \right>

If the inner product is symmetric, this proves the inequality.

An alternative proof follows from the expression

\displaystyle  \frac{{(a+b)}^2}{x+y}=\frac{a^2}{x}+\frac{b^2}{y},

valid for a and b real and $x>0$ and $y>0$. This expression is a restatement of (a y - b x)^2 \geq  0. From this one can get a general n-term expression

\displaystyle  \frac{{(a_1+a_2+\cdots+a_n)}^2}{x_1+x_2+\cdots x_n}=\frac{a_1^2}{x_1}+\frac{a_2^2}{x_2}+\cdots +\frac{a_n^2}{x_n}

To get cauchy-Scwarz inequality set a_k=\alpha_k \beta_k and x_k=\beta_k^2.

If the inner product is skew-symmetric, take

\displaystyle \alpha = -\frac{\left<X|Y\right>}{\left<X|X\right>}

\displaystyle    ( -\frac{\left< Y |X\right>}{\left< X |X\right>} \left<X\right| + \left<Y\right|)( -\frac{\left< X |Y\right>}{\left< X |X\right>} \left|X\right> + \left|Y\right> ) \geq 0

\displaystyle    \frac{\left< Y |X\right>\left< X |Y\right>}{\left< X |X\right>} -\frac{\left< Y |X\right>\left< X |Y\right>}{\left< X |X\right>} -\frac{\left< Y |X\right>\left< X |Y\right>}{\left< X |X\right>} + \left<Y|Y\right> \geq 0

\displaystyle    \left< X |X\right>\left<Y|Y\right> \geq \left< Y |X\right>\left< X |Y\right>
QED

The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities

Cauchy-Schwarz Inequality: Yet Another Proof

http://en.wikipedia.org/wiki/Cauchy%E2%80%93Schwarz_inequality

http://en.wikipedia.org/wiki/Jensen’s_inequality

For a real convex function \varphi, numbers x1x2, …, xn in its domain, and positive weights ai, Jensen’s inequality can be stated as:

\varphi\left(\frac{\sum a_i x_i}{\sum a_i}\right) \le \frac{\sum a_i \varphi (x_i)}{\sum a_i} \qquad\qquad (1)

and the inequality is reversed if \varphi is concave, which is

\varphi\left(\frac{\sum a_i x_i}{\sum a_i}\right) \geq \frac{\sum a_i \varphi (x_i)}{\sum a_i}.\qquad\qquad(2)
http://en.wikipedia.org/wiki/Inequality_of_arithmetic_and_geometric_means#Generalizations
The arithmetic mean, or less precisely the average, of a list of n numbers x1x2, . . ., xn is the sum of the numbers divided by n

\frac{x_1 + x_2 + \cdots + x_n}{n}.

The geometric mean is similar, except that it is only defined for a list of nonnegative real numbers, and uses multiplication and a root in place of addition and division:

\sqrt[n]{x_1 \cdot x_2 \cdots x_n}.

If x1x2, . . ., xn > 0, this is equal to the exponential of the arithmetic mean of the natural logarithms of the numbers:

\exp \left( \frac{\ln {x_1} + \ln {x_2} + \cdots + \ln {x_n}}{n} \right).

For any list of n nonnegative real numbers x1x2, . . ., xn,

\frac{x_1 + x_2 + \cdots + x_n}{n} \geq \sqrt[n]{x_1 \cdot x_2 \cdots x_n},

and that equality holds if and only if x1x2 = . . . = xn.

http://en.wikipedia.org/wiki/Young’s_inequality

In its standard form, the inequality states that if a and b are nonnegative real numbers and p and q are positive real numbers such that 1/p + 1/q = 1, then

ab \le \frac{a^p}{p} + \frac{b^q}{q}.

Equality holds if and only if apbq. This form of Young’s inequality is a special case of the inequality of weighted arithmetic and geometric means and can be used to prove Hölder’s inequality.

The claim is certainly true if a = 0 or b = 0. Therefore, assume a > 0 and b > 0 in the following. Put t = 1/p, and (1 − t) = 1/q. Then since the logarithm function is strictly concave

 \log(t a^p + (1-t) b^q) \ge t \log(a^p) + (1-t) \log(b^q) =                           \log(a) +\log(b) = \log(ab)

with equality if and only if apbq. Young’s inequality follows by exponentiating.

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s