Dual norm

In functional analysis, the dual norm is a measure of the "size" of continuous linear functionals. The formal proof for the dual norm depends upon the context in which it is introduced.

Definition

Topology

Let $X$ and $Y$ be topological vector spaces, and $L(X,Y)$ ^[1] be the collection of all bounded linear mappings (or operators) of $X$ into $Y$ . In the case where $X$ and $Y$ are normed vector spaces, $L(X,Y)$ can be normed in a natural way.

When $Y$ is a scalar field (i.e. $Y={\mathbb C}$ or $Y={\mathbb {R} }$ ) so that $L(X,Y)$ is the dual space $X^{*}$ of $X$ , the norm on $L(X,Y)$ defines a topology $X^{*}$ which turns out to be stronger than its weak-*topology.

Theorem 1: Let $X$ and $Y$ be normed spaces, and associate to each $f\in L(X,Y)$ the number:

\left\|f\right\|=\sup\{\left|f(x)\right|:x\in X,\left\|x\right\|\leq 1\}

We first establish that $L(X,Y)$ is bounded (using the triangle inequality), and complete (using Cauchy sequences) using our definition of $\|f\|$ , thereby making $L(X,Y)$ a normed space. If $Y$ is a Banach space, so is $L(X,Y)$ .^[2]

Proof:

A subset of a normed space is bounded if and only if it lies in some multiple of the unit sphere; thus $\lVert f\rVert <\infty$ for every $f\in L(X,Y)$
if $\alpha$ is a scalar, then $(\alpha f)(x)=\alpha \cdot fx$ so that
$\|\alpha f\|=|\alpha |\|f\|$

The triangle inequality in $Y$ shows that
${\begin{aligned}\|(f_{1}+f_{2})x\|&=\|f_{1}x+f_{2}x\|\leq \|f_{1}x\|+\|f_{2}x\|\\&\leq (\|f_{1}\|+\|f_{2}\|)\|x\|\leq \|f_{1}\|+\|f_{2}\|\end{aligned}}$

for every $x\in X$ with $\|x\|\leq 1$ . Thus
$\|f_{1}+f_{2}\|\leq \|f_{1}\|+\|f_{2}\|$

If $f\neq 0$ , then $fx\neq 0$ for some $x\in X$ ; hence $\|f\|>0$ . Thus, $L(X,Y)$ is a normed space.^[3]
Assume now that $Y$ is complete, and that $\{f_{n}\}$ is a Cauchy sequence in $L(X,Y)$ .
Since
$\|f_{n}x-f_{m}x\|\leq \|f_{n}-f_{m}\|\|x\|$

and it is assumed that $\|f_{n}-f_{m}\|\to 0$ as $n$ and $m$ tend to $\infty$ , $\{f_{n}x\}$ is a Cauchy sequence in $Y$ for every $x\in X$ .

Hence
$fx=\lim _{n\to \infty }f_{n}x$

exists. It is clear that $f:X\to Y$ is linear. If $\varepsilon >0$ , $\|f_{n}-f_{m}\|\|x\|\leq \varepsilon \|x\|$ for sufficiently large $n$ and $m$ . It follows
$\|fx-f_{m}x\|\leq \varepsilon \|x\|$

for sufficiently large $m$ .

Hence $\|fx\|\leq (\|f_{m}\|+\varepsilon )\|x\|$ , so that $f\in L(X,Y)$ and $\|f-f_{m}\|\leq \varepsilon$ .

Thus $f_{m}\to f$ in the norm of $L(X,Y)$ . This establishes the completeness of $L(X,Y)$ ^[4]

Theorem 2: Now suppose $B$ is the closed unit ball of normed space $X$ . Define

\|x^{*}\|=\sup\{|\langle {x,x^{*}}\rangle |:x\in B\}

for every $x^{*}\in X^{*}$

(a) This norm makes

X^{*}

into a Banach space.^[5]

(b) Let

B^{*}

be the closed unit ball of

X^{*}

. For every

x\in X

\|x\|=\sup\{|\langle {x,x^{*}}\rangle |:x^{*}\in B^{*}\}.

Consequently,

x^{*}\to \langle {x,x^{*}}\rangle

is a bounded linear functional on

X^{*}

, of norm

\|x\|

(c)

B^{*}

is weak*-compact.

Proof

Since

L(X,Y)=X^{*}

, when

Y

is the scalar field, (a) is a corollary of Theorem 1.

Fix

x\in X

. There exists^[6]

y^{*}\in B^{*}

such that

\langle {x,y^{*}}\rangle =\|x\|.

but,

|\langle {x,x^{*}}\rangle |\leq \|x\|\|x^{*}\|\leq \|x\|

for every

x^{*}\in B^{*}

. (b) follows from the above.

Since the open unit ball

U

X

is dense in

B

, the definition of

\|x^{*}\|

shows that

x^{*}\in B^{*}

if and only if

|\langle {x,x^{*}}\rangle |\leq 1

for every

x\in U

The proof for (c)^[7] now follows directly.^[8]

The second dual of a Banach space is an isometric isomorphism

The normed dual $X^{*}$ of a Banach space $X$ is also a Banach space, which means it has a normed dual, $X^{{**}}$ , of its own.

By part (b) of Theorem 2, every $x\in X$ defines a unique $\phi \in X^{**}$ by equation

\langle {x,x^{*}}\rangle =\langle {x^{*},\phi x}\rangle \;\;\;\;\;\;\;\;(x^{*}\in X^{*});

and

\lVert \phi x\rVert =\lVert x\rVert \;\;\;\;\;\;\;\;(x\in X).

It follows from the first and second equation that $\phi :X\to X^{**}$ is linear and $\phi$ is an isometry. Given that $X$ is assumed to be complete, $\phi (X)$ is closed in $X^{{**}}$ .

Thus, $\phi$ is an isometric isomorphism onto a closed subspace of $X^{{**}}$ .^[9]

The members of $\phi (x)$ are exactly the linear functionals on $X^{*}$ that are continuous with respect to its weak*-topology. Since the norm topology of $X^{*}$ is stronger, may happen that $\phi (X)$ is a proper subspace of $X^{{**}}$ .

However, there are many important spaces, such as the Lp spaces with $1 < p < \infty$ , where $\phi (X)=X^{**}$ ; these are called reflexive.

It is stressed that, for $X$ to be reflexive, the existence of some isometric isomorphism $\phi$ of $X$ onto $X^{{**}}$ is not enough; it is crucial that $\phi$ satisfies first equation in this section.^[10]

Mathematical Optimization

Let $||\cdot ||$ be a norm on $\mathbb {R} ^{n}$ . The associated dual norm, denoted $\|\cdot \|_{*}$ , is defined as

||z||_{*}=\sup\{z^{\intercal }x\;|\;||x||\leq 1\}.

(This can be shown to be a norm.) The dual norm can be interpreted as the operator norm of $z^{\intercal }$ , interpreted as a $1\times n$ matrix, with the norm $||\cdot ||$ on $\mathbb {R} ^{n}$ , and the absolute value on $\mathbb {R}$ :

||z||_{*}=\sup\{|z^{\intercal }x|\;|\;||x||\leq 1\}.

From the definition of dual norm we have the inequality

z^{\intercal }x\leq \lVert x\rVert \lVert z\rVert _{*}

which holds for all $x$ and $z$ .^[11] The dual of the dual norm is the original norm: we have $\lVert x\rVert _{**}=\lVert x\rVert$ for all $x$ . (This need not hold in infinite-dimensional vector spaces.)

The dual of the Euclidean norm is the Euclidean norm, since

\sup\{z^{\intercal }x\;|\;\lVert x\rVert _{2}\leq 1\}=\lVert z\rVert _{2}.

(This follows from the Cauchy-Schwarz inequality; for nonzero $z$ , the value of $x$ that maximises $z^{\intercal }x$ over $\lVert x\rVert _{2}\leq 1$ is ${\frac {z}{\lVert z\rVert _{2}}}$ .)

The dual of the $\ell _{1}$ -norm is the $\ell_\infty$ -norm:

\sup\{z^{\intercal }x\;|\;\lVert x\rVert _{\infty }\leq 1\}=\sum _{i=1}^{n}|z_{i}|=\lVert z\rVert _{1},

and the dual of the $\ell_\infty$ -norm is the $\ell _{1}$ -norm.

More generally, Hölder's inequality shows that the dual of the $\ell _{p}$ -norm is the $\ell _{q}$ -norm, where, $q$ satisfies ${\frac {1}{p}}+{\frac {1}{q}}=1$ , i.e., $q={\frac {p}{p-1}}.$

As another example, consider the $\ell _{2}$ - or spectral norm on $\mathbb {R} ^{m\times n}$ . The associated dual norm is

\lVert Z\rVert _{2*}=\sup\{\mathrm {\bf {tr}} (Z^{\intercal }X)|\;\lVert X\rVert _{2}\leq 1\},

which turns out to be the sum of the singular values,

\lVert Z\rVert _{2*}=\sigma _{1}(Z)+\ldots +\sigma _{r}(Z)=\mathrm {\bf {tr}} (Z^{\intercal }Z)^{\frac {1}{2}},

where $r=\mathrm {\bf {rank}} \;Z$ . This norm is sometimes called the nuclear norm.^[12]

Examples

Dual norm for matrices

Main articles: Hilbert-Schmidt operator and Matrix norm § Frobenius norm

The Frobenius norm defined by

\left\| A \right\| _{\text{F}} = \sqrt{\sum_{i=1}^m\sum_{j=1}^n \left| a_{ij} \right| ^2} = \sqrt{\operatorname{trace}(A^{{}^*}A)}=\sqrt{\sum_{i=1}^{\min\{m,\,n\}} \sigma_{i}^2}

is self-dual, i.e., its dual norm is

\left\|\cdot \right\|'_{\text{F}}=\left\|\cdot \right\|_{\text{F}}

The spectral norm, a special case of the induced norm when

p=2

, is defined by the maximum singular values of a matrix, i.e.,

\left\| A \right\| _2 = \sigma_{max}(A)

has dual norm defined by

\|B\|'_{2}=\sum _{i}\sigma _{i}(B)

for any matrix

B

where

\sigma _{i}(B)

denote the singular values.

Notes

↑ Each $L(X,Y)$ is a vector space, with the usual definitions of addition and scalar multiplication of functions; this only depends on the vector space structure of $Y$ , not $X$ .
↑ Rudin 1991, p. 92
↑ Rudin 1991, p. 93
↑ Rudin 1991, p. 93
↑ Aliprantis 2005, p. 230
6.7 Definition The norm dual $X^{*}$ of a normed space $(X,||\cdot ||)$ is Banach space $L(X,\mathbb {R} )$ . The operator norm on $X^{*}$ is also called the dual norm, also denoted $||\cdot ||$ . That is,
$||x^{*}||=\sup _{||x||\leq 1}|\langle {x^{*},x}\rangle |=\sup _{||x||=1}|\langle {x^{*},x}\rangle |$
The dual space is indeed a Banach space by Theorem 6.6.
↑ Rudin 1991, Theorem 3.3 Corollary, p. 59
↑ Rudin 1991, Theorem 3.15 The Banach-Alaoglu theorem algorithm, p. 68
↑ Rudin 1991, p. 94
↑ Rudin 1991, Theorem 4.5 The second dual of a Banach space, p. 95
↑ Rudin 1991, p. 95
↑ This inequality is tight, in the following sense: for any $x$ there is a $z$ for which the inequality holds with equality. (Similarly, for any $z$ there is an $x$ that gives equality.)
↑ Boyd & Vandenberghe 2004, p. 637

References

Aliprantis, Charalambos D.; Border, Kim C. (2007). Infinite Dimensional Analysis: A Hitchhiker's Guide (3rd ed.). Springer. ISBN 9783540326960.
Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization. Cambridge University Press. ISBN 9780521833783.
Kolmogorov, A.N.; Fomin, S.V. (1957). Elements of the Theory of Functions and Functional Analysis, Volume 1: Metric and Normed Spaces. Rochester: Graylock Press.
Rudin, Walter (1991), Functional analysis, McGraw-Hill Science, ISBN 978-0-07-054236-5 .

External links

Notes on the proximal mapping by Lieven Vandenberge

This article is issued from Wikipedia - version of the 7/28/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.