.md .pdf

repository open issue

Contents

8.9. Differentiability and Convex Functions

8.9.1. First Order Conditions

Let us look at the special case of real valued functions over ${\mathbb{R}}^n$ which are differentiable.

Theorem 8.98 (First order characterization of convexity)

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a real valued function which is differentiable at each point in $\mathrm{dom}f$ which is open.

Then $f$ is convex if and only if $\mathrm{dom}f$ is convex and

(8.4)$$f(\mathbf{y})\ge f(\mathbf{x})+\mathrm{\nabla }f(\mathbf{x}{)}^T(\mathbf{y}-\mathbf{x})$$

holds true for all $\mathbf{x},\mathbf{y}\in \mathrm{dom}f$.

Proof. To prove (8.4), we first show that a differentiable real function $f:\mathbb{R}\to \mathbb{R}$ is convex if and only if

$$f(y)\ge f(x)+f^{\prime }(x)(y-x)$$

holds true for all $x,y\in \mathrm{dom}f$.

Assume that $f$ is convex. Hence, $\mathrm{dom}f$ is convex too.

1. Let $x,y\in \mathrm{dom}f$.

2. Since $\mathrm{dom}f$ is convex, hence $(1-t)x+ty=x+t(y-x)\in \mathrm{dom}f$ for all $t\in [0,1]$.

3. By convexity of $f$, we have:

   $$f(x+t(y-x))\le (1-t)f(x)+tf(y).$$

4. If we divide by $t$ on both sides, we obtain:

   $$f(y)\ge f(x)+\frac{f(x+t(y-x))-f(x)}{t}.$$

5. Taking the limit as $t\to 0^{+}$, we obtain:

   $$f(y)\ge f(x)+f^{\prime }(x)(y-x).$$

For the converse, assume that $\mathrm{dom}f$ is convex and

$$f(y)\ge f(x)+f^{\prime }(x)(y-x)$$

holds true for all $x,y\in \mathrm{dom}f$.

1. Recall that in $\mathbb{R}$ the only convex sets are intervals. Thus, $\mathrm{dom}f$ is an open interval.

2. Choose any $x,y\in \mathrm{dom}f$ such that $x\ne y$.

3. Choose $t\in [0,1]$.

4. Let $z=tx+(1-t)y$.

5. By hypothesis, we have:

   $$f(x)\ge f(z)+f^{\prime }(z)(x-z)$$

   and

   $$f(y)\ge f(z)+f^{\prime }(z)(y-z).$$

6. Multiplying the first inequality with $t$ and second with $(1-t)$ and adding them yields:

   $$tf(x)+(1-t)f(y)\ge f(z)=f(tx+(1-t)y).$$

7. Thus, $f$ is convex.

We now prove for the general case with $f:{\mathbb{R}}^n\to \mathbb{R}$. Recall from Theorem 8.70 that for any $\mathbf{x},\mathbf{y}\in \mathrm{dom}f$ the restriction of $f$ on the line passing through $\mathbf{x}$ and $\mathbf{y}$ is given by:

$$g(t)=f(t\mathbf{y}+(1-t)\mathbf{x})=f(\mathbf{x}+t(\mathbf{y}-\mathbf{x})).$$

Note that, by chain rule (Example 5.11):

$$g^{\prime }(t)=\mathrm{\nabla }f(t\mathbf{y}+(1-t)\mathbf{x}{)}^T(\mathbf{y}-\mathbf{x})$$

Assume $f$ is convex.

1. Let $\mathbf{x},\mathbf{y}\in \mathrm{dom}f$ such that $\mathbf{x}\ne \mathbf{y}$.

2. Let $g$ be the restriction of $f$ on the line passing through $\mathbf{x},\mathbf{y}$ as described above.

3. Due to Theorem 8.70, $g$ is convex.

4. By the argument for real functions above:

   $$g(t^{\prime })\ge g(t)+g^{\prime }(t)(t^{\prime }-t)$$

   holds true for all $t,t^{\prime }\in \mathrm{dom}g$.

5. In particular, with $t^{\prime }=1$ and $t=0$, we have:

   $$g(1)\ge g(0)+g^{\prime }(0).$$

6. But $g^{\prime }(0)=\mathrm{\nabla }f(\mathbf{x}{)}^T(\mathbf{y}-\mathbf{x})$.

7. Also, $g(1)=f(\mathbf{y})$ and $g(0)=f(\mathbf{x})$.

8. Thus, we get:

   $$f(\mathbf{y})\ge f(\mathbf{x})+\mathrm{\nabla }f(\mathbf{x}{)}^T(\mathbf{y}-\mathbf{x})$$

   as desired.

For the converse, assume that this inequality holds for all $\mathbf{x},\mathbf{y}\in \mathrm{dom}f$ and $\mathrm{dom}f$ is convex.

1. Pick some $\mathbf{x},\mathbf{y}\in \mathrm{dom}f$ with $\mathbf{x}\ne \mathbf{y}$.

2. Let $g$ be the restriction of $f$ on the line passing through $\mathbf{x},\mathbf{y}$ as described above.

3. Pick $t_1,t_2\in \mathrm{dom}g$.

4. Then, ${\mathbf{z}}_1=t_1\mathbf{y}+(1-t_1)\mathbf{x}$ and ${\mathbf{z}}_2=t_2\mathbf{y}+(1-t_2)\mathbf{x}$ are in $\mathrm{dom}f$.

5. Consider $g(t_1)=f(t_1\mathbf{y}+(1-t_1)\mathbf{x})=f({\mathbf{z}}_1)$ and $g(t_2)=f(t_2\mathbf{y}+(1-2_1)\mathbf{x})=f({\mathbf{z}}_2)$.

6. Note that $g^{\prime }(t_2)=\mathrm{\nabla }f(t_2\mathbf{y}+(1-t_2)\mathbf{x}{)}^T(\mathbf{y}-\mathbf{x})=\mathrm{\nabla }f({\mathbf{z}}_2{)}^T(\mathbf{y}-\mathbf{x})$.

7. By hypothesis, we have:

   $$f({\mathbf{z}}_1)\ge f({\mathbf{z}}_2)+\mathrm{\nabla }f({\mathbf{z}}_2{)}^T({\mathbf{z}}_1-{\mathbf{z}}_2).$$

8. But ${\mathbf{z}}_1-{\mathbf{z}}_2=(t_1-t_2)(\mathbf{y}-\mathbf{x})$.

9. Thus, we get:

   $$g^{\prime }(t_1)\ge g^{\prime }(t_2)+g^{\prime }(t_2)(t_1-t_2).$$

10. This holds for every $t_1,t_2\in \mathrm{dom}g$.

11. But then, $g$ is convex by previous argument for real functions.

12. Since this is valid for every restriction of $f$ to a line passing through its domain, hence by Theorem 8.70 $f$ is convex.

8.9.2. Second Order Conditions

For functions which are twice differentiable, convexity can be expressed in terms of the positive-semidefiniteness of their Hessian matrices.

We start with a result on convexity of real functions on open intervals.

Theorem 8.99 (Convexity characterization for twice differentiable real functions on open intervals)

Let $f:\mathbb{R}\to \mathbb{R}$ be twice continuously differentiable on an open interval $(\alpha ,\beta )$; i.e., second derivative $f^{″}$ exists and is continuous at every point the open interval $(\alpha ,\beta )$.

Then, $f$ is convex if and only if its second derivative $f^{″}$ is non-negative for every $x\in (\alpha ,\beta )$:

$$f^{″}(x)\ge 0\mathrm{\forall }x\in (\alpha ,\beta ).$$

Proof. Assume that $f^{″}$ is nonnegative on $(\alpha ,\beta )$.

1. Then, $f^{\prime }$ is nondecreasing on $(\alpha ,\beta )$.

2. For any $x,y\in (\alpha ,\beta )$ with $x<y$ and $r\in (0,1)$, let $z=(1-r)x+ry$.

3. We have $z\in (x,y)$; i.e. $x<z<y$. Consequently,

   $$\begin{array}{rl}& f(z)-f(x)={\int }_x^z{f}^{\prime }(t)dt\le f^{\prime }(z)(z-x);\\ & f(y)-f(z)={\int }_z^y{f}^{\prime }(t)dt\ge f^{\prime }(z)(y-z).\end{array}$$

4. Since $z-x=r(y-x)$ and $y-z=(1-r)(y-x)$, we have

   $$\begin{array}{r}f(z)\le f(x)+r{f}^{\prime }(z)(y-x);\\ f(z)\le f(y)-(1-r)f^{\prime }(z)(y-x).\end{array}$$

   We wish to eliminate $f^{\prime }(z)$ from these inequalities.

5. Multiplying the two inequalities by $(1-r)$ and $r$ respectively, and adding them together, we obtain:

   $$(1-r)f(z)+rf(z)\le (1-r)f(x)+rf(y).$$

6. But $(1-r)f(z)+rf(z)=f(z)=f((1-r)x+ry)$.

7. Thus, $f((1-r)x+ry)\le (1-r)f(x)+rf(y)$.

8. This inequality is valid for the case where $x>y$ also.

9. Thus, $f$ is convex over $(\alpha ,\beta )$.

For the converse, assume that $f^{″}$ is not non-negative on $(\alpha ,\beta )$.

1. Then, since $f^{″}$ is continuous in $(\alpha ,\beta )$, hence $f^{″}$ is negative in some subinterval $({\alpha }^{\prime },{\beta }^{\prime })$.

2. Choose $x,y$ such that ${\alpha }^{\prime }<x<y<{\beta }^{\prime }$. Choose some $r\in (0,1)$.

3. Following an argument parallel to above, we have

   $$f((1-r)x+ry)>(1-r)f(x)+rf(y).$$

4. Thus, there exist $x,y\in (\alpha ,\beta )$ where the inequality (8.1) is not valid.

5. Consequently, $f$ is non-convex.

We continue further with real valued functions over ${\mathbb{R}}^n$ which are twice differentiable.

Theorem 8.100 (Second order characterization of convexity in Euclidean spaces)

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be twice continuously differentiable; i.e., its Hessian or second derivative ${\mathrm{\nabla }}^{2}f$ exists at every point in $\mathrm{dom}f$ which is open.

Then, $f$ is convex if and only if $\mathrm{dom}f$ is convex and its Hessian is positive semidefinite for every $\mathbf{x}\in \mathrm{dom}f$:

$${\mathrm{\nabla }}^{2}f(\mathbf{x})⪰\mathbf{O}\mathrm{\forall }\mathbf{x}\in \mathrm{dom}f.$$

Proof. The convexity of $f$ on its domain $C=\mathrm{dom}f$ is equivalent to the convexity of the restriction of $f$ to each line segment in $C$ due to Theorem 8.70.

We first note that if $f$ is convex then $C$ is convex and if $C$ is not convex, then $f$ is not convex. So, for the rest of the argument, we shall assume that $C$ is convex.

Consequently, for any $\mathbf{y}\in C$ and a nonzero $\mathbf{z}\in {\mathbb{R}}^n$ the intersection of the line $\{\mathbf{x}=\mathbf{y}+t\mathbf{z}|t\in \mathbb{R}\}$ and $C$ is an open line segment as $C$ is open and convex.

1. Let $\mathbf{y}\in C$.

2. Let $\mathbf{z}\in {\mathbb{R}}^n$ be an arbitrary (nonzero) direction.

3. Let $L=\{\mathbf{x}=\mathbf{y}+t\mathbf{z}|t\in \mathbb{R}\}$ be a line passing through $\mathbf{y}$ in the direction $\mathbf{z}$.

4. Consider the open real interval $S=\{t|\mathbf{y}+t\mathbf{z}\in C\}$. Since $L\cap C$ is an open line segment in ${\mathbb{R}}^n$, hence $S$ is indeed an open interval in $\mathbb{R}$.

5. Consider the parameterized restriction of $f$ on the open interval $S$ as:

   $$g(t)=f(\mathbf{y}+t\mathbf{z}),\mathrm{\forall }t\in S.$$

6. A simple calculation shows that

   $$g^{″}(t)=⟨\mathbf{z},{\mathrm{\nabla }}^{2}f(\mathbf{x})\mathbf{z}⟩$$

   where $\mathbf{x}=\mathbf{y}+t\mathbf{z}$.

7. By Theorem 8.99, $g$ is convex for each $\mathbf{y}\in C$ and nonzero $\mathbf{z}\in {\mathbb{R}}^n$ if and only if $⟨\mathbf{z},{\mathrm{\nabla }}^{2}f(\mathbf{x})\mathbf{z}⟩\ge 0$ for every $\mathbf{z}\in {\mathbb{R}}^n$ and $\mathbf{x}\in C$.

8. Thus, $f$ is convex if and only if ${\mathrm{\nabla }}^{2}f(\mathbf{x})⪰\mathbf{O}\mathrm{\forall }\mathbf{x}\in C$.

For real functions, the Hessian is simply the second derivative $f^{″}$.

Corollary 8.6 (Second order characterization of concavity)

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be twice continuously differentiable; i.e., its Hessian or second derivative ${\mathrm{\nabla }}^{2}f$ exists at every point in $\mathrm{dom}f$ which is open.

Then, $f$ is concave if and only if $\mathrm{dom}f$ is convex and its Hessian is negative semidefinite for every $\mathbf{x}\in \mathrm{dom}f$:

$${\mathrm{\nabla }}^{2}f(\mathbf{x})⪯\mathbf{O}\mathrm{\forall }\mathbf{x}\in \mathrm{dom}f.$$

Example 8.40 (Convexity of a quadratic function)

Let $\mathbf{P}\in {\mathbb{S}}^n$ be a symmetric matrix. Let $\mathbf{q}\in {\mathbb{R}}^n$ and $r\in \mathbb{R}$. Consider the quadratic functional $f:{\mathbb{R}}^n\to \mathbb{R}$ given as:

$$f(\mathbf{x})=\frac{1}{2}{\mathbf{x}}^T\mathbf{P}\mathbf{x}+{\mathbf{q}}^T\mathbf{x}+r.$$

As shown in Example 5.13, the Hessian of $f$ is:

$${\mathrm{\nabla }}^{2}f(\mathbf{x})=\mathbf{P}\mathrm{\forall }\mathbf{x}\in {\mathbb{R}}^n.$$

Thus, $f$ is convex if and only if $\mathbf{P}⪰\mathbf{O}$ (i.e., it is positive semidefinite).

In fact $f$ is strictly convex if and only if $P\mathrm{succ}\mathbf{O}$.

Example 8.41 (Identity is convex and concave)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=x.$$

We have $f^{\prime }(x)=1$ and $f^{″}(x)=0$.

$f$ is both convex and concave.

Example 8.42 (Exponential is convex)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=e^{ax}$$

with $\mathrm{dom}f=\mathbb{R}$.

We have $f^{\prime }(x)=a{e}^{ax}$ and $f^{″}(x)=a^2{e}^{ax}$.

For any $a,x\in \mathbb{R}$, $a^2{e}^{ax}>0$. Thus, $f$ is strictly convex.

Example 8.43 (Powers)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=x^a$$

with $\mathrm{dom}f={\mathbb{R}}_{++}$.

Now, $f^{\prime }(x)=a{x}^{a-1}$ and $f^{″}(x)=a(a-1)x^{a-2}$.

1. We have $x>0$.

2. For $a\ge 1$, $f^{″}(x)\ge 0$. $f$ is convex for $a\ge 1$.

3. For $a\le 0$, $a(a-1)\ge 0$. Thus, $f^{″}(x)\ge 0$. $f$ is convex for $a\le 0$.

4. For $0\le a\le 1$, $a(a-1)\le 0$. Thus, $f^{″}(x)\le 0$. $f$ is concave on $0\le a\le 1$.

Example 8.44 (Reciprocal powers)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=\frac{1}{x^r}=x^{-r}.$$

with $\mathrm{dom}f={\mathbb{R}}_{++}$.

Now, $f^{\prime }(x)=(-r)x^{-r-1}$ and $f^{″}(x)=(-r)(-r-1)x^{-r-2}=r(r+1)x^{-(r+2)}$.

1. We have $x>0$.

2. For $r\ge 0$, $f^{″}(x)\ge 0$. $f$ is convex for $r\ge 0$.

Example 8.45 (Logarithm is concave)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=\ln x.$$

with $\mathrm{dom}f={\mathbb{R}}_{++}$.

Now, $f^{\prime }(x)=\frac{1}{x}$ and $f^{″}(x)=\frac{-1}{x^2}$.

1. $f^{″}(x)<0$ for all $x>0$.

2. Thus, $f$ is concave for all $x>0$.

Example 8.46 (Negative entropy is convex)

Let $f:\mathbb{R}\to \mathbb{R}$ be:

$$f(x)=x\ln x.$$

with $\mathrm{dom}f={\mathbb{R}}_{++}$.

Now, $f^{\prime }(x)=\ln x+1$ and $f^{″}(x)=\frac{1}{x}$.

1. $f^{″}(x)>0$ for all $x>0$.

2. Thus, $f$ is convex for all $x>0$.

Example 8.47 (Quadratic over linear form is convex)

Let $f:\mathbb{R}\times \mathbb{R}\to \mathbb{R}$ be given by:

$$f(x,y)=\frac{x^2}{y}$$

with $\mathrm{dom}f=\{(x,y)|y>0\}$.

From Example 5.16, the Hessian is:

$$\begin{array}{r}{\mathrm{\nabla }}^{2}f(x,y)=\frac{2}{y^3}\left[\begin{array}{cc}{y}^2& -xy\\ -xy& x^2\end{array}\right]=\frac{2}{y^3}\left[\begin{array}{c}y\\ -x\end{array}\right]{\left[\begin{array}{c}y\\ -x\end{array}\right]}^T.\end{array}$$

Recall that for any $\mathbf{x}\in {\mathbb{R}}^n$, the matrix $\mathbf{x}{\mathbf{x}}^T$ is positive semi-definite. Hence,

$$\begin{array}{r}\left[\begin{array}{c}y\\ -x\end{array}\right]{\left[\begin{array}{c}y\\ -x\end{array}\right]}^T\end{array}$$

is positive semi-definite.

For $y>0$, $\frac{2}{y^3}>0$. Combining:

$${\mathrm{\nabla }}^{2}f(x,y)⪰\mathbf{O}.$$

Thus, $f$ is convex.

Example 8.48 (Log sum exponential is convex)

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be given by:

$$f(\mathbf{x})=\ln \left(\sum _{i=1}^n{e}^{x_i}\right)$$

with $\mathrm{dom}f={\mathbb{R}}^n$.

From Example 5.14, we have

$${\mathrm{\nabla }}^{2}f(\mathbf{x})=\frac{1}{({\mathbf{1}}^T\mathbf{z}{)}^2}(({\mathbf{1}}^T\mathbf{z})\mathrm{diag}(\mathbf{z})-\mathbf{z}{\mathbf{z}}^T)$$

where

$$\begin{array}{r}\mathbf{z}=\left[\begin{array}{c}{e}^{x_1}\\ ⋮\\ e^{x_n}\end{array}\right].\end{array}$$

To show that ${\mathrm{\nabla }}^{2}f(\mathbf{x})$ is p.s.d., it suffices to show that $({\mathbf{1}}^T\mathbf{z})\mathrm{diag}(\mathbf{z})-\mathbf{z}{\mathbf{z}}^T$ is p.s.d..

Now for any $\mathbf{v}\in {\mathbb{R}}^n$.

$$\begin{array}{rl}& {\mathbf{v}}^T(({\mathbf{1}}^T\mathbf{z})\mathrm{diag}(\mathbf{z})-\mathbf{z}{\mathbf{z}}^T)\mathbf{v}\\ & =({\mathbf{1}}^T\mathbf{z})({\mathbf{v}}^T\mathrm{diag}(\mathbf{z})\mathbf{v})-{\mathbf{v}}^T\mathbf{z}{\mathbf{z}}^T\mathbf{v}\\ & =({\mathbf{1}}^T\mathbf{z})({\mathbf{v}}^T\mathrm{diag}(\mathbf{z})\mathbf{v})-({\mathbf{v}}^T\mathbf{z}{)}^2\\ & =\left(\sum _{i=1}^n{z}_i\right)\left(\sum _{i=1}^n{v}_i^2{z}_i\right)-{\left(\sum _{i=1}^n{v}_i{z}_i\right)}^2.\end{array}$$

If we define vectors $\mathbf{a}$ and $\mathbf{b}$ with $a_i=v_i\sqrt{z_i}$ and $b_i=\sqrt{z_i}$, then by Cauchy-Schwartz inequality , we have:

$$({\mathbf{a}}^T\mathbf{a})({\mathbf{b}}^T\mathbf{b})\ge ({\mathbf{a}}^T\mathbf{b}{)}^2⟺({\mathbf{a}}^T\mathbf{a})({\mathbf{b}}^T\mathbf{b})-({\mathbf{a}}^T\mathbf{b}{)}^2\ge 0.$$

But this is exactly the expression above. Thus, ${\mathrm{\nabla }}^{2}f(\mathbf{x})⪰\mathbf{O}$.

Hence, $f$ is convex.

Example 8.49 (Log determinant function is concave)

Let $f:{\mathbb{S}}^n\to \mathbb{R}$ be:

$$f(\mathbf{X})=\log detX.$$

with $\mathrm{dom}f={\mathbb{S}}_{++}^n$ (the set of symmetric positive definite matrices).

Let any line in ${\mathbb{S}}^n$ be given by:

$$\mathbf{X}=\mathbf{Z}+t\mathbf{V}$$

where $\mathbf{Z},\mathbf{V}\in {\mathbb{S}}^n$.

Consider the restriction of $f$ on a line:

$$g(t)=\log det(\mathbf{Z}+t\mathbf{V})$$

to the interval of values where $\mathbf{Z}+t\mathbf{V}\mathrm{succ}\mathbf{O}$ (since $\mathrm{dom}f={\mathbb{S}}_{++}^n$ ). In other words,

$$\mathrm{dom}g=\{t\in \mathbb{R}|\mathbf{Z}+t\mathbf{V}\mathrm{succ}\mathbf{O}\}.$$

Without any loss of generality, we can assume that $t=0\in \mathrm{dom}g$; i.e. $\mathbf{Z}\mathrm{succ}\mathbf{O}$.

Recall that:

1. $det(AB)=det(A)det(B)$ for square matrices.

2. $det(A)=\prod _{i=1}^n{\lambda }_i$ for symmetric matrices with ${\lambda }_i$ being their eigen values.

3. If ${\lambda }_i$ are eigen values of $A$, then the eigen values of $I+tA$ are $1+t{\lambda }_i$.

Now

$$\begin{array}{rl}g(t)& =\log det(\mathbf{Z}+t\mathbf{V})\\ & =\log det({\mathbf{Z}}^{\frac{1}{2}}({\mathbf{Z}}^{\frac{1}{2}}+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}))\\ & =\log det({\mathbf{Z}}^{\frac{1}{2}}(I+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}}){\mathbf{Z}}^{\frac{1}{2}})\\ & =\log det({\mathbf{Z}}^{\frac{1}{2}})+\log det(I+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}})+\log det({\mathbf{Z}}^{\frac{1}{2}})\\ & =\log det(\mathbf{Z})+\log det(I+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}}).\end{array}$$

1. Let ${\lambda }_i$ be the eigen values of ${\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}}$.

2. Then, $1+t{\lambda }_i$ are eigen values of $I+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}}$.

3. Thus, $\log det(I+t{\mathbf{Z}}^{-\frac{1}{2}}\mathbf{V}{\mathbf{Z}}^{-\frac{1}{2}})=\sum _{i=1}^n\log det(1+t{\lambda }_i)$.

Thus,

$$g(t)=\sum _{i=1}^n\log det(1+t{\lambda }_i)+\log det(\mathbf{Z}).$$

Note that $\log det(\mathbf{Z})$ doesn’t depend on $t$. Similarly, ${\lambda }_i$ only depend on $\mathbf{Z}$ and $\mathbf{V}$, hence they don’t depend on $t$.

Differentiating $g$ w.r.t. $t$, we get:

$$g^{\prime }(t)=\sum _{i=1}^n\frac{{\lambda }_i}{1+t{\lambda }_i}.$$

Differentiating again, we get:

$$g^{″}(t)=-\sum _{i=1}^n\frac{{\lambda }_i^2}{(1+t{\lambda }_i{)}^2}.$$

Since $g^{″}(t)\le 0$, hence $f$ is concave.

previous

8.8. Convex Functions

next

8.10. Function Operations