A Special Case

Consider the optimal control problem of minimizing the functional

(1)   \begin{equation*}\frac 1{2}\int_{0}^{\infty}(\Vert y\Vert^2+\epsilon^2\Vert u\Vert^2)dt\end{equation*}

subject to the dynamics

(2)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1+A_2x_2\\\dot x_2&=&A_3x_1+A_4x_2+B_2u\\y&=&x_2,\end{array}\end{equation*}

so that

    \[A=\left(\begin{array}{cc}A_1& A_2\\ A_3& A_4\end{array}\right),~B=\left(\begin{array}{c}0\\ B_2\end{array}\right),~C=\left(\begin{array}{cc}0& I\end{array}\right).\]

We make the assumptions that the system (2) is time-invariant, has relative degree (1,…,1), (A,B) is stabilizable, (C,A) is detectable and that B_2 is nonsingular. (In lemma 3.1 below we show which systems can be represented in this form.) We want to study the limiting behavior of the solution to this problem, i.e. when the closed-loop system is stable, as \epsilon tends to zero.

We begin with a heuristic analysis of the problem of finding a feedback control law that minimizes the functional (1) and stabilizes the system. Since B_2 is nonsingular, x_2 is completely controllable. Thus, we can use x_2 to control x_1. If u is bounded, the cost of control is small for small \epsilon.

Suppose that A_1 is stable. Then we see that y=x_2=0 minimizes the functional. In this case we can find a bounded feedback control which makes the output zero. Hence the remaining dynamics

    \[\dot x_1=A_1x_1\]

are just the zero dynamics of the original system (see section 2.1), which are stable by assumption. On the other hand, if A_1 is antistable, stabilizability of the system implies that the pair (A_1,A_2) is controllable. Since x_2 is completely controllable, we can let x_2=v be a control for x_1 and find an optimal v by solving the reduced problem

(3)   \begin{equation*}\min \frac 1{2}\int_{0}^{\infty}\Vert v\Vert^2 dt\end{equation*}

subject to

(4)   \begin{equation*}\begin{array}{ccl}\dot x_1=A_1x_1+A_2v.\end{array}\end{equation*}

The facts that v=x_2 is optimal and that u is a feedback control of the variable x_2 suggest that the corresponding u for the original problem (1)-(2) is optimal too. Unfortunately, since the system (4) is not detectable, an optimal solution to the reduced problem (3)-
(4) may not exist, see Wonham [2, p. 280]. However, it is easy to see that an optimal solution to the problem (3)-(4) does exist if we make the assumption that A_1 has no eigenvalues on the imaginary axis. Nevertheless, we will give a detailed description of the solution in the end of this section.

First, we motivate our analysis of this special case by stating the following lemma.

Lemma 3.1: Consider the n-dimensional linear system

(5)   \begin{equation*}\begin{array}{ccl}\dot x&=&Ax+Bu\\y&=&Cx\end{array}\end{equation*}

and suppose that the system has relative degree (1,…,1). (It is also natural to assume that the input matrix B has
linearly independent columns.) Then the system (5) can always be transformed to the form (2)

Proof: We can always write

    \[\dot x=Ax+Bu\]

in the controllability canonical form (see e.g. Kwakernaak and Sivan [3, pp. 60-61]),

(6)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1\\\dot x_2&=&A_3x_1+A_4x_2+B_2u_2,\end{array}\end{equation*}

where x_1\in R^{n-m}, x_2\in R^m and B_2 is a non-singular m\times m matrix (since B has full column rank). Furthermore, the pair (A_4,B_2) is completely controllable. Partition y as

    \[y=C_1x_1+C_2x_2.\]

The condition that the system has relative degree (1,…,1) implies that CB\not= 0 and, by assumption, that the system has equal number of inputs and outputs (see section 2.1). Thus,

    \begin{equation*}CB=\left(\begin{array}{cc}C_1& C_2\end{array}\right)\left(\begin{array}{c}0\\ B_2\end{array}\right)=C_2B_2\not= 0,\end{equation*}

so that C_2 must be non-singular. Now we let

    \[z=C_1x_1+C_2x_2\]

or

    \[x_2=C_2^{-1}(z-C_1x_1),\]

which, after substitution into (6), yields

(7)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1\\\dot z&=&(C_1A_1+C_2A_3-C_2A_4C_2^{-1}C_1)x_1+\\&+&C_2A_4C_2^{-1}z+C_2B_2u_2\\y&=&z.\end{array}\end{equation*}

Renaming the parameters in (7) shows the statement.\bullet

Solving the optimal control problem (1)-(2), we define the Hamiltonian function

    \begin{equation*}\begin{array}{ccl}H&=&\frac 1{2}x_2^Tx_2+\frac 1{2}\epsilon^2 u^Tu\\&+&\left(\begin{array}{cc}\lambda_1^T& \lambda_2^T\end{array}\right)\left(\begin{array}{cc}A_1& A_2 \\ A_3& A_4\end{array}\right)\left(\begin{array}{c}x_1\\ x_2\end{array}\right)+\left(\begin{array}{c}0\\ B_2u\end{array}\right),\end{array}\end{equation*}

where \lambda_1 and \lambda_2 satisfy the adjoint equations

(8)   \begin{equation*}\begin{array}{ccl}\dot \lambda_1=-\partial{H}/\partial{x_1}&=&-A_1^T\lambda_1-A_3^T\lambda_2\\\dot \lambda_2=-\partial{H}/\partial{x_2}&=&-x_2-A_2^T\lambda_1-A_4^T\lambda_2.\end{array}\end{equation*}

Setting \partial{H}/\partial{u}=0, we find that the optimal control law is

(9)   \begin{equation*}u=-\frac 1{\epsilon^2}B_2^T\lambda_2.\end{equation*}

In order to study the behavior of the solution as \epsilon goes to zero, we use singular perturbation methods (see section 2.2). We rescale the adjoint variable \lambda_2 as \bar\lambda_2=\lambda_2/\epsilon, so that the adjoint equations (8) can be written

(10)   \begin{equation*}\begin{array}{ccl}\dot \lambda_1=-\partial{H}/\partial{x_1}&=&-A_1^T\lambda_1-\epsilon A_3^T\bar\lambda_2\\\dot{\bar \lambda}_2=-\partial{H}/\partial{x_2}&=&-x_2-A_2^T\lambda_1-\epsilon A_4^T\bar\lambda_2.\end{array}\end{equation*}

Multiplying the second equation in (2) by \epsilon and using (9) and (10), we
obtain the singularly perturbated system

(11)   \begin{equation*}\begin{array}{ccl}\left(\begin{array}{c}\dot x_1\\ \dot \lambda_1\end{array}\right)&=&\left(\begin{array}{cc}A_1& 0\\ 0& -A_1^T\end{array}\right)\left(\begin{array}{c}x_1\\ \lambda_1\end{array}\right)+\\ &+&\left(\begin{array}{cc}A_2& 0\\ 0& -\epsilon A_3^T\end{array}\right)\left(\begin{array}{c}{x_2\\ \bar\lambda_2\end{array}\right)\\\epsilon \left(\begin{array}{c}\dot x_2\\ \dot {\bar\lambda}_2\end{array}\right)&=&\left(\begin{array}{cc}\epsilon A_3& 0\\ 0& -A_2^T\end{array}\right)\left(\begin{array}{c}x_1\\ \lambda_1\end{array}\right)+\\ &+&\left(\begin{array}{cc}\epsilon A_4& B_2B_2^T\\ -I& -\epsilon A_4^T\end{array}\right)\left(\begin{array}{c}x_2\\ \bar\lambda_2\end{array}\right).\end{array}\end{equation*}

Since B_2 is nonsingular, we see that the system clearly satisfies the conditions of theorem 2.2 if we assume that the resulting Riccati equation has a unique positive semidefinite stabilizing solution. Therefore we set \epsilon=0 to obtain

(12)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1+A_2x_2\\\dot \lambda_1&=&-A_1^T\lambda_1\\B_2B_2^T\bar\lambda_2&=&0\\0&=&-A_2^T\lambda_1-x_2.\end{array}\end{equation*}

so that

(13)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1+A_2A_2^T\lambda_1\\\dot \lambda_1&=&-A_1^T\lambda_1\end{array}\end{equation*}

Now there exists an invariant subspace such that

(14)   \begin{equation*}\lambda_1=Px_1,\end{equation*}

and

(15)   \begin{equation*}\dot \lambda_1=P\dot x_1.\end{equation*}

Substitution of (14) and (15) into (13) gives the reduced system

(16)   \begin{equation*}\dot x_1=(A_1-A_2A_2^TP)x_1\end{equation*}

where P satisfies the Riccati equation

(17)   \begin{equation*}PA_1+A_1^TP+PA_2A_2^TP=0\end{equation*}

Referring to our heuristic solution, we show that the solution to the reduced problem (16)-(17) is exactly the solution to the problem

(18)   \begin{equation*}\min \frac 1{2}\int_{0}^{\infty}\Vert v\Vert^2 dt\end{equation*}

subject to

(19)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1+A_2v.\end{array}\end{equation*}

The Hamiltonian function for this problem is

    \[H=\frac 1{2}v^2+\lambda^TA_1x_1+\lambda^TA_2v,\]

where

    \[\dot \lambda=-\frac {\partial{H}}{\partial{x_1}}=-A_1^T\lambda.\]

The optimal control law is

    \[v=-A_2^T\lambda.\]

Letting

    \[\lambda=P_vx_1\]

and

    \[\dot \lambda=P_v\dot x_1,\]

we get the closed-loop system

    \[\dot x_1=(A_1-A_2A_2^TP_v)x_1,\]

where P_v is the positive semidefinite solution (if it exists) to the Riccati equation

    \[P_vA_1+A_1^TP_v-P_vA_2A_2^TP_v=0.\]

Thus, we have shown that the problem (1)-(2) reduces, as \epsilon \to 0, to the reduced problem (18)-(19). Now we will turn to the question of existence and uniqueness of the solution to the reduced problem.

Proposition 3.1: Consider the Riccati equation (17). If the matrix A_1 does not have any eigenvalues
on the imaginary axis, there exists a unique, positive semidefinite solution P to the Riccati equation which stabilizes the system (16).

Proof: Note that the detectability condition of theorem 2.5 is not satisfied for the problem (18)-(19). Therefore, we consider the three possible cases where A_1 is stable (all eigenvalues of A_1 lie in the open left-hand complex plane), antistable (all eigenvalues of A_1 lie in the open right-hand complex plane – we assumed that A_1 does not have any eigenvalue on the imaginary axis), and when A_1 has some eigenvalues in the open left-hand complex plane and some eigenvalues in the open right-hand complex plane.

Case 1:
Suppose that \sigma(A_1)\in\bf{C}^-. Then the Riccati equation (17) has the unique positive semidefinite solution P=0 and (16) becomes

    \[\dot x_1=A_1x_1.\]

Remark:
We see that this is just the zero dynamics of the system. Using the method outlined in section 2.1 on the system (2)

(20)   \begin{equation*}\begin{array}{ccl}\dot x_1&=&A_1x_1+A_2x_2\\\dot x_2&=&A_3x_1+A_4x_2+B_2u\\y&=&x_2\end{array}\end{equation*}

we set
y=0, so that \dot y=0 too. Then

    \[\dot y=\dot x_2=A_3x_1+A_4x_2+B_2u.\]

Since B_2 is nonsingular we choose

    \[u=-B_2^{-1}(A_3x_1+A_4x_2).\]

Changing coordinates according to

(21)   \begin{equation*}\begin{array}{ccl}z_1&=&y=x_2\\z_2&=&x_1,\end{array}\end{equation*}


we obtain

(22)   \begin{equation*}\begin{array}{ccl}\dot z_1&=&\dot y=A_3x_1+A_4x_2+B_2u\\\dot z_2&=&\dot x_1=A_1x_1+A_2x_2=A_1z_2+A_2z_1\end{array}\end{equation*}

Since z_1=\dot z_1=0, the zero dynamics of the system are described by

    \[\dot z_2=A_1z_2.\]

Case 2:
Suppose that \sigma(A_1)\in\bf{C}^+. Then (A_1,A_2) must be controllable due to the stabilizability condition. Consider the Lyapunov equation

(23)   \begin{equation*}-QA_1^T-A_1Q+A_2A_2^T=0.\end{equation*}

By integrating the expression

    \begin{equation*}\begin{array}{ccl}-\frac d{dt}(e^{-A_1t}Qe^{-A_1^Tt})&=&-e^{-A_1t}(-A_1Q-QA_1^T)e^{-A_1^Tt}=\\&=&e^{-A_1t}(A_2A_2^T)e^{-A_1^Tt},\end{array}\end{equation*}

using the Lyapunov equation (23), the stability of -A_1 and the fact that A_2A_2^T\geq 0, we see that

    \[Q=\int_{0}^{\infty}e^{-A_1t}A_2A_2^Te^{A_1^Tt}dt\]

is a solution. This is just the controllability Grammian of (A_1,A_2). (See e.g. Wonham [2,~pp.~277-278].) Furthermore, if (A_1,A_2) is controllable, we have that

    \[W(\tau)=\int_{0}^{\tau}e^{A_1t}A_2A_2^Te^{A_1^Tt}dt\]

is positive definite for every \tau >0 (see Wonham [2, p. 38]).

Thus Q is positive definite and therefore Q^{-1} exists. If we let Q^{-1}=P and multiply the Lyapunov equation by Q^{-1} from the right and from the left, we see that this is just the reduced Riccati equation (17). Since P is nonsingular, we have from (17)

    \[A_2A_2^TP=P^{-1}A_1^TP+A_1,\]

so that (16) becomes

    \[\dot x_1=(A_1-P^{-1}A_1^TP-A_1)x_1=-P^{-1}A_1^TPx_1.\]

Since the transpose of a matrix has the same eigenvalues as the original matrix and the similarity transformation does not change the eigenvalues, the reduced system has the same eigenvalues as A_1, but reflected in the imaginary axis.

Writing the equation as

    \[P\dot x_1=-A_1^TPx_1,\]

and referring to (13), we see that this is just the adjoint equation of x_1.

Case 3:
Finally, suppose that A_1 is unstable with some stable and some unstable eigenvalues. Define the modal subspaces
(see Francis [4, pp. 85-86]) of R^n as

(24)   \begin{equation*}\begin{array}{ccl}X_-(A)&:=&\ker~ \alpha_-(A)\\X_+(A)&:=&\ker~ \alpha_+(A),\end{array}\end{equation*}

where \alpha (s)=\alpha_-(s)\alpha_+(s) is the characteristic polynomial of A. The factor \alpha_- has all its zeros in Re s<0 and \alpha_+ has all its zeros in Re s>0. (There are no zeros on the imaginary axis.) It can be shown that X_-(A) is spanned by the generalized real eigenvectors of A corresponding to eigenvalues in
Re s<0 and similarly for X_+(A). These two modal subspaces are complementary, i.e. they are independent and their sum is all of R^n. Thus

    \[R^n=X_-(A)\oplus X_+(A).\]

Now, because A_1 is supposed not to have any eigenvalues on the imaginary axis, there exists a similarity transformation T such that T^{-1}A_1T has the form

    \[\left(\begin{array}{cc}A_{11}& 0\\ 0& A_{14}\end{array}\right),\]

where A_{11} is stable and A_{14} is antistable. Partitioning P and A_2 accordingly, the algebraic Riccati equation (17) can then be written

(25)   \begin{equation*}\begin{array}{ccl}\left(\begin{array}{cc}A_{11}^T& 0\\ 0& A_{14}^T\end{array}\right)\left(\begin{array}{cc}P_{1}& P_{2}\\P_{2}^T& P_{4}\end{array}\right)+\left(\begin{array}{cc}P_{1}& P_{2}\\ P_{2}^T& P_{4}\end{array}\right)\left(\begin{array}{cc}A_{11}& 0\\ 0& A_{14}\end{array}\right)-\\-\left(\begin{array}{cc}P_{1}& P_{2}\\P_{2}^T& P_{4}\end{array}\right)\left(\begin{array}{cc}A_{21}& A_{22}\\ A_{23}& A_{24}\end{array}\right)\left(\begin{array}{cc}A_{21}^T& A_{23}^T\\ A_{22}^T& A_{24}^T\end{array}\right)\left(\begin{array}{cc}P_{1}& P_{2}\cr P_{2}^T& P_{4}\end{array}\right)=0.\end{array}\end{equation*}

If we can find a positive semidefinite matrix P which stabilizes the system (16), i.e. makes A_1-A_2A_2^TP stable, and satisfies the Riccati equation (25), P must be unique. Suppose that P has the form

    \[\left(\begin{array}{cc}*& 0\\ 0& *\end{array}\right).\]

We see that we have the conditions

(26)   \begin{equation*}\begin{array}{ccl}{\rm (i)}&\quad &A_{11}-A_{21}A_{21}^TP_1-A_{22}A_{22}^TP_1\quad{\rm must~be~stable,}\\{\rm (ii)}&\quad &A_{14}-A_{23}A_{23}^TP_4-A_{24}A_{24}^TP_4\quad{\rm must~be~stable.}\end{array}\end{equation*}

From case 1 and 2, we see that the Riccati equation (25) and the stability conditions (i) and (ii) are satisfied if we choose P_1=0 and P_4 as the positive semidefinite solution to the equation

(27)   \begin{equation*}A_{14}^TP_4+P_4A_{14}-P_4A_{23}A_{23}^TP_4-P_4A_{24}A_{24}^TP_4=0.\end{equation*}

Such a solution exists, since the pair (A_1,A_2) is stabilizable by assumption. Thus, we have the resulting system

(28)   \begin{equation*}\left(\begin{array}{c}\dot x_{11}\\ \dot x_{12}\end{array}\right)=\left(\begin{array}{cc}A_{11}& -(A_{21}A_{23}^T+A_{22}A_{24}^T)P_4\\0& -P_4^{-1}A_{14}^TP_4\end{array}\right)\left(\begin{array}{c}x_{11}\\ x_{12}\end{array}\right).\end{equation*}

It is easily seen that the closed-loop system is stable since its eigenvalues are exactly those of the stable matrix A_{11} and those of the antistable matrix A_{14} with negative sign.\bullet

To interpret this result, first we observe that due to stabilizability, the unstable subspace must lie in the controllable subspace, R, but we can not conclude that the intersection of the stable subspace and the
controllable subspace is empty.

Consider the decomposition of case 3 into the stable and unstable modal subspaces. Suppose that X_-(A)\cap R=0, although this in general is not true. Then we simply have X_+(A)=R and we can put the system (19) in its controllability canonical form, i.e.

(29)   \begin{equation*}\dot x=\left(\begin{array}{cc}A_{11}& 0\\ 0& A_{14}\end{array}\right)x+\left(\begin{array}{c}0\\ A_2'\end{array}\right)v',\end{equation*}

where the zeros in the matrices follow from X_-(A)\cap X_+(A)=0 and the A-invariance of R. Therefore we have

(30)   \begin{equation*}\begin{array}{ccl}\dot x_{11}&=&A_{11}x_{11}\\\dot x_{12}&=&A_{14}x_{12}+A_2'v',\end{array}\end{equation*}

which corresponds to A_{21}A_{23}^T+A_{22}A_{24}^T=0 in case 3. This would give the reduced system

(31)   \begin{equation*}\begin{array}{ccl}\dot x_{11}&=&A_{11}x_{11}\\\dot x_{12}&=&-P_4^{-1}A_{14}^TP_4x_{12}.\end{array}\end{equation*}

Furthermore, we observe that R={\rm Im}~W(t), where W(t) is the controllability Grammian. Thus, we can interpret the above results as follows:

In the subspace X_-(A_1)/X_-(A_1)\cap R, the optimal control gives the zero dynamics restricted to the stable subspace, in X_+(A_1)\subseteq R we have the adjoint equation subject to a similarity transformation by the inverse of the controllability Grammian of X_+(A_1)\cap R, i.e. of R restricted to X_+(A_1). Finally, the subspace X_-(A_1)\cap R is naturally also affected by the minimum control which, however, does not change the stability properties of the matrix A_1.

To conclude, we observe that the conditions for L^2-bounding are satisfied. Considering the decomposition and lemma 2.2 of section 2.3 we see that

    \[{\rm rank~}G(s)={\rm rank~}B_2={\rm rank~}CB.\]

Furthermore, from the comment following theorem 2.4 and the fact that G(s) is square and has full rank, the condition that \gamma_r(s^2) has no zeros on the imaginary axis is exactly the condition that A_1 does not have any eigenvalues on the imaginary axis, since the eigenvalues of A_1 are the transmission zeros of the system.