Problem 1.
Consider the following optimal control problem
x. = u, x(0) = 1
J = 1/2 0∫Π (u2 - x2)dt
(a) Explain whether or not this is a linear quadratic regulator problem
(b) Find the solution to this optimal control problem (u*(t) and x*(t))
Problem 2.
Consider a fixed final-time nonlinear optimal control problem
x. = 2 sinu, x(0) = 0
-Π/2 ≤ u ≤ Π/2
J = 0∫1 cos2 u(t) dt
(a) The final state x(1) is unconstrained (free). Find all optimal control u*(t) and the corresponding state x*(t)
(b) Now the terminal constraint x(1) = 1 is required. Find all solutions to the control u*(t) and state x*(t) that satisfies the Minimum Principle to this problem.
Problem 3.
Given the system and performance index as follows:
x. = x - u, x(0) = 1
J = 1/20∫T (qx2 + u2)dt
(a) For a finite T>0 and q = 1, this is a finite-time LQR problem. Find the solution to the differential Riccati equation for T = 1, and the optimal closed-loop control law for this problem.
(b) For T = +∞, verify that all the conditions required for an infinite-horizon LQR problem are met with any q > 0. Set q = 1, find the solution to the Algebraic Riccati Equation, the constant-gain optimal closed-loop control law, and the eigenvalue of the closed-loop system. Show details of the work
(c) When a constraint x(1) = 0 is imposed for T = 1, this is no longer an LQR problem (why?). Set q = 0, and find the solution (control and state) to this problem by applying directly the Minimum Principle.
Problem 4.
Consider the following optimal control problem:
x.1 = x2
x.2 = u
x1(0) = 0, x2(0) = 0
x2(1) = 1
1 - x1(1) ≤ 0
J = 1/20∫T u2(t)dt
Obviously the "non-standard" part in this problem is the inequality terminal constraint 1 - x1(1) ≤ 0. In the absence of this constraint, the analytical solutions to the problem are available. Therefore it is possible to find the optimal solution of this problem by obtaining the analytical solution to optimal control problem and then solving a nonlinear programming problem, as described in the following.
(a) Ignore the constraint 1 - x1(1) ≤ 0 for now. Apply the Minimum Principle to obtain the expressions of the optimal control u(t), and states x1(t) and x2(t) as functions of time. They should contain two unknown constants c1 and c2.
(b) By using the expressions of x1(t), and x2(t) in part (a), you can now express the terminal equality constraint x2(1) = 1 as h(c1,c2)=0, and the terminal inequality constraint 1 - x1(1) ≤ 0 as g(c1,c2)≤0, where h and g are linear functions of c1 and c2. Furthermore, using the u(t) in part (a) to obtain the analytical expression of J in terms of c1 and c2. Denote it by f(c1, c2), which should be a quadratic function of c1 and c2.
(c) Now the performance index and terminal (equality and inequality) constraints in the original optimal control problem are equivalent to a nonlinear programing problem (i. e., a quadratic programming problem) where the two unknowns c1 and c2 are found by
where 
is a positive definite matrix, and h and g are the two linear functions of c1 and c2 from part (a). The expressions of Q, h and g should all be well defined from the results in (a) and (b).
(d) Solve the problem in part (c), by hand, to obtain the final solution for the original optimal control problem x1(t), x2(t) and u(t). Show the details of the work.