Adaptive and Learning Predictive Control


Our research lies at the intersection of control and machine learning. We focus of designing provably safe control algorithms that learn from data.

The Learning Model Predictive Control (LMPC) framework combines model-based control strategy and machine learning technique to provide a simple and systematic strategy to improve the control design using data. The controllers stores the trajectories of the system every time that a task is performed and uses these information to construct safety sets which are used to guarantee safety. Furthermore, the trajectories from previous task executions are used to evaluate a performance index which allows the controller to learn in which region of the state space the system should be operated. Finally, the data from the previous trajectories are used to learn the system model and the disturbance acting on it. For more details on the controller design we refer [1], [2], [3] and [4].

The LMPC framework accounts for disturbances acting on the system. The control designer can specify if the contestant must be satisfied robustly or stochastically (the closed-loop trajectory may violate the constraints with some probability). In the below left figure, the goal of the controller is to drive the system into the red region. The controller performs different trials (in blue), until it converges to a close-loop trajectory (in red). We noticed that the controller explores the state space before converging to a steady state trajectory which improves the system performance with respect to the previous trials. This example which that the controller improves the closed-loop performances and satisfies the state change constraint [3].

We tested the LMPC framework on our Berkeley Autonomous Race Car (BARC) platform. The goal of the controller is improve the lap time while safely driving the vehicle on the race track. In the above right figure, we see that the controller improves the lap time learning from experience. In particular, the controller understands the benefit of cutting the curves, moreover it accelerates the vehicle on the straight segments and it brakes just before entering the curves. For more details [5] and [6].


In the Adaptive Model Predictive Control (AMPC) framework we primarily focus on learning and improving the uncertain model of a dynamical sytem to improve controller performance. We systematically use input-output data from the system to synthesize maximum bounds on the uncertainties present in the model, which we adapt as we gather more and more data with time. The framework heavily relies on traditional model based approach for system identification and control synthesis, and hence, we can guarantee recursive satisfaction of imposed constraints on the system with a predictive controller in closed loop. We have developed theory for both robust and stochastic constraints with guarantees of recursive constraint satisfaction with the AMPC framework. Systems representable in both Finite Impulse Response (FIR) and State-Space  
models have been handled. For further details regarding the same, references [8,9] are useful. 

The figure on the left shows the basic architecture of the AMPC. The model used for the MPC controller is updated as shown from the data available over time. 

Moreover, the figure at the top depicts the simplest case of recursively estimating an unknown parameter which has a dimension 2. The parameter is kept constant with time. The estimated largest possible domain of the parameter is named Feasible Parameter Set, which is shrunk with available data (from light grey to dark red), without losing the guarantees that the actual unknown parameter (yellow) is contained within such an estimate. Hence, AMPC yields improved control performance, without compromising theoretical guarantees of constraint satisfaction. 


Our research has also focused on merging the above concepts of Learning MPC (LMPC) and Adaptive MPC (AMPC) for synthesizing an Adaptive Learning MPC (ALMPC) algorithm for robust constraint satisfaction under bounded uncertainties. The LMPC framework ensures iterative improvement of performance, while the AMPC framework simultaneously learns and improves the knowledge of uncertainties present in the system. The presence of AMPC in the loop enables the LMPC to further boost its exploration of state-space for even better trajectories with lower costs, as the available knowledge about the uncertainties is utilized for controller design. Details and numerical examples of this algorithm can be found in [9].

In the above two figures the resulting effect of adaptation of a Feasible Parameter Set is shown for a 2-D parameter case. AMPC framework ensures that LMPC is enabled to explore better trajectories with even lower iteration costs due to enhanced knowledge of the uncertainties. Thus, the explorative nature of the combined algorithm is improved.

Reference and related work:


   [1] Ugo Rosolia, Xiaojing Zhang, and Francesco Borrelli. "Data-Driven Predictive Control for Autonomous Systems." to appear on Annual Reviews on Control, Robotics, and Autonomous Systems,            2018.

    [2] Ugo Rosolia and Francesco Borrelli. "Learning Model Predictive Control for Iterative Tasks. A Data-Driven Control Framework." IEEE Transactions on Automatic Control (2017).

    [3] Ugo Rosolia, Xiaojing Zhang, and Francesco Borrelli. "Robust learning model predictive control for iterative tasks: Learning from experience." , 2017 IEEE 56th Annual Conference on Decision             and Control (CDC). IEEE, 2017.

    [4] Ugo Rosolia, Xiaojing Zhang, and Francesco Borrelli. "A Stochastic MPC Approach with Application to Iterative Learning Control." submitted.

    [5] Ugo Rosolia and Francesco Borrelli. "Learning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System." IFAC-PapersOnLine50.1 (2017): 3142-3147.

    [6] Ugo Rosolia, Ashwin Carvalho, and Francesco Borrelli. "Autonomous racing using learning model predictive control." American Control Conference (ACC), 2017. IEEE, 2017.

    [7] Maximilian Brunner, Ugo Rosolia, Jon Gonzales and Francesco. Borrelli. "Repetitive learning model predictive control: An autonomous racing example." Decision and Control (CDC), 2017 IEEE             56th Annual Conference on. IEEE, 2017.
    [8] Monimoy BujarbaruahXiaojing Zhangand Francesco Borrelli, ''Adaptive MPC with Chance Constraints for FIR Systems,'' Accepted for IEEE-American Control Conference, WI, June, 2018. [pdf]                      
    [9] Monimoy Bujarbaruah, Xiaojing Zhang, Ugo Rosolia, and Francesco Borrelli, ''Adaptive MPC for Iterative Tasks'', Submitted to IEEE-CDC, 2018. [pdf]