This paper develops a multisensor data fusion-based deep learning algorithm to locate and classify faults in a leader-following multiagent system. First, sequences of one-dimensional data collected from multiple sensors of followers are fused into a two-dimensional image. Then, the image is employed to train a convolution neural network with a batch normalisation layer. The trained network can locate and classify three typical fault types: the actuator limitation fault, the sensor failure and the communication failure. Moreover, faults can exist in both leaders and followers, and the faults in leaders can be identified through data from followers, indicating that the developed deep learning fault diagnosis is distributed. The effectiveness of the deep learning-based fault diagnosis algorithm is demonstrated via Quanser Servo 2 rotating inverted pendulums with a leader-follower protocol. From the experimental results, the fault classification accuracy can reach 98.9%.

Obtaining dynamics models is essential for robotics to achieve accurate model-based controllers and simulators for planning. The dynamics models are typically obtained using model specification of the manufacturer or simple numerical methods such as linear regression. However, this approach does not guarantee physically plausible parameters and can only be applied to kinematic chains consisting of rigid bodies. In this article, we describe a differentiable simulator that can be used to identify the system parameters of real-world mechanical systems with complex friction models, holonomic as well as non-holonomic constraints. To guarantee physically consistent parameters, we utilize virtual parameters and gradient-based optimization. The described Differentiable Newton-Euler Algorithm (DiffNEA) can be applied to a class of dynamical systems and guarantees physically plausible predictions. The extensive experimental evaluation shows, that the proposed model learning approach learns accurate dynamics models of systems with complex friction and non-holonomic constraints. Especially in the offline reinforcement learning experiments, the identified DiffNEA models excel. For the challenging ball in a cup task, these models solve the task using model-based offline reinforcement learning on the physical system. The black-box baselines fail on this task in simulation and on the physical system despite using more data for learning the model.

The unprecedented and exponentially growing global senior population is creating an exorbitant and unmet demand for physical rehabilitation. Telerehabilitation with robotic exoskeletons is an emerging, and compelling complementary rehabilitation modality. Some challenges are to overcome the effects of dynamic modeling uncertainties and ensure good tracking performance, stability, safe and compliant motion, and a high degree of telepresence between the two remotely-separated human-robot systems in the presence of nonlinearities, human torques, and communication constraints such as time delays. Two control methods were developed: Adaptive Robust Integral Impedance model (ARII) control and Adaptive Robust Integral Radial Basis Function Neural Networks-based Impedance model (RBFNN-I) control. Both methods implement compliant behaviour using an adjustable impedance model and revealed desirable performance. A novel human torque regulator (HTR) was developed, which provides higher fidelity telepresence for the therapist compared to existing methods to enhance the safety and perception of the closed-loop physical interaction. Unilateral and bilateral simulations were carried out using two-degrees-of-freedom (2-DOF) exoskeletons models and experiments were performed using single-joint robots. Excellent tracking performance, telepresence, and stability was achieved in the presence of large, variable and asymmetric time delays and human torques under numerous parameters variations.

The rotary inverted pendulum system is an inherently unstable system with highly nonlinear dynamics. It is used for design, testing, evaluating and comparing of different classical and contemporary control techniques. The goal of this project is to design an ADP based LQR controller for the rotary inverted pendulum system. Here model-based policy iteration algorithm is used to design the ADP based LQR controller. The swing-up and balance control is also implemented for the rotary inverted pendulum system using ADP based LQR controller gain. The response of the rotary inverted pendulum system with conventional LQR controller, ADP based LQR controller, swing-up and balance control is illustrated using MATLAB–SIMULINK platform. The result obtained after comparing the ADP based LQR controller response with conventional LQR controller, the rotary inverted pendulum system is stabilized faster with ADP based LQR controller and the swing-up and balance control response of the rotary inverted pendulum system has also improved due to ADP based LQR controller gain.

Sample efficiency is one of the key factors when applying policy search to real-world problems.
In recent years, Bayesian Optimization (BO) has become prominent in the field of robotics due
to its sample efficiency and little prior knowledge needed. However, one drawback of BO is its
poor performance on high-dimensional search spaces as it focuses on global search. In the policy
search setting, local optimization is typically sufficient as initial policies are often available, e.g.,
via meta-learning, kinesthetic demonstrations or sim-to-real approaches. In this paper, we propose
to constrain the policy search space to a sublevel-set of the Bayesian surrogate model’s predictive
uncertainty. This simple yet effective way of constraining the policy update enables BO to scale to
high-dimensional spaces (>100) as well as reduces the risk of damaging the system. We demonstrate
the effectiveness of our approach on a wide range of problems, including a motor skills task, adapting
deep RL agents to new reward signals and a sim-to-real task for an inverted pendulum system.
Keywords: Local Bayesian Optimization, Policy Search, Robot Learning

Deep learning has been widely used within learning algorithms for robotics. One disadvantage of deep networks is that these networks are black-box representations. Therefore, the learned approximations ignore the existing knowledge of physics or robotics. Especially for learning dynamics models, these black-box models are not desirable as the underlying principles are well understood and the standard deep networks can learn dynamics that violate these principles. To learn dynamics models with deep networks that guarantee physically plausible dynamics, we introduce physics-inspired deep networks that combine first principles from physics with deep learning. We incorporate Lagrangian mechanics within the model learning such that all approximated models adhere to the laws of physics and conserve energy. Deep Lagrangian Networks (DeLaN) parametrize the system energy using two networks. The parameters are obtained by minimizing the squared residual of the Euler-Lagrange differential equation. Therefore, the resulting model does not require specific knowledge of the individual system, is interpretable, and can be used as a forward, inverse, and energy model. Previously these properties were only obtained when using system identification techniques that require knowledge of the kinematic structure. We apply DeLaN to learning dynamics models and apply these models to control simulated and physical rigid body systems. The results show that the proposed approach obtains dynamics models that can be applied to physical systems for real-time control. Compared to standard deep networks, the physics-inspired models learn better models and capture the underlying structure of the dynamics.

In this paper, we propose a continuous finite-time-convergent active disturbance rejection control design scheme for a class of second order uncertain nonlinear systems. A continuous finite-time-convergent extended state observer is first developed to estimate the total disturbance which includes the nonlinear time-varying uncertain dynamics and the external disturbances. A continuous finite-time active disturbance rejection control design scheme is proposed by using saturation method. The key idea is using the estimate of the total disturbance to compensate the uncertainty such that the output tracks a given setpoint in finite time and stays at the setpoint afterwards, and all states of the closed-loop system remain uniformly bounded. By this dynamic compensation and saturation strategy, the finite-time stability of the error system is established under some weak restrictions on nonlinearities of the system. Experiments on a DC motor platform are carried out to illustrate the effectiveness of the proposed method.

Solving the Hamilton-Jacobi-Bellman equation is important in many domains including control, robotics and economics. Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs equation, is important as it yields the optimal policy that achieves the maximum reward on a give task. In the case of the Hamilton-Jacobi-Isaacs equation, which includes an adversary controlling the environment and minimizing the reward, the obtained policy is also robust to perturbations of the dynamics. In this paper we propose continuous fitted value iteration (cFVI) and robust fitted value iteration (rFVI). These algorithms leverage the non-linear control-affine dynamics and separable state and action reward of many continuous control problems to derive the optimal policy and optimal adversary in closed form. This analytic expression simplifies the differential equations and enables us to solve for the optimal value function using value iteration for continuous actions and states as well as the adversarial case. Notably, the resulting algorithms do not require discretization of states or actions. We apply the resulting algorithms to the Furuta pendulum and cartpole. We show that both algorithms obtain the optimal policy. The robustness Sim2Real experiments on the physical systems show that the policies successfully achieve the task in the real-world. When changing the masses of the pendulum, we observe that robust value iteration is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm.

This paper considers the problem of determining an optimal control action based on observed data. We formulate the problem assuming that the system can be modelled by a nonlinear state-space model, but where the model parameters, state and future disturbances are not known and are treated as random variables. Central to our formulation is that the joint distribution of these unknown objects is conditioned on the observed data. Crucially, as new measurements become available, this joint distribution continues to evolve so that control decisions are made accounting for uncertainty as evidenced in the data. The resulting problem is intractable which we obviate by providing approximations that result in finite dimensional deterministic optimisation problems. The proposed approach is demonstrated in simulation on a nonlinear system.

When learning policies for robot control, the required real-world data is typically prohibitively expensive to acquire, so learning in simulation is a popular strategy. Unfortunately, such polices are often not transferable to the real world due to a mismatch between the simulation and reality, called ‘reality gap’. Domain randomization methods tackle this problem by randomizing the physics simulator (source domain) during training according to a distribution over domain parameters in order to obtain more robust policies that are able to overcome the reality gap. Most domain randomization approaches sample the domain parameters from a fixed distribution. This solution is suboptimal in the context of sim-to-real transferability, since it yields policies that have been trained without explicitly optimizing for the reward on the real system (target domain). Additionally, a fixed distribution assumes there is prior knowledge about the uncertainty over the domain parameters. In this letter, we propose Bayesian Domain Randomization (BayRn), a black-box sim-to-real algorithm that solves tasks efficiently by adapting the domain parameter distribution during learning given sparse data from the real-world target domain. Bayesian Domain Randomization (BayRn) uses Bayesian optimization to search the space of source domain distribution parameters such that this leads to a policy which maximizes the real-word objective, allowing for adaptive distributions during policy optimization. We experimentally validate the proposed approach in sim-to-sim as well as in sim-to-real experiments, comparing against three baseline methods on two robotic tasks. Our results show that BayRn is able to perform sim-to-real transfer, while significantly reducing the required prior knowledge.

Artificial intelligence (AI) has an increasing influence in the manufacturing and processing industries. An increasingly interesting area of application is the design of controllers for technical systems through reinforcement learning. This allows the design of a controller with reduced effort for human experts, which is normally involved in the design process, for example by eliminating the determination of suitable controller parameters. In this paper a neural controller for a rotational inverted pendulum is developed by using artificial neural networks in combination with reinforcement learning. The rotational inverted pendulum is a classic example of a nonlinear and unstable system and has been little covered in previous publications on this subject. Furthermore, the behaviour of the neural controller is compared with a conventional controller when swinging up and balancing a rotational inverted pendulum.

Formulating a fractional-order controller and then implementing it in real-world environment is a challenging task for control professionals. Therefore, this paper proposes a simple, analytical and robust fractional-order controller synthesis scheme using Bode ideal transfer function in internal model control (IMC) paradigm. The controller acquires a PI form followed by a fractional-order integrator. The main advantage of the proposed scheme is that only two tuning parameters are required for satisfying the desired gain crossover frequency and phase margin. The properties of the proposed scheme are examined on the basis of tracking and disturbance rejection attributes. The effectiveness of the proposed method is verified through simulations of linear and nonlinear systems, and it is further experimentally validated on QUBE-Servo 2 set-up to control the velocity of DC motor. The proposed scheme exhibits satisfactory tracking and disturbance rejection performance when compared with integer-order and fractional-order PI controllers.

At our University, the Dynamic Systems classes utilize a combination of the Quanser® Qube servo 2 hardware, the National Instruments® myRIO units, and Mathworks® MATLAB/Simulink software in its lab portions. The combination of the Qube, the myRIO, and MATLAB has been especially useful in systems control classes. But this usefulness has been limited to the Qubes two attachments: the inertial disk and the inverted pendulum. The purpose of this project was to extend the usefulness of the Quanser Qube servo 2. This is achieved by the development of new attachments for it that use its existing equipment in various and new ways with some new programming in MATLAB. The first set of new attachments are different sized inertial disks with different inertias. The next set uses two Qubes to setup a drive gear and belt drive between the two using different sized gears. Another new attachment is a four-bar linkage for use on a single Qube, and the last one being a customizable wind turbine attachment.

This paper will first present the background of the implementation of the Qube and the myRIO in the course and why this project was started. Next it will introduce each of the custom-made attachments or sets of attachments in their own section. Each section will describe the attachment and what can be done with them and how they can be implemented into the course.

This paper is dedicated to the experimental analysis of discrete-time differentiators implemented in closed-loop control systems. To this end, two laboratory setups, namely an electro-pneumatic system and a rotary inverted pendulum have been used to implement 25 different differentiators. Since the
selected laboratory setups behave differently in the case of dynamic response and noise characteristics, it is expected that the results remain valid for a wide range of control applications. The validity of several theoretical results, which have been already reported in the literature using mathematical analysis and
numerical simulations, has been investigated, and several comments are provided to allow one to select an appropriate differentiation scheme in practical closed-loop control systems.

The main goal of the paper is to test the Embedded Model Control (EMC) design and implementation on a typical underactuated apparatus, like the Furuta pendulum, by comparing experimental results with a Linear Quadratic Regulator (LQR). EMC can be considered as a disturbance rejection control strategy, since the state predictor is extended to explicitly include disturbance dynamics, in charge of predicting the uncertainty to be rejected by control law. Essential in EMC design is the separation between controllable and not controllable dynamics, a task which allows us to find the controllable channel of underactuated systems from the low-dimensional command to the whole system degrees of freedom (DF). Pursuing this objective, a rather generic method is shown, which is applicable to other underactuated systems. The result is a very simple controllable dynamics from the single pendulum command to pendulum DF arranged in a single series of controllable integrators. The neglected feedback channels, including the unstable gravity feedback, are treated as unknown thus posing a challenge to disturbance prediction and closed loop stability. Typical in EMC, closed loop eigenvalues are chosen to guarantee stability, a pre-requisite to performance. Experimental results point out effectiveness and advantage, with respect to LQR, of design and implementation under adverse conditions, due to a disturbance pulse, in which command saturates.

As deep learning becomes more prevalent for prediction and control of real physical systems, it is important that these overparameterized models are consistent with physically plausible dynamics. This elicits a problem with how much inductive bias to impose on the model through known physical parameters and principles to reduce complexity of the learning problem to give us more reliable predictions. Recent work employs discrete variational integrators parameterized as a neural network architecture to learn conservative Lagrangian systems. The learned model captures and enforces global energy preserving properties of the system from very few trajectories. However, most real systems are inherently non-conservative and, in practice, we would also like to apply actuation. In this paper we extend this paradigm to account for general forcing (e.g. control input and damping) via discrete d'Alembert's principle which may ultimately be used for control applications. We show that this forced variational integrator networks (FVIN) architecture allows us to accurately account for energy dissipation and external forcing while still capturing the true underlying energy-based passive dynamics. We show that in application this can result in highly-data efficient model-based control and can predict on real non-conservative systems.

The objective of this work is to design a fuzzy rule-based set point weighting mechanism for fuzzy PID (FPID) controller so that an overall improved closed-loop performance may be achieved for linear as well as nonlinear process models. Till date, tuning criteria for FPID controllers are not well defined. Trial-and-error approach is primarily adopted and it is quite time-consuming and does not always ensure improved overall closed-loop behaviour. Hence, to ascertain satisfactory closed-loop performance with an initially tuned fuzzy controller, a fuzzy rule-based set point weighting mechanism is reported here. The proposed scheme is capable of providing performance enhancement with instantaneous weighting factor calculated online for each instant based on the latest process operating conditions. The proposed methodology is capable of ascertaining acceptable performances during set point tracking as well as load recovery phases. Efficacy of the proposed scheme is verified for linear as well as nonlinear process models through simulation study along with real-time verification on servo position control in comparison with the others’ reported performance augmentation schemes as well as fuzzy sliding mode control.

A robust tracking controller design was developed for a rotary motion control system. The friction force versus the angular velocity was measured and modeled as a combination of linear and nonlinear components. By adding a model-based friction compensator to a nominal proportional–integral–derivative controller, it was possible to build a simulated control system model that agreed well with the experimental results. A zero-phase error tracking controller was selected as the feedforward tracking controller and implemented based on the estimated closed-loop transfer function. To provide robustness against external disturbances and modeling uncertainties, a disturbance observer was added in the position feedback loop. The performance improvement of the overall tracking controller structure was verified through simulations and experiments.

Proximal and trust-region policy optimization methods (PPO and TRPO) belong to the standard reinforcement learning toolbox. Notably, PPO can be viewed as transforming the constrained TRPO problem into an unconstrained one, either via turning the constraint into a penalty or via objective clipping. In this chapter, an alternative problem reformulation is studied, where the information loss is bounded using a novel transformation of the KullbackLeibler (KL) divergence constraint. In contrast to PPO, the considered method does not require tuning of the regularization parameter, which is known to be hard due to its sensitivity to the reward scaling. The resulting algorithm, termed information-loss-bounded policy optimization (ILBPO), both enjoys the benefits of the first-order methods, being straightforward to implement using automatic differentiation, and maintains the advantages of the quasi-second order methods. It performs competitively in simulated OpenAI MuJoCo environments and achieves robust performance on a real robotic task of the Furuta pendulum swing-up and stabilization.

This paper deals with the practical implementation of implicit homogeneous controllers (IHCs) for linearized mechanical systems. The control design includes the methodology to get gains of the IHC based on the linearized approximation of the system. If the approximation error enforced by the linearization is vanishing with the state, locally measurable and bounded, the IHC can lead the state to the origin in finite-time. This IHC control allows accelerating the convergence rate of the states. A semi-explicit algorithm is provided to exert the digital implementation of the controller. The application of the bisection method estimates the controller gain ensuring the finite-time convergence of the state to the origin. A complementary analysis provides a simplified algorithm with a reduced number of computation stages but equally efficient gain estimation. The proposed IHC is applied to a rotary inverted pendulum QUBE Servo 2 platform of Quanser. The obtained results for the state convergence are compared with other classical feedback controllers to validate the effectiveness of the proposed scheme. The comparative analysis of the state convergence provides evidence of the faster convergence for the state trajectory to a zone centered on the origin with smaller hypervolume than the one gotten with the classical controllers.

This paper presents the implementation of IoT to the Quanser QUBE making them as twins to mimic. The IoT is done through various networking protocols. The communication between the master and slave is achieved using network-published shared variables. The shared variable is a streamlined programming interface for sharing information that was developed in LabVIEW. Utilizing the NI-PSP, you can undoubtedly pass information inside a device and between devices. The mimicking action of the Qube is achieved without any delay using NI-PSP. This is designed by integrating the Quanser QUBE with NI Elvis III and NI myRIO.

Deep Reinforcement Learning (DRL) has seen an uptick in publications due to its impressive performance in a variety of tasks. However, this comes at a cost since using a deep neural network policy requires a huge amount of data to learn from. Acquiring this data on a physical device is time and resource expensive. Thus, DRL often relies on simulations, since they provide vast amount of diverse training data faster than real time. A major problem in this research area is the reality gap, which describes the differences between the
simulated and the real world, making the policy transfer from the virtual environment to a real robot brittle and difficult. In this paper we propose a novel application of curriculum learning to the area of domain randomization called Self-Paced Domain Randomization (SPDR), which puts the Reinforcement Learning(RL) policy “in the loop”. By letting the policy influence the automatic generation of the curriculum of domain parameters based on its current performance we can show that this leads to performance increases and more stable policies when using DRL methods, both in the simulated environments and when applied to real-world platforms.

The goal of this paper is to investigate if it is possible to upgrade a given linear controller to a sliding mode one with an improvement of the control performance. Starting from a given linear controller, a design procedure for a sliding mode control having better performance than the linear one, is proposed. If the system has disturbances, which is always the case in experiments, the upgraded sliding mode control brings also a better robustness with respect to the given linear robust controller. The main idea is to divide the state-space into two areas, introducing a design parameter which separates the area of the linear control from the area of the sliding mode control. Some issues related to the chattering reduction are discussed. The control scheme’s efficiency is demonstrated experimentally on a rotary inverted pendulum. The experimental results demonstrate the effectiveness of the obtained controls, and show an improvement with respect to the given linear proportional control.

@conference{srinivasa_2020,
title = {A Hardware Proof of Concept of Networked Adaptive Systems},
author = {Srinivasa, S.B.; Makam, R.; George, K.},
booktitle = {2020 6th International Conference on Control, Automation and Robotics (ICCAR)},
year = {2020},
institution = {PES University, India},
abstract = {A hardware proof of concept of adaptation over the internet to achieve stability and output tracking despite the presence of both packet dropouts and delays induced by the network is provided in this paper. Assuming that the parameters of the models of the physical systems are unknown, a modified model reference adaptive control law for such networked adaptive systems ensures that the outputs of these systems asymptotically track specified desired trajectories. Further, we demonstrate that the multiple models, switching, and tuning methodology improves the transient performance. The example systems considered here are the Linear Servo Base Unit and the Qube Servo. },
keywords = {adaptive control, discrete-time systems, communication network, linear system},
language = {English},
publisher = {IEEE},
isbn = {978-1-7281-6140-2}
}

A hardware proof of concept of adaptation over the internet to achieve stability and output tracking despite the presence of both packet dropouts and delays induced by the network is provided in this paper. Assuming that the parameters of the models of the physical systems are unknown, a modified model reference adaptive control law for such networked adaptive systems ensures that the outputs of these systems asymptotically track specified desired trajectories. Further, we demonstrate that the multiple models, switching, and tuning methodology improves the transient performance. The example systems considered here are the Linear Servo Base Unit and the Qube Servo.

It is observed that classical controller design techniques, i.e., particularly fixed type of PI/PD/PID analytical control approach, fails for the parametric uncertainty issues. However, the graphical approach for PID tuning and advanced control techniques such as H infinity, Quantitative feedback technique (QFT) have already been developed for handling parametric uncertainty, but these approaches are complex. Because of this, instead of using advanced controller techniques and graphical PID tuning approach, a variant of PID, i.e., a fixed type proportional derivative (PD) controller design is proposed for a single input single output plant (SISO) interval system with no zeros using Krishnamurthy's approach based on Routh criterion, Kharitonov's theorem and model order reduction approach based on Routh criterion. The beauty of the proposed approach is that the direct formulae is proposed for the tuning of the PD controller for an interval system. The proposed controller design gives the necessary and sufficient condition of stability and also the desired performance. The proposed approach is validated through simulation, and further experimental validation is carried out on the Servo system.

@article{muratore_2020,
title = {Bayesian Domain Randomization for Sim-to-Real Transfer},
author = {Muratore, F.; Eilers, C.; Gienger, M.; Peters, J.},
journal = {arXiv},
year = {2020},
institution = {Technical University Darmstadt, Germany; Honda Research Institute Europe, Germany},
abstract = {When learning policies for robot control, the real-world data required is typically prohibitively expensive to acquire, so learning in simulation is a popular strategy. Unfortunately, such polices are often not transferable to the real world due to a mismatch between the simulation and reality, called 'reality gap'. Domain randomization methods tackle this problem by randomizing the physics simulator (source domain) according to a distribution over domain parameters during training in order to obtain more robust policies that are able to overcome the reality gap. Most domain randomization approaches sample the domain parameters from a fixed distribution. This solution is suboptimal in the context of sim-to-real transferability, since it yields policies that have been trained without explicitly optimizing for the reward on the real system (target domain). Additionally, a fixed distribution assumes there is prior knowledge about the uncertainty over the domain parameters. Thus, we propose Bayesian Domain Randomization (BayRn), a black box sim-to-real algorithm that solves tasks efficiently by adapting the domain parameter distribution during learning by sampling the real-world target domain. BayRn utilizes Bayesian optimization to search the space of source domain distribution parameters which produce a policy that maximizes the real-word objective, allowing for adaptive distributions during policy optimization. We experimentally validate the proposed approach by comparing against two baseline methods on a nonlinear under-actuated swing-up task. Our results show that BayRn is capable to perform direct sim-to-real transfer, while significantly reducing the required prior knowledge. },
keywords = {machine learning},
language = {English}
}

When learning policies for robot control, the real-world data required is typically prohibitively expensive to acquire, so learning in simulation is a popular strategy. Unfortunately, such polices are often not transferable to the real world due to a mismatch between the simulation and reality, called 'reality gap'. Domain randomization methods tackle this problem by randomizing the physics simulator (source domain) according to a distribution over domain parameters during training in order to obtain more robust policies that are able to overcome the reality gap. Most domain randomization approaches sample the domain parameters from a fixed distribution. This solution is suboptimal in the context of sim-to-real transferability, since it yields policies that have been trained without explicitly optimizing for the reward on the real system (target domain). Additionally, a fixed distribution assumes there is prior knowledge about the uncertainty over the domain parameters. Thus, we propose Bayesian Domain Randomization (BayRn), a black box sim-to-real algorithm that solves tasks efficiently by adapting the domain parameter distribution during learning by sampling the real-world target domain. BayRn utilizes Bayesian optimization to search the space of source domain distribution parameters which produce a policy that maximizes the real-word objective, allowing for adaptive distributions during policy optimization. We experimentally validate the proposed approach by comparing against two baseline methods on a nonlinear under-actuated swing-up task. Our results show that BayRn is capable to perform direct sim-to-real transfer, while significantly reducing the required prior knowledge.

@article{frohlich_2020,
title = {Bayesian Optimization for Policy Search in High-Dimensional Systems via Automatic Domain Selection},
author = {Fröhlich, L.P.; Klenske, E.D.; Daniel, C.G.; Zeilinger, M.N.},
journal = {arXiv},
year = {2020},
institution = {Bosch Center for Artificial Intelligence, Germany; ETH Zurich, Switzerland},
abstract = {Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (>10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combination with a model-based controller to simplify the BO problem by focusing onto the most relevant regions of the optimization domain. 2) Based on (1) we present a method to find an embedding in parameter space that reduces the effective dimensionality of the optimization problem. To evaluate the effectiveness of the proposed approach, we present an experimental evaluation on real hardware, as well as simulated tasks including a 48-dimensional policy for a quadcopter. },
language = {English}
}

Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (>10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combination with a model-based controller to simplify the BO problem by focusing onto the most relevant regions of the optimization domain. 2) Based on (1) we present a method to find an embedding in parameter space that reduces the effective dimensionality of the optimization problem. To evaluate the effectiveness of the proposed approach, we present an experimental evaluation on real hardware, as well as simulated tasks including a 48-dimensional policy for a quadcopter.

@article{de-maity2_2020,
title = {Comparative Performance Study of Optimal Interval Type-2 Fuzzy PID Controllers with Practical System},
author = {Rani De (Maity), R.; Mudi, R.K.; Dey, C.},
journal = {International Journal of Computer Sciences and Engineering},
year = {2020},
month = {03},
volume = {8},
number = {3},
institution = {Jadavpur University, India; University of Calcutta, India},
abstract = {In this paper, the input and output scaling factors of the type-2 fuzzy PID Controller (IT2-FPID) are determined using three different optimization algorithms (Cuckoo search (CS), Particle swarm optimization (PSO), and Bee colony algorithm (BCA)) for a first-order integrating plus dead time (FOIPD) model. A comparative performance study is made for these three optimization algorithms in terms of various transient performance indices. The comparative analysis on the experimental results reveals that BCA based optimal IT2-FPID shows better performance on a simulation model whereas CS based optimal IT2-FPID is found to be superior for practical system over other algorithms. },
keywords = {Particle swarm optimization(PSO), Cuckoo search algorithm (CS), Bee colony algorithm(BCA), Interval type-2 fuzzy controller},
language = {English}
}

In this paper, the input and output scaling factors of the type-2 fuzzy PID Controller (IT2-FPID) are determined using three different optimization algorithms (Cuckoo search (CS), Particle swarm optimization (PSO), and Bee colony algorithm (BCA)) for a first-order integrating plus dead time (FOIPD) model. A comparative performance study is made for these three optimization algorithms in terms of various transient performance indices. The comparative analysis on the experimental results reveals that BCA based optimal IT2-FPID shows better performance on a simulation model whereas CS based optimal IT2-FPID is found to be superior for practical system over other algorithms.

In this brief, we study the cooperative output regulation of linear multiagent systems under switching communication topology and exosystem detectability constraints, and applications to the synchronization of networked motors. Compared to similar works in the literature, in this brief, we consider the problem scenario in which none of the agents can estimate the exosystem states from their individual measurements on any of the switching configurations of the system. In other words, no agent in the system can solve the output regulation problem independently. By devising a distributed observer on the exosystem states, and under intuitive assumptions on the combined detectability of the multiagent system and joint connectivity of the information graphs, we are able to solve the problem by a distributed feedback protocol. Finally, the developed theoretical results are applied to the synchronization problem of a group of motors in a time-varying communication network.

This paper proposes a design of a robust controller for a rotary motion control system that includes a PID controller, a disturbance observer, and a friction compensator. Friction force versus angular velocity has been measured, and viscous, Coulomb friction and stiction components have been identified. With nominal PID (proportional- integral-derivative) controller, we have observed adverse effects due to friction such as excessive steady-state errors, oscillations, and limit-cycles. By adding a friction model as an augmented nonlinear dynamics of a plant, we are able to conduct a simulation study of a motion control system that matches very well with experimental results. The disturbance observer (DOB) based on simple and effective robust control theory has been implemented to make the rotary motion control system “robust” against inertia/load variations, external torque disturbances, and some of friction forces. Further performance enhancement of the DOB-based robust motion control system has been achieved by adding the friction compensator and experimentally verified.

Recently, reinforcement learning has been applied to various fields and shows better performance than humans. In particular, it is attracting attention in the fields of smart factories and robotics that require automatic control without human intervention. In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own IoT devices of the same type. There is no guarantee that the reinforcement learning agent that has learned the optimal control policy using one IoT device will perform optimal control of other IoT devices. Therefore, since reinforcement learning must be performed individually for each IoT device, it takes a lot of time and cost. To solve this problem, we propose a new method of federated reinforcement learning. In the proposed federated reinforcement learning, multiple agents have independent IoT devices, perform learning at the same time, and federate with each other to improve learning performance. Therefore, we apply a new gradient sharing method and transfer learning to reinforcement learning. In addition, Actor-Critic PPO, which shows good performance in reinforcement learning algorithms, is used. And, for smooth learning in the IoT environment where numerous devices exist, we propose an architecture based on Software-Defined Networking. Using multiple rotary inverted pendulum devices interconnected via a SDN, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple IoT devices.

@conference{lim2_2020,
title = {Federated Reinforcement Learning for Controlling Multiple Rotary Inverted Pendulums in Edge Computing Environments},
author = {Lim, H.-K.; Kim, J.-B.; Kim, C.-M.; Hwang, G.-Y.; Choi, H. ; Han, Y.-H.},
booktitle = {2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC)},
year = {2020},
institution = {Korea University of Technology and Education, Korea},
abstract = {Reinforcement learning has recently been studied in various fields and also used to optimally control real devices (e.g., robotic arms). In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own devices of the same type but with slightly different dynamics. For such multiple devices, there is no guarantee that an agent who interacts only with one device and learns the optimal control policy will also control another device well. Therefore, we may need to apply independent reinforcement learning to each device individually, which requires time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent device shares their learning experience with each other, and transfers a mature policy model parameters into other agents. We incorporate the Actor-Critic PPO algorithm into each agent in the proposed collaborative architecture, and propose an efficient procedure for the gradient sharing and the model transfer. We also use edge computing to solve network problems that occur when training multiple real devices at the same time. Using multiple rotary inverted pendulum devices, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple devices, and that the learning speed can be faster if more agents are involved. },
keywords = {Actor-Critic PPO, Edge computing, Federated reinforcement learning, Multiple RIP control},
language = {English},
publisher = {IEEE},
isbn = {978-1-7281-4986-8}
}

Reinforcement learning has recently been studied in various fields and also used to optimally control real devices (e.g., robotic arms). In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own devices of the same type but with slightly different dynamics. For such multiple devices, there is no guarantee that an agent who interacts only with one device and learns the optimal control policy will also control another device well. Therefore, we may need to apply independent reinforcement learning to each device individually, which requires time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent device shares their learning experience with each other, and transfers a mature policy model parameters into other agents. We incorporate the Actor-Critic PPO algorithm into each agent in the proposed collaborative architecture, and propose an efficient procedure for the gradient sharing and the model transfer. We also use edge computing to solve network problems that occur when training multiple real devices at the same time. Using multiple rotary inverted pendulum devices, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple devices, and that the learning speed can be faster if more agents are involved.

@article{lim_20220,
title = {Federated Reinforcement Learning for Training Control Policies on Multiple IoT Devices},
author = {Lim, H.-K.; Kim, J.-B.; Heo, J.-S.; Han, Y.-H. },
journal = {Sensors},
year = {2020},
volume = {20},
number = {5},
institution = {Korea University of Technology and Education, Korea},
abstract = {Reinforcement learning has recently been studied in various fields and also used to optimally control IoT devices supporting the expansion of Internet connection beyond the usual standard devices. In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own IoT devices of the same type but with slightly different dynamics. For such multiple IoT devices, there is no guarantee that an agent who interacts only with one IoT device and learns the optimal control policy will also control another IoT device well. Therefore, we may need to apply independent reinforcement learning to each IoT device individually, which requires a costly or time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent IoT device shares their learning experience (i.e., the gradient of loss function) with each other, and transfers a mature policy model parameters into other agents. They accelerate its learning process by using mature parameters. We incorporate the actor–critic proximal policy optimization (Actor–Critic PPO) algorithm into each agent in the proposed collaborative architecture and propose an efficient procedure for the gradient sharing and the model transfer. Using multiple rotary inverted pendulum devices interconnected via a network switch, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple IoT devices and that the learning speed can be faster if more agents are involved. },
issn = {1424-8220},
keywords = { Actor–Critic PPO; federated reinforcement learning; multi-device control},
language = {English},
publisher = {MDPI AG}
}

Reinforcement learning has recently been studied in various fields and also used to optimally control IoT devices supporting the expansion of Internet connection beyond the usual standard devices. In this paper, we try to allow multiple reinforcement learning agents to learn optimal control policy on their own IoT devices of the same type but with slightly different dynamics. For such multiple IoT devices, there is no guarantee that an agent who interacts only with one IoT device and learns the optimal control policy will also control another IoT device well. Therefore, we may need to apply independent reinforcement learning to each IoT device individually, which requires a costly or time-consuming effort. To solve this problem, we propose a new federated reinforcement learning architecture where each agent working on its independent IoT device shares their learning experience (i.e., the gradient of loss function) with each other, and transfers a mature policy model parameters into other agents. They accelerate its learning process by using mature parameters. We incorporate the actor–critic proximal policy optimization (Actor–Critic PPO) algorithm into each agent in the proposed collaborative architecture and propose an efficient procedure for the gradient sharing and the model transfer. Using multiple rotary inverted pendulum devices interconnected via a network switch, we demonstrate that the proposed federated reinforcement learning scheme can effectively facilitate the learning process for multiple IoT devices and that the learning speed can be faster if more agents are involved.

@inbook{mani_2020,
title = {Implementation of ACO Tuned Modified PI-like Position and Speed Control of DC Motor: An Application to Electric Vehicle},
author = {Mani, G.},
booktitle = {Soft Computing for Problem Solving},
year = {2020},
institution = {Vellore Institute of Technology, India},
abstract = {The modified PI-like control scheme has been derived from classic Internal Model Control (IMC), having two tuning parameters namely controller gain Kc and reset time τI. The controller action resembles like PI control action with inherent reset windup protection and dead time compensation, except the inverse of the process model gain is set as controller gain and dominant time constant of the process model set as reset time. The controller structure is feasible enough to implement in hardware. When an accurate process model is not available, the metaheuristic algorithms like Ant Colony Optimization (ACO) is applied to find the optimal tuning parameters. The performance of proposed control scheme for both speed control and position tracking is compared and analyzed under different operating conditions with well-known Proportional-Velocity (PV) control scheme and Two-Degree-of-Freedom (2-DoF) PID structure. The experimental evaluations have been demonstrated via computer-interfaced QUBE-servo DC Motor through Hardware in Loop (HiL) implementation. },
keywords = {Modified PI-like, 2-DoF, QUBE-servo DC motor, IAE, ISE, ACO },
language = {English},
series = {Advances in Intelligent Systems and Computing},
publisher = {Springer Nature Switzerland},
isbn = {978-981-15-0035-0},
pages = {629-645}
}

The modified PI-like control scheme has been derived from classic Internal Model Control (IMC), having two tuning parameters namely controller gain Kc and reset time τI. The controller action resembles like PI control action with inherent reset windup protection and dead time compensation, except the inverse of the process model gain is set as controller gain and dominant time constant of the process model set as reset time. The controller structure is feasible enough to implement in hardware. When an accurate process model is not available, the metaheuristic algorithms like Ant Colony Optimization (ACO) is applied to find the optimal tuning parameters. The performance of proposed control scheme for both speed control and position tracking is compared and analyzed under different operating conditions with well-known Proportional-Velocity (PV) control scheme and Two-Degree-of-Freedom (2-DoF) PID structure. The experimental evaluations have been demonstrated via computer-interfaced QUBE-servo DC Motor through Hardware in Loop (HiL) implementation.

@article{tran_2019,
title = {Implementing Homomorphic Encryption Based Secure Feedback Control for Physical Systems},
author = {Tran, .; Farokhi, F.; Cantoni, M.; Shames, I.},
journal = {Control Engineering Practice},
year = {2020},
month = {04},
volume = {97},
institution = {University of Melbourne, Australia},
abstract = {This paper is about an encryption based approach to the secure implementation of feedback controllers for physical systems. Specifically, Paillier's homomorphic encryption is used in a custom digital implementation of a class of linear dynamic controllers, including static gain as a special case. The implementation is amenable to Field Programmable Gate Array (FPGA) realization. Experimental results, including timing analysis and resource usage characteristics for different encryption key lengths, are presented for the realization of a controller for an inverted pendulum; as this is an unstable plant, the control is necessarily fast. },
keywords = {secure control, homomorphic encryption, Paillier encryption, digital design, FPGA},
language = {English},
publisher = {Elsevier B.V.}
}

This paper is about an encryption based approach to the secure implementation of feedback controllers for physical systems. Specifically, Paillier's homomorphic encryption is used in a custom digital implementation of a class of linear dynamic controllers, including static gain as a special case. The implementation is amenable to Field Programmable Gate Array (FPGA) realization. Experimental results, including timing analysis and resource usage characteristics for different encryption key lengths, are presented for the realization of a controller for an inverted pendulum; as this is an unstable plant, the control is necessarily fast.

Abstract: Sensors and encoders had become an essential part of all the machines. Without these, a closedloop feedback system cannot be made. However, the system is difficult to be controlled precisely due to the external noise corruption in the signal received in sensors or encoders. A common solution to reduce the noise is employing a low pass filter. Although low pass filter can helped to reduce noise, but it has side effect on lowering the response time and therefore it may not so suitable in most of the application by only deploying low pass filter itself. A filtering module usually integrated with other modules to improve a system. One of the famous modules is Proportional-Integral-Derivative (PID) controller. The plant to be controlled in this module was a DC motor which can be regarded as an inductive load, therefore a proportional-derivative (PD) controller is desired. Therefore, the objectives of this paper are to find the suitable value of cut-off frequency for the low-pass filter and find the optimum value of kp and kd in the PD controller which satisfy the transient and stead-state performance. The response time varies with the cutoff frequency in the low-pass filter. Since the response time does not differ much in between cut-off
frequency value, therefore, the cut-off frequency value was set at 150 rad s-1 in which the noise signal is significantly eliminated. As for the PD controller, the observation result showed that when kp increased, the settling time of the system and the percentage of peak overshoot also increased. Meanwhile, the increasing of value of kd reduce the settling time and the percentage of overshoot. The optimum value for kp and kd were calculated using formula and the results are 5.3678 and 0.2094 respectively. Since the formula does not include the external disturbance into the calculation, therefore the actual response will be slightly different from the target response. Therefore, the understanding of property changes in kp and kd able to help in fine-funing for these two values. After fine-tuning, the value for kp and kd are 7.0 and 0.2094 respectively.

@article{de-maity_2020,
title = {Nature-inspired and hybrid optimization algorithms on interval Type-2 fuzzy controller for servo processes: a comparative performance study},
author = {De (Maity), R.R.; Mudi, R.K.; Dey, C.},
journal = {SN Applied Sciences},
year = {2020},
institution = {Jadavpur University, India; University of Calcutta, India},
abstract = {In this paper, performance evaluations of six well-known nature-inspired algorithms have been reported containing genetic algorithm, cuckoo search, particle swarm optimization, differential evolution, bee colony, and combined particle swarm optimization and differential evolution (CPSODE) algorithms. Based on these optimization algorithms, input and output scaling factors of an interval Type-2 fuzzy PID controller (IT2-FLC) are chosen for closed-loop servo tracking. Optimal values of the scaling factors are chosen by minimization of the objective function which is defined based on the closed-loop controller performance criteria. A detailed comparative analysis is reported based on the simulation and experimental results. Performance analysis reveals that improved performance, reliability, robustness, and lesser noise sensitivity are reported by IT2-FLC with the optimal values obtained by the hybrid algorithm CPSODE. },
keywords = {Genetic algorithm (GA), Particle swarm optimization (PSO), Differential evolution (DE), Cuckoo search (CS), Bee colony (BC), Combined particle swarm optimization and differential evolution (CPSODE) algorithms, Interval Type-2 fuzzy controller, Servo tracking process},
language = {English},
publisher = {Springer Nature Switzerland}
}

In this paper, performance evaluations of six well-known nature-inspired algorithms have been reported containing genetic algorithm, cuckoo search, particle swarm optimization, differential evolution, bee colony, and combined particle swarm optimization and differential evolution (CPSODE) algorithms. Based on these optimization algorithms, input and output scaling factors of an interval Type-2 fuzzy PID controller (IT2-FLC) are chosen for closed-loop servo tracking. Optimal values of the scaling factors are chosen by minimization of the objective function which is defined based on the closed-loop controller performance criteria. A detailed comparative analysis is reported based on the simulation and experimental results. Performance analysis reveals that improved performance, reliability, robustness, and lesser noise sensitivity are reported by IT2-FLC with the optimal values obtained by the hybrid algorithm CPSODE.

@conference{wisotzki_2020,
title = {On IMC-Based PID Tuning Using Gain-Integrator-Delay Dynamics},
author = {Wisotzki, S.; Brahma, S.; Ossareh, H.R.},
booktitle = {2020 IEEE Conference on Control Technology and Applications (CCTA)},
year = {2020},
institution = {Booz Allen Hamilton, USA; University of Vermont, USA},
abstract = {The Internal Model Control (IMC)-based PID tuning using integrator dynamics is an effective technique for tuning the boost pressure control system in a turbocharged gasoline engine, as investigated in a previous work by the authors. In that work, two IMC-based PID tuning approaches were delineated: one that involved a post hoc modification of initial design to achieve a desired closed-loop performance, and another that assigned a desired closed-loop bandwidth exactly. This paper extends the results of that work by introducing a third design approach employing the technique: one that assigns the desired phase margin exactly. Also, the viability of all three approaches for use with more general plants that are less similar to the boost system is investigated through Monte Carlo simulations. Finally, to illustrate their effectiveness in practice, all three design approaches are applied to a DC motor control problem. },
keywords = {Bandwidth, Tuning, Relays, Engines, DC motors, Gain, Monte Carlo methods},
language = {English},
publisher = {IEEE},
isbn = {978-1-7281-7141-8}
}

The Internal Model Control (IMC)-based PID tuning using integrator dynamics is an effective technique for tuning the boost pressure control system in a turbocharged gasoline engine, as investigated in a previous work by the authors. In that work, two IMC-based PID tuning approaches were delineated: one that involved a post hoc modification of initial design to achieve a desired closed-loop performance, and another that assigned a desired closed-loop bandwidth exactly. This paper extends the results of that work by introducing a third design approach employing the technique: one that assigns the desired phase margin exactly. Also, the viability of all three approaches for use with more general plants that are less similar to the boost system is investigated through Monte Carlo simulations. Finally, to illustrate their effectiveness in practice, all three design approaches are applied to a DC motor control problem.

@article{lee_2020,
title = {Performance Enhancement of Motion Control Systems Through Friction Identification and Compensation},
author = {Lee, H. S.; Jung, S.; Ryu, S.},
journal = {Journal of the Korean Society of Manufacturing Process Engineers},
year = {2020},
volume = {19},
number = {6},
institution = {Gyeongsang National University, Korea},
abstract = {This paper proposes a method for measuring friction forces and creating a friction model for a rotary motion control system as well as an autonomous vehicle testbed. The friction forces versus the velocity were measured, and the viscous friction, Coulomb friction, and stiction were identified. With a nominal PID (proportional-integral-derivative) controller, we observed the adverse effects due to friction, such as excessive steady-state errors, oscillations, and limit-cycles. By adding an adequate friction model as part of the augmented nonlinear dynamics of a plant, we were able to conduct a simulation study of a motion control system that well matched experimental results. We have observed that the implementation of a model-based friction compensator improves the overall performance of both motion control systems, i.e., the rotary motion control system and the Altino testbed for autonomous vehicle development. By utilizing a better simulation tool with an embedded friction model, we expect that the overall development time and cost can be reduced. },
issn = {1598-6721},
keywords = {Position Control; Motion Control System; Friction Modeling; Friction Compensation },
language = {English},
publisher = {Korea Institute of Science and Technology Information}
}

This paper proposes a method for measuring friction forces and creating a friction model for a rotary motion control system as well as an autonomous vehicle testbed. The friction forces versus the velocity were measured, and the viscous friction, Coulomb friction, and stiction were identified. With a nominal PID (proportional-integral-derivative) controller, we observed the adverse effects due to friction, such as excessive steady-state errors, oscillations, and limit-cycles. By adding an adequate friction model as part of the augmented nonlinear dynamics of a plant, we were able to conduct a simulation study of a motion control system that well matched experimental results. We have observed that the implementation of a model-based friction compensator improves the overall performance of both motion control systems, i.e., the rotary motion control system and the Altino testbed for autonomous vehicle development. By utilizing a better simulation tool with an embedded friction model, we expect that the overall development time and cost can be reduced.

In this paper, a self-tuning rule-based position control algorithm is proposed for DC motors with system parameter estimation using the recursive least squares method. First, a mathematical model of the angular position control of a DC motor was derived. Next, the timevarying parameters including the rotational inertia in the model were estimated using the RLS method along with multiple forgetting factors without prior knowledge of the system. Based on the derived model and the parameter estimation, a sliding mode control algorithm was designed by applying a self-tuning rule that enables the magnitude of the voltage input to be adaptively adjusted for improvement of the energy efficiency. The performance of the designed control algorithm was then experimentally evaluated under several different load conditions. Finally, the evaluation results show that the designed controller achieves a satisfactory capability for a DC motor to deal with both tracking accuracy and energy efficiency without prior knowledge of the system.

@conference{truchetta_2019,
title = {Robust Model-free Reinforcement Learning with Multi-objective Bayesian Optimization},
author = {Turchetta, M.; Krause, A.; Trimpe, S.},
booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
year = {2020},
institution = {ETH Zurich, Switzerland; Max Planck ETH Center for Learning Systems, Germany},
abstract = {In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment. In real-world applications, test conditions may differ substantially from the training scenario and, therefore, focusing on pure reward maximization during training may lead to poor results at test time. In these cases, it is important to trade-off between performance and robustness while learning a policy. While several results exist for robust, model-based RL, the model-free case has not been widely investigated. In this paper, we cast the robust, model-free RL problem as a multi-objective optimization problem. To quantify the robustness of a policy, we use delay margin and gain margin, two robustness indicators that are common in control theory. We show how these metrics can be estimated from data in the model-free setting. We use multi-objective Bayesian optimization (MOBO) to solve efficiently this expensive-to-evaluate, multi-objective optimization problem. We show the benefits of our robust formulation both in sim-to-real and pure hardware experiments to balance a Furuta pendulum. },
keywords = {robotics, artificial intelligence, machine learning},
language = {English}
}

In reinforcement learning (RL), an autonomous agent learns to perform complex tasks by maximizing an exogenous reward signal while interacting with its environment. In real-world applications, test conditions may differ substantially from the training scenario and, therefore, focusing on pure reward maximization during training may lead to poor results at test time. In these cases, it is important to trade-off between performance and robustness while learning a policy. While several results exist for robust, model-based RL, the model-free case has not been widely investigated. In this paper, we cast the robust, model-free RL problem as a multi-objective optimization problem. To quantify the robustness of a policy, we use delay margin and gain margin, two robustness indicators that are common in control theory. We show how these metrics can be estimated from data in the model-free setting. We use multi-objective Bayesian optimization (MOBO) to solve efficiently this expensive-to-evaluate, multi-objective optimization problem. We show the benefits of our robust formulation both in sim-to-real and pure hardware experiments to balance a Furuta pendulum.

@article{zhang2_2020,
title = {Robust Non-Minimal State Feedback Control for a Furuta Pendulum with Parametric Modelling Errors},
author = {Zhang, L.; Dixon, R.},
journal = { IEEE Transactions on Industrial Electronics},
year = {2020},
institution = {National University of Defense Technology, China; Birmingham University, United Kingdom},
abstract = {Non-minimal state feedback (NMSF) control, also known as proportional-integral-plus (PIP) control, together with anti-windup integrator, is applied, for the first time, to stabilize a Furuta pendulum (FP). The key elements in the non-minimal state space (NMSS) state transition matrix is obtained directly from the original minimal state space (MSS) model. The NMSS model is proved to be controllable if the controllability conditions are met. In order to demonstrate the potential robustness of NMSF control, the controller is designed based on the model with two deliberate plant parameter mismatches. The feedback gains are optimized in an eigenvalue assignment way by minimizing a robust cost function, which is related to the closed-loop state matrix's eigenvalues' sensitivities to the two mismatched plant parameters. The sum of the normalized eigenvalue sensitivities (NES) is defined as cost function. Multiple simulations with different plant parameters are implemented to evaluate the robustness. The optimized FP control system shows smaller envelope of multiple time responses compared to the one before optimization. Experimental results also confirm the optimized controller's improved robustness under parametric mismatches. },
issn = {0278-0046},
keywords = {Non-minimal state space, parametric modelling errors, robust optimization, Proportional-Integral-Plus control, Furuta pendulum},
language = {English},
publisher = {IEEE}
}

Non-minimal state feedback (NMSF) control, also known as proportional-integral-plus (PIP) control, together with anti-windup integrator, is applied, for the first time, to stabilize a Furuta pendulum (FP). The key elements in the non-minimal state space (NMSS) state transition matrix is obtained directly from the original minimal state space (MSS) model. The NMSS model is proved to be controllable if the controllability conditions are met. In order to demonstrate the potential robustness of NMSF control, the controller is designed based on the model with two deliberate plant parameter mismatches. The feedback gains are optimized in an eigenvalue assignment way by minimizing a robust cost function, which is related to the closed-loop state matrix's eigenvalues' sensitivities to the two mismatched plant parameters. The sum of the normalized eigenvalue sensitivities (NES) is defined as cost function. Multiple simulations with different plant parameters are implemented to evaluate the robustness. The optimized FP control system shows smaller envelope of multiple time responses compared to the one before optimization. Experimental results also confirm the optimized controller's improved robustness under parametric mismatches.

@article{zhang3_2020,
title = {Switching Stabilization for Nonlinear Networked Control Systems with Delays and Packet Losses},
author = {Zhang, Q.; Liu, B.},
journal = {Information Technology and Control},
year = {2020},
volume = {49},
number = {2},
institution = {Engineering Research Center for Metallurgical Automation and Measurement Technology of Ministry of Education, China; Hubei Province Key Laboratory of Systems Science in Metallurgical Process, China},
abstract = {This paper studies the stabilization problem for nonlinear NCSs(NNCSs) with bilateral network-induced random delay and packet dropout. T-S fuzzy model is employed to represent the nonlinear controlled plant. Based on the T-S model, a discrete-time fuzzy switched system model with uncertain parameters is established by means of the uncertain method and switching system method. Furthermore, the exponential stability condition for the state of the fuzzy switched system is obtained by using the combination of slow switching model-dependent average dwell time (MDADT) method and fast switching MDADT method. Finally, a series of rotary inverted pendulum experiments are provided to illustrates the effectiveness of the proposed method and prove that the proposed fuzzy controller based on T-S fuzzy model can balance the rotary inverted pendulum in a greater state range rather than the linear controller based on linearization },
keywords = {nonlinear networked control systems, T-S fuzzy model, switched system, mode-dependent average dwell time},
language = {English},
pages = {302-316}
}

This paper studies the stabilization problem for nonlinear NCSs(NNCSs) with bilateral network-induced random delay and packet dropout. T-S fuzzy model is employed to represent the nonlinear controlled plant. Based on the T-S model, a discrete-time fuzzy switched system model with uncertain parameters is established by means of the uncertain method and switching system method. Furthermore, the exponential stability condition for the state of the fuzzy switched system is obtained by using the combination of slow switching model-dependent average dwell time (MDADT) method and fast switching MDADT method. Finally, a series of rotary inverted pendulum experiments are provided to illustrates the effectiveness of the proposed method and prove that the proposed fuzzy controller based on T-S fuzzy model can balance the rotary inverted pendulum in a greater state range rather than the linear controller based on linearization

@article{leamy_2020,
title = {Two-scale command shaping for arresting motion in nonlinear systems},
author = {Leamy, M.J.; Alyukov, A.},
journal = {Nonlinear Dynamics},
year = {2020},
institution = {Georgia Institute of Technology, USA; South Ural State University, Russia},
abstract = {This paper presents a feedforward technique for arresting motion in nonlinear systems based on two-scale command shaping (TSCS). The advantages of the proposed technique arise from its feedforward nature and ease of implementation in linear and nonlinear systems. Using the TSCS strategy, the control input required to arrest motion is decomposed into two scales—the first arrests dynamics associated with the linear subproblem, while the second eliminates response from the nonlinearities. Using direct numerical integration, the method is assessed using a traditional Duffing system and multi-degree-of-freedom nonlinear systems. Experiments are conducted on a compound pendulum attached to a servomotor, documenting effective arrest of the system in close agreement with theoretical predictions. },
keywords = {Nonzero initial conditions, Motion arrest, Nonlinear systems, Perturbation methods, Input shaping, Two-scale command shaping},
language = {English},
publisher = {Springer Nature Switzerland}
}

This paper presents a feedforward technique for arresting motion in nonlinear systems based on two-scale command shaping (TSCS). The advantages of the proposed technique arise from its feedforward nature and ease of implementation in linear and nonlinear systems. Using the TSCS strategy, the control input required to arrest motion is decomposed into two scales—the first arrests dynamics associated with the linear subproblem, while the second eliminates response from the nonlinearities. Using direct numerical integration, the method is assessed using a traditional Duffing system and multi-degree-of-freedom nonlinear systems. Experiments are conducted on a compound pendulum attached to a servomotor, documenting effective arrest of the system in close agreement with theoretical predictions.

@conference{eilers_2020,
title = {Underactuated Waypoint Trajectory Optimization for Light Painting Photography},
author = {Eilers, C.; Eschmann, J.; Menzenbach, R.; Belousov, B.; Muratore, F.; Peters, J.},
booktitle = {2020 IEEE International Conference on Robotics and Automation (ICRA)},
year = {2020},
institution = {Technical University Darmstadt, Germany},
abstract = {Despite their abundance in robotics and nature, underactuated systems remain a challenge for control engineering. Trajectory optimization provides a generally applicable solution, however its efficiency strongly depends on the skill of the engineer to frame the problem in an optimizer-friendly way. This paper proposes a procedure that automates such problem reformulation for a class of tasks in which the desired trajectory is specified by a sequence of waypoints. The approach is based on introducing auxiliary optimization variables that represent waypoint activations. To validate the proposed method, a letter drawing task is set up where shapes traced by the tip of a rotary inverted pendulum are visualized using long exposure photography. },
language = {English},
publisher = {IEEE}
}

Despite their abundance in robotics and nature, underactuated systems remain a challenge for control engineering. Trajectory optimization provides a generally applicable solution, however its efficiency strongly depends on the skill of the engineer to frame the problem in an optimizer-friendly way. This paper proposes a procedure that automates such problem reformulation for a class of tasks in which the desired trajectory is specified by a sequence of waypoints. The approach is based on introducing auxiliary optimization variables that represent waypoint activations. To validate the proposed method, a letter drawing task is set up where shapes traced by the tip of a rotary inverted pendulum are visualized using long exposure photography.

This paper considers parameter estimation for nonlinear state-space models, which is an important but challenging problem. We address this challenge by employing a variational inference (VI) approach, which is a principled method that has deep connections to maximum likelihood estimation. This VI approach ultimately provides estimates of the model as solutions to an optimisation problem, which is deterministic, tractable and can be solved using standard optimisation tools. A specialisation of this approach for systems with additive Gaussian noise is also detailed. The proposed method is examined numerically on a range of simulation and real examples with a focus on robustness to parameter initialisations; we additionally perform favourable comparisons against state-of-the-art alternatives.

This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable, distribution. The approach is deterministic and results in an optimisation problem of a standard form. Due to the parametrisation of the assumed density selected first- and second-order derivatives are readily available which allows for efficient solutions. The proposed method is compared against state-of-the-art Hamiltonian Monte Carlo in two numerical examples.

@article{jain_2020,
title = {Weighted Internal Model Control-Proportional Integral Derivative Control Scheme Via Fractional Gradient Descent Algorithm},
author = {Jain, S.; Hote, Y.V.},
journal = {Journal of Dynamic Systems, Measurement, and Control},
year = {2020},
month = {12},
volume = {142},
number = {2},
institution = {Indian Institute of Technology Roorkee , India},
abstract = {An adaptive controller design technique based on internal model control (IMC) scheme is proposed in this paper. Multiple IMC controllers having different values of filter time constants and exhibiting widely different performance are combined via weight update rule. The weight update rule, formulated via convex combination of integral and fractional order gradient descent algorithms, assigns time varying weights to individual candidate controllers to obtain an enhanced performance over the individual candidate controllers. The beauty of the proposed technique is that it employs the simplicity of one degree-of-freedom (1DOF) IMC structure to achieve an improved performance over existing 2DOF control schemes. The efficacy of the proposed technique is demonstrated via three illustrative examples and via experimental validation on the hardware setup of dc servosystem. An extensive comparative analysis in terms of simulation plots and performance indices offers a testimony to the effectiveness of the proposed scheme. },
issn = {0022-0434},
language = {English},
publisher = {ASME}
}

An adaptive controller design technique based on internal model control (IMC) scheme is proposed in this paper. Multiple IMC controllers having different values of filter time constants and exhibiting widely different performance are combined via weight update rule. The weight update rule, formulated via convex combination of integral and fractional order gradient descent algorithms, assigns time varying weights to individual candidate controllers to obtain an enhanced performance over the individual candidate controllers. The beauty of the proposed technique is that it employs the simplicity of one degree-of-freedom (1DOF) IMC structure to achieve an improved performance over existing 2DOF control schemes. The efficacy of the proposed technique is demonstrated via three illustrative examples and via experimental validation on the hardware setup of dc servosystem. An extensive comparative analysis in terms of simulation plots and performance indices offers a testimony to the effectiveness of the proposed scheme.

This work presents a miniaturized industrial plant built to portray a hydraulic and thermal system. The system was developed in a low-cost open-source platform for studies in the fields of automation, dynamic systems, and control. In order to test its capacity, different techniques of classic and modern control, estimation, modeling, and identification were applied. It should be noted that the work provides the ability to reproduction and continuous development of the plant, and also examples of educational applications, reducing the demand for laboratory equipment acquisition for teaching and research in Universities.

@article{rodriguez-cortes_2019,
title = {A swinging up controller for the Furuta pendulum based on the Total Energy Control System approach},
author = {Rodriguez-Cortes, H.},
journal = {Kybernetika},
year = {2019},
volume = {55},
number = {2},
institution = {CINVESTAV-IPN, Mexico},
abstract = {This paper considers the problem of swinging up the Furuta pendulum and proposes a new smooth nonlinear swing up controller based on the concept of energy. This new controller results from the Total Energy Control System (TECS) approach in conjunction with a linearizing feedback controller. The new controller commands to the desired reference the total energy rate of the Furuta pendulum; thus, the Furuta pendulum oscillates and reaches a neighborhood of its unstable configuration while the rotation of its base remains bounded. Once the Furuta pendulum configuration is in the neighborhood of its unstable equilibrium point, a linear controller stabilizes the unstable configuration of the Furuta pendulum. Real-time experiments are included to support the theoretical developments. },
keywords = {total energy control system, Furuta pendulum, swinging up control, real-time experiments},
language = {English},
publisher = {Institute of Information Theory and Automation AS CR},
pages = {402-421}
}

This paper considers the problem of swinging up the Furuta pendulum and proposes a new smooth nonlinear swing up controller based on the concept of energy. This new controller results from the Total Energy Control System (TECS) approach in conjunction with a linearizing feedback controller. The new controller commands to the desired reference the total energy rate of the Furuta pendulum; thus, the Furuta pendulum oscillates and reaches a neighborhood of its unstable configuration while the rotation of its base remains bounded. Once the Furuta pendulum configuration is in the neighborhood of its unstable equilibrium point, a linear controller
stabilizes the unstable configuration of the Furuta pendulum. Real-time experiments are included to support the theoretical developments.