For those already familiar with reinforcement learning, the challenges of bringing it to the classroom are clear. It is a complex area of research and a moving target for educators. And because it is only a tool with no single application, it is difficult – if not impossible – to create engaging experiences for students with solid intuitive learning outcomes.
Here at Quanser we have been working with reinforcement learning for years, and quite frankly, have struggled ourselves to bring it to pedagogical applications. It is far too easy to get caught up in your particular application, whether it be controls, process management, or data processing. But having worked closely with professors around the world, we have a unique perspective on the problem.
So lets start with the crux of the problem. How can one expose students to the concepts of reinforcement learning while avoiding overspecialization? I would like to propose three different approaches, which can be used in isolation or in concert to keep your students engaged with reinforcement learning as an abstract concept while preparing them for self-directed learning in the workplace.
The “good”
Because reinforcement learning is normally implemented using a specialized library, it is practical to use such a tool in your class. In particular with students primarily learning remotely, it is advantageous to use a tool which they can access for free for initial exploration and motivation. To this end we have focused on Python applications using free libraries such as Keras-RL or OpenAI Baselines.
For this initial exploration the application can be very abstract, such as a simple maze or reaction game. This has the advantage of being practical to implement entirely within the Python development environment. Furthermore, the experience students will gain working with an open library can be eye-opening in and of itself.
The “better”
If your students are already familiar with the basic concepts of RL, you may be looking for a toolbox with real-world applications in research and industry. This is where our partnership with MathWorks has really accelerated our in-house knowledge base. Using the Reinforcement Learning Toolbox™ we have implemented examples of research-grade RL applications in MATLAB® using virtual models of servomotors and other industrial systems. This takes the concept out of the “hobbyist” realm and introduces students to RL as a tool for developing better intelligent systems. While not as easily accessible to all students, the integration with community tools and a wide range of example applications make this an outstanding approach for a grad-level machine intelligence course.
The “best”
Nothing beats real hardware. And obviously as a hardware manufacturer, it would be strange if we did not have at least one solution which gave students physical hands-on experience. To avoid the aforementioned issue of overspecialization we have focused on a fairly universal and broad application: autonomous vehicles. Because access to hardware is a challenge for several reasons, our best solution for teaching RL in the classroom is a hybrid experience built both on a high-fidelity virtual platform as well as our extremely popular QCar self-driving research platform.
By allowing students to explore and experiment in a low-cost, low-risk virtual environment you can make practical RL implementation accessible to all your students. Once they are comfortable and have developed and tested their implementation in VR, the QUARC™ hardware interface allows them to seamlessly target a remote real-time target and see their code function in a lab environment with physical hardware, roads, and obstacles. The interface is nearly identical between virtual and hardware allowing for truly painless transitions.
Bringing it together
Because of the cross-platform design of the QUARC environment and our partnership with MathWorks, our mixed-reality solution is also compatible with code developed in either Python or Simulink allowing for a smooth transition from one or both of the other solutions if desired.
However if you choose to approach this challenge, know that you are pioneering the education of a whole new field of engineering. Students are going to come into the classroom with an enthusiasm for these topics driven by cutting-edge developments in AI. All we as educators need to do is give students the tools to explore and innovate. Hopefully these approaches to teaching reinforcement learning in the classroom help you to inspire your students to even greater achievements in the future.
