Design and optimisation of a thermoacoustic heat engine

The thermoacoustic heat engine (TAHE) is a type of prime mover which converts thermal power to acoustic power. It is composed of two heat exchangers (the devices heat source and sink), some kind of porous medium where the conversion of power takes place, and a tube which houses the acoustic wave produced. Its simple design and the fact that it is one of a few prime movers that does not require moving parts, make such a device an attractive alternative for many practical applications. The acoustic power produced by the TAHE can be used to generate electricity, drive a heat pump or a refrigeration system. Although the geometry of the TAHE is simple, the behavior of the engine is complex with 30+ design parameters that affect the performance of the device; therefore designing such a device remains a significant challenge. In this work a radical design methodology using reinforcement learning (RL) is employed for the design and optimization of a TAHE for the first time. Reinforcement learning is essentially the concept of learning by interacting with an environment, and although its framework is simple, it has proved to be a very powerful tool in solving a wide range of complex decision making/optimization problems. The reinforcement learning technique employed by the agent in this work is known as Q-learning. Preliminary results have shown the potential of the reinforcement learning technique to solve this type of complex design problem, and the learning agent was able to create a design that yielded an acoustic power output of 643.31 W. Eventually, it is hoped that with increased understanding of the design problem, in terms of the reinforcement learning framework, it will be possible to ultimately create an autonomous RL agent for the design and optimization of complex TAHEs with minimal pre-defined conditions/restrictions.