Quantum ML/RL

Motivation

Spurred by the recent experimental demonstration of so-called quantum advantage for a classically hard random-number generation problem with actual quantum hardware, the research community has increased its efforts to propose new types of quantum algorithms for the still imperfect, noisy intermediate scale quantum (NISQ) computing devices we have today.

One class of algorithms that have emerged as viable candidates for achieving quantum advantage with NISQ devices also in other application domains beyond random number generation are variational quantum circuits (VQCs). These adaptive quantum circuits lend themselves as a platform for quantum machine learning, here understood as the generation of quantum algorithms from data. While the requirements on the expressivity of these circuits to reach quantum advantage and related complexity-theoretic problems pose open research questions, experimentation with available architectures might uncover promising directions.

Project Idea

Within the framework of a joint master thesis project between the Chair for Quantum Theory at Friedrich- Alexander University Erlangen-Nürnberg and the Self-Learning Systems group at the Fraunhofer Institute for Integrated Circuits, the potential of VQCs for the task of learning and approximating complex multimodal distributions, as well as for the task of efficient sampling from these distributions, is to be investigated.

In the context of reinforcement learning, the learning of distributions from data plays a crucial role, both when generating surrogate models for the system dynamics from observational data, and when training the reinforcement-learning policy to approximately solve the underlying stochastic optimization problem. The latter case is closely tied to the trade-off between exploration and exploitation during the training phase of the reinforcement-learning procedure. To properly balance exploration and exploitation, it is particularly relevant to employ methods capable of learning conditional, multimodal distributions. In order to analyze the potential of quantum-enhanced learning algorithms in the context of model-free reinforcement learning, the goal of the advertised master thesis is to provide a proof-of-principle implementation of a VQC-based policy learning procedure. While actions are sampled according to the state prepared by the quantum circuit, the parameter updates for policy improvement need to be provided by an appropriate generalization of the so-called policy gradient.

The algorithm will be developed and tested for simple reinforcement-learning tasks (e.g. bandits and grid world problems) with discrete state and action spaces. In order to empirically gain insight on the possibility of quantum advantage, the quantum policy gradient approach is to be benchmarked against classical methods.

Required Skills: Basic knowledge of variational quantum circuits and machine learning, Python programming

More information here.

Motivation

Spurred by the recent experimental demonstration of so-called quantum advantage for a classically hard random-number generation problem with actual quantum hardware, the research community has increased its efforts to propose new types of quantum algorithms for the still imperfect, noisy intermediate scale quantum (NISQ) computing devices we have today.

One class of algorithms that have emerged as viable candidates for achieving quantum advantage with NISQ devices also in other application domains beyond random number generation are variational quantum circuits. These quantum circuits lend themselves as a platform for quantum machine learning, here understood as the generation of quantum algorithms from data. While the requirements on the expressivity of these circuits to reach quantum advantage and related complexity-theoretic problems pose open research questions, experimentation with available architectures might uncover promising directions.

Project Idea
In this regard, realizations of quantum generative adversarial networks (QGANs) – an extension of classical GANs to the domain of quantum data and quantum computation – provide an ideal testing ground for the exploitation of quantum resources for learning of and sampling from complex distributions – a ubiquitous task in statistical modelling and machine learning.
Within the framework of a joint master thesis project between the Chair for Quantum Theory at Friedrich- Alexander University Erlangen-Nürnberg and the Self-Learning Systems group at the Fraunhofer Institute for Integrated Circuits, the potential of QGANs for the task of learning and approximating distributions from data, as well as for the task of efficient sampling from these distributions, is to be investigated.
In the context of reinforcement learning, the learning of distributions from data plays a crucial role, both when generating surrogate models for the system dynamics from observational data, and when training the reinforcement-learning policy to approximately solve the underlying stochastic optimization problem. The first approach forms the basis for many model-based reinforcement-learning methods.
The goal of the advertised master thesis will be to implement and test QGAN architectures for learning distributions from a finite set of samples. A special emphasis will be put on the learning of state-transition probabilities from data generated by simulated reinforcement-learning environments with discrete state and action spaces. To leverage the potential quantum advantage offered by QGANs, the combination with dimensionality reduction techniques, such as representation learning approaches, will be explored.

Required Skills: Basic knowledge of variational quantum circuits and machine learning, Python programming

More information here.

Motivation

Spurred by the recent experimental demonstration of so-called quantum advantage for a classically hard random-number generation problem with actual quantum hardware, the research community has increased its efforts to propose new types of quantum algorithms for the still imperfect, noisy intermediate scale quantum (NISQ) computing devices we have today.

One class of algorithms that have emerged as viable candidates for achieving quantum advantage with NISQ devices also in other application domains beyond random number generation are variational quantum circuits (VQCs). These quantum circuits lend themselves as a platform for quantum machine learning, here understood as the generation of quantum algorithms from data. While the requirements on the expressivity of these circuits to reach quantum advantage and related complexity-theoretic problems pose open research questions, experimentation with available architectures might uncover promising directions.

Project Idea

Within the framework of a joint master thesis project between the Chair for Quantum Theory at Friedrich- Alexander University Erlangen-Nürnberg and the Self-Learning Systems group at the Fraunhofer Institute for Integrated Circuits, the potential of VQCs for the task of learning efficient and NISQ-compatible quantum algorithms is to be explored.

While quantum algorithms coming with the guarantee of exponential speed-ups seem to require fault-tolerant quantum computation, it remains an open question, whether shallow and thereby NISQ-compatible quantum algorithms with (maybe non-exponential) speed-up are possible. One approach to shorten existing algorithms is by employing optimization strategies to minimize gate count when expanding and approximating a quantum circuit by a given universal gate set.

Another approach is to try and learn a quantum algorithm for a specific purpose with a given gate budget from appropriately generated data via the paradigm of variational quantum circuits. As many machine learning and optimization algorithms rely on matrix inversion, it would be very desirable to employ such a procedure to arrive at NISQ implementation of, e.g., the HHL quantum algorithm, or similar algorithms, for matrix inversion. The goal of the master thesis project is the development of a structure- and parameter-learning method for VQCs, along with the required experimentation and benchmarking, for the purpose of learning efficient and NISQ implementable quantum circuits.

Required Skills: Basic knowledge of variational quantum circuits and machine learning, Python programming

More information here.

References

1. Mauro E.S. Morales, Timur Tlyachev, Jacob Biamonte, “Variationally Learning Grover’s Quantum Search Algorithm”, Phys. Rev. A 98, 062333 (2018)
2. Leonardo Banchi, Jason Pereira, Seth Lloyd, Stefano Pirandola, “Optimization and learning of quantum programs”, npj Quantum Information 6, 42 (2020)
3. Hsin-Yuan Huang, Kishor Bharti, Patrick Rebentrost, “Near-term quantum algorithms for linear systems of equations”, arXiv:1909.07344 [quant-ph]
4. Mateusz Ostaszewski, Edward Grant, Marcello Benedetti, “Quantum circuit structure learning”, arXiv:1905.09692 [quant-ph]
5. Carlos Bravo-Prieto, Ryan LaRose, M. Cerezo, Yigit Subasi, Lukasz Cincio, Patrick J. Coles, “Variational Quantum Linear Solver“, arXiv:1909.05820 [quant-ph]