Research

Broad Research Vision

We stand at the dawn of a technological and scientific revolution, with AI as its center. Despite its potential, our understanding of advanced AI systems is still very limited, as revealed by the following fact: the human brain, the inspiration behind modern neural networks, operates on a mere 12 watts, in stark contrast to the megawatts consumed by advanced neural networks. Therefore, state-of-the-art neural networks burn millions of times more power and data than a biological brain while coming short of simple reasoning and common sense tasks. Perhaps more importantly, the collective carbon footprint of AI has been increasing exponentially, rendering it unsustainable in the long term. This observation points to an urgent need for a deeper, fundamental understanding of neural networks to build models that are not just efficient but also sustainably intelligent. As a doctoral researcher, my broad research vision is to move towards such a fundamental understanding of neural networks to pave the way for their wider adoption, including their biomedical applications.

Mentorship

If you are a masters student looking for interesting projects to do during your studies or for your thesis, here is a list of available projects for MSc theses. Please send me an email with your CV and transcripts if you're interested!

Genomics:

• Learning metagenome graphs by GNNs, co-advised by Manuel Burger
• “Foundational sequence representations for single cell sequencing”, co-supervised by Stefan Stark

Neural Network Theory:

• Can a transformer learn sequence alignment?
• Replacing normalization with initialization
• Mystery of activation in neural nets
• Understanding pretraining speed in transformers.
• Zero-shot neural architecture search

Medical

• Doctor in the loop (Active Learning + Medical), co-advised by Hugo Yéche