Prof. Pier Luigi Dragotti

Imperial College London

Sparsity and Deep Neural Networks: a match made in Heaven

Abstract: Deep Neural Networks are currently able to achieve state-of-the-art performance in many imaging tasks including image denoising and image super-resolution. At the same time sparse representation theory has been central to many fundamental developments in image processing over the last two decades. In this talk, we argue that we can do far better in choosing deep-learning architectures by relying systematically on ideas in sparse representation and that this approach leads to deep neural networks which are easier to interpret. In the first part of the talk, we propose an approach based on deep dictionary learning to develop deep neural network for image super-resolution. The proposed architecture contains several layers of analysis dictionaries to extract high-level features and one synthesis dictionary which is designed to optimize the reconstruction task. Each analysis dictionary contains two sub-dictionaries: an information preserving analysis dictionary (IPAD) and a clustering analysis dictionary (CAD). The IPAD with its corresponding thresholds passes the key information from the previous layer, while the CAD with its properly designed thresholds provides a sparse representation of input data that facilitates discrimination of key features. We then look at the multi-modal case and use the dictionary learning framework as a tool to model dependency across modality, to dictate the architecture of a deep neural network and to initialize the parameters of the network. Numerical results show that this approach leads to state-of-the-art results.

Biography

Pier Luigi Dragotti is Professor of Signal Processing in the Electrical and Electronic Engineering Department at Imperial College London and a Fellow of the IEEE. He received the Laurea Degree (summa cum laude) in Electronic Engineering from the University Federico II, Naples, Italy, in 1997; the Master degree in Communications Systems from the Swiss Federal Institute of Technology of Lausanne (EPFL), Switzerland in 1998; and PhD degree from EPFL, Switzerland, in 2002. He has held several visiting positions. In particular, he was a visiting student at Stanford University, Stanford, CA in 1996, a summer researcher in the Mathematics of Communications Department at Bell Labs, Lucent Technologies, Murray Hill, NJ in 2000, a visiting scientist at Massachusetts Institute of Technology (MIT) in 2011 and a visiting scholar at Trinity College Cambridge in 2020.

Dragotti was Editor-in-Chief of the IEEE Transactions on Signal Processing (2018-2020), Technical CoChair for the European Signal Processing Conference in 2012, Associate Editor of the IEEE Transactions on Image Processing from 2006 to 2009. He was also Elected Member of the IEEE Computational Imaging Technical Committee and the recipient of an ERC starting investigator award for the project RecoSamp. Currently, he is IEEE SPS Distinguished Lecturer.

His research interests include sampling theory, wavelet theory and its applications, computational imaging and sparsity-driven signal processing

Assoc. Prof. Veronika Cheplygina

ITU Copenhagen

Cats, Crowds and other considerations for learning with limited labelled data

Abstract: Machine learning has vast potential in medical image analysis, improving possibilities for early diagnosis and prognosis of disease. Algorithms typically need large amounts of representative, annotated examples for good performance, which may be difficult to achieve, for example due to differences between image acquisition procedures, or the time and effort involved in annotation. To address these problems, several approaches have been proposed, which are either aimed at adapting to use other types of annotated data, and or at gathering annotations more efficiently.

In this talk I will highlight two such approaches: transfer learning from natural images such as cats, and crowdsourcing by annotators without medical expertise. I will also discuss more general issues we face we as a community face when addressing such problems.

Biography

Dr. Veronika Cheplygina's research focuses on limited labeled scenarios in machine learning, in particular in medical image analysis. She received her Ph.D. from Delft University of Technology in 2015. After a postdoc at the Erasmus Medical Center, in 2017 she started as an assistant professor at Eindhoven University of Technology. In 2020, failing to achieve various metrics, she left the tenure track of search of the next step where she can contribute to open and inclusive science. She recently started as an associate professor at IT University of Copenhagen.

Next to research and teaching, Veronika blogs about academic life on https://www.veronikach.com, and gives talks and workshops on failure and related topics. She also loves cats, which you will often encounter in her work.