Hyperspectral imaging can capture hundreds of images acquired for narrow and continuous spectral bands across the electromagnetic spectrum. Since the spectral profiles are specific for different materials, exploiting such high-dimensional data can help determine the characteristics of the objects of interest. A hyperspectral image can be interpreted as a data cube which couples spatial and spectral information captured for every pixel. Practical applications of such imagery are very vast and they spread across a number of fields, including, biology, medicine, forensics, and remote sensing.

The number of bands in hyperspectral images (HSI) can reach hundreds, and it is a very useful source of information in various remote sensing applications. However, its huge volume brings challenges in efficient analysis, transfer (especially sending the hyperspectral data acquired on board a satellite back to Earth is extremely time-consuming and costly), and storage of such imagery. Also, data redundancy is a serious practical issue. The neighboring bands are often correlated, therefore only their small subset contributes to the HSI classification process. Finally, generating ground-truth (manually-annotated) data for supervised classification and segmentation methods is extremely difficult, time-consuming, and prone to human errors, and exploiting small (in terms of the number of each-class examples) and very high-dimensional datasets can easily deteriorate the performance of supervised learners. To deal with these issues, HSI is subjected to either feature extraction (generating new, perhaps more informative, less redundant, and compressed features from HSI) or feature selection (determining a subset of all HSI bands which convey the most important and useful information). These techniques can drastically reduce the dimensionality of the original hyperspectral data, and – in ideal scenario – do not deteriorate the amount of information captured by such imagery.

In this talk, we review both feature extraction and band selection algorithms that benefit from deep learning. In the former case, deep learning-powered techniques elaborate new latent (and often significantly compressed) representations of the original data that can capture important information within the data. We focus on autoencoder-based deep network architectures, alongside various recurrent neural networks (long short-term memory- and gated recurrent unit-based models) that are applicable to both multi- and hyperspectral data, and show how they can be deployed in end-to-end data analysis pipelines. On the other hand, we review attention-based convolutional neural networks used for band selection, and show how understanding the influence of specific parts of the entire spectrum can help us select only a small subset of all bands that convey the most important information about the objects in the scene. Since the number of informative bands is often small compared to all available bands, the process of selecting “useful” bands may be considered as anomaly detection within all bands, according to the elaborated attention scores. We show how the dimensionality reduction of hyperspectral data affects all other steps in the hyperspectral image segmentation chain. Finally, as exploiting deep learning-powered techniques in hardware-constrained environments is challenging because of their memory and energy requirements, we perform quantization of the investigated models (to notably decrease their hardware requirements and make them more resource-frugal), and verify the influence of the quantization process on the overall quality of the feature extraction process. Our talk is concluded with a discussion on open questions and challenges in the state of the art that ultimately need to be tackled to allow us seamlessly deploy such deep learning-powered approaches on-board satellites.