The growing interest in on-board data processing for Earth observation satellites has been driven by both the needs of ground-based applications of satellite data and the increasing challenges borne by the latest on-board instrument technologies. On the application side, end users of satellite data are seeking methods of acquiring more timely data and lower latencies, supplied in a form that is immediately useful to them. On the technology side, instrument manufacturers are developing payloads with higher spatial and spectral resolutions and greatly increased acquisition rates. Such rates far exceed typical current and near-term downlink bandwidths, resulting in severe data bottlenecks.

Addressing the application needs leads to solutions which also address the challenges of adopting these new instruments. On-board processing activities which can extract information from data, reduce the data and prioritise useful information can not only deliver data to end users faster and in a more useful form, they can also filter and intelligently compress data to reduce or eliminate the downlink bottleneck. These on-board activities are many and variable, depending on the on-board functionality desired and the end user requirements to meet.

In this paper, generic on-board applications are presented which may be configured to target groupings of use cases and provide measurable end benefits to mission stakeholders. These use cases have been solicited from a comprehensive survey and a workshop and one-to-one engagements with remote sensing end users. These applications are underpinned by enabling technologies which leverage the state-of-the-art in embedded processing and AI. These technologies include machine learning algorithms (both traditional and deep learning), training datasets, processing architectures and embedded computing devices. The paper summarises the algorithms which can be used to implement the on-board applications and the datasets which can be used to train and test these algorithms. Finally, a summary of the requirements and further implications of adopting these technologies to implement the proposed applications is presented. This leads to several impact areas and corresponding needs that must be addressed in the design of future on-board processing systems.

This presentation aims to provide an overview of the required preparatory steps for an effective deployment of AI at the edge for Earth Observation mission. It is based primarily on the lessons learnt from Phisat-1, an enhancement of the Federated Satellite Systems (FSSCat) mission. Launched in September 2020, it was the first experiment to demonstrate how AI can be used for Earth observation. More specifically, an AI inference engine for cloud detection has been demonstrated thanks to an Intel® Movidius™ Myriad™ 2 Vision Processing Unit (VPU). In this presentation, we will also introduce the capabilities of the planned Phisat-2 mission (with Multispectral EO sensor with 7-bands in VNIR range).


More specifically, we will also address the following topics in this presentation:
- An analysis of the need for AI@edge is provided starting from end–user requirements. The AI methodologies (for instance mainly Deep Neural Network) being mostly supervised techniques, relevant databases for training are required. However, self-supervised or weakly supervised AI are of interest when sparse or no annotation / ground truth are available.
- We will also provide an overview of AI techniques to cope with uncertainties, especially when it is required to use synthetize EO data (e.g. multispectral optical image) that will be acquired by the upcoming flying platform subject to noise. New approaches can be investigated (e.g. NN Out-of-distribution detection-OOD, Bayesian DL) with the general idea to provide robust estimates with quantified uncertainties. Bayesian DL refers to merging deep learning architectures and Bayesian probability theory. Bayesian DL models typically derive estimations of uncertainty by either placing probability distributions over model weights, or by learning a direct mapping to probabilistic outputs.
- The design of Tiny ML models via binary dropout, pruning, and knowledge distillation is of high interest so as to be compliant with on-board requirement.
- We will report on current activities about the design of neuromorphic algorithms (spiking neural networks) for solving EO problems such as Temporal coding or rate-based coding)