Development and characterization of embedded memories based on ferroelectric transistors for neuromorphic applications

As part of CEA-LETI's Devices for Memory and Computation Laboratory (LDMC), you will be working on the development and optimization of FeFET transistors with amorphous oxide semiconductor channels for neuromorphic applications and near-memory computing.
The main challenge when co-integrating semiconductor and ferroelectric oxides is to perfectly assess and control a proper amount of oxygen vacancies, which govern both the ferroelectric properties of HfZrO2 and the conduction properties of semiconducting oxide, and impose major constraints on the manufacturing process steps.
The aim of the proposed internship is to conduct electrical measurements on various kind of elementary devices, stand-alone ferroelectric / semiconductive oxide films up to complete integreted FeFET devices. This will allow to propose an optimized process flow capable to provide both efficient ferroelectric switching performances (speed, low voltage capability…) together with state-of-the-art MOSFET performances (Ion/Ioff, subthreshold slope…).
The student will have access to a large amount of processed 200mm wafers, embedding a large variety of FeFET device flavors with different dimensions. Different process options will be available, either on already-available wafers or on request during the internship. For the latter, this will involve a close interaction with process experts (either deposition, annealing...) for modifying the FeFET process flow.
The student will benefit from state-of-the-art characterization platform, either for material characterization (XPS, UPS, XRD, TEM microscopy…) or for measuring the FeFET electrical performances.

Modeling and Simulation of Human Behavior for Human-Centric Digital Twins

Thanks to synchronized virtual representation, digital twins are a means to produce analyses, predictions and optimizations of real-world systems. However, some of these systems tightly engage with humans so that the role of the latter is determining in the system’s operation. This is for example the case in contexts such as industry 5.0 or the management of the control of critical systems, where the quality of collaboration between humans and machines will depend on the anticipation of their respective actions, interactions and decisions. Thus, to improve the accuracy of predictions and expand the applicability in various fields, it is necessary, based on knowledge from the human and social sciences, to develop digital twins that account for the complexity and richness of human behaviors (decision-making processes, interactions, emotions, etc.). These behavioral models may notably rely on machine learning, data mining, agent-based modeling and knowledge engineering. After having identified the useful human behavior models, we will study their conceptual articulation and their technical integration with the models of cyber-physical entities in the digital twin system. Additionally, we will explore how digital twin services are impacted and can be revised to account for these human-centric aspects. Finally, we will evaluate the effectiveness of human-centric digital twins in various applications by implementing experiments on representative real cases.
This research work aims to make the following contributions:
• The development of an approach based on human behavior models to achieve human-centric digital twins.
• New knowledge on the impact of human behavior on the control of a system and vice versa.
• Practical applications and guidelines for using human-centric digital twins in real-world scenarios.
This PhD will be carried out at Grenoble.

Security blind spots in Machine Learning systems: modeling and securing complex ML pipeline and lifecycle

With a strong context of regulation of AI at the European scale, several requirements have been proposed for the "cybersecurity of AI" and more particularly to increase the security of AI systems and not only the core ML models. This is important especially as we are experience an impressive development of large models that are deployed to be adapted to specific tasks in a large variety of platforms and devices. However, considering the security of the overall lifecycle of an AI system is far more complex than the constraint, unrealistic traditional ML pipeline, composed of a static training, then inference steps.

In that context, there is an urgent need to focus on core operations from a ML system that are poorly studied and are real blind spot for the security of AI systems with potentially many vulnerabilities. For that purpose, we need to model the overall complexity of an AI system thanks to MLOps (Machine Learning Operations) that aims to encapsulate all the processes and components including data management, deployment and inference steps as well as the dynamicity of an AI system (regular data and model updates).

Two major “blind spots” are model deployment and systems dynamicity. Regarding deployment, recent works highlight critical security issues related to model-based backdoor attacks processed after training time by replacing small parts of a deep neural network. Additionally, other works focused on security issues against model compression steps (quantization, pruning) that are very classical steps performed to deploy a model into constrained inference devices. For example, a dormant poisoned model may become active only after pruning and/or quantization processes. For systems dynamicity, several open questions remain concerning potential security regressions that may occur when core models of an AI system are dynamically trained and deployed (e.g., because of new training data or regular fine-tuning operations).

The objectives are:
1. model security of modern AI systems lifecycle with a MLOps framework and propose threat models and risk analysis related to critical steps, typically model deployment and continuous training
2. demonstrate and characterize attacks, e.g., attacks targeting the model optimization processes, fine tuning or model updating
3. propose and develop protection schemes and sound evaluation protocols.

In-memory analog computing for AI attention mechanisms

The aim of this thesis is to explore the execution of attention mechanisms for Artificial Intelligence directly within a cutting-edge Non-Volatile Memory (NVM) technology.

Attention mechanisms represent a breakthrough in Artificial Intelligence (AI) algorithms and represent the performance booster behind “Transformers” neural networks.
Initially designed for natural language processing, such as ChatGPT, these mechanisms are widely employed today in embedded application domains such as: predicting demand in an energy/heat network, predictive maintenance, and monitoring of transport infrastructures or industrial sites.
Despite their widespread use, attention-based workloads demand extensive data access and computing power, resulting in high power consumption, which may be impractical to target embedded hardware systems.

The non-volatile memristor technology offers a promising solution by enabling analog computing functions with minimal power consumption while serving as non-volatile storage for AI model parameters. Massive linear algebra algorithms can be executed faster, at an ultra-low energy cost, when compared with their fully-digital implementation.
However, the technology comes with limitations, e.g., variability, the number of bits to encode model parameters (i.e. quantization), the maximum size of vectors processed in parallel, etc.

This thesis focuses on overcoming these challenges in the context of embedded time-series analysis and prediction.
The key task is exploring the mapping of attention-based mechanisms to a spin-based memristor technology developed by the SPINTEC Laboratory.
This involves quantizing and partitioning AI models to align with the hardware architecture without compromising the performance of the prediction, and exploring the implementation of particular AI blocks into the memristor analog fabric.

This thesis is part of a collaboration between CEA List, Laboratoire d’Intelligence Intégrée Multi-Capteur, the Grenoble Institute of Engineering and Management and the SPINTEC Laboratory.
Joining this research presents a unique opportunity to work within an interdisciplinary and dynamic team at the forefront of the AI ecosystem in France, with strong connections to influential industrial players in the field.

Graph Neural Network-based power prediction of digital architectures

Performing power analysis is a major step during digital architecture development. This power analysis is needed as soon as the RTL (Register Transfer Level) coding starts, when the most rewarding changes can be made. As designs get larger, power analysis relies on longer simulation traces and becomes almost impossible, as the process generates huge simulation files (> gigabytes or terabytes of data) and long power analysis turnaround times (weeks or even months). Therefore, power models are used to speed up this step. There is a broad range of research on power modeling at RTL, mainly based on analytical or learning-based approaches. Analytical power modeling attempts to correlate application profiles such as memory behavior, branch behavior, and so on with the micro-architecture parameters to create a power model. Whereas, learning-based power modeling generates a model based on the simulation trace of the design and a reference power obtained from sign-off tools. Learning-based power modeling is gaining popularity because it is easier to implement than the analytical approach and does not require in-depth design knowledge. These ML-based methods have shown impressive improvement over analytical methods. However, the classical ML methods (linear regression, neural network, …) are more suitable to generate one model for one given architecture making them difficult to use to generate a generalizable model. Thus, in the last couple of years, a few studies have started to use Graph Neural Networks (GNN) to address model generalization in the field of electronic design automation (EDA). The advantage of a GNN over classical ML approaches is its ability to directly learn from graphs, making it more suitable for EDA problems.
The objective of this PhD is to develop a generalizable model of power consumption of digital electronic architecture, based on GNN. The developed model should be able to estimate, in addition to the average power consumption, the cycle-to-cycle power consumption of any digital electronic architecture. Very few works [1,2] exist in the state of the art on the use of GNNs for power estimation and the models developed in this work are limited to estimating the average power of an architecture. Moreover, several important research questions are not addressed in this work such as the number of data (architectures) needed for the generalization of the model, the impact of the graph structure during training, the selection of architectures used for training and for testing, the choice of features, etc.
Thus, during this PhD, these questions will be studied in order to know their impact during the generation of the model.
The work performed during this PhD thesis will be presented at international conferences and scientific journals. Certain results may be patented.

Quantum Machine Learning in the era of NISQ: can QML provide an advantage for the learning part of Neural Networks?

Quantum computing is believed to offer a future advantage in a variety of algorithms, including those challenging for traditional computers (e.g., Prime Factorization). However, in an era where Noisy Quantum Computers (QCs) are the norm, practical applications of QC would be centered around optimization approaches and energy efficiency rather than purely algorithmic performance.

In this context, this PhD thesis aims to address the utilization of QC to enhance the learning process of Neural Networks (NN). The learning phase of NN is arguably the most power-hungry aspect with traditional approaches. Leveraging quantum optimization techniques or quantum linear system solving could potentially yield an energy advantage, coupled with the ability to perform the learning phase with a less extensive set of training examples.

In-physics artificial intelligence using emerging nanodevices

Recent breakthroughs in models of AI are correlated with the energy burden required to define and run these models. GPUs are the goto hardware for these implementations, since they can perform configurable, highly parallelised and matrix multiplications using digital circuits. To go beyond the energy limits of GPUs however, it may be required to abandon the digital computing paradigm altogether.

A particularly elegant solution may be to exploit the intrinsic physics of electron devices in an analogue fashion. For example, early work has already proposed how physical entropy of silicon devices can realise probabilistic learning algorithms, how voltage relaxation in resistive networks may approximate gradients, and how the activity of interconnected oscillators may converge minima on energy surfaces.

The objective of this thesis will be to study existing, and propose new, in-physics computing primitives. Furthermore, like GPUs bias current AI to rely on matrix multiplications, the candidate must also consider how these new primitives will impact future AI algorithms. Particular attention will be given to emerging nanodevice technologies under development at CEA Grenoble. Depending on the interests of the PhD student, it may be possible to design, tape-out and test circuit concepts leveraging these in-house innovative technologies.

Deep Learning Inverse Problem Solving Applied to Interferometry

Multimodal continual learning under constraints

Standard deep learning methods are designed to use static data. This induces a significant practical limitation when they are deployed in dynamic environments and are confronted with unknown data. Continuous learning provides a solution to this problem, especially with the use of large, pre-trained models. However, deploying such models in stand-alone mode is currently impossible in many frugal applications that impose heavy computational and/or memory constraints. Furthermore, most current methods are developed for a single modality (text or visual), whereas the data captured is often multimodal.
This thesis proposes to address several objectives that enable the practical deployment of agents capable of updating their representations under constraints. This deployment involves the following objectives:(1) the collection of domain-oriented corpora and their augmentation based on generative multimodal models, (2) the compression of foundation models to adapt them to the domain and make them usable under computational and/or memory constraints, (3) the proposal of efficient continuous learning methods to manage new multimodal data, and (4) the management of realistic data flows to take into account the specificities of different application contexts.

Deep Learning Models for Decoding of LDPC Codes

Error correction coding (ECC) has an essential role in ensuring integrity of data in numerous applications, including data storage, transmission, and networking. Over the last few years, new interactions emerged between coding theory and machine learning, seen as a promising way to overcome the limitations of existing ECC solutions at short to medium code-lengths. For many known constructions of ECC codes, it turns out that these limitations are primarily due to the decoding algorithm, rather than the intrinsic error correction capability of the code. However, determining the appropriate machine learning models that apply to ECC decoding specificities is challenging, and current research still faces a significant gap to bridge to fundamental limits in the finite-length regime.

This PhD project aims at expanding the current knowledge on machine learning based decoding of low-density parity-check (LDPC) codes, in several directions. First, it will investigate ensemble learning methods, in which multiple models are trained to solve the decoding problem and combined to get better results. Specific methods will be devised to ensure diversity of the individual models and to cover all the variability of the code structure. Second, it will explore knowledge distillation to transfer the superior performance of an ensemble to a single model, or from a large model to a smaller one, which is known to boost the prediction performance in several cases. Finally, the project will investigate syndrome-based decoding strategies, as a way to enable the use of powerful deep neural network models, rather than current belief-propagation based models, thus unleashing the full power of the above ensemble learning and knowledge distillation methods.

The doctoral student will be hosted at CEA-Leti in Grenoble within a research team expert in signal processing for télécommunications (http://www.leti-cea.fr/cea-tech/leti/english/Pages/Applied-Research/Facilities/telecommunications-platform.aspx).

Top