Post-training neural architecture optimization for small language models
Generative AI, and particularly language models (LLM), have sparked a new revolution in AI with applications across all domains. However, LLMs are highly resource-intensive and, hence, difficult to implement on autonomous embedded systems. LLMs can be optimized by modifying their architecture to replace heavy Transformer layers with lighter alternatives. Given the difficulty of training LLM "from scratch," this thesis aims to develop post-training neural architecture optimization methods applicable to small LLM (SLM). Additionally, the thesis seeks to propose performance metrics of different layers of an SLM and their alternatives, to guide the replacement, and thus propose a comprehensive methodology for optimizing SLMs while considering hardware constraints. The work will be valorized through publications in major AI conferences and journals, and the developed codes and methods could be integrated into the tools developed at CEA.
An electrochemical flow microreactor for a greener synthesis of gold nanoparticles
Gold nanoparticles (AuNPs) possess unique electronic, photonic, and chemical properties of invaluable interest in a variety of medical and technological applications. They are typically produced by controlled chemical precipitation from a salt solution to achieve the precise size control critical for most applications. Continuous flow microreactors, which efficiently mix the salt solution and the reducing agent, are known to offer improved size control. However, even in these reactors, the smallest AuNPs can only be formed using powerful reducing agents that are harmful to human health or the environment. We propose to minimize their impact and to develop a more resource-efficient process by inserting an electrochemical cell into the reactor to form the reducing agent in-situ in the adjusted amount necessary to produce the desired AuNPs.
Your goal will be to test and adapt continuous-flow electrochemical cells for the synthesis of AuNPs, exploring various electrochemical reactions and cell designs. You will also explore the use of several capping agents of biological interest. A careful examination of AuNPs characteristics (size, interfacial and optical properties, etc.) will guide you in this research.
Multi-scale approach for ultrasonic propagation in inhomogeneous multiple-scattering media
Ultrasonic waves are strongly influenced by the microstructure of the materials through which they propagate, leading to attenuation, dispersion, and noise. Modeling these effects is essential, particularly in non-destructive testing, where they may either hinder defect detection or provide valuable information about the material. Analytical and numerical models help to better predict and interpret these phenomena. Homogeneous statistical properties are generally assumed in such approaches. In practice, however, microstructures often exhibit significant spatial variations, for instance due to manufacturing processes. Depending on the scale of these variations relative to the wavelength, they may induce either abrupt or gradual changes in effective properties. This PhD aims to establish a theoretical framework that accounts for both microstructural randomness and its spatial variations, in order to propose relevant simulation strategies depending on the scales involved. The approach will first be developed in 1D, then extended to 2D and 3D using tools developed in the laboratory, with numerical and possibly experimental validations.
Prediction of elastic wave dispersion effects using a semi-analytical model under high-frequency approximation
Ultrasonic testing (UT) methods are a fundamental component of non-destructive testing (NDT). They are widely used to inspect mechanical components such as welds (in nuclear and petrochemical industries) and composite material structures (in aeronautics). To understand the physical phenomena involved in a given configuration, simulation is a valuable tool and sometimes an essential step in implementing the inspection process.
Modeling approaches fall into two main categories: purely numerical models based on finite elements (FE) and semi-analytical methods derived from high-frequency (HF) approximations, such as paraxial rays. While the latter are often favored for their computational efficiency, they introduce simplifications that can compromise the quantitative accuracy of results, particularly for phenomena like dispersion (variation in wave speed with frequency), which are common in certain industrial contexts.
This thesis project aims to enhance the paraxial ray approach by integrating models of dispersive interfaces (composite interplies, coupling layers), dispersive viscoelastic media, and a modal guided wave model. The goal is to develop a simulation tool capable of faithfully reproducing realistic inspection configurations, thereby improving the representativeness of the results.
Reconciling predictability and performance in processor architectures for critical systems
Critical systems have both functional and timing requirements, the latter ensuring that deadlines are always met during operation; failure to do so may lead to catastrophic consequences. The critical nature of such systems demands specialized hardware and software solutions. This PhD thesis topic focuses on the development of computer architecture designs for critical systems, known as predictable architectures, capable of providing the necessary timing guarantees. Several such architectures exist, typically based on in-order pipelines and incorporating behavioral restrictions (e.g., disabling complex speculation mechanisms) or structural specializations (e.g., redesigned caches or deterministic arbitration for shared resources). These restrictions and specializations inevitably impact performance, and the design of predictable architectures must therefore address the predictability–performance tradeoff directly. This PhD thesis aims to explore this tradeoff in a novel way, by adapting a high-performance variant of an in-order processor (CVA6) and developing top-down techniques to make it predictable. Performance in such processors is usually achieved through mechanisms like branch prediction, prefetching, and value prediction, implemented via specialized storage elements (e.g., buffers) and supported by control mechanisms such as rollback on misprediction. Within this context, the goal of the thesis is to define a general predictability scheme for speculative execution, covering both storage organization and rollback behavior.
Out-of-Distribution Detection with Vision Foundation Models and Post-hoc Methods
The thesis focuses on improving the reliability of deep learning models, particularly in detecting out-of-distribution (OoD) samples, which are data points that differ from the training data and can lead to incorrect predictions. This is especially important in critical fields like healthcare and autonomous vehicles, where errors can have serious consequences. The research leverages vision foundation models (VFMs) like CLIP and DINO, which have revolutionized computer vision by enabling learning from limited data. The proposed work aims to develop methods that maintain the robustness of these models during fine-tuning, ensuring they can still effectively detect OoD samples. Additionally, the thesis will explore solutions for handling changing data distributions over time, a common challenge in real-world applications. The expected results include new techniques for OoD detection and adaptive methods for dynamic environments, ultimately enhancing the safety and reliability of AI systems in practical scenarios.
Sofware support for computing accelerators and memory transferts accelerators
For energy reasons, future computers will have to use accelerators for both computation and memory access (GPUs, TPUs, NPUs, smart DMAs). AI applications have intensive computational requirements in terms of both computing power and memory throughput.
These accelerators are not based on a simple instruction set (ISA), they break the Von Neuman model: they require specialized code to be written manually.
Furthermore, it is difficult to compare the use of these accelerators with code using a non-specialized processor, as the initial source codes are very different.
HybroLang is a hardware-close programming language that allows programs to be written using all of a processor's computing capabilities, while also allowing code to be specialized based on data known at runtime.
The HybroGen compiler has already demonstrated its ability to program in-memory computing accelerators, as well as to optimize code on conventional CPUs by performing innovative optimizations.
This thesis proposes to extend the HybroLang language in order to
- facilitate the programming of AI applications by providing support for complex data: stencils, convolution, sparse computing
- enable code generation both on CPUs and with hardware accelerators currently under development at the CEA (sparse computing, in-memory computing, memory access)
- allow to benchmark different computing architectures with the same initial source code
Ideally, a candidate should have knowledge of computer architecture, programming language implementation, code optimization and compilation.
LLM-Assisted Generation of Functional and Formal Hardware Models
Modern hardware systems, such as RISC-V processors and hardware accelerators, rely on functional simulators and formal verification models to ensure correct, reliable, and secure operation. Today, these models are mostly developed manually from design specifications, which is time-consuming and increasingly difficult as hardware architectures become more complex.
This PhD proposes to explore how Large Language Models (LLMs) can be used to assist the automatic generation of functional and formal hardware models from design specifications. The work will focus on defining a methodology that produces consistent and executable models while increasing confidence in their correctness. To achieve this, the approach will combine LLM-based generation with feedback from simulation and formal verification tools, possibly using reinforcement learning to refine the generation process.
The expected outcomes include a significant reduction in manual modeling effort, improved consistency between functional and formal models, and experimental validation on realistic hardware case studies, particularly RISC-V architectures and hardware accelerators.
Growth of 2D Ferromagnetic Chalcogenide Materials for Spintronics
Chalcogenide materials, particularly Ge-Sb-Te (GST) alloys, are essential for phase-change memory (PCMs).
Although high-performance, these memories consume a great deal of energy, which
is driving the search for alternative solutions. GST alloys offer unique opportunities in the field of spin-orbitronics as spin-charge conversion materials or as sources of spin-polarized current. Two-dimensional ferromagnetic alloys such as Fe-Ge-Te or Ge-Mn-Te offer promising avenues as sources of spin current for new types of more efficient memory devices. For efficient spin injection, we are seeking a material that not only exhibits a high Curie temperature (TC) and significant spin polarization, but is also fully compatible with existing silicon-based CMOS technology.
The aim of this thesis is to develop and master, on an industrial scale on 300 mm Si substrates, the van der Waals epitaxial growth of 2D ferromagnetic films based on Fe-Ge (Ga)Te2 (n=3, 5) or Ge_(1-x)Mn_xTe, for example to integrate them in situ with spin-charge conversion chalcogenide layers such as ferroelectric layers (a-GeTe(111)) or topological insulators (Bi_(2-x)Sb2Te3).
Architecture of small animal single photon emission tomograph.
Medical imaging, a source of major innovations, presents remarkable potential for meeting new challenges with the growing demand for precision medicine, which requires cutting-edge diagnostic and therapeutic approaches personalized for each patient.
In this context, CEA-Leti proposes a PhD internship to develop a dedicated preclinical SPECT (Single Photon Emission Tomography) imager that will provide the performance (spectral information, high resolution, and high sensitivity) needed by researchers developing new radiopharmaceuticals.
The laboratory has a recognized expertise on CZT (Cadmium Zinc Telluride) semiconductor imagers enabling better spatial and energy resolution than scintillators used by most systems. They open new opportunities for emission imaging like easier Compton imaging, multi-isotope imaging and better contrast.
The candidate will have to handle the following tasks:
1. Study the state of the art of small animal SPECT imagers to participate with the team to the choice of system specification and choice of a draft architecture.
2. Simulate this architecture by using Monte-Carlo codes and optimize free parameters.
3. Design and manufacture the prototype system, with the help of the team including system engineers.
4. Test and validate the imaging capabilities, using reconstruction algorithms provided by the team.
The PhD will be conducted inside an instrumentation laboratory with access to acquisition electronics, detectors, motorized mechanics, gamma-ray sources and processing/simulation software. The candidate will also work in collaboration with a clinical and preclinical centre (at Orsay’s hospital) for conducting imaging test on phantoms and animals.