About us
Espace utilisateur
INSTN offers more than 40 diplomas from operator level to post-graduate degree level. 30% of our students are international students.
Professionnal development
Professionnal development
Find a training course
INSTN delivers off-the-self or tailor-made training courses to support the operational excellence of your talents.
Human capital solutions
At INSTN, we are committed to providing our partners with the best human capital solutions to develop and deliver safe & sustainable projects.
Home   /   Thesis   /   Clip approach for improving energy efficiency of hardware embedding combinations

Clip approach for improving energy efficiency of hardware embedding combinations

Artificial intelligence & Data intelligence New computing paradigms, circuits and technologies, incl. quantum Technological challenges


In a global context of task automation, artificial neural networks are currently used in many domains requiring the processing of data from sensors: vision, sound, vibration.
Depending on different constraints, the information processing can be done on the Cloud (SIRI, AWS, TPU) or in an embedded way (NVidia's Jetson platform, Movidius, CEA-LIST's PNeuro/DNeuro). In this second case, many hardware constraints must be taken into account when dimensioning the algorithm. In order to improve the porting on hardware platforms, LIST has developed innovative state-of-the-art methods allowing to aggressively quantize the parameters of a neural network as well as to modify the coding of the activations to reduce the number of calculations to be performed.
The energy efficiency of neuromorphic architectures with equivalent technology is constrained by the classic paradigm of flexibility vs. efficiency. In other words, the more different tasks (and networks) an architecture is capable of performing, the less energy-efficient it becomes. While this relationship cannot be circumvented for a wide variety of algorithms, neural networks are parametric functions, learned for one and therefore potentially adaptable to other tasks by partial modification of the topology and/or parameters.
One technique, CLIP, seems to provide an answer, with a strong capacity for adaptation to a variety of tasks and the possibility of using multimodality. In its original form, CLIP is presented as a method for matching text and images to create a classification task.
The aim of this thesis is to study the hardware implementation of CLIP by proposing a dedicated architecture. The thesis is organized into 3 main phases, beginning with a study of CLIP's mechanisms, the operations to be performed and the consequences for embedding networks. Secondly, hardware optimizations applicable to CLIP, such as quantization (or others) and an estimation of flexibility vs. applicative generality. Finally, an architectural and implementation proposal to measure energy efficiency.


Département Systèmes et Circuits Intégrés Numériques (LIST)
Laboratoire Intelligence Artificielle Embarquée
Top envelopegraduation-hatlicensebookuserusersmap-markercalendar-fullbubblecrossmenuarrow-down