30 HP - Hybrid On-Device Language Model for In-Vehicle Voice Control on Android Automotive Introduction Are you exited about real life AI applications, and want to deep dive into how AI can be used to enhance user experience in commercials vehicles? Then this is a great opportunity to combine theory with practice to increase your AI skillset and build connections for your future career. Background Drivers have been able to control some vehicle and infotainment functions by voice for many years. Recent language models may make voice interaction more flexible, allowing drivers to use natural phrasing, combine requests, and ask follow-up questions. This thesis will investigate a hybrid approach that combines an existing embedded voice engine with a small language model SLM running locally in the vehicle. Objective The objective of this thesis is to design and build a proof of concept POC of a hybrid voice architecture. A small language model SLM that runs locally on the CPU of the Driver Display Unit DDU and works together with the existing embedded voice engine. The existing engine keeps responsibility for wake-up word, speech recognition, speech synthesis and well-defined commands. The SLM handles the requests the current solution cannot resolve. This includes free phrasing, several intents in one sentence, follow-up questions and knowledge-based questions. It turns them into validated, structured function calls toward infotainment and vehicle functions such as interior lighting, climate, media and truck controls. In this first phase, the POC runs only on the CPU. The design must still support a later move to a dedicated Neural Processing Unit NPU, and the CPU results serve as the baseline for that move. Job description This thesis focuses on practical system development, integration, benchmarking and evaluation on real target hardware. Your work will include: State-of-the-art survey. Study small language models about 0.5–3B parameters and on-device inference runtimes for example llama.cpp, ONNX Runtime, ExecuTorch, MLC LLM. Assess their performance on ARM CPUs, their function-calling ability and their readiness for NPU backends. Model selection and optimization. Hybrid architecture design. Define how the existing voice engine and the SLM work together. This covers routing and fallback logic, confidence thresholds, and how recognized text, intents and responses are exchanged. Platform integration. Integrate the SLM as an Android Automotive service with a clear interface AIDL. Function calling toward vehicle functions. Safety and determinism. Exclude high-risk functions from voice control. Performance benchmarking on the target. System impact analysis. Analyze how the SLM workload affects HMI rendering, display composition, audio and other IVI functions. Comparative evaluation. Compare the hybrid solution with the existing voice engine alone on task success rate, share of requests resolved, latency and resource usage. Your profile Y
