Autonomous systems (WP5)
WP5 focuses on applied research into safe, reliable and explainable autonomous systems for transport and robotics. The aim is to ensure the safe and reliable operation of autonomous vehicles and robotic systems in real-world scenarios. The research is structured into four main areas.
The first is real-time data processing with multimodal perception. Another area is research into autonomous traffic control and predictive models. The third area is the analysis of human-robot collaboration with autonomous robotic systems, based on the human-in-the-loop (HITL) approach. The final area is the investigation of the deployment of autonomous systems in complex environments and spatio-temporal domains.
What problem does the WP solve, and why is it important?
01
Poor safety and lack of transparency regarding the interactions between autonomous systems and vulnerable road users
02
Low levels of trust and social acceptance of autonomous transport systems
03
Research uncertainty arising from the combination of real-time control, perception and prediction in distributed edge computing systems
04
The limitations of current literature on immersive explicitness in human-machine interfaces to 2D or verbal formats
Examples of use / areas of application and benefits
Transport and Detection
- Reliable multimodal detection of people and estimation of forces acting on a robot
- Prediction of road user behaviour and traffic flows based on reinforcement learning methods
- Adaptive control in complex environments
Autonomous systems
- ADAS models
- Lane Keeping Assist
- Detection of obstacles on the road
Navigation
- Multi-agent algorithms for trajectory planning
- Transport optimisation and simulation of transport scenarios
Key people
Participating institutions
Used technologies and procedures
- Prediction of a robot’s interaction with the terrain and deformable objects, supplemented by a neuro-symbolic layer
- Real-time data processing with multimodal perception within the framework of Connected, Cooperative, and Automated Mobility (CCAM)
- Data-driven modelling and multimodal perception
- Development of digital twins for the effective testing and validation of systems
- The use of large language models (LLMs) for explainable predictions and solving transport problems
- Human-robot collaboration (Human-Robot Teaming) based on the human-in-the-loop (HITL) approach
- Active Bayesian inference and lifelong learning for adaptive locomotion control
Achieved and planned results
2026
AnomVision: software for semantic zero-shot anomaly detection in traffic video
R- software
2027
Software for the automatic generation of a virtual 3D road model from fleet vehicle data
R- software
The system of mutual perception between humans and robots, and between robots and humans
Gfunk – working sample
‘Industrial design’ of the new sensor system design for SAE J3016 L3–L4 automated driving, or remotely controlled transport systems
Fprum – industrial design
‘Utility model’ for new sensor systems for SAE J3016 L3–L4 automated driving, or remote traffic control systems (e.g. hardware topology and module interconnections, integration of active components, sensor cleaning control, and interconnection of the data logger and computing platform)
Fuzit – utility model
2028
Context-oriented human-robot collaboration in joint manipulation tasks
R- software
A set of algorithms and software libraries for planning and optimisation in transport scenarios
R- software
Software for the automatic generation of virtual 3D environments for testing ADAS automotive systems, created from fleet vehicle data
R- software
Software for the automatic creation of virtual 3D environments for automotive testingSoftware with defined functionality and demonstrable use cases (AI model, software package, fusion framework, edge AI module) for a pilot application of SAE J3016 L3–L4 automated driving, or remotely controlled vehicle systems, and ADAS systems created from fleet vehicle data
R- software
Software with defined functionality and demonstrable applications (AI model, software package) for metadata extraction (e.g. classification of longitudinal and vertical road signs; classification of areas with poor visibility; focusing on various approaches and methods of metadata extraction)
R- software
AnomEdge: scalable incident detection using edge computing
R- software
AI software modules for automating data labelling
R- software
2029
AnomInsight – software for gaining a deeper understanding of transport and incident patterns
R- software
2031
A tool for data management and training AI models, focused on the automotive sector (ADAS – Advanced Driver Assistance Systems)
R- software
Software for the automatic generation of highly realistic virtual 3D environments for testing ADAS automotive systems
R- software
CrossroadExpert: An agent-based system for the autonomous expert optimisation of transport junction design
R- software