Interacting with other traffic participants in urban environments is an elaborated task of the AV that requires holistic planning and timely communication. To that end, predicting their motion (i.e. recognizing pedestrian crossing intention and vehicles’ next maneuver), is not only important for safety in assisted and automated driving, but also beneficial for natural and smooth maneuvers and interaction of Automated Vehicles. This often requires a long prediction horizon, e.g., more than 3 seconds. In the interACT project one objective was to improve the prediction capability of the Automated Vehicle.

Figure 1: A typical scene where a pedestrian intends to cross.

• Controlled Markov chains [1] were used to predict where the pedestrian will be and when, as well as the corresponding probabilities—a probabilistic spatiotemporal result.
• Pedestrians’ intentions were estimated by incorporating the probability of colliding with other traffic participants (dynamic environments), their current dynamics, and topographic features of environments (semantic map).
• Model calibration was performed using recordings of real pedestrians and considering different objective functions during optimization [1].
• In addition, for further improvement of the pedestrian intention recognition the head orientation as well as hand waving gestures of pedestrians were recognized through deep neural networks.
• Hidden Markov model was employed for probabilistic recognition of vehicle’s maneuvers and generation of intention-aware map-conforming trajectory.
• Fusion of vehicle’s intention-based trajectory with typical motion-based (short-term) trajectory to extend the prediction time horizon.

Main Results

• Pedestrian Intention Recognition

Ability to perform frame-level action predictions, based on input video data, was demonstrated. Results have shown that as the pedestrian gets closer to road, the performance of our approach is further improved. Moreover, the abovementioned methods yielded more detailed probabilistic spatiotemporal predictions (see Figure 4). The suggested algorithms were incorporated into the interACT system enabling early and accurate detection of non-motorized traffic participants. System readiness, reaction and user-experience as a whole were extensively tested using real pedestrian-related scenarios, as part of WP6.

• Vehicle Intention Recognition

Simulation results demonstrated that the HMM enhanced trajectory prediction approach is able to predict driver intention to turn at least 3 seconds before the intersection entry, comparing favorably with the SoA. This enables a vehicle trajectory prediction considerably over 5 seconds as we can successfully use the map in the longer range while its accuracy in the short term is maintained by the kinematics-based trajectory fusion.

Integration in CRF demonstrator vehicle enabled to fine-tune model parameters using realistic scenarios, towards coordination between the Situation Awareness (intention-related) platform and the CCPU (decision and planning components).

Figure 2: Probabilistic spatiotemporal prediction results (green cells are
predicted future positions of the pedestrian, darker color means higher probability)
for 6 time steps (up to 3 seconds). The ground truth position is denoted by red points.

Figure 3: Simulation example Vehicle trajectory prediction (blue) versus
map-based trajectory (red) versus motion-model
trajectory (green) in 4-leg intersection

References

[1] J. Wu, J. Ruenz and M. Althoff, “Calibration of Controlled Markov Chains for Predicting Pedestrian Crossing Behavior Using Multi-objective Genetic Algorithms,” 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 2019, pp. 1032-1038

Read more

Ruenz, J., Wu, J., Zhang, J., Cao, Y., Schürmann, B., Althoff, M., Drainakis, G., Portouli, E. “interACT D2.3 Sensors and algorithms incorporating the developed models to be integrated into the demonstrator” (2019)