2nd Workshop On The Prediction Of Pedestrian Behaviors For Automated Driving

At IV 2022

June 5th, 2022 – Aachen, German

Important Information

About

With the progress of automated driving technologies, self-driving cars are driving safely on highways and freeways in most circumstances. However, the lack of safe and smooth interactions with pedestrians becomes one of the significant obstacles preventing fully autonomous vehicles in city streets. Protections of vulnerable road users like pedestrians are of the highest priority in traffic safety, and crashes with pedestrians will significantly impact the trust and public attitudes towards the new mobility technology. Disruptive interactions with pedestrians may also lower both the riding experiences and driving efficiency in the pedestrian-rich road environments. It is vital to understand pedestrians and make motion planning based on the predictions of their behaviors.

Besides pedestrian detection and tracking, many research efforts have been put into recognizing pedestrians' behaviors and predicting their trajectories in the past few years. The achievements can predict potential crashes and make motion-planning decisions accordingly in a longer duration. This Workshop will focus on the detection, recognition, and prediction of pedestrian behaviors for automated driving cars to interact with them smoothly. The goal is to build a platform for sharing state-of-the-art models, algorithms, and datasets in the field and identifying research needs and directions.

Topics

Format

The workshop will have two sections. The first section is invited talks, each of which will be 30 minutes including about 20 minutes of presentation and 10 minutes of Q/A. The second section will be discussions for everyone to participate about the research needs, open questions, existing resources, and future directions of the research area about pedestrian behavior predictions.

The workshop will be hybrid. Both participants and speakers can join in-person or remotely. Zoom link for the workshop will be provided to speakers and participants who cannot join in person.

Tentative Agenda

Speaker

Abstract

Title

1:00 pm
Renran Tian(In Person)
Introduction
A brief introduction to the workshop.

Time

Section 1: Invited Talks

Mohan Trivedi / Ross Greer
(In Person/Remote)
From Pedestrian Detection to Crosswalk Estimation: An EM Algorithm and Analysis on Diverse Datasets
1:45 pm
Amir Rasouli / Iuliia Kotseruba (In Person)
Intend-Wait-Cross: Towards Modeling Realistic Pedestrian Crossing Behavior
In this paper, we present a microscopic agent-based pedestrian behavior model Intend-Wait-Cross. The model is comprised of rules representing behaviors of pedestrians as a series of decisions that depend on their individual characteristics (e.g. demographics, walking speed, law obedience) and environmental conditions (e.g. traffic flow, road structure). The model’s main focus is on generating realistic crossing decision model, which incorporates an improved formulation of time-to-collision (TTC) computation accounting for context, vehicle dynamics, and perceptual noise.

Our model generates a diverse population of agents acting in a highly configurable environment. All model components, including individual characteristics of pedestrians, types of decisions they make, and environmental factors, are motivated by studies on pedestrian traffic behavior. Model parameters are calibrated using a combination of naturalistic driving data and estimates from the literature to maximize the realism of the simulated behaviors. A number of experiments validate various aspects of the model, such as pedestrian crossing patterns, and individual characteristics of pedestrians.
2:20 pm
In our work, we contribute an EM algorithm for estimation of corner points and linear crossing segments for both marked and unmarked pedestrian crosswalks using the detections of pedestrians from processed LiDAR point clouds. We demonstrate the algorithmic performance by analyzing two real-world datasets containing multiple periods of data collection for a four-corner and two-corner intersection. Additionally, we include a Python video tool to visualize the crossing parameter estimation, pedestrian trajectories, and phase intervals in our public source code.
Golam Md Muktadir/ Jim Whitehead
(In Person/Remote)
Adversarial jaywalker modeling for simulation-based testing of Autonomous Vehicle Systems
We present an approach for creating adversarial jaywalkers, autonomous pedestrian models which intentionally act to create unsafe situations involving other vehicles. An adversarial jaywalker employs a hybrid state-model with social forces and state transition rules. The parameters (for social forces and state transitions) of this model are tuned via reinforcement learning to create risky situations faster with synthetic yet plausible behavior. The resulting jaywalkers are capable of realistic behavior while still engaging in sufficiently risky actions to be useful for testing. These adversarial pedestrian models are useful in a wide range of scenario-based tests for autonomous vehicles
1:10 pm

Break

Renran Tian (In Person)
PSI: A Pedestrian Behavior Dataset for Socially Intelligent Autonomous Car
3:55 pm
Interpretable Pedestrian Intent Prediction
Autonomous driving attracts lots of interest in interpretable pedestrian intent models with a desirable capability that mimic human cognition. Such reasoning enables forecasting of the next actions in pedestrian videos. In recent years, various models have been developed based on convolution operations for prediction or forecasting, with exploring the spatiotemporal relationships or mining implicit visual cues. In this talk, we present to discuss the motivation of interpretation, and review several existing proposed model, with further presenting the challenges for autonomous driving. To evaluate the AI models, we proposes and shares another benchmark dataset called the IUPUI-CSRC Pedestrian Situated Intent (PSI) data with two innovative labels besides comprehensive computer vision labels. The first novel label is the dynamic intent changes for the pedestrians to cross in front of the ego-vehicle, achieved from 24 drivers with diverse backgrounds. The second one is the text-based explanations of the driver reasoning process when estimating pedestrian intents and predicting their behaviors during the interaction period.
Prediction of pedestrian behavior is critical for fully autonomous vehicles to drive in busy city streets safely and efficiently. The future autonomous cars need to fit into mixed conditions with not only technical but also social capabilities. As more algorithms and datasets have been developed to predict pedestrian behaviors, these efforts lack the benchmark labels and the capability to estimate the temporal-dynamic intent changes of the pedestrians, provide explanations of the interaction scenes, and support algorithms with social intelligence. This paper proposes and shares another benchmark dataset called the IUPUI-CSRC Pedestrian Situated Intent (PSI) data with two innovative labels besides comprehensive computer vision labels. The first novel label is the dynamic intent changes for the pedestrians to cross in front of the ego-vehicle, achieved from 24 drivers with diverse backgrounds. The second one is the text-based explanations of the driver reasoning process when estimating pedestrian intents and predicting their behaviors during the interaction period. These innovative labels can enable several computer vision tasks, including pedestrian intent/behavior prediction, vehicle-pedestrian interaction segmentation, and video-to-language mapping for explainable algorithms. The released dataset can fundamentally improve the development of pedestrian behavior prediction models and develop socially intelligent autonomous cars to interact with pedestrians efficiently. The dataset has been evaluated with different tasks and is released to the public to access.
3:20 pm

Section II: Discussions

All the participants
5:00 pm
Wrap Up
4:30 pm

Speakers

Invited Speaker

Mohan Trivedi

University of California – San Diego
Website

Short bio: Mohan Trivedi received his PhD in Electrical Engineering from Utah State University in 1979, after completing undergraduate work in India. He has published extensively and has edited over a dozen volumes including books, special issues, video presentations, and conference proceedings. Trivedi is a recipient of the Pioneer Award and the Meritorious Service Award from the IEEE ComputerSociety; and the Distinguished Alumnus Award from Utah State University. He isa Fellow of the International Society for Optical Engineering (SPIE). He is a founding member of the Executive Committee of the UC System-wide Digital MediaInnovation Program (DiMI). Trivedi is also Editor-in-Chief of Machine Vision& Applications (http://link.springer-ny.com/link/service/journals/00138/index.htm).

Workshop Organizers

Josh Domeyer

Toyota Motor North America
Website

Short bio: Josh Domeyer is a senior researcher in the Toyota Collaborative Safety Research Center. He received a Ph.D. in Industrial and Systems Engineering at the University of Wisconsin-Madison in 2021 and B.S. and M.S. degrees in Experimental Psychology from Central Michigan University in 2009 and 2011. He has been a human factors researcher at Toyota since 2011 where his research has focused on driver-vehicle interface, distraction, and human-automation interaction. He joined the Collaborative Safety Research Center in 2017 to explore the communication between drivers and pedestrians with the goal of informing the development of vehicle automation. He is the Chair of SAE’s Safety and Human Factors Steering Committee and a U.S. expert for the ISO TC22/SC39/WG8 standards group.

Amir Rasouli

Huawei Technologies Canada
Website

Short bio: Amir Rasouli is a senior research scientist leading the behavior prediction team atNoah’s Ark Laboratory, Huawei, Canada. He received B.Eng. degree in computer systems engineering and B.A. in Business Management from the Royal MelbourneInstitute of Technology. He obtained M.A.Sc. degree in computer engineering under the supervision of Prof. John Tsotsos at York University studying the role of attention in active visual object search for mobile robots. He completed his Ph.D. degree in the same lab on the topic of pedestrian behavior understanding and prediction in the context of autonomous driving. His current work focuses on various aspects of road user behavior prediction including analysis of naturalistic driving data, understanding theoretical foundations of interactions between heterogeneous traffic participants, modeling vehicle and pedestrian behavior, simulation, and development of predictive models for intelligent driving systems.

Zhengming (Allan) Ding

Tulane University
Website

Short bio: Dr.Ding is a Tenure-Track Assistant Professor at the Department of ComputerScience, Tulane University. He is an active researcher in the fields of face recognition, transfer learning, multi-view learning, and has published over 50 articles in the flagship conferences/journals and books. He serves as Associate Editor of IET ImageProcessing, and Journal of Electronic Imaging. He also serves many leading conferences/workshops in the vision and data mining fields, e.g., the Publicity Chair of AMFG2017, program co-chair of IEEE Big Data Workshop 2017, and workshop Chairs ofAMFG 2021.He organizes four tutorials on multi-view learning in the FG 2017, CVPR 2018,IEEE BigData 2018, AAAI 2019 and IJCAI 2020. He also serves as senior program committee for AAAI 2019/2020, IJCAI 2021, and program committee for CVPR2018-2021, FG 2017-2019, and reviewers of many prestigious journals includingIEEE TPAMI, TNNLS, TKDE, TIP, TMM.

Renran Tian

Indiana University Purdue University Indianapolis
Website

Short bio: Dr. Renran Tian is an assistant professor in the Department of Computer Information & Graphics Technology and the Transportation & Autonomous Systems Institute (TASI) at Indiana University-Purdue University Indianapolis (IUPUI). He received his Ph.D. degree from the School of Industrial Engineering at Purdue University –West Lafayette in 2013, and B.S. and M.S. in Mechanical
Engineering from Tsinghua University – Beijing, China in 2002 and 2005. His research interests include human-centered computing, cognitive ergonomic, computational behavior analysis, human-AI interaction, and autonomous driving. He serves as the co-chair for Technical Activities Committee on Human Factors in Intelligent Transportation Systems (HFITS), IEEE Intelligent Transportation Systems Society.