Reason-Aware Annotation for Contextualizing Driving Scenarios

A major part of our work is to annotate part of the collected data. We have taken a reason-based approach in annotating driving data. In our annotation system, we annotate the driver’s state, actions, and activities, as well as events in both in-cabin and outside environments. We perform a detailed annotation to be able to describe a driving story based on the annotated data. This is the first work that provides multi-modal data including videos, physiological, and environmental sensing together with labels for every segment of driving. We use simple excel sheet, but we ask human annotators to record every single event with details. 

In our work, for instance, a change lane action is annotated together with the reason behind it. This can then help us understand the interplay of driver’s actions, environment, and reasoning underlying those actions.

So far we have annotated multiple activities and events in our data. This is an on-going work, but the most recent detail of annotation, based on the number of instances annotated from each participant can be seen below.