tion 3 presents the back end of our visual analytics
framework briefly to motivate the application. Sec-
tion 4 presents the steps and methods for trajectory
generation and processing. Section 5 presents the de-
sign of the graphical user interface (GUI). We finally
conclude in Section 6.
2 RELATED WORK
There has been prior research in the field of trajec-
tory analysis for vehicle and pedestrian trajectories,
but only a few of these have developed a visual ana-
lytics framework. For example, Sha et al. (Sha et al.,
2011) used laser data to perform trajectory analysis of
moving objects at an intersection. Xu et al. (Xu et al.,
2015) performed the clustering of the trajectories at
an intersection. While the authors in (Xu et al., 2015)
and (Sha et al., 2011) developed trajectory analysis al-
gorithms, they have not developed a visual analytics
software. In the rest of this section, we present exist-
ing work in trajectory analysis that also have a visual
analytics system.
Dohuki et al. (AL-Dohuki et al., 2019) have de-
veloped an open source visual analytics software, Tra-
jAnalytics, for exploring urban trajectories that per-
forms the tasks of modeling, transformation, and vi-
sualization of urban trajectory data. The goal of the
software is to allow stakeholders to have an under-
standing of the the population mobility data of the
city. A fundamental difference of this work and that
of ours is that in the former, the authors capture and
analyze the whole trajectory of vehicles or humans
through the road network of a city. In our case, we
focus only on the trajectories that appear at traffic in-
tersections.
Kim et al. (Kim et al., 2017) presented a vi-
sual analysis tool for exploring trajectories. Their
visual analytics workflow consists of steps such as
map view, trajectory view, analysis view, and time-
line view. The authors provide a way for the user to
filter areas of interest in the map view, generate the
trajectories of interest, and study the timeline of these
trajectories. This work focuses on entire trajectories,
and, hence, their metrics of interest are different than
ours.
3 BACKGROUND
In this section, we present some preliminary back-
ground of video processing, a fundamental compo-
nent for trajectory generation. Figure 1 demonstrates
the overall pipeline. First, we take the raw fisheye
video as input and perform video processing using
computer vision techniques for detecting and track-
ing road objects (cars, pedestrians, etc.). Then, we
process and cluster trajectories for final visualization
purposes.
3.1 Video Analysis
The raw video that is an input to our software is cap-
tured using fisheye cameras installed at traffic inter-
sections. Compared to an ordinary video camera, a
fisheye camera can capture the whole intersection in
a wide panoramic and non-rectilinear image using its
wide-angled fisheye lens. Fisheye cameras are advan-
tageous because a single camera can capture a com-
plete view of the entire intersection. If the intersec-
tion is large, two fisheye cameras may be installed to
capture the complete intersection.
Processing the video obtained from a fisheye cam-
era enables us to create trajectories of moving objects
at an intersection. A trajectory of a moving object is
its path represented by timestamped location coordi-
nates of the object. For a typical, moderately busy
intersection we studied, the volume of traffic is enor-
mous, with over 10,000 trajectories being generated
on a weekday. Thus, a visualization system is critical
to understand the traffic behavior for a given intersec-
tion for a specific period. For privacy protection, the
information about the moving object is automatically
anonymized by saving only the location coordinates
of objects and, in the cases of vehicles, their size and
color to our database.
Video processing generates frame-by-frame de-
tection and tracking of all the moving objects in an
intersection. It also uses a temporal superpixel (su-
pervoxel) method (Huang et al., 2018) to extract an
accurate mask for object representation. These can
be converted into trajectories that represent the spa-
tial and temporal movement of traffic. A trajectory is
a path traversed by a moving object that is represented
as successive spatial coordinates and corresponding
timestamps. Details of the video processing and anal-
ysis are described briefly below and provided in detail
in (Huang et al., 2020b).
3.2 Description of Database
The trajectories generated by video processing are up-
loaded in a MySQL database on the cloud. SPaT (Sig-
nal Phase and Timing) information is extracted from
high-resolution controller logs and stored is a sepa-
rate database as well. Figure 6 presents the key at-
tributes. The fields of a single row in TrackInfo are -
frame id, identifies the current video frame, track id
A Visual Analytics System for Processed Videos from Traffic Intersections
69