3 and 4. The results of the Intel RealSense reveal that
in no frames markers were detected. The reason for
the very poor detection performance based on the data
provided by the Intel RealSense can be seen in Table
4: Due to the reduced illumination in the room also
the infrared light by external emitters is reduced, so
that the structured-light dominates in the IR-images.
That is the reason why the markers are pictured with
bright dots. As the detection algorithm is based on
the gradient image, the necessary edges for extracting
the marker are not calculated appropriately. Accord-
ing to the expectations the RGB-image based detec-
tion does not work fine with low illumination. But
interestingly the Kinect still detects the markers since
a small light emitter was placed in the room. Conse-
quently, the exposure adjustment of the Kinect sup-
ports the marker detection. The Kinect shows a rea-
sonable performance with the Retro 1 marker regard-
ing the IR-image. The detection behaviour with IR-
images of the paper printed marker in the underex-
posed room is similar to the results described in Table
1. The detection of the Retro 2 marker is due to the
already explained effects of the reflectance not accu-
rate.
Conclusively the best marker detection performance
can be achieved with the Kinect camera and the
Retro 1 marker. Due to the very accurate detection
with daylight and the reasonable behaviour on the IR-
image data a parallel detection in both images and a
following merging step stabilizes the detection due to
redundancy.
5 CONCLUSION
We presented an augmented reality marker based on
ArUco markers that can be detected in RGB- and
IR-images. The usage of retroreflective material im-
proves the marker detection in IR-images remarkably.
Due to active perception by IR-capable camera sys-
tems the negative impact of external disturbances like
change of light conditions on the marker detection is
minimized. As different retroreflective materials in-
fluence the image quality depending on the camera
system, we also examined two different retroreflec-
tive materials and compared the performance of the
Kinect V2 and the Intel RealSense D435 regarding
the detection probability depending on the distance of
the augmented reality marker to the camera. Accord-
ing to our results the Kinect together with the micro-
prisms based material for the markers shows the best
detection performance. Since the marker detection is
parallel processed on the IR- and RGB-image, it is
stabilized due to redundant sensor fusion.
REFERENCES
Broggi, A. and Dickmanns, E. D. (2000). Applications of
computer vision to intelligent vehicles. Image and Vi-
sion Computing, 5(18):365–366.
Dorfm
¨
uller, K. and Wirth, H. (1998). Real-time hand
and head tracking for virtual environments using in-
frared beacons. In Modelling and Motion Capture
Techniques for Virtual Environments, pages 113–127.
Springer.
Draelos, M., Qiu, Q., Bronstein, A. M., and Sapiro, G.
(2015). Intel realsense= real low cost gaze. In ICIP,
pages 2520–2524.
Fankhauser, P., Bloesch, M., Rodriguez, D., Kaestner, R.,
Hutter, M., and Siegwart, R. Y. (2015). Kinect v2 for
mobile robot navigation: Evaluation and modeling. In
2015 International Conference on Advanced Robotics
(ICAR), pages 388–394. IEEE.
Fiala, M. (2010). Designing highly reliable fiducial mark-
ers. IEEE Transactions on Pattern analysis and ma-
chine intelligence, 32(7):1317–1324.
Garrido-Jurado, S., Mu
˜
noz-Salinas, R., Madrid-Cuevas,
F. J., and Mar
´
ın-Jim
´
enez, M. J. (2014). Auto-
matic generation and detection of highly reliable fidu-
cial markers under occlusion. Pattern Recognition,
47(6):2280–2292.
Kato, H. and Billinghurst, M. (1999). Marker tracking
and hmd calibration for a video-based augmented re-
ality conferencing system. In Augmented Reality,
1999.(IWAR’99) Proceedings. 2nd IEEE and ACM In-
ternational Workshop on, pages 85–94. IEEE.
Munoz-Salinas, R. (2012). Aruco: a minimal library for
augmented reality applications based on opencv. Uni-
versidad de C
´
ordoba.
Naimark, L. and Foxlin, E. (2002). Circular data ma-
trix fiducial system and robust image processing for
a wearable vision-inertial self-tracker. In Proceedings
of the 1st International Symposium on Mixed and Aug-
mented Reality, page 27. IEEE Computer Society.
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T.,
Leibs, J., Wheeler, R., and Ng, A. Y. (2009). Ros: an
open-source robot operating system. In ICRA work-
shop on open source software, volume 3, page 5.
Kobe, Japan.
Rossner, H.-C. (2014). Retro-reflective marker. US Patent
8,915,599.
Schoenborn, D. and Schulte, W. (1995). RSA Handbuch-
sicherung von Arbeitsstellen an Strassen, Band 1:
RSA mit Kommentar.
Sim, R. and Little, J. J. (2006). Autonomous vision-based
exploration and mapping using hybrid maps and rao-
blackwellised particle filters. In Intelligent Robots and
Systems, 2006 IEEE/RSJ International Conference on,
pages 2082–2089. IEEE.
Zhang, Z. (2012). Microsoft kinect sensor and its effect.
IEEE multimedia, 19(2):4–10.
An Approach to Marker Detection in IR- and RGB-images for an Augmented Reality Marker
197