Mobile Augmented Reality

Computer Vision & Geovisualization

Project Name: Research on Spatial Information Visualization and Multi-modal Interaction Mechanism Based on Outdoor Augmented Reality

Funding Source

National Natural Science Foundation of China
Project No. 41371427
My Role

Core Member
(October 2016 - June 2019)
My Contribution

Data Collection & Preprocessing
Deep Neural Network Model
Android Development
Paper Writing

Reference:

Rao, J.✉, Gao, S.*, Kang, Y. and Du, Q. (2020). Landmarks as Beacons: Pedestrian Navigation Based on Landmark Detection and Mobile Augmented Reality. AutoCarto 2020.

Rao, J., Qiao, Y., Ren, F., Wang, J. and Du, Q.*✉ (2017). A Mobile Outdoor Augmented Reality Method Combining Deep Learning Object Detection and Spatial Relationships for Geovisualization. Sensors, 17(9), 1951.

Qiao, Y., Rao, J., Wang, J., Du, Q.*✉ and Ren, F. (2017). Geographic Object Detection for Outdoor Augmented Reality. Geomatics World, 5, 011.

Introduction

The purpose of this study was to develop a robust, fast and markerless mobile augmented reality method for registration, geovisualization, and interaction in uncontrolled outdoor environments. We propose a lightweight deep-learning-based object detection approach for mobile or embedded devices; the vision-based detection results of this approach are combined with spatial relationships by means of the host device’s built-in Global Positioning System receiver, Inertial Measurement Unit, and magnetometer. Virtual objects generated based on geospatial information are precisely registered in the real world, and an interaction method based on touch gestures is implemented. The entire method is independent of the network to ensure robustness to poor signal conditions.

A mobile prototype system based on Android was developed and tested on the Wuhan University campus to evaluate the method and validate its results. The findings demonstrate that our method achieves a high detection accuracy, stable geovisualization results and interaction. Lately, we experimentally integrated a high-speed tracking algorithm (Kernelized Correlation Filters, KCF) in the latest system, and it can run at approximately 10 FPS currently.

UPDATE: Recently, we proposed an augmented reality pedestrian navigation method that uses landmarks as beacons to guide users’ wayfinding in urban areas. Specifically, landmarks are detected using vision-based models and then augmented by virtual arrows and panels with navigation information.


Back to Home Page

Selected Works

Recent research works and projects

Stay-at-Home Order Effects

COVID-19 & Mobile Phone Location Data

Association of mobility data indications of travel and stay-at-Home mandates with COVID-19 infection rates in the US.

Read Me

Trajectory Privacy Protection

Deep Learning & Geo-Privacy Protection

LSTM-TrajGAN: A deep learning method combining LSTM and generative adversarial network for trajectory privacy protection.

Read Me

Mapping Mobility Changes

ESRI ArcGIS Online Dashboard

Mapping county-level mobility pattern changes in the United States in response to COVID-19.

Read Me

Mobile Augmented Reality

Computer Vision & Geovisualization

Focus on geovisualization and human-computer interaction through computer vision methods.

Read Me

Geospatial Data Processing

Python & Parallel Data Processing

Using Python and plenty of open source libraries to achieve parallel geospatial data processing.

Read Me

More Projects

More Research Works and Projects.

More projects, including research works, thesis projects, course projects, and personal projects.

Read Me