Navigation System Applying Augmented Reality

• US 20170116783A1
• Filed: 09/06/2016
• Published: 04/27/2017
• Est. Priority Date: 10/26/2015
• Status: Active Grant

• Current Assignee: National Yunlin University of Science and Technology
• Original Assignee: Institute Of Nuclear Energy Research Atomic Energy Council Executive Yuan, National Yunlin University of Science and Technology
• Inventors: Huang, Chung-Hao, Lee, Lun-Hui, Ho, Chian-Cheng, Wang, Bo-Kai
• Granted Patent: US 9,953,461 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/18 Stabilised platforms, e.g. by gyroscope
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/36 Indoor scenes

Abstract

A navigation system includes a measurement unit that outputs a positioning signal associated with positioning coordinates, a navigation unit that obtains guidance indication and that outputs a navigation signal, and an image unit that includes a capturing module capturing a series of planar images, a computation module and an augmented reality module. The computation module generates an optical flow signal for obtaining the positioning coordinates, and further outputs a registration signal. The augmented reality module performs image registration processing based on the navigation signal and the registration signal, so that the guidance indication is superimposed on the series of planar images of real-world scene for subsequent display.

Claims

1. A navigation system which applies augmented reality, comprising:

a measurement unit including a positioning module that is configured to output a positioning signal associated with positioning coordinates of said navigation system;

a navigation unit coupled to said measurement unit, and configured to perform a path planning algorithm according to the positioning signal, so as to obtain guidance indication for guidance of a user of said navigation system, said navigation unit being further configured to output a navigation signal associated with the guidance indication; and

an image unit coupled to said measurement unit and said navigation unit, and including

a capturing module that is configured to capture a series of planar images of real-world scene,

a computation module that is configured to perform an optical flow analysis upon the series of planar images so as to generate an optical flow signal associated with a result of performance of the optical flow analysis and used to obtain the positioning coordinates, said computation module being further configured to perform plane detection processing upon each planar image of the series of planar images so as to output a registration signal associated with a result of performance of the plane detection processing, and

an augmented reality module that is configured to receive the navigation signal and the registration signal, and to perform image registration processing based on the navigation signal and the registration signal, so that the guidance indication is superimposed on the series of planar images of real-world scene for subsequent display.

2. The navigation system of claim 1, wherein to realize the optical flow analysis for generation of the optical flow signal, said computation module is further configured to:

obtain a plurality of analysis features for each planar image of the series of planar images, and recording pixel coordinates for each of the analysis features;

for each planar image, calculate a sum of x-coordinates of the analysis features and a sum of y-coordinates of the analysis features; and

subtract the sum of x-coordinates of the analysis features on a current planar image of the series of planar images from the sum of x-coordinates of the analysis features on a previous planar image of the series of planar images, divide a result of subtraction of the sums of x-coordinates over a number of the analysis features, subtract the sum of y-coordinates of the analysis features on the current planar image of the series of planar images from the sum of y-coordinates of the analysis features on the previous planar image of the series of planar images, and divide a result of subtraction of the sums of y-coordinates over the number of the analysis features, so as to generate the optical flow signal.

3. The navigation system of claim 1, wherein to realize the plane detection processing for outputting the registration signal, said computation module is further configured to:

convert each planar image into a grayscale image;

perform an algorithm associated with Wiener deconvolution for enhancing edges in the grayscale image;

detect the edges thus enhanced by using a Canny detector;

connect the edges in a same straight line using the Hough transform;

determine if each of the edges is a wall-floor boundary based on a position and a direction in which the edge is located in the grayscale image; and

define a floor plane in the spatial environment based on pairwise wall-floor boundary so as to output the registration signal associated with the floor plane.

4. The navigation system of claim 1,

wherein said measurement unit further includes a posture module which is configured to generate a posture signal associated with angles of rotation of said posture module;

and

wherein said image unit further includes a display interface, and to realize the image registration processing, said augmented reality module is further configured to

generate, according to the posture signal and the registration signal, a mapping matrix which is associated with a mapping relationship between different coordinate systems, and

perform an algorithm associated with Affine transformation upon the guidance indication by using the mapping matrix thus generated so as to superimpose the guidance indication on the series of planar images of real-world scene for subsequent display on said display interface.

5. The navigation system of claim 1,

wherein said measurement unit stores map data associated with the spatial environment;

wherein said computation module is configured to determine a displacement vector associated with displacement of said capturing module according to the optical flow signal;

wherein said positioning module is configured to perform, according to the displacement vector determined by said computation module, position estimation processing with reference to a coordinate system of the map data, so as to output the positioning signal associated with the positioning coordinates of said navigation system located at said coordinate system of the map data.

6. The navigation system of claim 5, wherein to realize the position estimation processing for outputting the positioning signal, said positioning module is further configured to:

obtain starting coordinates of said navigation system by detecting a positioning landmark in the spatial environment; and

obtain estimated coordinates by mapping the displacement vector to said coordinate system of the map data, and by adding the displacement vector thus mapped to the starting coordinates, so as to output the positioning signal which is associated with the estimated coordinates, the estimated coordinates serving as the positioning coordinates.

7. The navigation system of claim 6, wherein said positioning module is further configured to:

determine if there exist the estimated coordinates;

when it is determined that there do not exist the estimated coordinates, obtain the starting coordinates by detecting the positioning landmark; and

when it is determined that there exist the estimated coordinates, obtain updated estimated coordinates by adding the displacement vector, which is determined by said computation module according to the optical flow signal, to the estimated coordinates, and by mapping a result of the addition to said coordinate system of the map data, so as to continuously update the estimated coordinates and to output the positioning signal associated with the estimated coordinates.

8. The navigation system of claim 5, wherein to realize the path planning algorithm for obtaining the guidance indication, said navigation unit is further configured to:

establish an adjacency list based on the map data;

map a current position, which is associated with the positioning coordinates contained in the positioning signal, and a target position to two nodes of the adjacency list, respectively; and

find a path which has a shortest distance or time between the two nodes according to the Dijkstra'"'"'s algorithm, so as to obtain the guidance indication associated with the path thus found.

9. The navigation system of claim 8, wherein the guidance indication superimposed on the series of planar images of real-world scene is represented in a manner of at least one of a graphic or a word for guiding a user of said navigation system to move from the current position to the target position along the path thus found.

10. The navigation system of claim 1, wherein said measurement unit further includes a posture module which generates a posture signal associated with angles of rotation of said posture module, said posture module includes a gyroscope which outputs a posture angle signal associated with each axis of a self-defined coordinate system of said gyroscope, and said posture module, based on the posture angle signal and a gravitational acceleration, outputs the posture signal associated with angles of rotation of said posture module at respective axes of a world coordinate system.