System and method for linking real-world objects and object representations by pointing
First Claim
1. A computer-implemented method for determining a likelihood an object in a real-world scene is the object being pointed at by a mobile device that includes capabilities of a pointing device, comprising the steps of:
- (A) pointing a mobile electronic device that includes capabilities of a pointing device (mobile device) at an object of interest in the real-world scene;
(B) generating with the mobile device a mobile device geodetic position and pointing direction, and transmitting the mobile device geodetic position and pointing direction, and standard deviations relating to generating the mobile device geodetic position and pointing direction to a system server;
(C) mapping a three-dimensional representation of a real-world scene containing the objects in the real-world scene, including the object of interest, and transmitting the three-dimensional representation to the system server;
(D) a system server performing the substeps of,(1) generating a two-dimensional digital representation of the three-dimensional scene mapped at step (C), with the two-dimensional digital representation including at least a digital representation of the objects in the real-world scene, including object of interest,(2) determining for each object, including the object of interest, in the two-dimensional digital representation a distance estimate from the mobile device geodetic position to each such object, including the object of interest,(3) determining for each object, including the object of interest, in the two-dimensional digital representation an accuracy of actual pointing and in determining the accuracy of pointing according to substep (D)(3) using standard deviations transmitted at step (B) caused by the mobile device in generating the mobile device geodetic position and pointing direction and the distance estimate for each object determined at substep (D)(2),(4) defining for each object, including the object interest, in the two-dimensional digital representation an optimal pointing location relating to a visible part of each object, including the object of interest,(5) determining for each object, including the object of interest, in the two-dimensional digital representation a probability surface for optimal pointing from the mobile device geodetic position applying the definition for each object defined at substep (D)(4),(6) determining for each object, including the object of interest, in the two-dimensional digital representation, a numeric correspondence between actual pointing accuracy and optimal pointing location by comparing a standard deviation for the actual pointing accuracy determined at substep (D)(3) and the probability surface for optimal pointing determined at substep (D)(5),(7) selecting as the object of interest the object with the highest numeric correspondence determined at substep (D)(6), and(8) the system server communicating to the mobile device an identity of the object of interest.
3 Assignments
0 Petitions
Accused Products
Abstract
A system and method are described for selecting and identifying a unique object or feature in the system user'"'"'s three-dimensional (“3-D”) environment in a two-dimensional (“2-D”) virtual representation of the same object or feature in a virtual environment. The system and method may be incorporated in a mobile device that includes position and orientation sensors to determine the pointing device'"'"'s position and pointing direction. The mobile device incorporating the present invention may be adapted for wireless communication with a computer-based system that represents static and dynamic objects and features that exist or are present in the system user'"'"'s 3-D environment. The mobile device incorporating the present invention will also have the capability to process information regarding a system user'"'"'s environment and calculating specific measures for pointing accuracy and reliability.
-
Citations
22 Claims
-
1. A computer-implemented method for determining a likelihood an object in a real-world scene is the object being pointed at by a mobile device that includes capabilities of a pointing device, comprising the steps of:
-
(A) pointing a mobile electronic device that includes capabilities of a pointing device (mobile device) at an object of interest in the real-world scene; (B) generating with the mobile device a mobile device geodetic position and pointing direction, and transmitting the mobile device geodetic position and pointing direction, and standard deviations relating to generating the mobile device geodetic position and pointing direction to a system server; (C) mapping a three-dimensional representation of a real-world scene containing the objects in the real-world scene, including the object of interest, and transmitting the three-dimensional representation to the system server; (D) a system server performing the substeps of, (1) generating a two-dimensional digital representation of the three-dimensional scene mapped at step (C), with the two-dimensional digital representation including at least a digital representation of the objects in the real-world scene, including object of interest, (2) determining for each object, including the object of interest, in the two-dimensional digital representation a distance estimate from the mobile device geodetic position to each such object, including the object of interest, (3) determining for each object, including the object of interest, in the two-dimensional digital representation an accuracy of actual pointing and in determining the accuracy of pointing according to substep (D)(3) using standard deviations transmitted at step (B) caused by the mobile device in generating the mobile device geodetic position and pointing direction and the distance estimate for each object determined at substep (D)(2), (4) defining for each object, including the object interest, in the two-dimensional digital representation an optimal pointing location relating to a visible part of each object, including the object of interest, (5) determining for each object, including the object of interest, in the two-dimensional digital representation a probability surface for optimal pointing from the mobile device geodetic position applying the definition for each object defined at substep (D)(4), (6) determining for each object, including the object of interest, in the two-dimensional digital representation, a numeric correspondence between actual pointing accuracy and optimal pointing location by comparing a standard deviation for the actual pointing accuracy determined at substep (D)(3) and the probability surface for optimal pointing determined at substep (D)(5), (7) selecting as the object of interest the object with the highest numeric correspondence determined at substep (D)(6), and (8) the system server communicating to the mobile device an identity of the object of interest. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21)
-
-
16. A system for determining a likelihood an object in a real-world scene is the object being pointed at by a mobile device that includes capabilities of a pointing device, comprising:
-
a mobile device that includes the capabilities of a pointing device (mobile device) for generating a geodetic position and pointing direction for the mobile device, and transmitting the mobile device geodetic position and pointing direction, and standard deviations relating to generating the mobile device geodetic position and pointing direction to a system server; and a system server further comprising, a mapping module for receiving and processing a three-dimensional representation of an environment that contains a real-world scene with objects, including the object of interest, a scene generator module that connects to the mapping module and receives an output from the mapping module that generates a two-dimensional digital representation of the real-world scene including objects, including the object of interest, and determines for each object, including the object of interest, in the two-dimensional digital representation a distance estimate from the mobile device geodetic position to each such object, including the object of interest, a pointing accuracy module that determines for each object, including the object of interest, in the two-dimensional digital representation an accuracy of actual pointing and in determining accuracy of pointing including the standard deviations transmitted from the mobile device caused by the mobile device in generating the mobile device geodetic position and pointing direction, and the distance estimate, optimal pointing module that defines for each object, including the object interest, in the two-dimensional digital representation an optimal pointing location relating to a visible part of each object, including the object of interest, and for each object, including the object of interest, in the two-dimensional digital representation a probability surface for optimal pointing from the mobile device geodetic position to such objects, including the object of interest, a comparison module that determines for each object, including the object of interest, in the two-dimensional digital representation, a numeric correspondence between actual pointing accuracy and optimal pointing location by comparing a standard deviation for actual pointing accuracy and a probability surface for optimal pointing for each such object, including the object of interest, and that selects as the object of interest the object with the highest numeric correspondence according to a comparison by the comparison module, and a communications module that communicates with the mobile device with the communications including transmitting an identity of the object of interest to the mobile device. - View Dependent Claims (22)
-
Specification