METHOD AND APPARATUS FOR MARKING A POSITION OF A REAL WORLD OBJECT IN A SEE-THROUGH DISPLAY

• US 20100125812A1
• Filed: 11/17/2008
• Published: 05/20/2010
• Est. Priority Date: 11/17/2008
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Hartman, Randolph G., Buster, Duke
• Granted Patent: US 8,397,181 B2
• Time in Patent Office: Days
• US Class Current: 715/848
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A method for marking a position of a real world object on a see-through display is provided. The method includes capturing an image of a real world object with an imaging device. A viewing angle and a distance to the object are determined. A real world position of the object is calculated based on the viewing angle to the object and the distance to the object. A location on the see-through display that corresponds to the real world position of the object is determined. A mark is then displayed on the see-through display at the location that corresponds to the real world object.

Claims

1. A method for marking a position of a real world object on a see-through display comprising:

capturing an image of a real world object with an imaging device;

determining a viewing angle to the object;

determining a distance to the object;

calculating a real world position of the object based on the viewing angle to the object and the distance to the object;

determining a location on the see-through display that corresponds to the real world position of the object; and

displaying a mark on the see-through display at the location that corresponds to the real world object.

2. The method of claim 1, further comprising:

identifying a pointing device within the field of view of an imaging device;

wherein determining a viewing angle determines along a line from the user to a tip of the pointing device to the object;

determining an orientation for the user; and

wherein calculating the relative position of the object calculates based on the viewing angle, the distance, and the orientation of the user.

3. The method of claim 1, further comprising:

correlating a location of the pointing device relative to the field of view of the imaging device to a location on the see-through display; and

placing a mark on the see-through display at the correlated location on the see-through display.

4. The method of claim 3, further comprising:

updating the correlation of the pointing device relative to the field of view of the imaging device; and

updating the location on the see-through display in which the mark is placed.

5. The method of claim 3, further comprising:

updating at least one of an orientation or a position for the user;

updating a viewing angle for the user, wherein the updated viewing angle is determined along a line from the user to the relative position of the object;

determining an updated location on the see-through display based on the updated viewing angle; and

changing the location of the mark on the see-through display.

6. The method of claim 1, wherein the see-through display located within the field of view of the user such that determining a viewing angle determines an angle from an eye of the user to the location in the see-through display.

7. The method of claim 1, wherein determining a real world position determines a position in terms of an earth coordinate system.

8. A system for marking a position of a real world object on a see-through display comprising:

a processor;

a see-through display communicatively coupled to the processor;

an imaging device communicatively coupled to the processor; and

a ranging device communicatively coupled to the processor;

wherein the imaging device is configured to;

capture an image of a real world object with the imaging device;

wherein the processor is configured to;

determine a viewing angle to the object;

determine a distance to the object;

calculate a real world position of the object based on the viewing angle to the object and the distance to the object;

determine a location on the see-through display that corresponds to the real world position of the object; and

display a mark on the see-through display at the location that corresponds to the real world object.

9. The system of claim 8, wherein the processor is further configured to:

identify a pointing device within the field of view of an imaging device;

wherein determining a viewing angle determines along a line from the user to a tip of the pointing device to the object;

determine an orientation for the user; and

wherein calculating the relative position of the object calculates based on the viewing angle, the distance, and the orientation of the user.

10. The system of claim 8, wherein the processor is further configured to:

correlate location of the pointing device relative to the field of view of the imaging device to a location on the see-through display; and

place a mark on the see-through display at the correlated location on the see-through display.

11. The system of claim 10, wherein the processor is further configured to:

update the correlation of the pointing device relative to the field of view of the imaging device; and

update the location on the see-through display in which the mark is placed.

12. The system of claim 10, wherein the processor is further configured to:

update at least one of an orientation or a position for the user;

update a viewing angle for the user, wherein the updated viewing angle is determined along a line from the user to the relative position of the object;

determine an updated location on the see-through display based on the updated viewing angle; and

change the location of the mark on the see-through display.

13. The system of claim 8, wherein the see-through display located within a field of view of the user such that determining a viewing angle determines an angle from an eye of the user to the location in the see-through display.

14. The system of claim 8, wherein the processor is configured to determine a relative position in terms of an earth coordinate system.

15. A program product comprising a processor-readable medium on which program instructions are embodied, wherein the program instructions are operable to:

capture an image of a real world object with the imaging device;

determine a viewing angle to the object;

determine a distance to the object;

calculate a real world position of the object based on the viewing angle to the object and the distance to the object;

determine a location on the see-through display that corresponds to the real world position of the object; and

display a mark on the see-through display at the location that corresponds to the real world object.

16. The program product of claim 15, wherein the program instructions are further operable to:

identify a pointing device within the field of view of an imaging device;

wherein determining a viewing angle determines along a line from the user to a tip of the pointing device to the object;

determine an orientation for the user; and

wherein calculating the relative position of the object calculates based on the viewing angle, the distance, and the orientation of the user.

17. The program product of claim 15, wherein the program instructions are further operable to:

correlate location of the pointing device relative to the field of view of the imaging device to a location on the see-through display; and

place a mark on the see-through display at the correlated location on the see-through display.

18. The program product of claim 17, wherein the program instructions are further operable to:

update the correlation of the pointing device relative to the field of view of the imaging device; and

update the location on the see-through display in which the mark is placed.

19. The program product of claim 17, wherein the processor is further configured to:

update at least one of an orientation or a position for the user;

update a viewing angle for the user, wherein the updated viewing angle is determined along a line from the user to the relative position of the object;

determine an updated location on the see-through display based on the updated viewing angle; and

change the location of the mark on the see-through display.

20. The program product of claim 15, wherein the program instructions are further operable to:

determine a relative position in terms of an earth coordinate system.

21. A method of determining a distance to a point comprising:

identifying a point within a field of view of an imaging device;

determining an orientation between the imaging device and the point within the field of view of the imaging device based on a position of the point within the field of view of the imaging device;

steering a laser such that a laser beam from the laser propagates at the determined orientation; and

determining a distance from the laser to an object located at the determined orientation.

22. The method of claim 21, wherein determining the orientation includes:

determining an azimuth angle for the point based on the position of the point within the field of view of the imaging device; and

determining the elevation angle for the point based on the position of the point within the field of view of the imaging device.

23. The method of claim 22, wherein steering the laser includes orienting the laser based on the azimuth angle and the elevation angle.

24. The method of claim 21, wherein the point within the field of view of the imaging device corresponds to a real world object within the field of view of the imaging device;

and

wherein steering the laser includes orienting the laser such that a laser beam from the laser is directed towards the real world object.

25. An apparatus comprising:

a processing unit;

a laser coupled to the processing unit, wherein the laser is steerable, such that a laser beam from the laser is capable of being directed towards a particular orientation; and

an imaging device coupled to the processing unit;

wherein the processing unit is configured to;

identify a point within a field of view of an imaging device;

determine an orientation between the imaging device and the point within the field of view of the imaging device based on a position of the point within the field of view of the imaging device;

provide the determined orientation to the laser; and

determine a distance from the laser to an object located at the determined orientation based on information from the laser.

26. The apparatus of claim 25, wherein the processing unit is configured to determine an orientation by being configured to:

determine an azimuth angle for the point based on the position of the point within the field of view of the imaging device;

determine the elevation angle for the point based on the position of the point within the field of view of the imaging device; and

wherein the processing unit is configured to provide the determined orientation to the laser by being configured to provide the azimuth and the elevation to the laser.

27. The apparatus of claim 25, wherein the point within the field of view of the imaging device corresponds to a real world object within the field of view of the imaging device;

and

wherein the laser is oriented such that a laser beam from the laser is directed towards the real world object.