Method and apparatus to facilitate visual augmentation of perceived reality

• US 20060262140A1
• Filed: 05/18/2005
• Published: 11/23/2006
• Est. Priority Date: 05/18/2005
• Status: Abandoned Application

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Motorola Solutions, Inc.
• Inventors: Lagunas, Abelardo Lopez, Ahmed, Mohamed Imtiaz, Bellas, Nikos, Chai, Sek M., Kujawa, Gregory A., Lee, King F.
• Application Number:
  US11/132,124
• Publication Number:
  US 20060262140A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/1529 Head-up displays
• CPC Class Codes: B60K 2370/177 Augmented reality
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A visual reality augmentation apparatus (300) comprises one or more (substantially) real time reality context input stages (301, 302) that provide corresponding reality context information to a reality content detector (303). The latter provides detected object information to an augmented reality content display (304) that provides augmentation information (via, for example, projection display techniques) to augment the real world scene being viewed by a viewer. In a preferred approach a direction-of-gaze detector (305) detects the viewer'"'"'s gaze direction. That information then serves to facilitate positional synchronization of the augmentation information to the viewer'"'"'s point of view of the corresponding real world information.

Claims

1. A method comprising:

capturing, substantially in real time, information regarding a given reality context within a given field of view;

processing, substantially in real time, the information regarding a given reality context to provide detected reality content for the given field of view;

using, substantially in real time, the detected reality content for the given field of view to provide visually perceivable reality content augmentation to a person viewing the given field of view wherein the visually perceivable reality content augmentation is positionally visually synchronized with respect to at least one element of the given reality context.

2. The method of claim 1 wherein the given field of view comprises at least one of:

a forward-looking view as corresponds to a vehicle operator'"'"'s view while operating a vehicle;

a rearward-looking view as corresponds to a vehicle operator'"'"'s view while operating a vehicle;

a mirrored view as corresponds to a vehicle operator'"'"'s view while operating a vehicle.

3. The method of claim 1 wherein capturing, substantially in real time, information regarding a given reality context within a given field of view comprises capturing the information using at least one camera.

4. The method of claim 1 further comprising:

capturing, substantially in real time, information regarding a viewer'"'"'s present gaze direction with respect to the given field of view;

and wherein using, substantially in real time, the detected reality content for the given field of view to provide visually perceivable reality content augmentation to a person viewing the given field of view wherein the visually perceivable reality content augmentation is positionally visually synchronized with respect to at least one element of the given reality context comprises using the viewer'"'"'s present gaze direction with respect to the given field of view in conjunction with the detected reality content for the given field of view to achieve visual positional synchronization between the given reality context as viewed by the viewer and the visually perceivable reality content augmentation.

5. The method of claim 4 wherein achieving the visual positional synchronization comprises at least one of:

translating;

rotating; and

skewing;

the visually perceivable reality content augmentation based on at least one of;

gaze directionality as pertains to the person;

eye position of the person;

head position of the person; and

a distance from at least one eye of the person to a display of the visually perceivable reality content augmentation.

6. The method of claim 1 wherein processing, substantially in real time, the information regarding a given reality context to provide detected reality content for the given field of view comprises processing the information regarding a given reality context to detect at least one of:

object edges;

object shape;

object distance;

relative position of objects;

textual information;

object recognition;

at least one color;

a temporally dynamic object.

7. The method of claim 1 wherein providing visually perceivable reality content augmentation to a person viewing the given field of view comprises providing a display of the visually perceivable reality content augmentation.

8. The method of claim 7 wherein providing a display of the visually perceivable reality content augmentation comprises providing the display on at least one of:

a substantially transparent surface; and

a mirror.

9. The method of claim 8 wherein providing the display on a substantially transparent surface comprises projecting the display on the substantially transparent surface.

10. The method of claim 9 wherein the substantially transparent surface comprises at least one of:

a vehicle operator'"'"'s windscreen;

corrective lens eyewear;

sunglasses.

11. The method of claim 1 further comprising:

automatically controlling provision of the visually perceivable reality content augmentation to a person viewing the given field of view as a function, at least in part, of;

a level of confidence with respect to likely accuracy of the detected reality content for the given field of view;

distance to a detected object;

a personal preference of the person;

the person'"'"'s level of experience with respect to a particular activity;

the person'"'"'s level of skill with respect to a particular activity;

the person'"'"'s age;

an object'"'"'s occlusion;

at least one environmental condition.

12. The method of claim 1 wherein providing the visually perceivable reality content augmentation to a person viewing the given field of view further comprises using color to visually augment at least one real object in the given field of view.

13. The method of claim 12 wherein using color to visually augment at least one real object in the given field of view further comprises selecting from a plurality of candidate colors to provide a selected color to use when visually augmenting the at least one real object in the given field of view.

14. The method of claim 1 wherein providing the visually perceivable reality content augmentation to a person viewing the given field of view further comprises using at least one of:

a line;

a curve;

a two-dimensional shape;

text;

to visually augment at least one real object in the given field of view.

15. The method of claim 1 wherein providing the visually perceivable reality content augmentation to a person viewing the given field of view further comprises using at least one of:

a blinking property;

a selectively variable opaqueness property;

to visually augment at least one real object in the given field of view.

16. A visual reality augmentation apparatus comprising:

a substantially real time reality context input stage having a field of view input and a captured reality context information output;

a substantially real time reality content detector having an input operably coupled to the captured reality content information output of the substantially real time reality context input stage and having a detected content output;

a substantially real time and substantially transparent augmented reality content display responsive to the detected content output of the reality content detector wherein at least one real object within a field of view as corresponds to the field of view input appears visually augmented by a positionally synchronized augmentation element when view by a viewer.

17. The visual reality augmentation apparatus of claim 16 wherein the substantially real time reality context input stage comprises at least one camera.

18. The visual reality augmentation apparatus of claim 16 further comprising:

a viewer'"'"'s present direction-of-gaze detector;

and wherein the substantially real time and substantially transparent augmented reality content display is further responsive to the viewer'"'"'s present direction-of-gaze detector.

19. The visual reality augmentation apparatus of claim 16 wherein the substantially real time and substantially transparent augmented reality content display further comprises means for positionally synchronizing the at least one real object within the field of view with the augmentation element as a function, at least in part, of at least one of:

the viewer'"'"'s gaze direction;

a relative position of a viewer'"'"'s eyes with respect to the substantially transparent augmented reality content display.

20. The visual reality augmentation apparatus of claim 16 wherein the substantially real time and substantially transparent augmented reality content display further comprises a vehicle operator'"'"'s windscreen.