OBJECT DISPLAY WITH VISUAL VERISIMILITUDE

• US 20140347262A1
• Filed: 05/24/2013
• Published: 11/27/2014
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Paek, Timothy S., Apacible, Johnson
• Granted Patent: US 9,348,411 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2340/0464 Positioning
• CPC Class Codes: G09G 3/20 for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
• CPC Class Codes: G09G 5/32 with means for controlling the display position see provisionally G09G5/42

Abstract

Described herein are technologies relating to display of a representation of an object on a display screen with visual verisimilitude to a viewer. A location of eyes of the viewer relative to a reference point on the display screen is determined. Additionally, a direction of gaze of the eyes of the viewer is determined. Based upon the location and direction of gaze of the eyes of the viewer, the representation of the object can be displayed at a scale and orientation such that it appears with visual verisimilitude to the viewer.

Claims

1. A method performed by a processor of a computing device, the method comprising:

receiving image data for display on a display screen, the image data comprising a representation of an object;

receiving data from at least one sensor, the data being indicative of;

location of eyes of a viewer relative to the display screen; and

direction of gaze of the eyes of the viewer;

processing the image data based upon the data received from the at least one sensor, wherein processing the image data comprises computing a perspective and scale pertaining to the representation of the object based upon the data received from the at least one sensor; and

displaying the representation of the object on the display screen using the perspective and the scale such that from the perspective of the viewer the representation of the object is displayed with visual verisimilitude.

2. The method of claim 1, the image data being video data, and the processing of the image data and the displaying of the representation of the object are performed in real-time.

3. The method of claim 1, the data from the at least one sensor being an image received from a camera that is directed at the viewer, the method further comprising computing the direction of gaze of the eyes of the viewer based upon the image received from the camera.

4. The method of claim 3, wherein computing the direction of gaze of the eyes of the viewer comprises:

identifying a head of the viewer in the image;

computing a tilt of the head of the viewer in the image; and

determining the direction of the gaze of the eyes of the viewer based upon the computing of the tilt of the head of the viewer in the image.

5. The method of claim 3, wherein computing the direction of gaze of the eyes of the viewer comprises:

identifying a first feature and a second feature of a face of the viewer in the image;

computing data indicative of an orientation of the first feature relative to the second feature responsive to identifying the first feature and the second feature of the face of the viewer; and

determining the direction of the gaze of the eyes of the viewer based at least in part upon the data that is indicative of the orientation of the first feature relative to the second feature.

6. The method of claim 3, wherein the display screen is a translucent display screen, and the camera is positioned such that the display screen is between the viewer and the camera.

7. The method of claim 6, the display screen being an organic light emitting diode (OLED) display screen.

8. The method of claim 3, further comprising:

transmitting the video received from the camera to another display screen, the another display screen associated with the image data received for display on the display screen.

9. The method of claim 1, further comprising:

receiving audio data, the audio data corresponding in time to the image data; and

controlling operation of a directional speaker based upon the data from the at least one sensor to cause the directional speaker to output the audio data toward the viewer.

10. The method of claim 1, further comprising:

detecting a gesture set forth by the viewer based upon the data received from the at least one sensor; and

updating content displayed on the display screen based at least in part upon the gesture detected as being set forth by the viewer.

11. The method of claim 1, further comprising:

detecting a voice command set forth by the viewer based upon the data received from the at least one sensor; and

updating content displayed on the display screen based at least in part upon the voice command detected as being set forth by the viewer.

12. The method of claim 1, further comprising:

receiving data from a treadmill upon which the viewer is moving, wherein the processing of the image data is based upon the data received from the treadmill.

13. A system that facilitates presenting imagery on a display screen, the system comprising:

a processor; and

a memory that comprises a plurality of components that are executed by the processor, the plurality of components comprising;

a receiver component that receives an image from a camera, the camera configured to capture images of a viewer of the display screen;

a determiner component that is in communication with the receiver component and computes a location of eyes of the viewer and direction of gaze of the viewer relative to the display screen based upon the image received from the camera;

an image processor component that processes image data based upon the location of the eyes of the viewer and the direction of the gaze of the viewer computed by the determiner component, the image data comprising a representation of an object, wherein processing performed by the image processor component comprises computing an orientation and scale of the representation of the object; and

a display component that displays the representation of the object with the orientation and the scale such that the representation of the object is displayed with visual verisimilitude to the viewer.

14. The system of claim 13, wherein the receiver component receives the image data, the image data comprising a plurality of video frames from a respective plurality of video cameras.

15. The system of claim 13 comprised by a television.

16. The system of claim 13 comprised by a mobile computing device.

17. The system of claim 13, wherein the display screen is a transparent OLED display screen, and wherein the camera is positioned such that the display screen is between the camera and the viewer.

18. The system of claim 17, wherein the receiver component receives images from a respective plurality of cameras, the display screen positioned between the viewer and the plurality of cameras.

19. The system of claim 13, further comprising a transmitter component that transmits the image to another computing device of another viewer to facilitate creation of a portal between the viewer and the another viewer.

20. A display device, comprising:

a transparent OLED display screen;

a camera positioned to capture an image through the OLED display screen;

a memory comprising instructions that, when executed by the processor, cause the processor to perform acts comprising;

receiving the image captured by the camera, the image comprising a viewer of the display device;

computing a location of eyes of the viewer based upon the image captured by the camera;

computing a direction of gaze of the eyes of the viewer based upon the image captured by the camera;

receiving image data that is to be displayed to the viewer on the OLED display screen, the image data comprising a representation of an object;

responsive to computing the location of the eyes of the viewer and the direction of the gaze of the viewer, computing an orientation and a scale of the representation of the object based upon the location of the eyes of the viewer and the direction of the gaze of the viewer, such that when the representation of the object is displayed on the OLED display screen with the orientation and the scale, and when the eyes of the viewer are at the computed location and the gaze of the viewer is in the computed direction, the representation of the object is displayed with visual verisimilitude; and

displaying the object on the OLED display screen with the orientation and the scale until one of the location of the eyes of the viewer alters or the direction of the gaze of the viewer alters.