Display with built in 3D sensing capability and gesture control of TV

• US 9,247,236 B2
• Filed: 08/21/2012
• Issued: 01/26/2016
• Est. Priority Date: 03/07/2008
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Intellectual Ventures Holding 81 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew
• Primary Examiner(s): JOSHI, SURAJ M
• Application Number:
  US13/591,054
• Publication Number:
  US 20120317511A1
• Time in Patent Office: 1,253 Days
• Field of Search: 348/51
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

Information from execution of a vision processing module may be used to control a 3D vision system.

Claims

1. A system comprising:

a display; and

one or more hardware processors configured to execute instructions stored on a tangible computer readable medium in order to cause the system to;

access image data of an interactive space;

detect physical characteristics of a user within the interactive space based on the image data;

monitor movement of the user in the interactive space based on the image data;

identify the user based on the detected physical characteristics of the user;

automatically load display settings associated with the identified user, the display settings including an input to provide images to a display;

initiate display of media content on the display according to the display settings associated with the identified user and according to a media adjustment included in the display settings, the media adjustment identified by the movement of the user, the media adjustment identifying an alteration within the media content; and

in response to detecting the user leaving the interactive space, cause the display to enter an energy saving state,

wherein the media content comprises an interactive game, and wherein the media adjustment comprises altering, in response to the monitored movement of the user, a level of difficulty for the interactive game displayed.

2. The system of claim 1, wherein the instructions further cause the system to:

automatically access a user profile associated with the identified user; and

select the media content based on attributes of the user profile.

3. The system of claim 1, further comprising:

an illuminator configured to emit a predetermined pattern of light in the interactive space.

4. The system of claim 3, wherein the instructions further cause the system to:

detect the physical characteristics of the user by detecting textures formed on the user by the predetermined pattern of light.

5. The system of claim 1, wherein the instructions further cause the system to:

identify one or more gestures of the user and generate a control signal for controlling a function of the display based on the identified one or more gestures.

6. The system of claim 5, wherein the control signal comprises a signal to change one or more of a volume, a channel, an input, or a display setting of the display.

7. The system of claim 5, wherein a thumb down gesture of the user initiates a display power off control signal;

and wherein a thumb up gesture of the user initiates a display power on control signal.

8. The system of claim 1, wherein the instructions further cause the system to transmit a control message to an environmental control device, the control message being associated with the identified user.

9. The system of claim 1, further comprising a camera for capturing the image data of the interactive space.

10. The system of claim 1, wherein the media content further comprises a digital avatar, and wherein the media adjustment further comprises altering a position of the digital avatar based on the monitored movement of the user such that the position of the digital avatar maps to a skeletal model of a position of the user.

11. A method comprising:

accessing image data of an interactive space;

detecting physical characteristics of a user within the interactive space based on the image data;

monitoring movement of the user in the interactive space based on the image data;

identifying the user based on the detected physical characteristics of the user;

automatically accessing a user profile associated with the identified user, the user profile including an input to provide images to an output display;

identifying media content associated with the user profile;

initiating playing the media content on the output device based on display settings included in the user profile, the display settings including a media adjustment identified by the movement of the user, said media adjustment identifying an alteration to the media content; and

in response to detecting the user leaving the interactive space, stopping playing of the media content associated with the user profile,

wherein the media content comprises an interactive game, and wherein the media adjustment comprises altering, in response to a monitored movement of the user, a level of difficulty for the interactive game displayed based on the monitored movement of the user.

12. The method of claim 11, wherein said stopping playing of the content comprises pausing the media content.

13. The method of claim 11, wherein the media content comprises one or more of:

a song, a movie or a slideshow.

14. The method of claim 11, further comprising emitting a predetermined pattern of light into the interactive space and wherein said detecting the physical characteristics of the user comprises detecting textures formed on the user by the predetermined pattern of light.

15. The method of claim 11, further comprising:

identifying one or more gestures of the user; and

generating a control signal for controlling a function of the output device based on the identified one or more gestures.

16. The method of claim 15, wherein the control signal comprises a signal to pause, fast forward, or navigate through the media content.

17. A non-transitory computer readable storage having executable instructions stored thereon, the executable instructions configured to cause a computing system having one or more hardware processors to perform operations comprising:

accessing image data of an interactive space;

detecting physical characteristics of a user within the interactive space based on the image data;

monitoring movement of the user in the interactive space based on the image data;

identifying the user based on the detected physical characteristics of the user;

accessing a user profile associated with the identified user, the user profile including an input to provide images to a display;

altering the display based on the data stored in the user profile, said altering including initiating display of media content associated with the user profile on the display based on display settings included in the user profile, the display settings including a media adjustment identified by the movement of the user, said media adjustment identifying an alteration to the media content; and

in response to detecting the user leaving the interactive space, further altering the display, wherein the media content comprises an interactive game, and wherein the media adjustment comprises altering, in response to a monitored movement of the user, a level of difficulty for the interactive game displayed based on the monitored movement of the user.

18. The non-transitory computer readable storage of claim 17, wherein said further altering the display comprises stopping display of the media content.

19. The non-transitory computer readable storage of claim 17, wherein said further altering the display comprises causing the display to enter an energy saving state.

20. The non-transitory computer readable storage of claim 17, wherein the computing system performs operation comprising:

identifying one or more gestures of the user; and

generating a control signal for controlling a function of the display based on the identified one or more gestures.

21. The non-transitory computer readable storage of claim 20, wherein the control signal comprises a signal to pause, fast forward, or navigate through content.

22. The non-transitory computer readable storage of claim 20, wherein the control signal comprises a signal to change one or more of a volume, a channel, the input, or a setting of the display.

23. The non-transitory computer readable storage of claim 17, wherein the computing system performs operation comprising:

initiating emission of a predetermined pattern of light into the interactive space;

wherein said detecting the physical characteristics of the user comprises detecting textures formed on the user by the predetermined pattern of light.