SYSTEM FOR CONTROLLING LIGHT ENABLED DEVICES

• US 20130131836A1
• Filed: 11/21/2011
• Published: 05/23/2013
• Est. Priority Date: 11/21/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Katz, Sagi, Yahav, Giora, Adler, Avishai
• Granted Patent: US 9,628,843 B2
• Time in Patent Office: Days
• US Class Current: 700/12
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/42221 Transmission circuitry, e.g. infrared [IR] or radio frequency [RF]
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

A system for controlling infrared (IR) enabled devices by projecting coded IR pulses from an active illumination depth camera is described. In some embodiments, a gesture recognition system includes an active illumination depth camera such as a depth camera that utilizes time-of-flight (TOF) or structured light techniques for obtaining depth information. The gesture recognition system may detect the performance of a particular gesture associated with a particular electronic device, determine a set of device instructions in response to detecting the particular gesture, and transmit the set of device instructions to the particular electronic device utilizing coded IR pulses. The coded IR pulses may imitate the IR pulses associated with a remote control protocol. In some cases, the coded IR pulses transmitted may also be used by the active illumination depth camera for determining depth information.

Claims

1. A method for controlling one or more electronic devices, comprising:

detecting an end user command associated with the one or more electronic devices;

determining one or more device instructions for communicating with the one or more electronic devices based on the end user command;

determining one or more coded light pulses associated with the one or more device instructions; and

projecting a sequence of light patterns originating from an active illumination depth camera into an environment, the sequence of light patterns includes the one or more coded light pulses associated with the one or more device instructions.

2. The method of claim 1, wherein:

the detecting an end user command includes detecting a particular gesture associated with the one or more electronic devices.

3. The method of claim 1, wherein:

the detecting an end user command includes detecting a particular gesture and determining a particular electronic device of the one or more electronic devices based on a direction of motion associated with the particular gesture.

4. The method of claim 1, further comprising:

generating depth information associated with the environment, the generating depth information includes capturing a first image of the environment associated with a first light pattern of the sequence of light patterns, the first light pattern is associated with a first pulse of the one or more coded light pulses.

5. The method of claim 1, further comprising:

generating depth information associated with the environment, the generating depth information overlaps in time with the projecting a sequence of light patterns, the generating depth information includes capturing one or more images associated with reflections of at least a subset of the sequence of light patterns within the environment.

6. The method of claim 4, further comprising:

detecting a new end user command based on the first image, the detecting a new end user command is performed subsequent to the detecting an end user command, the sequence of light patterns includes one or more new light pulses associated with the new user command.

7. The method of claim 1, wherein:

the determining one or more device instructions includes acquiring a predetermined set of instructions associated with the end user command.

8. The method of claim 1, wherein:

the determining one or more coded light pulses includes acquiring one or more IR transmission protocols associated with the one or more electronic devices.

9. The method of claim 1, wherein:

the determining one or more coded light pulses includes automatically identifying the one or more electronic devices.

10. The method of claim 9, wherein:

the automatically identifying the one or more electronic devices includes transmitting one or more commands associated with a particular electronic device, the one or more commands cause the particular electronic device to generate sound, the automatically identifying the one or more electronic devices includes detecting and localizing sounds within the environment in response to the transmitting one or more commands.

11. The method of claim 1, further comprising:

determining environmental characteristics associated with the environment; and

modifying a light intensity associated with one or more light patterns of the sequence of light patterns based on the environmental characteristics.

12. The method of claim 1, wherein:

the active illumination depth camera utilizes time-of-flight techniques for obtaining depth information;

the one or more coded light pulses correspond with a remote control protocol; and

the one or more device instructions include a first instruction associated with turning on a television, a second instruction associated with setting a volume setting of the television, a third instruction associated with turning on a DVD player, a fourth instruction for instructing the DVD player to play, and a fifth instruction for dimming one or more lights associated with the environment.

13. A system for controlling one or more electronic devices, comprising:

an active illumination depth camera, the active illumination depth camera captures a first image of an environment; and

one or more processors, the one or more processors in communication with the active illumination depth camera, the one or more processors detect an end user command based on at least the first image, the one or more processors determine one or more device instructions for communicating with the one or more electronic devices, the one or more processors direct the active illumination depth camera to project a sequence of light patterns into the environment, the sequence of light patterns includes one or more coded IR pulses associated with the one or more device instructions.

14. The electronic device of claim 13, wherein:

the one or more processors detect the end user command by recognizing a particular gesture associated with the one or more electronic devices.

15. The electronic device of claim 14, wherein:

the active illumination depth camera captures a second image of the environment, the second image is associated with a first light pattern of the sequence of light patterns, the first light pattern is associated with a first pulse of the one or more coded IR pulses, the one or more processors generate depth information associated with the environment based on the second image.

16. One or more storage devices containing processor readable code for programming one or more processors to perform a method for controlling one or more electronic devices comprising the steps of:

detecting a particular gesture associated with the one or more electronic devices;

determining a particular electronic device of the one or more electronic devices based on a direction of motion associated with the particular gesture;

determining one or more instructions for controlling the particular electronic device; and

modulating one or more light patterns projected from an active illumination depth camera, the modulating one or more light patterns generates one or more coded IR pulses associated with the one or more instructions.

17. The one or more storage devices of claim 16, further comprising:

generating depth information associated with the environment, the generating depth information includes capturing a first image of the environment associated with a first light pattern of the one or more light patterns, the first light pattern is associated with a first pulse of the one or more coded IR pulses.

18. The one or more storage devices of claim 17, wherein:

the modulating one or more light patterns includes modulating an amplitude of at least a subset of the one or more light patterns projected from the active illumination depth camera.

19. The one or more storage devices of claim 17, wherein:

the modulating one or more light patterns includes modulating a frequency of the one or more light patterns projected from the active illumination depth camera.

20. The one or more storage devices of claim 17, wherein:

the determining one or more instructions includes automatically identifying the one or more electronic devices, the automatically identifying the one or more electronic devices includes transmitting one or more commands associated with a particular electronic device, the one or more commands cause the particular electronic device to generate sound, the automatically identifying the one or more electronic devices includes detecting and localizing sounds within the environment in response to the transmitting one or more commands.