Power consumption in motion-capture systems

• US 10,609,285 B2
• Filed: 01/07/2014
• Issued: 03/31/2020
• Est. Priority Date: 01/07/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Ultrahaptics Ip Two Limited (Ultraleap Holdings Ltd.)
• Inventors: Holz, David
• Primary Examiner(s): Billah, Masum
• Application Number:
  US14/149,663
• Publication Number:
  US 20140192206A1
• Time in Patent Office: 2,275 Days
• Field of Search: 348155, 348208, 348169
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera

Abstract

The technology disclosed relates to reducing the overall power consumption of motion-capture system without compromising the quality of motion capture and tracking In general, this is accomplished by operating the motion-detecting cameras and associated image-processing hardware in a low-power mode unless and until a moving object is detected. Once an object of interest has been detected in the field of view of the cameras, the motion-capture system is “woken up,” i.e., switched into a high-power mode, in which it acquires and processes images at a frame rate sufficient for accurate motion tracking.

Claims

1. A method of operating a motion-capture system having at least one camera to minimize power consumption while assuring high-quality motion capture to track motions of an object, the method including:

monitoring using an optically sensitive sensor-a region of interest including a field of view of the at least one camera of a motion-capture system;

wherein the region of interest is set as a volume of approximately 1 meter in length and a light source positioned near the camera(s) of the motion-capture system to shine light onto the object thereby enabling objects of interest to be distinguished from background objects using falloff of light intensity with distance;

using the optically sensitive sensor, measuring brightness of light in a field of view of the at least one camera of the motion capture system at a first measurement and a second measurement conducted subsequent to the first measurement;

comparing brightness of light measured using the sensor at the second measurement and brightness of light measured using the sensor at the first measurement to determine a difference in brightness of light measured at first and second measurements;

determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view of the at least one camera of the motion-capture system between the first measurement and the second measurement;

responsive to determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view, (i) detecting subsequent motions of the object using the motion capture system;

(ii) determining speed of the subsequent motions detected by the motion capture system, and (iii) switching the motion-capture system to operate in one of a set of intermediate modes, each intermediate mode having a rate of image capture different from a rate of image capture of other intermediate modes, the intermediate modes arranged between a low-power mode to a high-power mode; and

repeating the (i) detecting, (ii) determining, and (iii) switching until the motion capture system determines that the object has left the region of interest, to reconstruct 3D motion of the object;

thereby enabling the motion capture system to minimize power consumption while assuring high-quality motion capture to track motions of the object.

2. The method of claim 1, further including optically sensitive sensor monitoring environmental reflectivity in the region of interest.

3. The method of claim 2, wherein monitoring the environmental reflectivity further includes blinking a light source and measuring a brightness of a reflection signal using the optically sensitive sensor.

4. The method of claim 3, wherein the light source is turned off, dimmed, or blinked when the at least one camera operates in low-power mode.

5. The method of claim 1, further including setting a threshold so as to be indicative of at least one of presence or movement of an object or person in the region of interest.

6. The method of claim 1, further including setting a threshold so as to be indicative of geometric configuration of the region of interest.

7. The method of claim 1, further including switching the motion capture system into the low-power mode responsive to receiving a user'"'"'s command.

8. The method of claim 1, further including successively decreasing processing of captured images in intermediate modes having successively reduced frame rates.

9. The method of claim 1, further including monitoring brightness in the region of interest at certain time intervals when in lowest power mode.

10. The method of claim 1, further including tracking objects moving in the region of interest using a frame rate of several tens or hundreds of frames per second at a high power mode.

11. The method of claim 1, further including evaluating images captured by the at least one camera in low-power mode for presence or movement of an object of interest dependent on distinguishing object pixels from background pixels by applying a brightness threshold to each image pixel.

12. The method of claim 1, further including filtering out visible light so that only infrared light is registered in images captured by cameras of the motion capture system.

13. The method of claim 1, wherein the low-power mode is a standby or sleep mode during which motion capture does not take place.

14. A non-transitory computer readable medium storing instructions for operating a motion-capture system having at least one camera to minimize power consumption while assuring high-quality motion capture to track motions of an object, the instructions when executed by one or more processors perform:

monitoring using an optically sensitive sensor, a region of interest including a field of view of the at least one camera of a motion-capture system;

wherein the region of interest is set as a volume of approximately 1 meter in length and a light source positioned near the camera(s) of the motion-capture system to shine light onto the object thereby enabling objects of interest to be distinguished from background objects using falloff of light intensity with distance;

using the optically sensitive sensor, measuring brightness of light in a field of view of the at least one camera of the motion capture system at a first measurement and a second measurement conducted subsequent to the first measurement;

comparing brightness of light measured using the sensor at the second measurement and brightness of light measured using the sensor at the first measurement to determine a difference in brightness of light measured at first and second measurements;

determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view of the at least one camera of the motion-capture system between the first measurement and the second measurement;

responsive to determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view, (i) detecting subsequent motions of the object using the motion capture system;

(ii) determining speed of the subsequent motions detected by the motion capture system, and (iii) switching the motion-capture system to operate in one of a set of intermediate modes, each intermediate mode having a rate of image capture different from a rate of image capture of other intermediate modes, the intermediate modes arranged between a low-power mode to a high-power mode; and

repeating the (i) detecting, (ii) determining, and (iii) switching until the motion capture system determines that the object has left the region of interest, to reconstruct 3D motion of the object;

thereby enabling the motion capture system to minimize power consumption while assuring high-quality motion capture to track motions of the object.

15. A motion-capture system, including:

at least one camera for capturing images of an object within a field of view thereof;

an image-analysis module comprising a processor coupled to a memory storing instructions that, when executed by the processor, track motion of the object based on the images and interprets as input a user'"'"'s hand gestures tracked;

an optically sensitive sensor for measuring brightness of light within the field of view of the at least one camera at a first measurement and a second measurement subsequent to the first measurement; and

a control module comprising a processor coupled to a memory storing instructions that, when executed by the processor, perform;

monitoring using the optically sensitive sensor, a region of interest including a field of view of the at least one camera of a motion-capture system;

wherein the region of interest is set as a volume of approximately 1 meter in length and a light source positioned near the camera(s) of the motion-capture system to shine light onto the object thereby enabling objects of interest to be distinguished from background objects using falloff of light intensity with distance;

using the optically sensitive sensor, measuring brightness of light in a field of view of the at least one camera of the motion capture system at a first measurement and a second measurement conducted subsequent to the first measurement;

comparing brightness of light measured using the sensor at the second measurement and brightness of light measured using the sensor at the first measurement to determine a difference in brightness of light measured at first and second measurements;

determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view of the at least one camera of the motion-capture system between the first measurement and the second measurement;

responsive to determining that the difference in brightness of light measured using the sensor at first and second measurements exceeds a predetermined threshold indicating an object has entered the field of view, (i) detecting subsequent motions of the object using the motion capture system;

(ii) determining speed of the subsequent motions detected by the motion capture system, and (iii) switching the motion-capture system to operate in one of a set of intermediate modes, each intermediate mode having a rate of image capture different from a rate of image capture of other intermediate modes, the intermediate modes arranged between a low-power mode to a high-power mode; and

repeating the (i) detecting, (ii) determining, and (iii) switching until the motion capture system determines that the object has left the region of interest, to reconstruct 3D motion of the object;

thereby enabling the motion capture system to minimize power consumption while assuring high-quality motion capture to track motions of the object.