Portable remote control device enabling three-dimensional user interaction with at least one appliance
First Claim
1. A method to enable at least one user interaction made in a three-dimensional hover zone to be recognized by a hand holdable remote control device that creates a detectable event, corresponding to said interaction, useable to remotely control an appliance, the method including the following steps:
- (a) providing said hand holdable device system with an acquisition system that includes at least a first camera have a first FOV to acquire at least a number N of data points of two-dimensional image information of at least a portion of said user in said FOV, each said camera being attached to said remote control device;
wherein said at least first camera has a characteristic selected from a group consisting of (i) said camera is a time-of-flight (TOF) camera that acquires three-dimensional image information of at least a portion of said user, (ii) said first camera includes a structured light camera system, and (iii) said camera is a two-dimensional camera that acquires two-dimensional image information of at least a portion of said user;
(b) processing said at least two-dimensional image information acquired from each said camera at step (a) so as to represent an imaged portion of said user using a set of landmark points sufficient to recognize a gesture made by said user, said processing discarding at least about 90% of said number N data points;
said landmark points being three-dimensional if said acquisition system includes at least one of (i) a TOF camera, (ii) at least two two-dimensional cameras, and (iii) a structure light system; and
said landmark points being two-dimensional if said acquisition system includes only one two-dimensional camera;
wherein said processing has at least one characteristic selected from a group consisting of (i) processing occurs within said remote control device, (ii) processing occurs externally to said remote control device, and (iii) at least a portion of said processing occurs within said remote control device and at least a portion of said processing occurs externally to said remote control device;
(c) translating each user gesture identified in step (b) to a command to control said appliance, wherein said translating has a characteristic selected from a group consisting of (i) translating occurs within said remote control device, (ii) translating occurs externally to said remote control device, and (iii) at least a portion of said translating occurs within said remote control device and at least a portion of said translating occurs externally to said remote control device; and
(d) transmitting each said user gesture translated in step (c) to said appliance to control at least one aspect of said appliance, wherein said transmitting has a characteristic selected from a group consisting of (i) transmitting occurs within said remote control device, (ii) transmitting occurs externally to said remote control device, and (iii) at least a portion of said transmitting occurs within said remote control device and at least a portion of said transmitting occurs externally to said remote control device.
5 Assignments
0 Petitions
Accused Products
Abstract
A portable remote control device enables user interaction with an appliance by detecting user gestures made in a hover zone, and converting the gestures to commands that are wirelessly transmitted to the appliance. The remote control device includes at least two cameras whose intersecting FOVs define a three-dimensional hover zone within which user interactions are imaged. Separately and collectively image data is analyzed to identify a relatively few user landmarks. Substantially unambiguous correspondence is established between the same landmark on each acquired image, and a three-dimensional reconstruction is made in a common coordinate system. Preferably cameras are modeled to have characteristics of pinhole cameras, enabling rectified epipolar geometric analysis to facilitate more rapid disambiguation among potential landmark points. As a result processing overhead and latency times are substantially reduced. Landmark identification and position information is convertible into commands that alter the appliance behavior as intended by the user'"'"'s gesture.
-
Citations
20 Claims
-
1. A method to enable at least one user interaction made in a three-dimensional hover zone to be recognized by a hand holdable remote control device that creates a detectable event, corresponding to said interaction, useable to remotely control an appliance, the method including the following steps:
-
(a) providing said hand holdable device system with an acquisition system that includes at least a first camera have a first FOV to acquire at least a number N of data points of two-dimensional image information of at least a portion of said user in said FOV, each said camera being attached to said remote control device; wherein said at least first camera has a characteristic selected from a group consisting of (i) said camera is a time-of-flight (TOF) camera that acquires three-dimensional image information of at least a portion of said user, (ii) said first camera includes a structured light camera system, and (iii) said camera is a two-dimensional camera that acquires two-dimensional image information of at least a portion of said user; (b) processing said at least two-dimensional image information acquired from each said camera at step (a) so as to represent an imaged portion of said user using a set of landmark points sufficient to recognize a gesture made by said user, said processing discarding at least about 90% of said number N data points; said landmark points being three-dimensional if said acquisition system includes at least one of (i) a TOF camera, (ii) at least two two-dimensional cameras, and (iii) a structure light system; and said landmark points being two-dimensional if said acquisition system includes only one two-dimensional camera; wherein said processing has at least one characteristic selected from a group consisting of (i) processing occurs within said remote control device, (ii) processing occurs externally to said remote control device, and (iii) at least a portion of said processing occurs within said remote control device and at least a portion of said processing occurs externally to said remote control device; (c) translating each user gesture identified in step (b) to a command to control said appliance, wherein said translating has a characteristic selected from a group consisting of (i) translating occurs within said remote control device, (ii) translating occurs externally to said remote control device, and (iii) at least a portion of said translating occurs within said remote control device and at least a portion of said translating occurs externally to said remote control device; and (d) transmitting each said user gesture translated in step (c) to said appliance to control at least one aspect of said appliance, wherein said transmitting has a characteristic selected from a group consisting of (i) transmitting occurs within said remote control device, (ii) transmitting occurs externally to said remote control device, and (iii) at least a portion of said transmitting occurs within said remote control device and at least a portion of said transmitting occurs externally to said remote control device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A handholdable remote control device to enable at least one user interaction, made in a three-dimensional hover zone, with an appliance, said interaction creating a detectable event useable by said appliance, where at least a portion of said user is representable by at least one three dimensional landmark, the handholdable remote control device including:
-
an acquisition system comprising at least a first two-dimensional camera having a first FOV and a first resolution, and a second two-dimensional camera having a second FOV and a second resolution, said first and second camera disposed on said remote control device such an intersection of said first FOV and second FOV defines said three-dimensional hover zone, said acquisition system acquiring at least a number N of data points of two-dimensional image information from at least a portion of said user in said three-dimensional hover zone; means for processing said at least two-dimensional image information acquired from said acquisition system so as to represent an imaged portion of said user using a set of three-dimensional landmark points sufficient to recognize a gesture made by said user, said means for processing discarding at least about 90% of said number N data points; wherein said means for processing has at least one characteristic selected from a group consisting of (i) said means for processing is disposed within said handholdable remote control device, and (ii) at least a portion of said means for processing is disposed external to said handholdable remote control device; means for translating each user gesture recognized by said means for processing to a command to control said appliance, said means for translating disposed within said handholdable remote control device; wherein said means for translating has a characteristic selected from a group consisting of (i) said means for translating is disposed within said handholdable remote control device, (ii) translating occurs externally to said remote control device, and (iii) at least a portion of said translating occurs within said remote control device and at least a portion of said translating occurs externally to said remote control device; and a transmitting system to transmit each said user gesture translated by said means for translating to said appliance to control at least one aspect of said appliance, wherein said transmitting system has a characteristic selected from a group consisting of (i) said transmitting system is disposed within said handholdable remote control device, (ii) at least a portion of said transmitting system is disposed externally to said handholdable remote control device, and (iii) at least a portion of said transmitting system is disposed within said handholdable remote control device and at least a portion of said transmitting system is disposed externally to said handholdable remote control device. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification