Method and system enabling control of different digital devices using gesture or motion control

• US 10,078,374 B2
• Filed: 01/03/2014
• Issued: 09/18/2018
• Est. Priority Date: 01/03/2013
• Status: Active Grant

• Current Assignee: Harshit Shrivastava, Saurav Suman
• Original Assignee: Harshit Shrivastava, Saurav Suman
• Inventors: Suman, Saurav, Shrivastava, Harshit
• Primary Examiner(s): Tung, David
• Application Number:
  US14/655,005
• Publication Number:
  US 20150355719A1
• Time in Patent Office: 1,719 Days
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A system for controlling operation of digital devices according to gesture of its user comprising: at least one gesture indicating hardware; at least one imaging device for capturing at least one first array of images of the at least one gesture indicating hardware; and a processor for processing the captured stream of images for; identifying at least one second array of images of the at least one gesture indicating hardware from the at least one first array of images captured by the at least one imaging device; determining at least one geometrical characteristic of bodies present in the at least one second array of images and their variation to construct at least one motion path of the at least one gesture indicating hardware; determining at least one key coordinates point representing at least one motion path; and generating at least one operating instruction to control at least one operation of at least one digital device. Noise cancellation while gesture detection is carried out by evaluating errors as a function of relative distances, boundary distances, and characteristics of various bodies identified beyond a threshold image size. Another approach of removing noise in gesture detection is to fix the emitting device in a still environment and identifying still bodies except emitter and thereby removing those stilt objects while capturing emitting device in motion.

Claims

1. A system for controlling operation of digital devices according to gesture of its user comprising:

at least one gesture indicating hardware;

at least one imaging device for capturing at least one first array of images of the at least one gesture indicating hardware; and

a processor for processing the captured stream of images for;

identifying at least one second array of images of the at least one gesture indicating hardware from the at least one first array of images captured by the at least one imaging device;

determining at least one geometrical characteristic of bodies present in the at least one second array of images and their variation to construct at least one motion path of the at least one gesture indicating hardware, wherein while constructing the at least one motion path of the at least one gesture indicating hardware, carrying out noise cancellation by evaluating errors as a function of size of bodies beyond a threshold size, directional radius at different angles with respect to a body considered, relative distances, boundary distances, ratio of radius perpendicular to a direction of velocity of previously selected body to the body considered, motion characteristics of various bodies identified beyond a threshold, and deviation of the body from a predicted position;

determining at least one key coordinates point representing at least one motion path; and

generating at least one operating instruction to control at least one operation of at least one digital device.

2. The system as claimed in claim 1, wherein the at least one gesture indicating hardware comprises at least one electromagnetic signal emitter for generating and emitting electromagnetic signal detectable by the at least one imaging device.

3. The system as claimed in claim 2, wherein the at least imaging device comprises at least one image sensor adapted to capture at least one image of the at least one gesture indicating hardware and detect the electromagnetic signal emitted by the at least one electromagnetic signal emitter of the at least one gesture indicating hardware.

4. The system as claimed in claim 3, wherein the at least one imaging device is provided in a separated module adapted to be disposed in operative communication with the at least one digital device.

5. The system as claimed claim 3, wherein the at least one imaging device is built in the at least one digital devices.

6. The system as claimed claim 5, wherein the at least one imaging device further comprises at least one electromagnetic signal filter adapted to block any other electromagnetic signal except the electromagnetic signal emitted from the at least one electromagnetic emitter, wherein the electromagnetic signal filter is configured to filter out all wavelength except when sun'"'"'s IR radiation falls lower than intensity of the emitter or is in the range of 1340-1460 nanometer or between 1740-2000 nanometer.

7. The system as claimed in claim 6, wherein the at least one imaging device further comprises at least two image sensors adapted to constitute a single image sensor with high imaging capturing rate for smoothing imaging of motion of the at least one gesture indicating hardware.

8. The system as claimed in the claim 7, wherein the at least one imaging device comprises two image sensors with imaging capturing rate of value m are adapted to operate as individual imaging devices, each comprising single image sensors with imaging capturing rate of 2 m by initiating one of the image sensors ½

m seconds before the second image sensor.

9. The system as claimed in the claim 8, wherein the two image sensors are placed relatively at a position to obtain standard stereo vision is obtained.

10. The system as claimed in claim 9, wherein the processor for identifying and separating the images of the gesture indicating hardware comprises:

at least one pixel intensity computing module adapted to determine intensity distribution of pixels in an array of images captured by at least one image sensor to locate the at least one electromagnetic signal emitter in the image;

at least one computation unit adapted to calculate different geometrical characteristics of different bodies represented by set of connected pixels in the array of images and ratio of the geometrical characteristics of the different bodies in an image frame to that of the body identified as the at least one electromagnetic signal emitter in an previous frame; and

at least one comparator module adapted to compare the ratio values and identify at least one electromagnetic signal emitter as a body for which the ratio shows least deviation in the array of images.

11. The system as claimed in claim 10, wherein the at least one pixel intensity computing module is adapted to locate the at least one electromagnetic signal emitter in the image by determining the pixels in the image having highest intensity with surrounding pixels having decreasing intensity as a function of the distance from the said highest intensity pixels.

12. The system as claimed in claim 11, wherein the at least one computation unit is adapted to calculate point in the bodies in the image where first or higher moment of a set of connected pixels representing the bodies in the captured image is zero or a point where an intensity or intensity of selected colours is highest for assigning a centre in the body.

13. The system as claimed in the claim 11, wherein the at least one computation unit is adapted to calculate the geometrical characteristics including radii of the bodies, distance between two bodies present in the image streams with reference to the assigned centre.

14. The system as claimed in claim 13, wherein the at least one computation unit is adapted to determine motion state of the bodies in the array of captured images by calculating a ratio of total number of pixels in body ‘

a’

minus n to the total number of pixels in body ‘

a’

;

wherein, ‘

a’

represents a particular body in later frame of any two consecutive image frames selected from the array of captured images and n is total number of pixels that are found to be a part of any body in an earlier frame of the two consecutive image frames.

15. The system as claimed in claim 14, wherein the at least one processor is adapted to detect the movement of the at least one gesture indicating hardware including its distance from the at least one imaging device by continuously calculating variation in geometrical characteristics of the at least one electromagnetic signal emitter in the array of captured images.

16. The system as claimed in claim 13, wherein the processor further comprises coordinate computation module adapted to identify the motion path representing key coordinates points by finding pair of coordinate points in the motion path at which angle between tangents is equal to or greater than a user defined threshold value.

17. The system as claimed in claim 16, wherein the processor comprises memory devices adapted to store user specific gesture assigned to perform any particular operation of the digital device as a sequence of the key coordinate points;

wherein the stored sequence of the key coordinate points is compared with the determined key coordinates of the motion path of the gesture indicating hardware and performs the assigned operation in the digital device on matching of the sequence of the stored and the determined key coordinates.

18. A method for operating digital devices according to gesture of its user involving the system as claimed claim 1 comprising:

continuously capturing images of the at least one gesture indicating hardware;

locating a specific intensity distribution of the pixels of captured images to identify an image of the at least one electromagnetic signal emitter associated with the at least one gesture indicating hardware in the array of captured images;

determining the geometric characteristics of the set of connected pixels representing different bodies in the array of captured images;

identifying the emitter in the subsequent image streams by comparing the geometric characteristics of the bodies with the same of the body identified as the at least one electromagnetic signal emitter in the previous image frame;

determining variation in the geometric characteristics of the body identified as the at least one electromagnetic signal emitter in the array of captured images to detect the movement of the gesture indicating hardware including its distance from the imaging device, wherein while detecting the movement of the gesture indicating hardware, carrying out noise cancellation by evaluating errors as a function of size of bodies beyond a threshold size, directional radius at different angles with respect to a body considered, relative distances, boundary distances, ratio of radius perpendicular to a direction of velocity of previously selected body to the body considered, motion characteristics of various bodies identified beyond a threshold, and deviation of the body from a predicted position;

constituting the motion path of the at least one gesture indicating hardware and determining sequence of the key coordinate points in the motion path; and

generating operating instructions to control the at least one digital devices in accordance with the sequence of the at least one key coordinate points in the motion path.