CONTROLLING A SYSTEM BASED ON USER BEHAVIORAL SIGNALS DETECTED FROM A 3D CAPTURED IMAGE STREAM

• US 20080170749A1
• Filed: 01/12/2007
• Published: 07/17/2008
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Goldman, Steven D., Sessa, Anthony J., Arnold, Kenneth C., Paolini, Michael A., Albertson, Jacob C
• Granted Patent: US 7,801,332 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A computer-implemented method, system, and program includes a behavior processing system for capturing a three-dimensional movement of a user within a particular environment, wherein the three-dimensional movement is determined by using at least one image capture device aimed at the user. The behavior processing system identifies a three-dimensional object properties stream using the captured movement. The behavior processing system identifies a particular defined behavior of the user representing a particular behavioral signal from the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with multiple behavior definitions each representing a separate behavioral signal for directing control of at least one machine. In response to identifying the particular defined behavior, a machine control system generates a control signal triggered by the particular behavioral signal for directing control of the at least one machine.

Claims

1. A computer-implemented method for generating control signals for a machine based on user behavior, comprising:

capturing a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the user;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular defined behavior of the user representing a particular behavioral signal from the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions each representing a separate behavioral signal for directing control of at least one machine;

in response to identifying the particular defined behavior of the user representing the particular behavioral signal from among the plurality of behavior definitions, generating a control signal triggered by the particular behavioral signal for directing control of at least one machine.

2. The method according to claim 1, wherein capturing a three-dimensional movement of the user further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

3. The method according to claim 1, wherein capturing a three-dimensional movement of the user further comprises capturing at least one image frame of a user at a first point in time and comparing at least one additional image frame from a second point in time with the image frame from the first point in time.

4. The method according to claim 1, wherein identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises calculating a percentage probability that the captured three-dimensional movement represents a particular behavior and represents the behavioral signal defined in the particular behavior definition.

5. The method according to claim 1, wherein identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises identifying at least one body movement of the user comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the user within the three-dimensional object properties stream matching at least one of the plurality of behavior definitions.

6. The method according to claim 1, wherein identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises identifying at least one behavior by the user signaling the user is experiencing a medical emergency, wherein the particular behavioral signal of a medical emergency directs control of the at least one machine for automatically generating a notification to at least one other user of the medical emergency.

7. The method according to claim 1, wherein identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises identifying at least one behavior by the user signaling control of a particular function of the machine.

8. The method according to claim 1, wherein generating a control signal triggered by the particular behavioral signal for directing control of at least one machine further comprises:

sending the particular defined behavior to a machine control system for generating the control signal; and

translating, at the machine control system, the particular defined behavior representing the particular behavioral signal into at least one control signal for directing control of the at least one machine from among a plurality of machines controllable by the machine control system.

9. A system for generating control signals for a machine based on user behavior, comprising:

a behavior processing system communicative connected to a network, further comprising;

means for capturing a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the user;

means for identifying a three-dimensional object properties stream using the captured movement; and

means for identifying a particular defined behavior of the user representing a particular behavioral signal from the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions each representing a separate behavioral signal for directing control of at least one machine;

a machine control system communicatively connected to the behavior processing system via the network for generating a control signal triggered by the particular behavioral signal for directing control of at least one machine.

10. The system according to claim 9, wherein the means for capturing a three-dimensional movement of the user further comprises means for capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

11. The system according to claim 9, wherein the means for identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises means for calculating a percentage probability that the captured three-dimensional movement represents a particular behavior and represents the behavioral signal defined in the particular behavior definition.

12. The system according to claim 9, wherein the means for identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises means for identifying at least one body movement of the user comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the user within the three-dimensional object properties stream matching at least one of the plurality of behavior definitions.

13. The system according to claim 9, wherein the means for identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises means for identifying at least one behavior by the user signaling the user is experiencing a medical emergency, wherein the particular behavioral signal of a medical emergency directs control of the at least one machine for automatically generating a notification to at least one other user of the medical emergency.

14. The system according to claim 9, wherein the means for identifying a particular defined behavior representing a particular behavioral signal from the three-dimensional object properties stream further comprises means for identifying at least one behavior by the user signaling control of a particular function of the machine.

15. The system according to claim 9, wherein the machine control system for generating a control signal triggered by the particular behavioral signal for directing control of at least one machine further comprises:

means for translating the particular defined behavior representing the particular behavioral signal into at least one control signal for directing control of the at least one machine from among a plurality of machines controllable by the machine control system.

16. A program product comprising a storage-type computer-usable medium including a computer-readable program for generating control signals for a machine based on user behavior, wherein the computer-readable program when executed on a computer cause the computer to:

capture a three-dimensional movement of a user, wherein the three-dimensional movement is determined using at least one image capture device aimed at the user;

identify a three-dimensional object properties stream using the captured movement;

identify a particular defined behavior of the user representing a particular behavioral signal from the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions each representing a separate behavioral signal for directing control of at least one machine;

in response to identifying the particular defined behavior of the user representing the particular behavioral signal from among the plurality of behavior definitions, generate a control signal triggered by the particular behavioral signal for directing control of at least one machine.

17. The program product of claim 16, wherein said computer-readable program is transmitted over a network.

18. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to capture the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the user.

19. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to calculate a percentage probability that the captured three-dimensional movement represents a particular behavior and represents the behavioral signal defined in the particular behavior definition.

20. The program product of claim 16, wherein the computer-readable program when executed on the computer further causes the computer to identify at least one behavior by the user signaling the user is experiencing a medical emergency, wherein the particular behavioral signal of a medical emergency directs control of the at least one machine for automatically generating a notification to at least one other user of the medical emergency.