USER INTERFACE SYSTEM AND METHOD USING THERMAL IMAGING

• US 20120075463A1
• Filed: 12/15/2010
• Published: 03/29/2012
• Est. Priority Date: 09/23/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Osman, Steven, CHEN, RUXIN
• Granted Patent: US 8,638,364 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A thermal imaging interface for control of a computer program may obtain one or more thermal infrared images of one or more objects with one or more thermographic cameras. The images may be analyzed to identify one or more characteristics of the objects. Such characteristics may be used as a control input in the computer program.

Claims

1. A thermal imaging interface method for control of a computer program, comprising:

a) obtaining one or more thermal infrared images of one or more objects with one or more thermographic cameras;

b) analyzing the one or more thermal infrared images;

c) identifying one or more characteristics of the one or more objects from the one or more thermal infrared images; and

d) using the one or more characteristics as a control input in the computer program.

2. The method of claim 1, wherein the one or more objects include one or more users.

3. The method of claim 2, wherein the one or more characteristics in c) include one or more vital signs for the one or more of the users determined by measuring a temperature or temperature change of the one or more users.

4. The method of claim 3, wherein the one or more vital signs is used to control a pace of actions occurring during execution of the program.

5. The method of claim 4, wherein the pace of actions includes a tempo of music.

6. The method of claim 3, wherein the one or more vital signs is used to control one or more graphics associated with the computer program.

7. The method of claim 3, wherein the one or more vital signs is used to measure a performance of the one or more users in conjunction with the computer program.

8. The method of claim 3, wherein the one or more vital signs is used to distinguish living objects from inanimate objects.

9. The method of claim 3, wherein the one or more vital signs is used to control a response speed of one or more user interfaces associated with the computer program.

10. The method of claim 3 wherein the one or more vital signs is mapped to trigger an action of a virtual object.

11. The method of claim 3 wherein the one or more vital signs is sampled and used by the computer program to generate a sound.

12. The method of claim 3, wherein the one or more vital signs is a heart beat rate, respiration rate, body temperature, or blood pressure.

13. The method of claim 12, further comprising analyzing the one or more vital signs and determining a health condition of a user from results of analysis of the vital signs.

14. The method of claim 1, wherein the one or more characteristics in c) include a respiration pattern for one or more users determined by measuring the airflow and temperature change around the nose and mouth of the one or more users.

15. The method of claim 14, wherein the respiration pattern for the one or more users is used to identify a source of sounds originating amongst the one or more users.

16. The method of claim 14, wherein the respiration pattern for the one or more users is used to determine whether the one or more of the users is an authorized user.

17. The method of claim 1, wherein the one or more characteristics in c) include a depth of the one or more objects in relation to the one or more thermographic cameras determined by infrared reflection coupled with interpolation using the one or more thermal infrared images.

18. The method of claim 17, wherein d) includes using the one or more characteristics for each of the one or more users in combination with one or more normal images of the one or more users to track the movement of one or more body parts associated with the one or more users.

19. The method of claim 1, wherein the one or more characteristics in c) include a segmentation cue that distinguishes the one or more objects from their background.

20. An apparatus for control of a computer program, comprising:

one or more thermographic cameras;

one or more processors coupled to the one or more thermographic cameras; and

instructions executable by the one or more processors configured to;

a) obtain one or more thermal infrared images of one or more objects with the one or more thermographic cameras;

b) analyze the one or more thermal infrared images;

c) identify one or more characteristics of the one or more objects from the one or more thermal infrared images;

d) use the one or more characteristics as a control input for a computer program.

21. The apparatus of claim 20, wherein the one or more objects include one or more users.

22. The apparatus of claim 21, wherein the one or more characteristics in c) include a one or more vital signs for the one or more users determined by measuring a temperature or temperature change of the one or more users.

23. The apparatus of claim 20, wherein the one or more characteristics in c) include a respiration rate for each of the one or more users determined by measuring the airflow and temperature change around the nose and mouth of the one or more users.

24. The apparatus of claim 20, wherein the one or more characteristics in c) include a depth of the one or more objects in relation to the one or more thermographic cameras determined by infrared reflection coupled with interpolation using the one or more thermal infrared images.

25. The apparatus of claim 20, wherein the one or more thermographic cameras include a thermographic camera mounted to a video display that is coupled to the processor.

26. The apparatus of claim 20, wherein the one or more processors include one or more dedicated processors, wherein each of the one or more thermographic cameras is coupled to a corresponding dedicated processor.

27. The apparatus of claim 26, wherein b), c) and d) are implemented by the one or more dedicated processors.

28. A computer program product, comprising:

a non-transitory computer-readable storage medium having computer readable program code embodied in said medium for controlling a computer program using a thermal image interface, said computer program product having;

a) computer readable program code means for obtaining one or more thermal infrared images of one or more objects with one or more thermographic cameras;

b) computer readable program code means for analyzing the one or more thermal infrared images;

c) computer readable program code means for identifying one or more characteristics of the one or more objects from the one or more thermal infrared images; and

d) computer readable program code means for using the one or more characteristics as a control input in the computer program.