BLOW TRACKING USER INTERFACE SYSTEM AND METHOD

• US 20120075462A1
• Filed: 09/23/2010
• Published: 03/29/2012
• Est. Priority Date: 09/23/2010
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Chen, Ruxin, Osman, Steven
• Granted Patent: US 8,786,698 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A blow tracking user interface method and apparatus may detect an orientation of blowing of a user'"'"'s breath and a magnitude of blowing of the user'"'"'s breath. A blow vector may be generated from the orientation and magnitude of the blowing of the user'"'"'s breath. The blow vector may be used as a control input in a computer program.

Claims

1. A blow tracking interface method for control of a computer program, comprising:

detecting orientation of blowing of a breath;

detecting a magnitude of blowing of the breath;

generating a blow vector from the orientation and magnitude; and

using the blow vector as a control input in a computer program.

2. The method of claim 1, further comprising determining a timing of the blowing of the breath, wherein the timing of the blowing of the breath determines timing of an action in the computer program.

3. The method of claim 2, further comprising synchronizing timing of actions in the computer program to the timing of the blowing of breath.

4. The method of claim 1, wherein detecting the orientation and/or magnitude of the blowing of the breath includes obtaining one or more thermal infrared images of a user with one or more thermographic cameras, analyzing the one or more infrared images, and identifying one or more patterns in the one or more images corresponding to the blowing of breath from the one or more thermal infrared image.

5. The method of claim 4 wherein detecting the orientation and/or magnitude of the breath includes analyzing thermal contours in the one or more thermal infrared images, identifying one or more patterns characteristic of the blowing of the breath from the thermal infrared image, and determining the orientation from the one or more patterns.

6. The method of claim 4, wherein the thermographic camera is mounted to a pair of glasses worn by the user.

7. The method of claim 4, wherein generating the blow vector includes augmenting information obtained from the one or more thermal infrared images with information relating to the breath from one or more auxiliary sensors.

8. The method of claim 6, wherein the one or more auxiliary sensors includes a microphone or camera.

9. The method of claim 6, wherein the auxiliary detector is an additional thermographic camera mounted to a pair of glasses worn by the user.

10. The method of claim 1 wherein generating a blow vector includes generating a blow vector for breath through the user'"'"'s nose and determining an orientation of the user'"'"'s face from an orientation of the blow vector for breath through the user'"'"'s nose.

11. The method of claim 1, wherein detecting the orientation and/or magnitude of the blowing of the breath includes detecting sounds of the blowing of the breath with a microphone.

12. The method of claim 1 wherein detecting the orientation of the breath includes determining a source of the blowing of the breath.

13. The method of claim 6 wherein determining the source of the breath includes determining which user from among a plurality of users is the source of the blowing of the breath.

14. The method of claim 6 wherein determining the source of the breath includes determining whether the source of the blowing of the breath is a user'"'"'s mouth or nose.

15. The method of claim 1, further comprising mapping the timing and intensity of a user'"'"'s breathing to simulated breathing of an avatar controlled by a user so that the avatar appears to breathe in synchronization with user.

16. The method of claim 1 wherein detecting orientation of a breath includes determining whether a user is blowing breath through the user'"'"'s mouth or nose.

17. The method of claim 1 wherein detecting orientation of a breath includes determining whether a user is breathing in or out.

18. The method of claim 1 wherein detecting orientation or magnitude of a breath includes determining an intended temperature of the breath.

19. The method of claim 1 wherein using the blow vector as a control input includes using the orientation of the breath to determine a direction of an action taken by the computer program.

20. The method of claim 1 wherein using the blow vector as a control input includes using the magnitude of the breath to determine the magnitude of an action taken by the computer program.

21. The method of claim 1 wherein the orientation of the breath is determined relative to a video display.

22. The method of claim 1 wherein generating the blow vector includes using a past breath history to improve an estimation of the blow vector.

23. A blow tracking interface apparatus for control of a computer program, comprising:

a blow detector configured to detect one or more signals corresponding to blowing of a breath; and

a processor coupled to the blow detector, wherein the processor is configured to determine an orientation and magnitude of the blowing of the breath from the one or more signals, and generate a blow vector from the orientation and magnitude, wherein the processor is configured to use the blow vector as a control input in a computer program.

24. The apparatus of claim 23 wherein the breath detector includes one or more thermographic cameras.

25. The apparatus of claim 24, wherein the processor is configured to detect the orientation and/or magnitude of the breath by analyzing thermal contours in the image to identify one or more patterns characteristic the breath from the thermal infrared image.

26. The apparatus of claim 24 wherein the one or more thermographic cameras include a thermographic camera mounted to a video display that is coupled to the processor.

27. The apparatus of claim 24 wherein the one or more thermographic cameras include a thermographic camera mounted to a pair of glasses.

28. The apparatus of claim 23 wherein the blow detector includes one or more microphones.

29. A non-transitory computer readable medium having computer executable instructions embodied therein, wherein the instructions are configured to implement a blow tracking interface method for control of a computer program when executed by a computer processor, wherein the method implemented by the execution of the instructions comprises:

detecting orientation of blowing of a breath;

detecting a magnitude of blowing of the breath;

generating a blow vector from the orientation and magnitude; and

using the blow vector as a control input in a computer program.