Technique for controlling a presentation of a computer generated object having a plurality of movable components

• US 6,249,292 B1
• Filed: 05/04/1998
• Issued: 06/19/2001
• Est. Priority Date: 05/04/1998
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Christian, Andrew D., Avery, Brian L.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Padmanabhan, Mano
• Application Number:
  US09/071,037
• Time in Patent Office: 1,142 Days
• Field of Search: 345/473, 345/302, 345/474
• US Class Current: 345/473
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel

Abstract

A technique for controlling a presentation of a computer generated object having a plurality of movable components is disclosed. In one embodiment, the technique is realized by receiving a gesture element and an audio element at a processing device. The gesture element represents a gesture involving one or more of the plurality of movable components. The audio element represents an audio signal. The processing device receives the gesture element and the audio element in a sequential order. The processing device processes the gesture element and the audio element in the sequential order so that each of the plurality of movable components associated with the gesture element are moved to perform the gesture and the audio signal associated with the audio element is generated during a presentation of the computer generated object. The gesture can be performed and the audio signal can be generated simultaneously.

Claims

1. A method for controlling a presentation of a computer generated object having a plurality of movable components, the method comprising the steps of:

receiving a gesture element, an associated modifier and an audio element at a processing device, the gesture element representing a gesture involving one or more of the plurality of movable components, the audio element representing an audio signal, the gesture element and the audio element being received in a sequential order, and wherein the associated modifier controls a performance rate of the gesture element or the audio signal, and wherein the sequential order of receipt of the gesture element and the associated modifier and the audio signal determines the timing of the presentation of the gesture element with respect to the audio element including inclusion of the associated modifier; and

processing the gesture element and the audio element in the sequential order so that each of the plurality of movable components associated with the gesture element are moved to perform the gesture and the audio signal associated with the audio element is generated in accordance with the timing indicated by the sequence order and the associated modifier during a presentation of the computer generated object.

2. The method as defined in claim 1, further comprising the step of indicating when the sequential order is to be disregarded so that the gesture and the audio signal are performed simultaneously.

3. The method as defined in claim 1, wherein the computer generated object is a computer generated face.

4. The method as defined in claim 3, wherein the plurality of movable components include facial muscles.

5. The method as defined in claim 3, wherein the plurality of movable components include eyes.

6. The method as defined in claim 3, wherein the plurality of movable components include a head upon which the computer generated face is formed.

7. The method as defined in claim 3, wherein the gesture is a facial expression.

8. The method as defined in claim 1, wherein the audio signal is a speech signal.

9. The method as defined in claim 1, of wherein the associated modifier corresponds to a performance extent of the gesture.

10. The method as defined in claim 1, wherein the gesture element is defined to have a temporal duration.

11. The method as defined in claim 1, wherein the gesture element is defined using at least one gesture component.

12. The method as defined in claim 11, wherein each gesture component has an associated modifier.

13. The method as defined in claim 12, wherein the associated modifier corresponds to a performance extent of the gesture component.

14. The method as defined in claim 1, further comprising the step of:

storing the gesture element and the audio element in a memory;

wherein the gesture element and the audio element are retrieved from the memory by the processing device.

15. The method as defined in claim 1, further comprising the step of:

creating a text file containing the gesture element and the audio element.

16. The method as defined in claim 15, wherein the step of receiving the gesture element and the audio element includes reading the gesture element and the audio element from the text file.

17. The method as defined in claim 1, further comprising the steps of:

receiving a configuration element at the processing device, the configuration element representing a characteristic of the computer generated object, the configuration element being received sequentially with the gesture element and the audio element; and

processing the configuration element in the sequence in which it is received so that the characteristic is generated during the presentation of the computer generated object.

18. The method as defined in claim 19, wherein the characteristic is one of a face type, a voice type, and a speech rate.

19. The method as defined in claim 1, wherein the gesture element is one of a plurality of gesture elements, and wherein the gesture element is defined using another of the plurality of gesture elements.

20. The method as defined in claim 1, wherein the gesture element is processed such that an additional movement is superimposed upon the movement of at least one of the plurality of movable components associated with the gesture element.

21. An apparatus for controlling a presentation of a computer generated object having a plurality of movable components, the apparatus comprising:

a receiver for receiving a gesture element, a associated modifier and an audio element at a processing device, the gesture element representing a gesture involving one or more of the plurality of movable components, the audio element representing an audio signal, the gesture element and the audio element being received in a sequential order, and wherein the associated modifier controls a performance rate of the gesture element or the audio signal, and wherein the sequential order of receipt of the gesture element and the duration modifier and the audio signal determines the timing of the presentation of the gesture element with respect to the audio element including inclusion of the associated modifier; and

a processor for processing the gesture element and the audio element in the sequential order so that each of the plurality of movable components associated with the gesture element are moved to perform the gesture and the audio signal associated with the audio element is generated in accordance with the timing indicated by the sequence order and the associated modifier during a presentation of the computer generated object.

22. The apparatus as defined in claim 21, further comprising means for indicating when the sequential order is to be disregarded so that the gesture and the audio signal are performed simultaneously.

23. The apparatus as defined in claim 21, wherein the computer generated object is a computer generated face.

24. The apparatus as defined in claim 23, wherein the plurality of movable components include facial muscles.

25. The apparatus as defined in claim 23, wherein the plurality of movable components include eyes.

26. The apparatus as defined in claim 23, wherein the plurality of movable components include a head upon which the computer generated face is formed.

27. The apparatus as defined in claim 23, wherein the gesture is a facial expression.

28. The apparatus as defined in claim 21, wherein the audio signal is a speech signal.

29. The apparatus as defined in claim 23, wherein the associated modifier corresponds to a performance extent of the gesture.

30. The apparatus as defined in claim 21, wherein the gesture element is defined to have a temporal duration.

31. The apparatus as defined in claim 21, wherein the gesture element is defined using at least one gesture component.

32. The apparatus as defined in claim 31, wherein each gesture component has an associated modifier.

33. The apparatus as defined in claim 32, wherein the associated modifier corresponds to a performance extent of the gesture component.

34. The apparatus as defined in claim 21, further comprising:

a memory for storing the gesture element and the audio element;

wherein the gesture element and the audio element are retrieved from the memory by the processor.

35. The apparatus as defined in claim 21, further comprising:

a text file containing the gesture element and the audio element.

36. The apparatus as defined in claim 35, wherein the processor reads the gesture element and the audio element from the text file.

37. The apparatus as defined in claim 21, wherein the receiver receives a configuration element at the processing device, the configuration element representing a characteristic of the computer generated object, the configuration element being received sequentially with the gesture element and the audio element, and wherein the processor processes the configuration element in the sequence in which it is received so that the characteristic is generated during the presentation of the computer generated object.

38. The apparatus as defined in claim 37, wherein the characteristic is one of a face type, a voice type, and a speech rate.

39. The apparatus as defined in claim 21, wherein the gesture element is one of a plurality of gesture elements, and wherein the gesture element is defined using another of the plurality of gesture elements.

40. The apparatus as defined in claim 21, wherein the processor processes the gesture element such that an additional movement is superimposed upon the movement of at least one of the plurality of movable components associated with the gesture element.

41. An article of manufacture for controlling a presentation of a computer generated object having a plurality of movable components, the article of manufacture comprising:

a computer readable storage medium; and

computer programming stored on the storage medium;

wherein the stored computer programming is configured to be readable from the computer readable storage medium by a computer and thereby cause the computer to operate so as to;

receive a gesture element, an associated modifier and an audio element at a processing device, the gesture element representing a gesture involving one or more of the plurality of movable components, the audio element representing an audio signal, the gesture element and the audio element being received in a sequential order, and wherein the associated modifier controls a performance rate of the gesture element or the audio signal, and wherein the sequential order of receipt of the gesture element and the associated modifier and the audio signal determines the timing of the presentation of the gesture element with respect to the audio element including inclusion of the associated modifier; and

process the gesture element and the audio element in the sequential order so that each of the plurality of movable components associated with the gesture element are moved to perform the gesture and the audio signal associated with the audio element is generated in accordance with the timing indicated by the sequence order and the associated modifier during a presentation of the computer generated object.

42. The article of manufacture as defined in claim 45, wherein the gesture is performed and the audio signal is generated simultaneously.

43. The article of manufacture as defined in claim 41, wherein the computer generated object is a computer generated face.

44. The article of manufacture as defined in claim 43, wherein the plurality of movable components include facial muscles.

45. The article of manufacture as defined in claim 43, wherein the plurality of movable components include eyes.

46. The article of manufacture as defined in claim 43, wherein the plurality of movable components include a head upon which the computer generated face is formed.

47. The article of manufacture as defined in claim 43, wherein the gesture is a facial expression.

48. The article of manufacture as defined in claim 44, wherein the audio signal is a speech signal.

49. The article of manufacture as defined in claim 45, wherein the associated modifier corresponds to a performance extent of the gesture.

50. The article of manufacture as defined in claim 44, wherein the gesture element is defined to have a temporal duration.

51. The article of manufacture as defined in claim 41, wherein the gesture element is defined using at least one gesture component.

52. The article of manufacture as defined in claim 51, wherein each gesture component has an associated modifier.

53. The article of manufacture as defined in claim 52, wherein the associated modifier corresponds to a performance extent of the gesture component.

54. The article of manufacture as defined in claim 41, further causing the computer to operate so as to:

store the gesture element and the audio element in a memory;

wherein the gesture element and the audio element are retrieved from the memory by the processing device.

55. The article of manufacture as defined in claim 41, further causing the computer to operate so as to:

create a text file containing the gesture element and the audio element.

56. The article of manufacture as defined in claim 55, further causing the computer to operate so as to:

read the gesture element and the audio element from the text file.

57. The article of manufacture as defined in claim 41, further causing the computer to operate so as to:

receive a configuration element at the processing device, the configuration element representing a characteristic of the computer generated object, the configuration element being received sequentially with the gesture element and the audio element; and

process the configuration element in the sequence in which it is received so that the characteristic is generated during the presentation of the computer generated object.

58. The article of manufacture as defined in claim 57, wherein the characteristic is one of a face type, a voice type, and a speech rate.

59. The article of manufacture as defined in claim 41, wherein the gesture element is one of a plurality of gesture elements, and wherein the gesture element is defined using another of the plurality of gesture elements.

60. The article of manufacture as defined in claim 41, further causing the computer to operate so as to:

process the gesture element such that an additional movement is superimposed upon the movement of at least one of the plurality of movable components associated with the gesture element.