Authoring system and authoring method, and storage medium

• US 20030023347A1
• Filed: 05/24/2002
• Published: 01/30/2003
• Est. Priority Date: 09/28/2000
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ogure, Satoko, Konno, Reizo
• Granted Patent: US 6,718,231 B2
• Time in Patent Office: Days
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: G05B 19/0426 Programming the control sequence
• CPC Class Codes: G05B 2219/23267 Program derived from sequence time diagram and stored in table
• CPC Class Codes: G05B 2219/2666 Toy

Abstract

An action editing window (200) is formed as a time table of a two-dimensional time line style, made up by a horizontal time axis and a vertical channel. Within the time line table, respective constituent elements of an action of a robot, such as a time ruler (201), a keyframe channel (202), a motion channel (203), a sound channel (204) or an LED operation channel (205), are chronologically displayed. A user is able to edit as synchronization between the respective components of the action is checked. This supports the creation and the editing of a series of commands and data stating a pattern of robot movements.

Claims

1. An authoring system for supporting the creation and editing of an action of a multi-joint structure comprised of a combination of a plurality of chronological data, comprising:

a user inputting unit for inputting commands or data from a user;

a user presenting unit for presenting an editing area in which chronological data making up the action are arrayed chronologically along the time axis; and

a chronological data editing unit provided for each chronological data making up the action, said chronological data editing unit creating or editing relevant chronological data based on a user input through said user presenting unit.

2. The authoring system according to claim 1 wherein one of the chronological data making up the action is motion data stating the chronological movements of respective joints of the multi-joint structure.

3. The authoring system according to claim 1 wherein one of the chronological data making up the action is motion data which prescribes chronological movements of respective joints of the multi-joint structure and which, by arraying two or more keyframes each representing the multi-joint structure striking a preset pose on the time axis, smoothly joins the respective keyframes.

4. The authoring system according to claim 1 wherein one of the chronological data making up the action is sound data output as sound in timed relation to reproduction of said action.

5. The authoring system according to claim 4 wherein the sound data is stated in the MIDI (Musical Instrumental Digital Interface) form or WAVE form.

6. The authoring system according to claim 1 wherein one of the chronological data making up the action is indicator display data stating the turning on/off of display indicators displayed in timed relation to reproduction of said action.

7. The authoring system according to claim 1 wherein indicator display data is stated in the MIDI (Musical Instrumental Digital Interface) form.

8. The authoring system according to claim 1 wherein the editing area presented by said user presenting unit is comprised of chronological data based chronological data display channels, said data extending along the horizontally arrayed time axis, said channels being arrayed in the vertical direction.

9. The authoring system according to claim 8 wherein the editing area presented by said user presenting unit includes a time ruler made up by graduations representing the time axis in real-time.

10. The authoring system according to claim 8 wherein the editing area presented by said user presenting unit includes one or more time display lines extending in the vertical direction for indicating the relevant time prescribed by a time ruler.

11. The authoring system according to claim 8 wherein the editing area presented by said user presenting unit includes a vertically extending current time display line for indicating the current time on the time axis prescribed by the time ruler;

said current time display line being shifted on the time ruler to a position corresponding to user actuation.

12. The authoring system according to claim 8 wherein the editing area presented by said user presenting unit includes a display window for previewing movements of the multi-joint structure at the current time.

13. The authoring system according to claim 8 wherein one of the chronological data making up the action is motion data which prescribes the chronological movements of respective joints of the multi-joint structure and which arrays two or more keyframes, each representing the multi-joint structure striking a preset pose, to smoothly join the respective keyframes;

said editing area presented by said user presenting unit including a keyframe channel representing each keyframe or its thumbnail along the time axis prescribed by said time ruler.

14. The authoring system according to claim 13 wherein the keyframe time is changed so as to follow up with dragging of said keyframe or its thumbnail within the keyframe channel.

15. The authoring system according to claim 13 wherein a pose editing picture for editing a relevant pose is started responsive to a selecting operation by a user of a keyframe or its thumbnail within the keyframe channel.

16. The authoring system according to claim 8 wherein one of the chronological data making up the action is motion data stating the chronological movements of respective joints of the multi-joint structure;

the editing area presented by said user presenting unit includes a motion channel for editing and displaying the contents of a motion along the time axis prescribed by the time ruler.

17. The authoring system according to claim 16 wherein said motion channel is comprised of respective timing charts representing chronological movements of respective joints of the multi-joint structure, said timing charts being arrayed in the vertical direction.

18. The authoring system according to claim 17 wherein the movements of joints at a relevant timing are changed so as to follow up with dragging on the timing chart in said motion channel.

19. The authoring system according to claim 8 wherein one of the chronological data making up the action is sound data issued as sound in timed relation to reproduction of an action;

the editing area presented by said user presenting unit including a sound channel for indicating the contents of the sound along the time axis prescribed by the time ruler.

20. The authoring system according to claim 8 wherein one of the chronological data making up the action is indicator display data stating the operation of the turning on/off of display indicators, as output in timed relation to the reproduction of an action;

the editing area presented by said user presenting unit including a display indicator channel for indicating the contents of the indicator data along the time axis prescribed by the time ruler.

21. The authoring system according to claim 1 wherein one of the chronological data making up the action is sound data output as sound in timed relation to the reproduction of an action;

said user presenting unit further indicating a sound editing area for indicating and editing the sound contents along the time axis prescribed by said time ruler.

22. The authoring system according to claim 21 wherein said sound editing area includes a score channel constituted by a basic grid along the time axis direction and a piano keyboard;

a score being formed by the sound duration as the reference of the time axis and the pitch of the piano keys;

the sound being edited by arraying the color corresponding to sound notes in a cell corresponding to the desired time point and the sound scale.

23. The authoring system according to claim 21 wherein said sound editing area includes a velocity channel for representing the intensity of each sound along the time axis.

24. The authoring system according to claim 1 wherein one of the chronological data making up an action is indicator data stating the turning on/off of an indicator, output in timed relation to the reproduction of an action;

said user presenting unit further indicating an indicator editing area for displaying and editing the contents of indicator data along the time axis prescribed by the time ruler.

25. The authoring system according to claim 24 wherein said indicator editing area includes a score channel constituted by a list of sites of arraying of display indicators and by the basic grid along the time axis direction;

a score being edited from one site list to another by indicating the turned-on state of the indicator in each site on the time axis on a score channel.

26. The authoring system according to claim 1 wherein said user presenting unit further displays a preview window for visually checking an action of the multi-joint structure generated based on each chronological data as edited by the chronological data editing unit.

27. The authoring system according to claim 26 wherein one of the chronological data making up an action is motion data stating the chronological movements of the respective joints of the multi-joint structure;

said preview window including a 3D view area for three-dimensional display of movements of the multi-joint structure generated based on motion data.

28. The authoring system according to claim 26 wherein one of the chronological data making up an action is indicator data stating the turning on/off of an indicator, output in timed relation to the reproduction of an action;

said preview window including an indicator operation preview area for displaying the operation of the indicators corresponding to the indicator display data in synchronism with the preview of other chronological data.

29. The authoring system according to claim 1 wherein said user presenting unit further displays a pose window for editing the pose of the multi-joint structure by the GUI operation.

30. The authoring system according to claim 29 wherein said pose window includes an entity specifying area for displaying the multi-joint structure in a developed plan view for accepting user selection of editable sites.

31. The authoring system according to claim 29 wherein said pose window includes a list specifying area for displaying a list of editable sites of the multi-joint structure and setting values thereof.

32. The authoring system according to claim 29 wherein said pose window includes a setting value area for displaying a list of the names of setting sites, setting values, settable maximum values and settable minimum values of editable sites of the multi-joint structure.

33. The authoring system according to claim 29 wherein said pose window includes a 3D display area for 3D displaying a full image of the multi-joint structure generated by 3D graphics and for accepting user selection on said 3D display.

34. The authoring system according to claim 29 further comprising:

data inputting means for inputting chronological data making up an action of the multi-joint structure from outside;

said pose window displaying the pose generated based on data input from said data inputting means.

35. The authoring system according to claim 1 wherein one of the chronological data making up an action is motion data which prescribes chronological movements of respective joints of the multi-joint structure and which arrays two or more keyframes each representing the multi-joint structure striking a preset pose to smoothly join the respective keyframes;

said user presenting unit further displaying a motion preview window arraying one or more key frames making up the motion or thumbnails thereof in accordance with the chronological sequence used in motion reproduction.

36. An authoring method for supporting the creation and editing of an action of a multi-joint structure comprised of a combination of a plurality of chronological data, comprising:

a user presenting step of presenting an editing area in which the chronological data making up the action are arrayed chronologically along the time axis; and

a chronological data editing step for creating or editing relevant chronological data based on a user input through the editing area, provided for each chronological data making up the action, via said user presenting step.

37. The authoring method according to claim 36 wherein one of the chronological data making up the action is motion data stating the chronological movements of respective joints of the multi-joint structure.

38. The authoring method according to claim 36 wherein one of the chronological data making up the action is motion data which prescribes chronological movements of respective joints of the multi-joint structure and which, by arraying two or more keyframes each representing the multi-joint structure striking a preset pose on the time axis, smoothly joins the respective keyframes.

39. The authoring method according to claim 36 wherein one of the chronological data making up the action is sound data output as sound in timed relation to reproduction of said action.

40. The authoring method according to claim 39 wherein the sound data is stated in the MIDI (Musical Instrumental Digital Interface) form or WAVE form.

41. The authoring method according to claim 36 wherein one of the chronological data making up the action is indicator display data stating the turning on/off of display indicators, said display data being displayed in timed relation to reproduction of said action.

42. The authoring method according to claim 36 wherein indicator display data is stated in the MIDI (Musical Instrumental Digital Interface) form.

43. The authoring method according to claim 36 wherein the editing area presented in said user presenting step is comprised of chronological data based chronological data display channels, said data extending along the horizontally arrayed time axis, said channels being arrayed in the vertical direction.

44. The authoring method according to claim 43 wherein the editing area presented in said user presenting step includes a time ruler made up by graduations representing the time axis in real-time.

45. The authoring method according to claim 43 wherein the editing area presented by said user presenting step includes one or more time display lines extending in the vertical direction for indicating the relevant time prescribed by a time ruler.

46. The authoring method according to claim 43 wherein the editing area presented by said user presenting step includes a vertically extending current time display line for indicating the current time on the time axis prescribed by the time ruler;

there being included a further step of shifting said current time display line on the time ruler to a position corresponding to user actuation.

47. The authoring method according to claim 43 further comprising a step of presenting a display window for previewing movements of the multi-joint structure at the current time.

48. The authoring method according to claim 43 wherein one of the chronological data making up the action is motion data which prescribes the chronological movements of respective joints of the multi-joint structure and which arrays two or more keyframes each representing the multi-joint structure striking a preset pose to smoothly join the respective keyframes;

said user presenting step including presenting a keyframe channel representing each keyframe or its thumbnail along the time axis prescribed by said time ruler.

49. The authoring method according to claim 48 wherein the keyframe time is changed so as to follow up with dragging of said keyframe or its thumbnail within the keyframe channel.

50. The authoring method according to claim 48 wherein a pose editing picture for editing a relevant pose is started responsive to a user selecting operation of a keyframe or its thumbnail within the keyframe channel.

51. The authoring method according to claim 43 wherein one of the chronological data making up the action is motion data stating the chronological movements of respective joints of the multi-joint structure;

said user presenting step including a motion channel for editing and displaying the contents of a motion along the time axis prescribed by the time ruler.

52. The authoring method according to claim 51 wherein said user presenting step represents a motion channel by arraying respective timing charts representing chronological movements of respective joints of the multi-joint structure in the vertical direction.

53. The authoring method according to claim 52 further comprising a step of changing the movements of joints at a relevant time so as to follow up with dragging on the timing chart in said motion channel.

54. The authoring method according to claim 43 wherein one of the chronological data making up the action is sound data issued as sound in timed relation to reproduction of an action;

said user presenting step presenting an editing area including a sound channel for indicating the contents of the sound along the time axis prescribed by the time ruler.

55. The authoring method according to claim 43 wherein one of the chronological data making up the action is indicator data stating the operation of the turning on/off of the indicator, as output in timed relation to the reproduction of an action;

said user presenting step presenting an editing area including a display indicator channel for indicating the contents of the indicator display data along the time axis prescribed by the time ruler.

56. The authoring method according to claim 36 wherein one of the chronological data making up the action is sound data output as sound in timed relation to the reproduction of an action;

there being further provided a step of indicating a sound editing area for indicating and editing the sound contents along the time axis prescribed by said time ruler.

57. The authoring method according to claim 56 wherein said sound editing area includes a score channel constituted by a basic grid along the time axis direction and a piano keyboard;

a score being formed by the sound duration as the reference of the time axis and the pitch of the piano keys;

the sound being edited by arraying the color corresponding to sound notes in a cell corresponding to the desired time point and the sound scale on said score channel.

58. The authoring method according to claim 56 wherein said sound editing area includes a velocity channel for representing the intensity of each sound along the time axis.

59. The authoring method according to claim 36 wherein one of the chronological data making up an action is indicator display data stating the turning on/off of a display indicator, output in timed relation to the reproduction of an action;

there being further provided a step of indicating an indicator editing area for displaying and editing the contents of indicator data along the time axis prescribed by the time ruler.

60. The authoring method according to claim 59 wherein said indicator editing area includes a score channel constituted by a list of arraying sites of the display indicators and by the basic grid along the time axis direction;

a score being edited from one site list to another by indicating the turned-on state of the display indicators in respective sites on the time axis on a score channel.

61. The authoring method according to claim 36 further comprising a step of displaying a preview window for visually checking an action of the multi-joint structure generated based on each chronological data as edited in the chronological data editing step.

62. The authoring method according to claim 61 wherein one of the chronological data making up an action is motion data stating the chronological movements of respective joints of the multi-joint structure;

said preview window indicating step three-dimensionally displaying movements of the multi-joint structure, generated based on motion data, in a 3D view area.

63. The authoring method according to claim 61 wherein one of the chronological data making up an action is indicator data stating the turning on/off of display indicators, output in timed relation to the reproduction of an action;

said preview window indicating step displaying the operation of the display indicators corresponding to the indicator display data in synchronism with the preview of other chronological data.

64. The authoring method according to claim 36 further comprising a step of displaying a pose window for editing the pose of the multi-joint structure by the GUI operation.

65. The authoring method according to claim 64 wherein said pose window includes an entity specifying area for displaying the multi-joint structure in a developed plan view for accepting user selection of editable sites.

66. The authoring method according to claim 64 wherein said pose window includes a list specifying area for displaying a list of editable sites of the multi-joint structure and setting values thereof.

67. The authoring method according to claim 64 wherein said pose window includes a setting value area for displaying a list of the names of setting sites, setting values, settable maximum values and settable minimum values of editable sites of the multi-joint structure.

68. The authoring method according to claim 64 wherein said pose window includes a 3D display area for 3D displaying a full image of the multi-joint structure generated by 3D graphics and for accepting user selection of said editable sites on said 3D display.

69. The authoring method according to claim 64 further comprising a data inputting step of inputting chronological data making up an action of the multi-joint structure from outside;

said pose window displaying the pose generated based on data input from said data inputting means.

70. The authoring method according to claim 36 wherein one of the chronological data making up an action is motion data which prescribes chronological movements of respective joints of the multi-joint structure and which arrays two or more keyframes each representing the multi-joint structure striking a preset pose to smoothly join the respective keyframes;

there being further provided a step of displaying a motion preview window arraying one or more key frames making up the motion or thumbnails thereof in accordance with the chronological sequence used in motion reproduction.

71. A recording medium having physically stored thereon, in a computer-readable form, a computer software stating the processing for supporting the creation and editing of an action of a multi-joint structure, comprised of the combination of plural chronological data, on a computer system, said computer software comprising:

a user presenting step of presenting an editing area in which chronological data making up the action are arrayed chronologically along the time axis; and

a chronological data editing step for creating or editing relevant chronological data based on a user input through an editing area by said user presenting step, said editing area being provided for each chronological data making up the action.