Video or information processing method and processing apparatus, and monitoring method and monitoring apparatus using the same

• US 20020075244A1
• Filed: 05/01/2001
• Published: 06/20/2002
• Est. Priority Date: 04/08/1991
• Status: Active Grant

• Current Assignee: Atsuhiko Hirota, Atsuhiko Nishikawa, Kimiya Yamaashi, Koichiro Tanikoshi, Masayasu Futakawa, Masayuki Tani, Shinya Tanifuji
• Original Assignee: Atsuhiko Hirota, Atsuhiko Nishikawa, Kimiya Yamaashi, Koichiro Tanikoshi, Masayasu Futakawa, Masayuki Tani, Shinya Tanifuji
• Inventors: Tanikoshi, Koichiro, Tanifuji, Shinya, Tani, Masayuki, Yamaashi, Kimiya, Futakawa, Masayasu, Hirota, Atsuhiko, Nishikawa, Atsuhiko
• Granted Patent: US 6,965,376 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G05B 23/0216 Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration
• CPC Class Codes: G05B 23/0267 Fault communication, e.g. human machine interface [HMI]
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

In a remote operation monitoring system and the like, it is a video processing apparatus capable of intuitively grasping an object operated by an operator and an operation result. The video processing apparatus includes a unit (310, 320, 2104, 2202) for storing information about at least one object displayed on a screen of a display unit; a unit (12, 2105) for designating information about the object; a unit (300, 2201) for searching the store unit based upon the designated information, and for obtaining information within the store unit corresponding to the designated information; and also a unit (20, 2103) for performing a process related to the object based on the obtained information. An operator can readily grasp an object to be operated and a result.

Claims

1. A video processing apparatus for performing a process related to a video image of at least one object displayed on a screen of display means, comprising:

means for storing information about said object; and

means for performing a process related to said object based upon said information.

2. A video processing apparatus as claimed in claim 1, further comprising:

means for designating the information about said object; and

means for searching said store means based upon said designated information to obtain information in said store means, corresponding to said designated information, said process performing means executing a process related to said object based on said obtained information.

3. A video processing apparatus as claimed in claim 2, wherein said process performing means causes an image of an object corresponding to said obtained information to be displayed on said display means.

4. A video processing apparatus as claimed in claim 2, wherein means for designating information about said object is further comprised, and said process performing means includes means for reading out the information about said designated object from said store means to be displayed on said display means.

5. A video processing apparatus as claimed in claim 2, wherein means for designating information about said object is further included, and said process performing means includes means for reading out a list of executable processes as the information about said designated object from said store means to be displayed on said display means.

6. A video processing apparatus as claimed in any one of the preceding claims 1 to 5, wherein means for designating a search key by inputting either a text and a pattern is further comprised, and said process performing means includes means for reading out the information about the object, corresponding to said designated search key, from said store means to be displayed on said display means.

7. A video processing apparatus as claimed in claim 6, wherein said process performing means causes a picture of a camera corresponding to said obtained information among pictures of plural cameras which photograph objects different from each other, to be displayed on said display means.

8. A video processing apparatus as claimed in claim 1, further comprising:

means for designating a region of a picture of an object displayed on said display means; and

means for defining a process of said object corresponding to said designated region.

9. A video processing apparatus for displaying a video image of at least one object on a screen of display means, comprising:

means for storing information about said object; and

means for displaying a graphics related to said object based on said information on said display means.

10. A video processing apparatus as claimed in claim 9, further comprising:

means for designating information about said object; and

means for searching said store means based on said designated information so as to obtain information within said store means, corresponding to said designated information, said display means displaying a graphics related to said object based on said obtained information.

11. A video processing apparatus as claimed in claim 9, further comprising:

means for operating said object via the graphics displayed on said screen.

13. A video processing apparatus as claimed in claim 12, wherein said identifying means includes:

means for designating an image on said screen;

means for searching said store means based on the positional information of said designated image to obtain positional information within said store means, corresponding to the positional information of said designated image; and

means for identifying the object on said screen based on said obtained positional information.

14. A video processing apparatus as claimed in claim 13, wherein said process performing means displays a corresponding graphics based on said identified object on said screen.

15. A video processing apparatus as claimed in claim 13, wherein said process performing means displays the image of said corresponding object based upon said identified object.

16. A video processing apparatus as claimed in claim 13, wherein said process performing means operates said corresponding object based upon said identified object.

12. A video processing apparatus for performing a process related to a video image of at least one object displayed on a screen of display means, comprising:

means for storing positional information related to said object;

means for identifying the object on said screen based on said positional information; and

means for performing a process based on said identified object.

17. A video processing method for performing a process related to a video image of at least one object displayed on a screen of display means, comparison the steps of:

storing information related to said object; and

performing a process related to said object based on said information.

18. A video processing method for displaying a video image of at least one object comprising the steps of:

storing information related to said object; and

displaying a graphics related to said object on said display means based on said information.

19. A video processing method for executing a process related to a video image of at least one object displayed on a screen of display means, comprising the steps of:

storing positional information related to said object;

identifying an object on said screen based on said positional information; and

performing a process based on said identified object.

20. An image operating method for operating a video image of at least one object displayed on a screen of display means, comprising:

an object designating step for designating an object within a video image; and

a process performing step for performing a process corresponding to the designated object.

21. An image operating method as claimed in claim 20, wherein said object designating step includes a step for designating an object of an image inputted from a video camera and being displayed on said screen.

22. An image operating method as claimed in claim 20 or claim 21, wherein said process performing step includes a step for executing a process to change a video image based on the designated object.

23. An image operating method as claimed in claim 21, wherein said process performing step includes a step for performing a process to operate a video camera based on the designated object.

24. An image operating method as claimed in claim 21, wherein said process performing step includes a step for performing a process to operate the designated object.

25. An image operating method as claimed in claim 20, or claim 21, wherein said process performing step includes a step for superimposing a graphics corresponding to the designated object on said image to be displayed, and for clearly indicating which object has been designated.

26. An image operating method as claimed in claim 20, or claim 21, wherein said process performing step includes a step for menu displaying a list of executable processes corresponding to the designated object, and capable of selecting a process to be executed.

27. An image operating method as claimed in claim 20, or claim 21, wherein said process performing step includes:

a graphics displaying step for synthesizing a graphics corresponding to the designated object with said video image to be displayed;

a graphics element selecting step for selecting a constructive element of a graphics; and

a performing step for performing a process based on the constructive element selected by said graphics element selecting step.

28. An image operating method as claimed in claim 20, or claim 21, wherein said object designating step includes an object clearly indicating step for clearly indicating an object designatable within an image.

29. An image operating method as claimed in claim 20, or claim 21, wherein said process performing step includes a step for performing a process for displaying information related to said designated object.

39. A remote operation monitoring method comprising:

an input step for photographing an object corresponding to an object to be controlled;

a display step for displaying said photographed object; and

the image operating method as recited in any one of the preceding claims 20 to 29.

30. An image operating apparatus comprising:

display means for displaying a video image of at least one object on a screen;

object designating means for designating the object within the video image displayed on the screen of said display means; and

process performing means for performing a process corresponding to the designated object.

40. A remote operation monitoring apparatus comprising:

a video camera for photographing an object corresponding to an object to be controlled;

display means for displaying an object photographed by said video camera; and

the image operating apparatus as recited in claim 30.

31. An image searching method for searching a video image and for displaying the searched image, comprising:

a search key designating step for designating a search key by inputting either a text or a pattern; and

an image searching step for displaying a video image in which an object adapted to the search key designated by said search key designating step is being displayed.

32. An image searching method as claimed in claim 31, wherein said image searching step includes a step for selecting a video camera based upon said search key.

33. An image searching method as claimed in claim 31 or claim 32, wherein said image searching step includes a step for operating a video camera based upon the search key.

34. An image searching method as claimed any one of the preceding claims 31 to 33, wherein said image searching step includes a step for clearly indicating an object adapted to the search key by synthesizing a graphics with the searched image to be displayed.

35. An image searching method as claimed any one of the preceding claims 31 to 34, wherein said search key input step includes a step for inputting a search key by a voice.

36. An image searching apparatus for searching a video image and for displaying the search image, comprising:

search key designating means for designating a search key by inputting either a text or a pattern; and

image searching means for displaying a video image in which an object adapted to the search key designated by said search key designating means is being displayed.

37. An image process defining method comprising:

an image display step for displaying an image on a screen or display means;

a region designating step for designating a region of an image displayed by said image display step; and

a process defining step for defining a process corresponding to a region designated by said region designating step.

38. An image process defining apparatus comprising:

display means for displaying an image on a screen;

region designating means for designating a region of an image displayed by said display means; and

process defining means for defining a process corresponding to a region designated by said region designating mean.

41. A remote operation monitoring apparatus comprising:

a video camera for photographing an object to be controlled;

display means for displaying the objected to be controlled, which is photographed by the video camera;

designating means for designating the object to be controlled, which is displayed by said display means, on a screen; and

executing means for executing a desirable process corresponding to the object to be controlled, which is designated on said screen.

42. A remote operation monitoring apparatus comprising:

an object to be controlled;

a video camera for photographing the object to be controlled, which is photographed by said video camera;

designating means for designating the object to be controlled, which is displayed by said display means, on a screen;

defining means for defining a desirable process corresponding to the object to be controlled, which is designated on the screen; and

executing means for executing said desirable process corresponding to the object to be controlled, which is designated on the screen.

43. An information processing apparatus for storing data to control an object (referred to “

control data”

hereinafter), and also for storing either audio, or video data related to said object, comprising;

means for relating the control data with the audio, or video data; and

means for relating the control data with the audio, or video data based upon said relating means to be outputted.

44. An information processing apparatus as claimed in claim 43, wherein;

first reproducing means for reproducing said control data, and second reproducing means for reproducing at least one of said audio data and said video data are further comprised;

said relating means includes means for storing information for relating said control data to said audio data, or said video data;

said output means includes;

first output means for outputting said reproduced measurement data;

second output means for outputting said reproduced sound data, or said reproduced video data;

synchronizing means for controlling said first and second output means in such a manner that said outputted control data is synchronized with said audio data, or said video data based on said relating information to be outputted.

45. An information processing apparatus as claimed in claim 44, wherein said second means further includes:

means for designating at least one of a reproducing direction and a reproducing speed with respect to said sound data and said video data; and

means for reproducing said audio and video data at the designated reproducing speed and the designated reproducing direction.

46. An information processing apparatus as claimed in claim 44, wherein means for designating both of control data to be outputted and a time instant thereof, and

said synchronizing means instructs said first output means in such a manner that either video data or audio data is outputted at a time instant very close to a time instant when the output-designated control data has been recorded.

47. An information processing apparatus as claimed in claim 46, wherein said designating means includes means for accepting a search value of the control data;

said synchronizing means includes means for searching a time instant when the designated control data takes this search value; and

said first output means includes means for outputting both of said time instant and the data value at this time instant.

48. An information processing apparatus as claimed in any one of the preceding claims 45 to 47, wherein

said synchronizing means includes:

means for determining a display degree of the control data in accordance with the reproducing speed of the video data, or audio data; and

means for displaying the control data at the determined display degree in synchronization with the reproduction of the audio data or the video data.

49. An information processing apparatus as claimed in any one of the preceding claims 46 to 48, wherein said designating means includes means for accepting the display degree of the control data;

said first output means includes means for displaying the control data in accordance with the determined display degree;

said synchronizing means includes means for determining a reproducing speed from the designated display degree; and

said first output means includes means for outputting the video data and the audio data at the determined speed.

50. An information processing apparatus as claimed in claim 44, wherein third storage means for storing operation input information of an operator is further comprised, and

said output means includes third output means for outputting the operation input information of the operator in synchronism with the reproduction of the video data or the audio data.

51. An information processing apparatus as claimed in claim 50, wherein means for accepting operation input information is further comprised;

said synchronizing means includes means for searching video and audio data acquired at a time instant very close to a time instant when the designated operation is inputted; and

said second output means includes means for controlling the third output means to output the operation input information in synchronism with the reproduction of the video data, or the audio data by the second output means.

52. An information processing apparatus as claimed in claim 44, further comprising:

means for designating an object to be recorded within an image, for designating a recording condition to designate when the object to be recorded is recorded, and for accepting at least one designation of the recording methods for the image data,

said synchronizing means controlling a read of said audio data or said video data by determining at least one of the designations of the recording object, the recording condition, and the recording method.

62. A monitoring apparatus as claimed in claim 51, wherein said relating means includes means for storing information to relate the monitoring object with the control data;

said control means includes means for selecting a monitoring object related to said selected control data based on said information; and

said second output means includes means for displaying said selected monitoring object.

63. A monitoring apparatus as claimed in claim 62, wherein said second output means includes:

means for displaying the control data in a trend graph, and for selecting the displayed data item; and

means for performing such a representation that the monitoring object related to said control data has been selected in the picture.

64. A monitoring apparatus as claimed in claim 62, wherein said second output means includes means for displaying the control data within a trend graph and for selecting the displayed data item, and

said control means includes means for changing such a picture where there is a related monitoring object when there is no monitoring object related to said control data in the picture;

and

said second output means performs such a representation that the monitoring object related to said control data has been selected within the picture.

53. A monitoring apparatus for processing audio data or video data of an object to be monitored, comprising:

means for relating data to control a control system (referred to “

control data”

) with the audio data, or the video data;

means for selecting the object to be monitored on an image;

first output means for outputting the control data;

second output means for outputting the audio data, or the video data; and

means for controlling said first output means in such a manner that the control data related to the selected object to be monitored is outputted.

54. A monitoring apparatus as claimed in claim 53, wherein said relating means includes storage means for storing information for relating the object to be monitored to said control data, and information about the object to be monitored;

said control means includes;

means for selecting the control data related to the selected object to be monitored based on the information about said object to be monitored, and the relating information, which have been stored; and

means for displaying said selected control data by said first output means.

55. A monitoring apparatus as claimed in claim 54, wherein in case that there are a plurality of control data related to the selected object to be monitored,

said second output means includes;

means for displaying a selection menu of the selected object to be monitored, and the related control data positioned near this object;

means for accepting a designation of at least one item from said selecting menu; and

means for displaying the control data corresponding to the accepted item.

56. A monitoring apparatus as claimed in claim 54, wherein said second output means includes means for displaying the control data related to the selected object to be monitored within a trend graph.

57. A monitoring apparatus as claimed in claim 54, wherein said second output means includes means for displaying both of the selected object and at least one control data related to this selected object near said object.

58. A monitoring apparatus as claimed in claim 54, wherein said second output means includes means for representing the control data related to the selected monitoring object as a graphics, and for superimposing the graphics on an image of said selected monitoring object for a display purpose.

59. A monitoring apparatus as claimed in claim 53, wherein said relating means includes storage means for storing both information about a monitoring object, and information for relating the monitoring object with the related monitoring object;

said controlling means includes means for selecting a monitoring object related to the selected monitoring object based on the stored information about the monitoring object and the relating information; and

said second output means includes means for displaying on a picture that said selected monitoring object has been selected, and also for representing said selected monitoring object related to said selected monitoring object.

60. A monitoring apparatus as claimed in claim 53, wherein said relating means includes memory means for storing information about a monitoring object, and information for relating said monitoring object with additional information related thereto;

said control means selects the additional information related to the selected monitoring object based upon the information about the stored monitoring object and the relating information; and

said second output means includes means for representing on a picture that said selected monitoring object has been selected, and for indicating said selected additional information related to said selected monitoring object.

61. A monitoring apparatus for processing audio, or video data of a monitoring object, comprising:

means for relating data to control a control system (control data) with the audio or video data;

means for selecting at least one control data;

first output means for outputting the control data;

second output means for outputting the audio data, or the video data; and

means for controlling said first output means in such a manner that audio data, or video data related to said selected control data is outputted.