CAMERA CONTROL SYSTEM, CAMERA SERVER, CAMERA CLIENT, CONTROL METHOD, AND STORAGE MEDIUM

• US 20010024233A1
• Filed: 10/14/1997
• Published: 09/27/2001
• Est. Priority Date: 10/15/1996
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: SUZUKI, KAZUKO, KAWAI, TOMOAKI, URISAKA, SHINYA, YANO, KOICHI
• Granted Patent: US 6,529,234 B2
• Time in Patent Office: Days
• US Class Current: 348/211.6
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

In a system in which a plurality of cameras and microphones, the image pickup directions and audio input directions of which can be remote-controlled, are arranged on a network, when an image pickup inhibited region is set, and when a network client remote-controls a certain camera to pickup a range including the image pickup inhibited region, a portion corresponding to the image pickup inhibited region is not displayed, and sound corresponding to the region is not output.

Claims

1. A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers a pickup image by the camera to clients on the network to display the image on display screens of the client devices, comprising:

storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;

detection means for detecting a level of the connected client by referring to said storage means;

determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level of the client; and

inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred.

2. The system according to

claim 1

, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

3. The system according to

claim 2

, wherein said modification means comprises mask means for masking an image.

4. The system according to

claim 2

, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

5. A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers a pickup image by the camera to clients on the network to display the image on display screens of the client devices, comprising:

storage means for storing information indicating an image pickup inhibited region;

audio detection means for detecting sound;

determination means for determining if an image pickup range of the camera includes the image pickup inhibited region stored in said storage means; and

inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred, and also inhibiting the sound detected by said audio detection means from being transferred.

6. The system according to

claim 5

, wherein said audio detection means includes a microphone, a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

7. The system according to

claim 6

, wherein said microphone is mounted on a single panhead together with the camera.

8. The system according to

claim 5

, further comprising storage means for storing level information in units of a plurality of clients, and image pickup inhibited regions in units of levels, and

wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

9. The system according to

claim 8

, wherein said modification means comprises mask means for masking an image.

10. The system according to

claim 8

, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

11. A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers an image pickup by the camera to clients on the network to display the image on display screens of the client devices, comprising:

storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;

detection means for detecting a level of the connected client by referring to said storage means;

determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level of the client; and

inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred in units of users of the level.

12. The system according to

claim 11

, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the pickup image to image quality corresponding to a level of the connected client.

13. The system according to

claim 12

, wherein said modification means comprises mask means for masking an image.

14. The system according to

claim 12

, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

15. The system according to

claim 12

, further comprising audio detection means for detecting sound, and

wherein when said determination means determines that the image pickup range includes the image pickup inhibited region, said inhibition means also inhibits audio information detected by said audio detection means from being transferred.

16. The system according to

claim 12

, wherein said audio detection means includes a microphone, a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

18. The system according to

claim 16

, wherein said inhibition means includes modification means for modifying an image in a region corresponding to the image pickup inhibited region in the picked up image to image quality corresponding to the detected lowest level.

20. The system according to

claim 16

, further comprising audio detection means for detecting sound, and

wherein when said determination means determines that the image pickup range includes the image pickup inhibited region, said inhibition means also inhibits audio information detected by said audio detection means from being transferred.

21. The system according to

claim 20

, wherein said audio detection means includes a microphone, audio detection characteristics including a directivity, detection direction, and input sensitivity of which can be controlled, at least the detection direction being changed in correspondence with an image pickup direction of the camera.

22. The system according to

claim 21

, further comprising status information transmission means for transmitting information indicating the audio detection characteristics of said microphone and angle information of the camera to the connected clients, and

wherein each of the clients displays camera and microphone icons on a predetermined map on the basis of the received angle information of the camera and the audio detection characteristic information of said microphone.

17. A camera control system, which has a camera, an angle of which can be controlled, is connected to a predetermined network, and transfers an image pickup by the camera to clients on the network to display the image on display screens of the client devices, comprising:

storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions for substantially inhibiting pickup image in units of levels;

detection means for detecting a lowest level among the connected clients by referring to said storage means;

determination means for determining if an image pickup range includes the image pickup inhibited region corresponding to the level detected by said detection means; and

inhibition means for, when said determination means determines that the image pickup range includes the image pickup inhibited region, inhibiting image data corresponding to the image pickup inhibited region in the image to be picked up from being transferred in units of users of the level.

19. The system according to

claim 17

, wherein said modification means comprises mosaic means for applying a mosaic pattern to an image, and a pixel block size of the mosaic pattern becomes larger for a client with lower level.

23. A camera control system, which has a camera, a image pickup direction of which can be changed by remote control from a client on a network, and is constituted by a camera server for transferring an image pickup by the camera, and a client for displaying the transferred image and controlling the image pickup direction of the camera, comprising:

display means for displaying a graphical scroll bar used for change control of the image pickup direction of the camera on a display screen of the client;

storage means for storing information indicating levels in units of clients on the network, and information indicating image pickup inhibited regions corresponding to the levels; and

display control means for distinguishably displaying a portion indicating an image pickup grant range of the camera and a portion indicating an image pickup inhibited range on the basis of the image pickup inhibited region stored in said storage means.

24. The system according to

claim 23

, wherein said storage means also stores information indicating levels of the clients, and said display control means distinguishably displays the portion indicating the image pickup grant range of the camera and the portion indicating the image pickup inhibited range in correspondence with the levels of the clients.

25. The system according to

claim 23

, wherein said display control means distinguishably displays movable and immovable ranges of rectangular symbols on pan and tilt angle scroll bars for changing an angle.

26. The system according to

claim 23

, wherein a plurality of clients can be connected to the camera server.

27. The system according to

claim 26

, wherein the camera server comprises means for giving a right of camera access to one connected client, and

means for, when an image pickup by operating the scroll bars by the client given the right of camera access includes the image pickup inhibited region of a client of another level, modifying a portion corresponding to the image pickup inhibited region of the client of the other level in the pickup image not to be substantially displayed, and transferring the modified image to the client of the other level.

28. A camera server for controlling a camera in accordance with instructions of an angle and a zoom ratio from a client on a network to pickup an image, and transmitting the pickup image to the client, comprising:

storage means for storing a plurality of regions which define upper limit values of the zoom ratio within a potential image pickup grant region within which pickup image can be done by changing the angle of the camera, the upper limit zoom ratios of the individual regions being different from each other; and

zoom control means for, when the angle of the camera is controlled in accordance with the instructions from the client to pickup an image, controlling an upper limit value of the zoom ratio in a direction of the camera based on the angle not to exceed the upper limit value stored in said storage means.

29. The server according to

claim 28

, wherein said storage means stores regions having different patterns and different sizes in units of levels of clients, and

said zoom control means determines an upper limit of the zoom ratio in accordance with the region information corresponding to the level of the client who remote-controls the camera.

30. A control method for a camera server for controlling a camera in accordance with instructions of an angle and a zoom ratio from a client on a network to pickup an image, and transmitting the pickup image to the client, comprising:

the storage step of storing, in predetermined storage means, a plurality of regions which define upper limit values of the zoom ratio within a potential image pickup grant region within which pickup image can be done by changing the angle of the camera, the upper limit zoom ratios of the individual regions being different from each other; and

the zoom control step of controlling an upper limit value of the zoom ratio in a direction of the camera based on the angle not to exceed the upper limit value stored in the storage means when the angle of the camera is controlled in accordance with the instructions from the client to pickup an image.

32. The device according to

claim 30

, wherein the status information includes information indicating image pickup directions of the cameras, and detection directions and detection ranges of the microphones, and

said first and second display means display the camera and microphone icons in accordance with the image pickup direction information and the detection direction information, respectively, and also display each of the microphone icons using two line segments that define the detection range.

31. A client device which is arranged on a network and remote-controls one of a plurality of camera servers each having a camera and microphone via the network, comprising:

output means for receiving an image and sound from the selected camera server, displaying the image, and outputting the sound;

first display means for receiving status information including position information of all the cameras and all the microphones transferred via the network, and displaying icons representing the cameras and microphones on a predetermined map on the basis of the received status information; and

second display means for displaying the icons of the camera and microphone as sources of the image and sound output by said output means to be distinguished from other icons.

33. A control method for a client device which is arranged on a network and remote-controls one of a plurality of camera servers each having a camera and microphone via the network, comprising:

the output step of receiving an image and sound from the selected camera server, displaying the image, and outputting the sound;

the first display step of receiving status information including position information of all the cameras and all the microphones transferred via the network, and displaying icons representing the cameras and microphones on a predetermined map on the basis of the received status information; and

the second display step of displaying the icons of the camera and microphone as sources of the image and sound output in the output step to be distinguished from other icons.

34. A camera control system for selecting at least one of cameras connected on a network, and transmitting control command information of the camera and audio information near the camera, comprising:

setting means for setting a display inhibition region of an image input from the camera;

display inhibition means for, when a direction range of the display inhibition region set by said setting means overlaps an image pickup range, substantially inhibiting a display of the display inhibition region in the direction range;

audio input means for inputting sound near the camera; and

audio output inhibition means for, when said display inhibition means substantially inhibits the display of the display inhibition region in the direction range, inhibiting the sound input by said audio input means from being output.

35. The system according to

claim 34

, further comprising control means for controlling directivity of the sound in accordance with the image input by the camera.

36. The system according to

claim 34

, further comprising:

camera index means for indicating at least one of a position and image pickup direction of the camera;

microphone index means for indicating at least one of a position and directivity of said audio input means; and

map display means for displaying a map on which said camera index means and said microphone index means are superimposed.

37. The system according to

claim 36

, wherein said setting means sets the display inhibition region of the image on the map display by said map display means.

38. The system according to

claim 37

, further comprising user storage means for storing at least one user of said camera control system, and

wherein said setting means can select display inhibition regions of an image in units of users stored in said user storage means.

39. The system according to any one of claims 34, further comprising:

region storage means for storing the display inhibition region of an image set by said setting means; and

changing means for changing the display inhibition region stored in said region storage means.

40. The system according to

claim 34

, wherein said display inhibition means makes a display in the display inhibition region of an image set by said setting means different from a display of an image input by the camera.

41. The system according to

claim 35

, wherein the directivity of said audio input means includes at least one of an audio input range, audio input direction, and audio input sensitivity.

42. The system according to

claim 41

, wherein said microphone index means that indicates said audio input means, which is inputting sound in communication, is displayed as an index different from other microphone index means.

43. A camera server which has an image sensing device and transfers sensed image data to one or a plurality of remote clients, comprising:

storage means for storing groups to which the clients belong, and control limitation information in units of groups;

determination means for determining the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and

control means for performing camera control in accordance with the control limitation information based on the determination result of said determination means upon reception of a camera control request from the client who has the right of operation.

44. The server according to

claim 43

, wherein the control limitation information includes a pan range, tilt range, and zoom range of the image sensing device.

45. The server according to

claim 44

, wherein the zoom range varies depending on a view point direction of the image sensing device.

46. The server according to

claim 43

, wherein the control limitation information includes a region where a normal display can be done and a region where a resolution is dropped when pan and tilt angles are changed.

47. A control method for a camera server which has an image sensing device and transfers sensed image data to one or a plurality of remote clients, comprising:

the determination step of determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and

the control step of performing camera control in accordance with the control limitation information based on the determination result in the determination step upon reception of a camera control request from the client who has the right of operation.

48. The method according to

claim 47

, wherein the control limitation information includes a pan range, tilt range, and zoom range of the image sensing device.

49. The method according to

claim 48

, wherein the zoom range varies depending on a view point direction of the image sensing device.

50. The method according to

claim 47

, wherein the control limitation information includes a region where a normal display can be done and a region where a resolution is dropped when pan and tilt angles are changed.

51. A storage medium which stores a program that implements a server for transferring an image sensed by an image sensing device to a remote client in accordance with an instruction from the remote client, storing program codes that implement:

determination means for determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and

control means for performing camera control in accordance with the control limitation information based on the determination result in said determination means upon reception of a camera control request from the client who has the right of operation.

52. A camera client connected to a camera server for transferring an image pickup by a camera, comprising:

reception means for receiving data including angle information and image information;

first display means for displaying a current angle state and an angle change range of the camera in accordance with the angle information and angle change allowable range information of image data included in the received angle information; and

second display means for displaying the image data included in the received data.

53. A control method for a camera client connected to a camera server for transferring an image pickup by a camera, comprising:

the reception step of receiving data including angle information and image information;

the first display step of displaying a current angle state and an angle change range of the camera in accordance with the angle information and angle change allowable range information of image data included in the received angle information; and

the second display step of displaying the image data included in the received data.

54. A storage medium which stores a program that implements a camera client for receiving and displaying an image from a remote camera server, and controlling a camera of the camera server, storing program codes that implement:

determination means for determining, with reference to storage means which stores groups to which the clients belong, and control limitation information in units of groups, the group to which the client who has a right of operation to the image sensing device among the connected clients belongs; and

control means for performing camera control in accordance with the control limitation information based on the determination result in said determination means upon reception of a camera control request from the client who has the right of operation.

55. A camera control system for selecting at least one of cameras connected on a network, and transmitting control command information of the camera and audio information near the camera, comprising:

setting means for setting a display inhibition region of an image input from the camera;

display inhibition means for, when a direction range of the display inhibition region set by said setting means overlaps an image pickup range, substantially inhibiting a display of the display inhibition region in the direction range;

audio input means for inputting sound near the camera; and

audio input inhibition means for, when said display inhibition means substantially inhibits the display of the display inhibition region in the direction range, inhibiting said audio input means from inputting the sound.

56. The system according to

claim 55

, further comprising control means for controlling directivity of the sound in accordance with the image input by the camera.

57. The system according to

claim 55

, further comprising:

camera index means for indicating at least one of a position and image pickup direction of the camera;

microphone index means for indicating at least one of a position and directivity of said audio input means; and

map display means for displaying a map on which said camera index means and said microphone index means are superimposed.

58. The system according to

claim 57

, wherein said setting means sets the display inhibition region of the image on the map display by said map display means.