Two-way interactive system, terminal equipment and image pickup apparatus having mechanism for matching lines of sight between interlocutors through transmission means

• US 5,815,197 A
• Filed: 02/16/1996
• Issued: 09/29/1998
• Est. Priority Date: 02/16/1995
• Status: Expired due to Term

• Current Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Original Assignee: Sumitomo Electric Industries Limited (Sumitomo Corporation)
• Inventors: Kakii, Toshiaki
• Primary Examiner(s): Kuntz, Curtis A.
• Assistant Examiner(s): Mei, Xu
• Application Number:
  US08/606,044
• Time in Patent Office: 956 Days
• Field of Search: 381/14-20
• US Class Current: 348/14.16
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

This invention is to provide a system having a mechanism for matching lines of sight between interlocutors through transmission means to realize smooth counseling between the interlocutors in remote areas. In this system, a terminal equipment is located at the point of each interlocutor, having a monitor for displaying at least the image of a partner, a position regulation means for regulating the position of the interlocutor with respect to a monitor screen, and an image pickup apparatus having an image pickup unit for picking up the image of the interlocutor and a support mechanism for setting the image pickup unit at a predetermined position between the interlocutor and the monitor. Particularly, the image pickup unit is set in a line-of-sight matching area defined by the distance from the interlocutor to the monitor, which is specified by the position regulation means, and a parallactic angle for determining that the line of sight of the interlocutor matches the line of sight of the monitor image of the partner.

Claims

1. A two-way interactive system comprising a first terminal equipment located at a first point where a first interlocutor is present, a second terminal equipment located at a second point where a second interlocutor is present, and transmission means, arranged between said first terminal equipment and said second terminal equipment, for enabling two-way communication of at least image data of the first interlocutor and the second interlocutor, at least said first terminal equipment comprising:

a display unit for displaying an image of the second interlocutor, which is received through said transmission means, on a monitor;

position regulation means for regulating a position of the first interlocutor with respect to a monitor screen of said display unit; and

an image pickup apparatus for enabling transmission of an image of the first interlocutor to said second terminal equipment through said transmission means, said image pickup apparatus having an image pickup unit for directly picking up the image of the first interlocutor and a support mechanism for setting said image pickup unit at a predetermined position between the first interlocutor and said monitor while an image pickup direction of said image pickup unit is set directly toward the first interlocutor,

wherein said image pickup unit is set in a predetermined area defined by a distance from said monitor to the first interlocutor, which is specified by said position regulation means, and a predetermined parallactic angle for determining that a line of sight of the first interlocutor matches a line of sight of the monitor image of the second interlocutor.

2. A two-way interactive system according to claim 1, wherein said image pickup unit is set in the predetermined area defined by the distance specified by said position regulation means and the predetermined parallactic angle, whereby the monitor image of the second interlocutor is partially shielded when viewed from the first interlocutor.

3. A two-way interactive system according to claim 1, wherein the parallactic angle for matching the line of sight of the first interlocutor and the line of sight of the monitor image of the second interlocutor is not more than 4.5°

in a horizontal direction of said monitor screen, not more than 12°

in an upper direction, and not more than 8°

in a lower direction with respect to the line of sight of the first interlocutor when viewed from the first interlocutor.

4. A two-way interactive system according to claim 3, wherein

the parallactic angle for matching the line of sight of the first interlocutor and the line of sight of the monitor image of the second interlocutor is not more than 3°

in the horizontal and vertical directions of said monitor screen with respect to the line of sight of the first interlocutor when viewed from the first interlocutor.

5. A two-way interactive system according to claim 1, wherein said first terminal equipment further comprises a processing unit for recognizing a position of the image of the first interlocutor on said monitor screen, the image being displayed on said monitor, and

wherein said image pickup apparatus of said first terminal equipment further comprises a driving mechanism for relatively moving a position of said image pickup unit with respect to the first interlocutor whose position is recognized by said processing unit, or changing an image pickup angle of said image pickup unit.

6. A two-way interactive system according to claim 5, wherein said second terminal equipment comprises at least a second point image pickup unit for picking up the image of the second interlocutor, and a second point driving mechanism for moving a position of said second point image pickup unit or changing an image pickup angle of said second point image pickup unit, and

wherein said processing unit of said first terminal equipment recognizes a position of the image of the second interlocutor on said monitor screen, the image being received through said transmission means and displayed on said monitor, and transmits a control signal for relatively moving the position of said second point image pickup unit with respect to the second interlocutor or changing the image pickup angle of said second point image pickup unit to said second terminal equipment through said transmission means.

7. A two-way interactive system according to claim 1, wherein said first terminal equipment further comprises a processing unit for recognizing a position of the image of the first interlocutor on said monitor screen, the image being displayed on said monitor, and

wherein said position regulation means comprises a mechanism for adjusting a position of an eye of the first interlocutor to be picked up with respect to a position of an eye of the monitor image of the first interlocutor whose position is recognized by said processing unit.

8. A two-way interactive system according to claim 7, wherein said second terminal equipment comprises at least a second point display unit for displaying the image of the first interlocutor on a monitor, and second point position regulation means for regulating a position of the second interlocutor, said second point regulation means having a mechanism for adjusting a position of an eye of the second interlocutor with respect to a position of an eye of the monitor image, and

wherein said processing unit of said first terminal equipment recognizes a position of the image of the second interlocutor on said monitor, the image being received through said transmission means and displayed on said monitor, and transmits a control signal for adjusting the position of the eye of the second interlocutor with respect to the position of the eye of the monitor image on said second point display unit to said second terminal equipment through said transmission means.

9. A two-way interactive system according to claim 1, wherein said support mechanism of said image pickup apparatus comprises a transparent member for directly supporting said image pickup unit.

10. A two-way interactive system according to claim 1, wherein said position regulation means includes a chair having a sitting height adjustment mechanism.

11. A two-way interactive system according to claim 1, wherein said position regulation means includes a table set between the first interlocutor and said display unit and having a predetermined height.

12. A two-way interactive system according to claim 1, wherein said position regulation means includes a table having a predetermined height, on which said display unit is directly set.

13. A two-way interactive system according to claim 1, wherein said first terminal equipment comprises an interactive operation processing unit for counting a total interactive operation time, an average interactive operation time, and the number of interactive operations during a two-way interactive operation between the first interlocutor and the second interlocutor, and sequentially displaying the total interactive operation time, the average interactive operation time, the number of interactive operations, and a distribution of the number of interactive operations on said display unit.

14. A two-way interactive system according to claim 1, wherein first terminal equipment comprises a sound absorbing member having external sound insulating properties and at least partially arranged around the first interlocutor.

15. A two-way interactive system according to claim 1, wherein said first terminal equipment comprises a voice/acoustic data processing unit for receiving, as an electrical signal, voice of the first interlocutor, which is gathered by a voice input unit, and a voice/acoustic output control unit for causing a voice output unit to reproduce voice data of the second interlocutor, which is transmitted through said transmission means.

16. A two-way interactive system according to claim 15, wherein said voice output unit includes a pair of speakers arranged near ears aside the first interlocutor.

17. A two-way interactive system according to claim 15, wherein said voice input unit includes a directional high-sensitive microphone.

18. A two-way interactive system according to claim 15, wherein said voice/acoustic data processing unit detects an average level of input volume levels for a predetermined time and automatically adjusts the volume when a difference between the input volume level and the average level exceeds a predetermined value.

19. A two-way interactive system according to claim 15, wherein said voice/acoustic output control unit detects that an interactive operation volume is substantially zero for a predetermined time and, on the basis of a detection result, at least starts a background music or performs adjustment to increase or decrease a volume of the background music.

20. A two-way interactive system according to claim 15, wherein said voice/acoustic data processing unit performs statistical analysis processing in which a voice spectrum of the first interlocutor or a voice spectrum according to transmitted voice information of the second interlocutor is analyzed over time, and detects an average spectrum and a voice spectrum changed beyond a predetermined value.

21. A two-way interactive system according to claim 1, wherein said display unit has a through hole causing a rear surface of said display unit to communicate with said monitor screen on a front surface of said display unit, and

wherein said image pickup unit is attached to said display unit through said support mechanism while a distal end of said image pickup unit extends through the through hole of said display unit and projects forward from said monitor screen.

22. A two-way interactive system according to claim 21, wherein said image pickup unit is attached to said display unit through said support mechanism to freely rotate about a predetermined portion of said image pickup unit, whereby the image pickup direction of said image pickup unit is changed in a desired direction.

23. A two-way interactive system according to claim 22, wherein said first terminal equipment comprises a driving mechanism for changing an attachment angle between said display unit and said image pickup unit.

24. A two-way interactive system according to claim 1, wherein said first terminal equipment further comprises a processing unit for adjusting a display position of the image of the second interlocutor, which is displayed on said monitor, said processing unit

specifying a reference point at a virtual position on the monitor image where said image pickup unit should be originally present on the basis of the image of the second interlocutor, the image being received through said transmission means and displayed on said monitor,

detecting a shift of the monitor image of the second interlocutor on the basis of coordinates on said monitor corresponding to an actual set position of said image pickup unit and coordinates of the obtained reference point on said monitor,

calculating a vector starting from the coordinates of the reference point on said monitor and terminating at the coordinates on said monitor corresponding to the set position of said image pickup unit, and

translating the monitor image of the second interlocutor on the basis of the obtained vector.

25. A two-way interactive system according to claim 24, wherein said processing unit detects display positions of both eyes of the image of the second interlocutor, the image being received through said transmission means and displayed on said monitor, and specifies the reference point at a position on said monitor, which is separated upward from a center between both the eyes by a predetermined distance on said monitor.

26. A two-way interactive system according to claim 24, wherein said processing unit specifies the reference point at a position on said monitor corresponding to a parietal portion of the monitor image of the second interlocutor, the image being received through said transmission means and displayed on said monitor.

27. A two-way interactive system according to claim 1, wherein said first terminal equipment comprises a display control unit for enlarging or reducing a person'"'"'s image displayed on said monitor, said display control unit detecting part of the person'"'"'s image corresponding to an actual set position of said image pickup unit near a parietal portion of the person'"'"'s image and enlarging or reducing the entire person'"'"'s image while fixing the detected part of the person'"'"'s image.

28. A two-way interactive system according to claim 27, wherein said display control unit displays at least characters or graphics in a predetermined area on said monitor, except for an area where the person'"'"'s image is displayed.

29. A two-way interactive system according to claim 1, wherein said first terminal equipment comprises a person authentication unit for authenticating a user of said first terminal equipment before an interactive operation is started.

30. A two-way interactive system according to claim 29, wherein said first terminal equipment comprises a read unit for reading a predetermined item from a card-shaped external recording medium having at least an IC, in which the predetermined item is recorded in advance.

31. A two-way interactive system according to claim 30, wherein said external recording medium records, as the predetermined item, at least one of reservation information and information for performing person authentication of a user of said second terminal equipment as a partner for an interactive operation.

32. A two-way interactive system according to claim 30, wherein said external recording medium records, as the predetermined item, interactive operation conditions including at least a reserved interactive operation start time, a reserved interactive operation time, and interactive operation contents.

33. A two-way interactive system according to claim 30, wherein said external recording medium records, as the predetermined item, interactive operation reservation data including at least a reserved interactive operation start time and information associated with a partner for an interactive operation, and

wherein said first terminal equipment starts two-way communication with said second terminal equipment after said first terminal equipment is line-connected with said second terminal equipment searched through a communication network serving as said transmission means on the basis of the interactive operation reservation data.

34. A two-way interactive system according to claim 33, wherein said external recording medium records, as the predetermined item, at least one of character information, graphic information, and image information, and said first terminal equipment displays part of the information recorded in said external recording medium on said monitor of said first terminal equipment at least before an interactive operation, during an interactive operation, or after an interactive operation.

35. A two-way interactive system according to claim 34, wherein said first terminal equipment transmits control information for causing said second terminal equipment to display and output part of the information recorded in said external recording medium to said second terminal equipment through said transmission means at least before an interactive operation, during an interactive operation, or after an interactive operation.

36. A two-way interactive system according to claim 34, wherein said first terminal equipment outputs an acoustic medium including at least one of a background music and a voice recorded/stored inside or outside said first terminal equipment in advance in a manner interlocked with a display operation of said first terminal equipment of claim 32.

37. A two-way interactive system according to claim 34, wherein said first terminal equipment transmits a control signal for causing said second terminal equipment to output an acoustic medium including at least one of a background music and a voice recorded/stored inside or outside said first terminal equipment in advance to said second terminal equipment through said transmission means in a manner interlocked with a transmission operation of said first terminal equipment of claim 33.

38. An image pickup apparatus adapted to a two-way interactive system according to claim 1, comprising:

an image pickup unit for picking up an image of a first interlocutor;

a support mechanism having a transparent member to which said image pickup unit is attached while an image pickup direction of said image pickup unit is set directly toward the first interlocutor, said support mechanism setting said image pickup unit at a predetermined position; and

a processing unit for transmitting the image of the first interlocutor to predetermined transmission means as image data, the image being picked up by said image pickup unit, and receiving image data of a second interlocutor, the image data being transmitted from said transmission means.