Adjustment of perceived roundness in stereoscopic image of a head

• US 10,356,383 B2
• Filed: 12/24/2015
• Issued: 07/16/2019
• Est. Priority Date: 12/24/2014
• Status: Active Grant

• Current Assignee: Reald Spark, LLC
• Original Assignee: Reald Spark, LLC
• Inventors: Woodgate, Graham J., Robinson, Michael G., Sommerlade, Eric
• Primary Examiner(s): Tran, Loi H
• Application Number:
  US14/757,893
• Publication Number:
  US 20160219258A1
• Time in Patent Office: 1,300 Days
• Field of Search: 348 43
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

In telecommunication video calling and videoconferencing systems, it is strongly desirable for remote observers to interact with natural gaze cues. In natural gaze interaction, the camera for a source observer appears to be co-located in the eye region of a destination observer image and vice versa. The appearance of camera co-location is achieved for stereoscopic camera pair that are placed either side of an autostereoscopic 3D display. Such cameras typically provide stereoscopic images that have disparity distributions that provide unnatural perceived head roundness. The present embodiments achieve perceived head roundness that is closer to that expected in natural face to face interaction by modification of perceived local depth in detected regions of the head.

Claims

1. A method of adjusting perceived roundness in a stereoscopic pair of images that image a head, the method comprising processing one or each image by the following steps:

identifying, by a computer apparatus, at least one region of the image that images a part of the head; and

locally transforming, by the computer apparatus, the at least one region in a manner that adjusts the perceived local depth in accordance with expected local depth in the imaged part of the head,

wherein the at least one region of the image includes a reference feature between first and second head features, and

wherein the transforming of the at least one region includes;

translating pixels of the at least one region in a first direction in a first image of the pair of images and in a second direction in a second image of the pair of images; and

resampling pixels of a second region adjacent the at least one region in both of the pair of images, the resampling including shifting pixels by respective distances that vary depending on respective distances from the at least one region.

2. A method according to claim 1, wherein the step of locally transforming the at least one region is performed in a manner that adjusts the perceived local depth by adjusting the disparity between the stereoscopic pair of images.

3. A method according to claim 1, wherein the step of locally transforming the at least one region is performed in a manner that adjusts the perceived local depth by re-sampling parts of a region that are occluded in the other image.

4. A method according to claim 1, wherein the at least one region of the image includes a nasal region that images a part of the head including the nose.

5. A method according to claim 4, wherein

the step of identifying the nasal region further comprises identifying a nasal end region that images the end of the nose, the nasal region encompassing the nasal end region, and

the step of locally transforming the nasal region comprises:

translating the nasal end region; and

re-sampling an outer region of the nasal region outside the nasal end region.

6. A method according to claim 4, wherein the step of locally transforming the nasal region comprises:

detecting image features from the image;

on the basis of the detected image features, selecting a transformation from a stored set of transformations of the nasal region; and

performing the selected transformation.

7. A method according to claim 1, wherein the at least one region of the image includes a side region that images the side of the head closest to the image capture position of the image and extends to a boundary at the extremity of the head in the image.

8. A method according to claim 7, wherein the step of locally transforming the side region comprises:

translating the boundary of the side region at the extremity of the head in the image; and

re-sampling the side region.

9. A method according to claim 8, wherein the step of locally transforming the side region comprises:

filling the background of the image that is uncovered by re-sampling a region of the background outside the boundary of the side region.

10. A method according to claim 1, wherein the step of identifying at least one region of the image comprises:

detecting facial features of the imaged head in the image; and

identifying the at least one region in relation to the detected facial features.

11. A method according to claim 1, wherein said step of processing one or each image comprises processing each image.

12. A method according to claim 1, wherein the step of locally transforming the at least one region is performed in a manner that adjusts the perceived local depth by reducing the magnitude of the disparity between the stereoscopic pair of images by an amount that depends on the magnitude of an expected depth in the imaged part of the head with respect to a reference feature of the head, and wherein a larger magnitude of the expected depth in the imaged part of the head with respect to the reference feature of the head results in a greater reduction of the magnitude of the disparity between the stereoscopic pair of images.

13. A method according to claim 12, wherein the reference feature is the eyes of the head.

14. A method according to claim 12, wherein the reference feature is the outside of the cheeks of the head.

15. A method according to claim 12, further comprising, after said processing of one or both of the images, stereoscopically displaying the stereoscopic pair of images with an overall disparity between the stereoscopic pair of images that provides a disparity of zero between the stereoscopic pair of images for the reference feature of the head.

16. A method according to claim 12, further comprising, after said processing of one or both of the images, stereoscopically displaying the stereoscopic pair of images.

17. A method according to claim 15, wherein said step of stereoscopically displaying the stereoscopic pair of images comprises stereoscopically displaying the stereoscopic pair of images on a destination telecommunication device to which the stereoscopic pair of images are transmitted over a telecommunication network.

18. A method according to claim 1, further comprising, before said processing of one or both of the images, capturing the stereoscopic pair of images.

19. A method according to claim 17, wherein said step of capturing the stereoscopic pair of images comprises capturing the stereoscopic pair of images on a source telecommunication device which transmits the stereoscopic pair of images over a telecommunication network.

20. A non-transitory computer readable medium storing a computer-executable program, the computer-executable program, when executed, causing a computer apparatus to adjust a stereoscopic pair of images that image a head by performing the following steps:

processing one or each image by the following steps;

identifying at least one region of the image that images a part of the head; and

locally transforming the at least one region in a manner that adjusts the perceived local depth in accordance with expected local depth in the imaged part of the head,

wherein the at least one region of the image includes a reference feature between first and second head features, and

wherein the transforming of the at least one region includes;

translating pixels of the at least one region in a first direction in a first image of the pair of images and in a second direction in a second image of the pair of images; and

resampling pixels of a second region adjacent the at least one region in both of the pair of images, the resampling including shifting pixels by respective distances that vary depending on respective distances from the at least one region.

21. A device for adjusting perceived roundness in a stereoscopic pair of images that image a head, the device comprising at least one computer apparatus arranged to process one or each image by:

identifying, by the computer apparatus, at least one region of the image that images a part of the head;

locally transforming, by the computer apparatus, the at least one region in a manner that adjusts the perceived local depth in accordance with expected local depth in the imaged part of the head,

wherein the at least one region of the image includes a reference feature between first and second head features, and

wherein the transforming of the at least one region includes;

translating pixels of the at least one region in a first direction in a first image of the pair of images and in a second direction in a second image of the pair of images; and

resampling pixels of a second region adjacent the at least one region in both of the pair of images, the resampling including shifting pixels by respective distances that vary depending on respective distances from the at least one region.

22. A device according to claim 21, wherein the device is communicatively coupled via a telecommunications network.