Overlay for camera field of vision

• US 10,395,431 B2
• Filed: 12/08/2017
• Issued: 08/27/2019
• Est. Priority Date: 11/02/2015
• Status: Active Grant

• Current Assignee: Roblox Corp.
• Original Assignee: International Business Machines Corporation
• Inventors: Akselrod, Ben Z., Di Loreto, Anthony, McDuff, Steve, Robeson, Kyle D.
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US15/835,549
• Publication Number:
  US 20180101992A1
• Time in Patent Office: 627 Days
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

One or more computer processors generate an augmented reality overlay for camera field of vision based on a perspective of a display device. The one or more processors generate an overlay image that represents a field of view of an imaging device based at least on an orientation of the imaging device at a first location. The one or more processors generate a modified overlay image by modification of one or both of a size and an orientation of the overlay image based, at least in part, on differences between (i) a second location and an orientation of an image display device relative to (ii) the first location and the orientation of the imaging device. The one or more processors the modified overlay image using the image display device.

Claims

1. A method for generating an augmented reality overlay for camera field of vision based on a perspective of a display device, the method comprising:

generating, by one or more processors, a shape as a visual guide in an overlay image, wherein the shape indicates what tangible content is included within a field of view of an imaging device based at least on an orientation of the imaging device at a first location;

generating, by the one or more processors, a modified overlay image by modification of one or both of a size and an orientation of the shape in the overlay image based, at least in part, on differences between (i) a second location and an orientation of an image display device relative to (ii) the first location and the orientation of the imaging device, wherein the image display device and the imaging device are at different locations; and

displaying, by the one or more processors, the modified overlay image using the image display device.

2. The method of claim 1, wherein the image display device is worn by a user such that the vision of that user includes the modified overlay image.

3. The method of claim 1, the method further comprising:

determining, by the one or more processors, the first location of the imaging device based, at least in part, on a data from at least one of an indoor positioning system, a global positioning system, and one or more orientation sensors, wherein a measure of a depth of field of the field of view of the imaging device is based, at least in part, on the data.

4. The method of claim 1, the method further comprising:

identifying, by the one or more processors, a known object included in an image data received from the imaging device; and

determining, by the one or more processors, the first location of the imaging device based, at least in part, on a comparison between a known orientation and known size of the known object and an apparent orientation and apparent size of the known object as included in the image data.

5. The method of claim 1, wherein the modified overlay image includes at least one of a) three dimensional shape with one or more edges that outline at least a portion of the field of view of the imaging device, b) a first content that is viewable by the imaging device, c) a second content as seen through an obstacle, d) a depth view of the imaging device, and e) a visual identifier that indicates an object that is obstructed from the field of view of the imaging device.

6. The method of claim 1, the method comprising:

determining, by the one or more processors, that a type of criterion has been met that dictate that the overlay image is to be updated; and

updating, by the one or more processors, the overlay image based, at least in part, on the type of criterion.

7. The method of claim 6, wherein the type of criterion includes at least one of a) passage of a predetermined time period, b) a change in one or both of the second location of the image display device and the first location of the imaging device, c) a change in a content being captured by the imaging device, and d) a change in one or more settings of the imaging device.

8. A computer program product for generating an augmented reality overlay for camera field of vision based on a perspective of a display device, the computer program product comprising:

one or more computer-readable storage media and program instructions stored on the one or more computer-readable storage media for execution by one or more processors, the program instructions comprising;

program instructions to generate a shape as a visual guide in an overlay image, wherein the shape indicates what tangible content is included within a field of view of an imaging device based at least on an orientation of the imaging device at a first location;

program instructions to generate a modified overlay image by modification of one or both of a size and an orientation of the shape in the overlay image based, at least in part, on differences between (i) a second location and an orientation of an image display device relative to (ii) the first location and the orientation of the imaging device, wherein the image display device and the imaging device are at different locations; and

program instructions to display the modified overlay image using the image display device.

9. The computer program product of claim 8, wherein the image display device is worn by a user such that the vision of that user includes the modified overlay image.

10. The computer program product of claim 8, the program instructions further comprising:

program instructions to determine the first location of the imaging device based, at least in part, on a data from at least one of an indoor positioning system, a global positioning system, and one or more orientation sensors, wherein a measure of a depth of field of the field of view of the imaging device is based, at least in part, on the data.

11. The computer program product of claim 8, the program instructions further comprising:

program instructions to identify a known object included in an image data received from the imaging device; and

program instructions to determine the first location of the imaging device based, at least in part, on a comparison between a known orientation and known size of the known object and an apparent orientation and apparent size of the known object as included in the image data.

12. The computer program product of claim 8, wherein the modified overlay image includes at least one of a) three dimensional shape with one or more edges that outline at least a portion of the field of view of the imaging device, b) a first content that is viewable by the imaging device, c) a second content as seen through an obstacle, d) a depth view of the imaging device, and e) a visual identifier that indicates an object that is obstructed from the field of view of the imaging device.

13. The computer program product of claim 8, the program instructions comprising:

program instructions to determine that a type of criterion has been met that dictates that the overlay image is to be updated; and

program instructions to update the overlay image based, at least in part, on the type of criterion.

14. The computer program product of claim 13, wherein the type of criterion includes at least one of a) passage of a predetermined time period, b) a change in one or both of the second location of the image display device and the first location of the imaging device, c) a change in a content being captured by the imaging device, and d) a change in one or more settings of the imaging device.

15. A computer system for generating an augmented reality overlay for camera field of vision based on a perspective of a display device, the computer system comprising:

one or more computer processors;

one or more computer readable storage medium;

program instructions stored on the computer readable storage medium for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to generate a shape as a visual guide in an overlay image, wherein the shape indicates what tangible content is included within a field of view of an imaging device based at least on an orientation of the imaging device at a first location;

program instructions to generate a modified overlay image by modification of one or both of a size and an orientation of the shape in the overlay image based, at least in part, on differences between (i) a second location and an orientation of an image display device relative to (ii) the first location and the orientation of the imaging device, wherein the image display device and the imaging device are at different locations; and

program instructions to display the modified overlay image using the image display device.

16. The computer system of claim 15, wherein the image display device is worn by a user such that the vision of that user includes the modified overlay image.

17. The computer system of claim 15, the program instructions further comprising:

program instructions to determine the first location of the imaging device based, at least in part, on a data from at least one of an indoor positioning system, a global positioning system, and one or more orientation sensors, wherein a measure of a depth of field of the field of view of the imaging device is based, at least in part, on the data.

18. The computer system of claim 15, the program instructions further comprising:

program instructions to identify a known object included in an image data received from the imaging device; and

program instructions to determine the first location of the imaging device based, at least in part, on a comparison between a known orientation and known size of the known object and an apparent orientation and apparent size of the known object as included in the image data.

19. The computer system of claim 15, wherein the modified overlay image includes at least one of a) three dimensional shape with one or more edges that outline at least a portion of the field of view of the imaging device, b) a first content that is viewable by the imaging device, c) a second content as seen through an obstacle, d) a depth view of the imaging device, and e) a visual identifier that indicates an object that is obstructed from the field of view of the imaging device.

20. The computer system of claim 15, the program instructions comprising:

program instructions to determine that a type of criterion has been met that dictates that the overlay image is to be updated; and

program instructions to update the overlay image based, at least in part, on the type of criterion.