Localisation and mapping

• US 9,779,509 B2
• Filed: 03/13/2015
• Issued: 10/03/2017
• Est. Priority Date: 07/30/2012
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Martini, Antonio
• Primary Examiner(s): WU, XIAO MIN
• Assistant Examiner(s): Elbinger, Steven
• Application Number:
  US14/657,296
• Publication Number:
  US 20150186745A1
• Time in Patent Office: 935 Days
• Field of Search: None
• CPC Class Codes: A63F 13/25 Output arrangements for video game devices
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 30/422 : Technical drawings; Geographical maps
• CPC Class Codes: H04N 13/207 : using a single 2D image sensor
• CPC Class Codes: H04N 13/211 : using temporal multiplexing
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method of detecting the presence of a building in a captured image is provided. The method includes dividing the captured image into a plurality of image regions, detecting a dominant angle of detected edges within each image region, and detecting the presence of a building in an image region if the dominant angle of detected edges within that image region, excluding edges within a first threshold angle of vertical, is within a second threshold angle of horizontal and if the distribution of detected edges within the second threshold angle of horizontal is substantially the same as that of at least one adjacent image region.

Claims

1. A method of detecting a presence of a building in a captured image, the method comprising:

dividing, by a processor, the captured image into a plurality of image regions;

detecting, by the processor, a dominant angle of detected edges within each of the plurality of image regions; and

detecting, by the processor, the presence of a building in a given one of the plurality of image regions if the dominant angle of detected edges within the given image region, excluding all substantially vertical edges, being edges that are within a first threshold angle of vertical, is substantially horizontal, being an angle within a second threshold angle of horizontal, and if a distribution of the substantially horizontal detected edges is substantially the same as that of at least one adjacent image region in both a horizontal and a vertical direction.

2. A method according to claim 1, in which the step of detecting the dominant angle comprises detecting a median value of detected edge directions.

3. A method according to claim 1, comprising the steps of:

capturing one or more images of the building using a camera;

detecting a gravitational vertical direction with respect to the camera so as to detect a direction of features within the first threshold angle of vertical in the captured images; and

removing features within the first threshold angle of vertical from the captured images.

4. A method according to claim 3, comprising:

detecting image features in the one or more captured images representing horizontal features of the building; and

detecting an orientation of the building, in the one or more captured images, from the detected vertical direction and the detected orientation of the horizontal features.

5. A method according to claim 4, comprising:

applying an image transformation based upon the detected orientation of the building, prior to the step of detecting the presence of the building.

6. A method according to claim 1, comprising the step of performing the step of detecting the presence of a building in respect of an array of image regions.

7. A method of detecting an orientation of a building in an image captured by a camera, the method comprising the steps of:

detecting, by a processor, a gravitational vertical direction with respect to the camera so as to detect a direction of vertical features of the building in the captured image;

detecting, by a processor, image features in the captured image representing horizontal features of the building, excluding image features that are substantially vertical, being within a threshold angle of vertical according to the detected gravitational vertical direction; and

detecting, by a processor, the orientation of the building, in the captured image, from the detected vertical direction and a detected orientation of the horizontal features.

8. A non-transitory computer readable medium storing computer instructions thereon, the instructions, when executed by a processor, cause the processor to carry out a method of detecting a presence of a building in a captured image, the method comprising:

dividing the captured image into a plurality of image regions;

detecting a dominant angle of detected edges within each of the plurality of image regions; and

detecting the presence of a building in a given one of the plurality of image regions if the dominant angle of detected edges within the given image region, excluding all substantially vertical edges, being edges that are within a first threshold angle of vertical, is substantially horizontal, being an angle within a second threshold angle of horizontal, and if a distribution of the substantially horizontal detected edges is substantially the same as that of at least one adjacent image region in both a horizontal and a vertical direction.

9. A non-transitory computer readable medium storing computer instructions thereon, the instructions, when executed by a processor, cause the processor to carry out a method of detecting an orientation of a building in an image captured by a camera, the method comprising the steps of:

detecting a gravitational vertical direction with respect to the camera so as to detect a direction of vertical features of the building in the captured image;

detecting image features in the captured image representing horizontal features of the building, excluding image features that are substantially vertical, being within a threshold angle of vertical according to the detected gravitational vertical direction; and

detecting the orientation of the building, in the captured image, from the detected vertical direction and a detected orientation of the horizontal features.

10. Image processing apparatus operable to detect a presence of a building in a captured image, the apparatus comprising:

an image region processor configured to divide the captured image into a plurality of image regions and to detect a dominant angle of detected edges within each of the plurality of image regions; and

a processor configured to detect the presence of a building in a given one of the plurality of image regions if the dominant angle of detected edges within the given image region, excluding all substantially vertical edges, being edges that are within a first threshold angle substantially of vertical, is substantially horizontal, being an angle within a second threshold angle of horizontal, and if a distribution of the substantially horizontal detected edges is substantially the same as that of at least one adjacent image region in both a horizontal and a vertical direction.

11. Image processing apparatus operable to detect an orientation of a building in an image captured by a camera, the apparatus comprising:

a gravity detector configured to detect a gravitational vertical direction with respect to the camera so as to detect a direction of vertical features of the building in the captured image; and

an orientation detector configured to detect image features in the captured image representing horizontal features of the building excluding image features that are substantially vertical, being within a threshold angle of vertical as detected by the gravity detector, and to detect the orientation of the building, in the captured image, from the detected vertical direction and a detected orientation of the horizontal features.

12. Computer games apparatus comprising:

an image processing apparatus configured to detect a presence of a building in a captured image, the image processing apparatus comprising;

an image region processor configured to divide the captured image into a plurality of image regions and to detect a dominant angle of detected edges within each of the plurality of image regions, and

a processor configured to detect the presence of a building in a given one of the plurality of image regions if the dominant angle of detected edges within the given image region, excluding all substantially vertical edges, being edges that are within a first threshold angle of vertical, is substantially horizontal, being an angle within a second threshold angle of horizontal, and if a distribution of the substantially horizontal detected edges is substantially the same as that of at least one adjacent image region in both a horizontal and a vertical direction; and

an image renderer configured to render an image for display by combining a particular image with one or more computer-generated image portions generated with respect to detected buildings in the captured image.

13. Apparatus according to claim 12, comprising a camera configured to capture the images.

14. Computer games apparatus comprising:

an image processing apparatus configured to detect an orientation of a building in an image captured by a camera, the apparatus comprising;

a gravity detector configured to detect a gravitational vertical direction with respect to the camera so as to detect a direction of vertical features of the building in the captured image, and

an orientation detector configured to detect image features in the captured image representing horizontal features of the building excluding image features that are substantially vertical, being within a threshold angle of vertical as detected by the gravity detector, and to detect the orientation of the building, in the captured image, from the detected vertical direction and a detected orientation of the horizontal features; and

an image renderer configured to render an image for display by combining a particular image with one or more computer-generated image portions generated with respect to detected buildings in the captured image.

15. Apparatus according to claim 14, comprising a camera configured to capture the images.