Privacy filtering of area description file prior to upload

• US 10,033,941 B2
• Filed: 05/11/2015
• Issued: 07/24/2018
• Est. Priority Date: 05/11/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Williams, Brian Patrick, Hickman, Ryan Michael
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US14/708,955
• Publication Number:
  US 20160337599A1
• Time in Patent Office: 1,170 Days
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
• CPC Class Codes: H04W 64/00 Locating users or terminals or network equipment for network management purposes, e.g. mobility management

Abstract

A mobile device includes at least one imaging sensor to capture imagery of an environment of the mobile device, a privacy filter module, a spatial feature detection module, an assembly module, and a network interface. The privacy filter module is to perform at least one image-based privacy filtering process using the captured imagery to generate filtered imagery. The spatial feature detection module is to determine a set of spatial features in the filtered imagery. The assembly module is to generate an area description file representative of the set of spatial features. The network interface is to transmit the area description file to a remote computing system. The assembly module may select only a subset of the set of spatial features for inclusion in the area description file.

Claims

1. At a mobile device communicatively coupled to a remote computing system, a method comprising:

capturing imagery of an area at which the mobile device is located;

performing at least one image-based privacy filtering process using the captured imagery to generate filtered imagery;

determining a set of spatial features in the filtered imagery;

selecting a subset having fewer spatial features than the determined set of spatial features for inclusion in an area description file, the subset based on a maximum spatial feature density criterion; and

transmitting the area description file representative of the subset having fewer spatial features than the determined set of spatial features to the remote computing system.

2. The method of claim 1, wherein performing the at least one image-based privacy filtering process comprises:

performing a text detection process to detect one or more first regions of the imagery that potentially represent text content; and

modifying the imagery to remove image content from the detected one or more first regions.

3. The method of claim 2, wherein performing the at least one image-based privacy filtering process further comprises:

performing a face detection process to detect one or more second regions of the imagery that potentially represent human faces; and

modifying the imagery to remove image content from the detected one or more second regions.

4. The method of claim 1, wherein performing the at least one image-based privacy filtering process comprises:

performing a face detection process to detect one or more regions of the imagery that potentially represent human faces; and

modifying the imagery to remove image content from the detected one or more regions.

5. The method of claim 1, wherein the maximum spatial feature density criterion comprises a specified maximum number of spatial features per cubic unit.

6. At a mobile device communicatively coupled to a remote computing system, a method comprising:

capturing imagery of an area in which the mobile device is located;

determining a set of spatial features in the imagery;

downsampling the set of spatial features to generate a sparse set of spatial features, wherein the sparse set represents fewer spatial features than the determined set of spatial features in the imagery and is based on a maximum spatial feature density criterion; and

transmitting an area description file representing the sparse set of spatial features to the remote computing system.

7. The method of claim 6, wherein the maximum spatial feature density criterion comprises a specified maximum number of spatial features per cubic unit.

8. A mobile device comprising:

at least one imaging sensor to capture imagery of an environment of the mobile device;

at least one processor;

a storage component to store a set of executable instructions, the set of executable instructions configured to manipulate the at least one processor to;

perform at least one image-based privacy filtering process using the captured imagery to generate filtered imagery;

determine a set of spatial features in the filtered imagery;

select only a subset having fewer spatial features than the determined set of spatial features for inclusion in an area description file by selecting spatial features for the subset based on a maximum spatial feature density criterion;

generate the area description file representative of the subset having fewer spatial features than the determined set of spatial features; and

a network interface coupled to the at least one processor, the network interface to transmit the area description file to a remote computing system.

9. The mobile device of claim 8, wherein the set of executable instructions are further configured to manipulate the at least one processor to:

detect one or more first regions of the imagery that potentially represent text content and to modify the imagery to remove image content the detected one or more first regions.

10. The mobile device of claim 9, wherein the set of executable instructions are configured to manipulate the at least one processor to:

detect one or more second regions of the imagery that potentially represent human faces and to modify the imagery to remove image content the detected one or more second regions.

11. The mobile device of claim 8, wherein the set of executable instructions are configured to manipulate the at least one processor to:

detect one or more regions of the imagery that potentially represent human faces and to modify the imagery to remove image content the detected one or more regions.

12. The mobile device of claim 8, wherein the maximum spatial feature density criterion comprises a specified maximum number of spatial features per cubic unit.

13. A mobile device comprising:

at least one imaging sensor to capture imagery of an environment of the mobile device;

at least one processor;

a storage component to store a set of executable instructions, the set of executable instructions configured to manipulate the at least one processor to;

determine a set of spatial features in the imagery;

downsample the set of spatial features to determine a sparse set of spatial features having fewer spatial features than the determined set of spatial features in the imagery, the sparse set based on a maximum spatial feature density criterion;

generate an area description file representative of the sparse set of spatial features; and

a network interface coupled to the at least one processor, the network interface to transmit the area description file to a remote computing system.

14. The mobile device of claim 13, wherein the maximum spatial feature density criterion comprises a specified maximum number of spatial features per cubic unit.