EXPANDING A DIGITAL REPRESENTATION OF A PHYSICAL PLANE

• US 20150062166A1
• Filed: 08/28/2014
• Published: 03/05/2015
• Est. Priority Date: 08/30/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Pan, Qi, Siklossy, Istvan, Reisner-Kollmann, Irene, Gervautz, Michael, Nielsen, Per Orum, Lawrence Ashok Inigo, Roy
• Granted Patent: US 9,595,125 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

Techniques are presented for expanding a digital representation of a physical plane from a physical scene. In some aspects, a method may include determining an orientation and an initial portion of a physical plane in the scene, and subdividing a rectified image for the scene into a plurality of grid cells. For the grid cells, an image signature may be generated. A grid cell contiguous to the obtained initial portion of the plane is determined to include part of the plane. An iterative process may be performed for each neighboring grid cell from the grid cell contiguous to at least part of the obtained initial portion, determining whether the neighboring grid cell is to be included as part of the plane if the image signature of said neighboring grid cell is similar to the image signature of a grid cell already determined to be included as part of the plane.

Claims

1. A method for constructing a digital representation of a physical plane from a physical scene, the method comprising:

determining an orientation and an initial portion of a physical plane in the image of the physical scene in three dimensional space;

generating a rectified image, wherein the rectified image contains a fronto-parallel view of at least part of the physical plane;

subdividing the rectified image into a plurality of grid cells;

for each of the plurality of grid cells, generating an image signature based on at least one image characteristic associated with said grid cell;

selecting one or more initial grid cells contiguous to at least part of the obtained initial portion of the physical plane to include as part of the digital representation of the physical plane;

comparing the image signature for each neighboring grid cell contiguous to the one or more initial grid cells to the image signature of one or more grid cells currently included in the digital representation of the physical plane; and

adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

2. The method of claim 1, further comprising:

comparing the image signature for each neighboring grid cell contiguous to a grid cell already included as part of the digital representation of the physical plane to the contiguous grid cell; and

adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

3. The method of claim 1, wherein the initial portion of the physical plane is based on a known object positioned on the physical plane.

4. The method of claim 1, wherein the initial portion of the physical plane is based on fitting the physical plane to a point cloud model.

5. The method of claim 1, wherein the image signature comprises at least one histogram indicative of image information associated with the grid cell.

6. The method of claim 5, wherein the at least one histogram comprises a statistical representation of an amount and type of colors present in the grid cell.

7. The method of claim 5, wherein the at least one histogram comprises a statistical representation of a gradient orientation of the grid cell.

8. The method of claim 5, wherein the at least one histogram comprises a statistical representation of an amount and type of gray scale shades representative of colors present in the grid cell.

9. The method of claim 1, wherein the at least one image characteristic associated with the said grid cell comprises three-dimensional information.

10. The method of claim 1, further comprising, for each neighboring grid cell determined to not be included as part of the digital representation of the physical plane:

generating a plurality of secondary grid cells, wherein a position of each of the plurality of secondary grid cells is based on an offset from a position of said neighboring grid cell, wherein the offset is within a width and a length of said neighboring grid cell; and

for each of the plurality of secondary grid cells;

generating an image signature based on at least one image characteristic associated with the secondary grid cell; and

determining whether said secondary grid cell is to be additionally included as part of the digital representation of the physical plane if the image signature of said secondary grid cell is similar to the image signature of a grid cell already determined to be included as part of the digital representation of the physical plane, wherein the similarity is based on at least one predetermined threshold criterion.

11. An apparatus for constructing a digital representation of a physical plane from a physical scene, comprising:

memory;

one or more processors configured to;

determine an orientation and an initial portion of a physical plane in the physical scene in three dimensional space;

receive on or more images of the physical scene;

generate a rectified image, wherein the rectified image contains a fronto-parallel view of at least part of the physical plane;

subdivide the rectified image into a plurality of grid cells;

for each of the plurality of grid cells, generate an image signature based on at least one image characteristic associated with said grid cell;

select one or more initial grid cells contiguous to at least part of the obtained initial portion of the physical plane to include as part of the digital representation of the physical plane;

compare the image signature for each neighboring grid cell contiguous to the one or more initial grid cells to the image signature of one or more grid cells currently included in the digital representation of the physical plane; and

add each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

12. The apparatus of claim 11, further comprising:

comparing the image signature for each neighboring grid cell contiguous to a grid cell already included as part of the digital representation of the physical plane to the contiguous grid cell; and

adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

13. The apparatus of claim 11, wherein the initial portion of the physical plane is based on a known object positioned on the physical plane.

14. The apparatus of claim 11, wherein the initial portion of the physical plane is based on fitting the physical plane to a point cloud model.

15. The apparatus of claim 11, wherein the image signature comprises at least one histogram indicative of image information associated with the grid cell.

16. The apparatus of claim 15, wherein the at least one histogram comprises a statistical representation of an amount and type of colors present in the grid cell.

17. The apparatus of claim 15, wherein the at least one histogram comprises a statistical representation of a gradient orientation of the grid cell.

18. The apparatus of claim 15, wherein the at least one histogram comprises a statistical representation of an amount and type of gray scale shades representative of colors present in the grid cell.

19. The apparatus of claim 11, wherein the at least one image characteristic associated with the said grid cell comprises three dimensional information.

20. The apparatus of claim 11, for each neighboring grid cell determined to not be included as part of the digital representation of the physical plane, the one or more processors further configured to:

generate a plurality of secondary grid cells, wherein a position of each of the plurality of secondary grid cells is based on an offset from a position of said neighboring grid cell, wherein the offset is within a width and a length of said neighboring grid cell; and

for each of the plurality of secondary grid cells;

generate an image signature based on at least one image characteristic associated with the secondary grid cell; and

determine whether said secondary grid cell is to be additionally included as part of the digital representation of the physical plane if the image signature of said secondary grid cell is similar to the image signature of a grid cell already determined to be included as part of the digital representation of the physical plane, wherein the similarity is based on at least one predetermined threshold criterion.

21. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises instructions executable by a processor, the instructions comprising instructions to:

determine an orientation and an initial portion of a physical plane in the physical scene in three dimensional space;

receive on or more images of the physical scene;

generate a rectified image, wherein the rectified image contains a fronto-parallel view of at least part of the physical plane;

subdivide the rectified image into a plurality of grid cells;

for each of the plurality of grid cells, generate an image signature based on at least one image characteristic associated with said grid cell;

select one or more initial grid cells contiguous to at least part of the obtained initial portion of the physical plane to include as part of the digital representation of the physical plane;

compare the image signature for each neighboring grid cell contiguous to the one or more initial grid cells to the image signature of one or more grid cells currently included in the digital representation of the physical plane; and

add each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

22. The non-transitory computer-readable storage medium of claim 21, further comprising:

further comprising comparing the image signature for each neighboring grid cell contiguous to a grid cell already included as part of the digital representation of the physical plane to the contiguous grid cell; and

adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

23. The non-transitory computer-readable storage medium of claim 21, wherein the image signature comprises at least one histogram indicative of image information associated with the grid cell.

24. The non-transitory computer-readable storage medium of claim 21, wherein the at least one image characteristic associated with the said grid cell comprises three-dimensional information.

25. The non-transitory computer-readable storage medium of claim of claim 21, wherein for each neighboring grid cell determined to not be included as part of the digital representation of the physical plane, the one or more processors further configured to:

generate a plurality of secondary grid cells, wherein a position of each of the plurality of secondary grid cells is based on an offset from a position of said neighboring grid cell, wherein the offset is within a width and a length of said neighboring grid cell; and

for each of the plurality of secondary grid cells;

generate an image signature based on at least one image characteristic associated with the secondary grid cell; and

determine whether said secondary grid cell is to be additionally included as part of the digital representation of the physical plane if the image signature of said secondary grid cell is similar to the image signature of a grid cell already determined to be included as part of the digital representation of the physical plane, wherein the similarity is based on at least one predetermined threshold criterion.

26. The apparatus for constructing a digital representation of a physical plane from a physical scene, the apparatus comprising

means for determining an orientation and an initial portion of a physical plane in the image of the physical scene in three dimensional space;

means for generating a rectified image, wherein the rectified image contains a fronto-parallel view of at least part of the physical plane;

means for subdividing the rectified image into a plurality of grid cells;

for each of the plurality of grid cells, means for generating an image signature based on at least one image characteristic associated with said grid cell;

means for selecting one or more initial grid cells contiguous to at least part of the obtained initial portion of the physical plane to include as part of the digital representation of the physical plane;

means for comparing the image signature for each neighboring grid cell contiguous to the one or more initial grid cells to the image signature of one or more grid cells currently included in the digital representation of the physical plane; and

means for adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

27. The non-transitory computer-readable storage medium of claim 26, further comprising:

means for comparing the image signature for each neighboring grid cell contiguous to a grid cell already included as part of the digital representation of the physical plane to the contiguous grid cell; and

means for adding each grid cell that meets a predetermined threshold to the digital representation of the physical plane.

28. The apparatus of claim 26, wherein the image signature comprises at least one histogram indicative of image information associated with the grid cell.

29. The apparatus of claim 26, wherein the at least one image characteristic associated with the said grid cell comprises three dimensional information.

30. The apparatus of claim 26, further comprising, for each neighboring grid cell determined to not be included as part of the digital representation of the physical plane:

means for generating a plurality of secondary grid cells, wherein a position of each of the plurality of secondary grid cells is based on an offset from a position of said neighboring grid cell, wherein the offset is within a width and a length of said neighboring grid cell; and

for each of the plurality of secondary grid cells;

means for generating an image signature based on at least one image characteristic associated with the said secondary grid cell; and

means for determining whether said secondary grid cell is to be additionally included as part of the digital representation of the physical plane if the image signature of said secondary grid cell is similar to the image signature of a grid cell already determined to be included as part of the digital representation of the physical plane, wherein the similarity is based on at least one predetermined threshold criterion.