Devices, methods, and apparatuses for homography evaluation involving a mobile device

• US 9,342,886 B2
• Filed: 04/23/2012
• Issued: 05/17/2016
• Est. Priority Date: 04/29/2011
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Grzechnik, Slawomir K.
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Potts, Ryan P
• Application Number:
  US13/453,812
• Publication Number:
  US 20120301011A1
• Time in Patent Office: 1,485 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

Components, methods, and apparatuses are provided that may be used to access information pertaining to a two-dimensional image of a three-dimensional object, to detect homography between said image of said three-dimensional object captured in said two-dimensional image indicative of said three-dimensional object and a reference object image and to determine whether said homography indicates pose suitable for image augmentation based, at least in part, on characteristics of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image.

Claims

1. A method comprising:

accessing, from a video frame stored in a memory, a two-dimensional image indicative of a three-dimensional object;

detecting homography between said two-dimensional image indicative of said three-dimensional object and a reference object image;

determining whether said homography indicates pose suitable for image augmentation based, at least in part, on a ratio of a first span to a second span of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image; and

responsive to determining that said homography indicates pose suitable for image augmentation;

tracking said two-dimensional image, from said video frame, in one or more successive video frames; and

inserting said reference object image into at least one of said one or more successive video frames.

2. The method of claim 1, wherein said at least some of said plurality of inliers are modeled as point masses that contribute to a moment of rotational inertia about an axis of said elliptically-shaped area that encompasses at least some of said plurality of inliers.

3. The method of claim 2, wherein said determining further comprises:

computing at least one eigenvalue of a matrix that corresponds to said moment of rotational inertia about said axis.

4. The method of claim 3, further comprising:

computing an eigenvector from said at least one eigenvalue, said eigenvector determining said first span of said elliptically-shaped area that encompasses said at least some of said plurality of inliers distributed within said elliptically-shaped area of said two-dimensional image.

5. The method of claim 2, wherein said determining further comprises:

detecting that a ratio of said first span of said elliptically-shaped area to said second span of said elliptically-shaped area is at least 0.3.

6. The method of claim 5, wherein determining further comprises:

detecting that at least one of said first span and said second span includes at least 20% of a length said two-dimensional image.

7. The method of claim 1, further comprising:

applying a convex hull technique to at least some of said plurality of inliers.

8. The method of claim 7, wherein said applying said convex hull technique comprises:

generating an area that approximates said elliptically-shaped area that encompasses at least some of said plurality of inliers distributed in said two-dimensional image.

9. The method of claim 1, wherein said at least some of said plurality of inliers represent keypoints having individual reprojection errors of 5-15 pixels.

10. The method of claim 1, further comprising:

extracting keypoints from said information states pertaining to said two-dimensional image.

11. An apparatus comprising:

means for accessing, from a video frame stored in a memory, a two-dimensional image indicative of a three-dimensional object;

means for detecting homography between a reference object image and said two-dimensional image indicative of said three-dimensional object;

means for determining whether said homography indicates pose suitable for image augmentation based, at least in part, on a ratio of a first span to a second span of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image;

means for tracking said two-dimensional image, from said video frame, in one or more successive video frames responsive to determining that said homography indicates pose suitable for image augmentation; and

means for inserting said reference object image into at least one of said one or more successive video frames.

12. The apparatus of claim 11, wherein said elliptically-shaped area is to encompass

at least 20% of a length of said two-dimensional image.

13. The apparatus of claim 11, further comprising:

means for computing at least one eigenvalue of a matrix that corresponds to a moment of rotational inertia of point masses about an axis of said area encompassing said at least some of said plurality of inliers within said area.

14. The apparatus of claim 11, further comprising:

means for applying a convex hull technique to said at least some of said plurality of inliers within said area.

15. The apparatus of claim 11, further comprising:

means for comparing keypoints extracted from said two-dimensional image with key points of said reference object image.

16. An article comprising:

a non-transitory storage medium comprising machine-readable instructions stored thereon which are executable by a processor of a mobile device to;

access, from a video frame stored in a memory, a two-dimensional image of a three-dimensional object;

detect homography between said image of said three-dimensional object representative of said two-dimensional image and a reference object image; and

determine whether said homography indicates pose suitable for image augmentation based, at least in part, on a ratio of a first span to a second span of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image; and

responsive to determining that said homography indicates pose suitable for image augmentation;

track said two-dimensional image, from said video frame, in one or more successive video frames; and

insert said reference object image into said one or more successive video frames.

17. The article of claim 16, wherein said non-transitory storage medium further comprises machine-readable instructions stored thereon which are executable by a processor to:

model said at least some of said plurality of inliers as point masses that contribute to a moment of rotational inertia about an axis.

18. The article of claim 17, wherein said non-transitory storage medium further comprises machine-readable instructions stored thereon which are executable by processor to:

compute at least one eigenvalue of a matrix that corresponds to said moment of rotational inertia of said point masses.

19. The article of claim 18, wherein said non-transitory storage medium further comprises machine-readable instructions stored thereon which are executable by processor:

compute an eigenvector from said at least one eigenvalue, said eigenvector determining said first span of said elliptically-shaped area that includes said at least some of said plurality of inliers.

20. The article of claim 19, wherein said non-transitory storage medium further comprises machine-readable instructions stored thereon which are executable by processor to:

apply a convex hull technique to said at least some of said plurality of inliers.

21. The article of claim 16, wherein said non-transitory storage medium further comprises machine-readable instructions stored thereon which are executable by processor to:

extract keypoints from said two-dimensional image.

22. A mobile device comprising:

one or more sensors for evaluating homography of an image of a three-dimensional object captured by said mobile device; and

one or more processors to;

detect homography between said image of said three-dimensional object captured in said two-dimensional image, from a video frame stored in a memory, and a reference object image;

determine whether said homography indicates pose suitable for image augmentation based, at least in part, on a ratio of a first span to a second span of an elliptically-shaped area that encompasses at least some of a plurality of inliers distributed in said two-dimensional image; and

responsive to determining that said homography indicates pose suitable for image augmentation;

track said two-dimensional image, from said video frame, in one or more successive video frames; and

insert said reference object image into said one or more successive video frames.

23. The mobile device of claim 22, wherein said one or more processors further models at least some of said plurality of inliers as point masses that contribute to a moment of rotational inertia about an axis.

24. The mobile device of claim 23, wherein said one or more processors further computes at least one eigenvalue of a matrix that corresponds to said moment of rotational inertia about said axis.

25. The mobile device of claim 24, wherein said one or more processors further computes an eigenvector from said at least one eigenvalue, said eigenvector determining said first span of said elliptically-shaped area that encompasses said at least some of said plurality of inliers.