METHODS FOR FACILITATING COMPUTER VISION APPLICATION INITIALIZATION

• US 20140327792A1
• Filed: 05/02/2014
• Published: 11/06/2014
• Est. Priority Date: 05/02/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: RAMACHANDRAN, Mahesh, WAGNER, Daniel, MULLONI, Alessandro, REITMAYR, Gerhard
• Granted Patent: US 9,667,873 B2
• Time in Patent Office: Days
• US Class Current: 348/211.8
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

Embodiments disclosed pertain to systems, method s and apparatus for the initialization of Computer Vision (CV) applications on user devices (UDs) comprising a camera and a display. In some embodiments, an optimal camera trajectory for initialization of a Computer Vision (CV) application may be determined based on an initial camera pose and an estimated pivot distance. For example, the initial camera pose may be estimated based on a first image captured by the camera. Further, the display may be updated in real-time with an indication of a desired movement direction for the camera. In some embodiments, the indication of desired movement direction may be based, in part, on a current camera pose and the optimal trajectory, where the current camera pose may be estimated based on a current image captured by the camera.

Claims

1. A method on a user device (UD) comprising a camera and a display, the method comprising:

determining an optimal camera trajectory for initialization of a Computer Vision (CV) application based on an initial camera pose and an estimated pivot distance, the initial camera pose estimated based on a first image captured by the camera; and

updating the display in real-time with an indication of a desired movement direction for the camera, the indication of movement direction being based, in part, on a current camera pose and the optimal trajectory, the current camera pose estimated based on a current image captured by the camera.

2. The method of claim 1, wherein the desired movement direction for the camera is obtained based in part, on a comparison of a current trajectory of the camera with the optimal trajectory of the camera, the current trajectory of the camera being determined from a sequence of prior camera poses.

3. The method of claim 1, further comprising:

further updating display in real-time with a sequence of indications of updated desired movement directions for the camera, wherein each desired movement direction in the sequence is updated based, in part, on a corresponding camera pose associated with further movement of the camera.

4. The method of claim 1, wherein the pivot distance is determined as at least one of:

an arbitrary positive value;

an average distance of a subset of observed feature points in the first image;

a value above a threshold.

5. The method of claim 1, further comprising:

computing a measure of quality of a current trajectory of the camera, the current trajectory of the camera being determined based, in part, on the current camera pose and a sequence of prior camera poses, and wherein the measure of quality is based on a comparison of the current trajectory with the optimal trajectory; and

displaying an indication of the measure of quality of the current trajectory of the camera.

6. The method of claim 1, wherein prior to determining an optimal camera trajectory, the method comprises:

determining the suitability of a camera captured image for initialization based, in part, on the number of feature points in the camera captured image; and

selecting the camera captured image as the first image, when the camera captured image is determined to be suitable for initialization.

7. The method of claim 6, further comprising:

providing, upon selection of the first image, an indication of suitability of the first image for initialization by augmenting a display of the camera captured image with a first virtual object;

or

providing an indication of unsuitability of the camera captured image for initialization by augmenting a display of the camera captured image with a second virtual object.

8. The method of claim 1, wherein the indication of movement direction is provided using at least one of:

a Screen-aligned 2-Dimensional (2D) guidance scheme comprising 2D screen aligned UI elements to provide the indication of movement direction;

or

a Screen-aligned 3-Dimensional (3D) guidance scheme comprising 3D screen aligned UI elements to provide the indication of movement direction;

or

a World aligned 3D guidance scheme, comprising 3D UI elements aligned to real world objects to provide the indication of movement direction or

a World aligned 2.5-Dimensional (2.5-D) guidance scheme, comprising UI elements displayed in 2.5-D, the UI elements aligned to real world objects to provide the indication of movement direction.

9. The method of claim 1, wherein the indication of movement direction is provided for both sideward translational and pivot initialization motion sequences.

10. The method of claim 1, wherein the indication of movement direction comprises an indication of past movement and an indication of a preferred future movement direction.

11. The method of claim 1, wherein the indication of movement direction also indicates rotational movement of the camera.

12. The method of claim 1, further comprising:

providing at least one of auditory or tactile feedback to the user in conjunction with the indication of movement direction.

13. A user device (UD) comprising:

an image sensor to capture a sequence of images;

a display, and

a processor coupled to the image sensor and the display, wherein the processor is configured to;

determine an optimal image sensor trajectory for initialization of a Computer Vision (CV) application based on an initial image sensor pose and an estimated pivot distance, the initial image sensor pose estimated based on a first image captured by the camera, and

update the display in real-time with an indication of a desired movement direction for the camera, the indication of desired movement direction being based, in part, on a current image sensor pose and the optimal trajectory, the current image sensor pose estimated based on a current image captured by the camera.

14. The UD of claim 13, wherein to obtain the desired movement direction for the image sensor, the processor is configured to:

compare the optimal trajectory of the camera with a current trajectory of the camera, the current trajectory of the camera being determined from a sequence of prior camera poses.

15. The UD of claim 13, the processor is further configured to:

further update the display in real-time with a sequence of indications of updated desired movement directions for the camera, wherein each desired movement direction in the sequence is updated based, in part, on a corresponding camera pose associated with further movement of the camera.

16. The UD of claim 13, wherein the processor is configured to:

determine the pivot distance as at least one of;

an arbitrary positive value;

an average distance of observed feature points in the first image;

a value above a threshold.

17. The UD of claim 13, wherein the processor is further configured to:

compute a measure of quality of a current trajectory of the camera, the current trajectory of the camera being determined based, in part, on the current camera pose and a sequence of prior camera poses, and wherein the measure of quality is based on a comparison of the current trajectory with the optimal trajectory; and

display an indication of the measure of quality of the current trajectory of the camera.

18. The UD of claim 13, wherein prior to determining an optimal camera trajectory, processor is configured to:

determine the suitability of a camera captured image for initialization based, in part, on the number of feature points in the camera captured image; and

select the camera captured image as the first image, when the camera captured image is determined to be suitable for initialization.

19. The UD of claim 18, wherein the processor is further configured to:

provide, upon selection of the first image, an indication of suitability of the first image for initialization by augmenting a display of the camera captured image with a first virtual object;

or

provide an indication of unsuitability of the camera captured image for initialization by augmenting a display of the camera captured image with a second virtual object.

20. The UD of claim 13, wherein the indication of movement direction is provided using at least one of:

a Screen-aligned 2-Dimensional (2D) guidance scheme comprising 2D screen aligned UI elements to provide the indication of movement direction;

or

a Screen-aligned 3-Dimensional (3D) guidance scheme comprising 3D screen aligned UI elements to provide the indication of movement direction;

or

a World aligned 3D guidance scheme, comprising 3D UI elements aligned to real world objects to provide the indication of movement direction or

a World aligned 2.5-Dimensional (2.5-D) guidance scheme, comprising UI elements displayed in 2.5-D, the UI elements aligned to real world objects to provide the indication of movement direction.

21. The UD of claim 13, wherein the indication of movement direction is provided for both sideward translational and pivot initialization motion sequences.

22. The UD of claim 13, wherein the indication of movement direction comprises an indication of past movement and an indication of a preferred future movement direction.

23. The UD of claim 13, wherein the indication of movement direction also indicates rotational movement of the UD.

24. The UD of claim 13, wherein the processor is further configured to:

cause the provision of at least one of auditory or tactile feedback to the user in conjunction with the indication of movement direction.

27. The apparatus of claim 21, wherein the indication of desired movement direction is provided using at least one of:

a Screen-aligned 2-Dimensional (2D) guidance scheme comprising 2D screen aligned UI elements to provide the indication of movement direction;

or

a Screen-aligned 3-Dimensional (3D) guidance scheme comprising 3D screen aligned UI elements to provide the indication of movement direction;

or

a World aligned 3D guidance scheme, comprising 3D UI elements aligned to real world objects to provide the indication of movement direction or

a World aligned 2.5-Dimensional (2.5-D) guidance scheme, comprising UI elements displayed in 2.5-D, the UI elements aligned to real world objects to provide the indication of movement direction.

25. An apparatus comprising:

image capture means, the image capture means to capture a sequence of images;

display means;

means for determining an optimal trajectory of the image capture means for initialization of a Computer Vision (CV) application based on an initial pose of the image capture means and an estimated pivot distance, the initial pose of the image capture means estimated based on a first image captured by the image capture means; and

means for updating the display means in real-time with an indication of a desired movement direction for the image capture means, the indication of desired movement direction being based, in part, on a current pose of the image capture means and the optimal trajectory, the current pose of the image capture means estimated based on a current image captured by the image capture means.

26. The apparatus of claim 25, wherein means for updating the display means comprises:

means for obtaining the desired movement direction for the image sensor, wherein the means for obtaining the desired movement direction for the image sensor comprises;

means for comparing the optimal trajectory of the camera with a current trajectory of the camera, the current trajectory of the camera being determined from a sequence of prior camera poses.

28. A computer-readable medium comprising instructions, which, when executed by a processor, perform a method on a user device (UD) comprising a camera and a display, the method comprising:

determining an optimal camera trajectory for initialization of a Computer Vision (CV) application based on an initial camera pose and an estimated pivot distance, the initial camera pose estimated based on a first image captured by the camera; and

updating the display in real-time with an indication of a desired movement direction for the camera, the indication of desired movement direction being based, in part, on a current camera pose and the optimal trajectory, the current camera pose estimated based on a current image captured by the camera.

29. The computer-readable medium of claim 28, wherein the desired movement direction for the camera is obtained based in part, on a comparison of a current trajectory of the camera with the optimal trajectory of the camera, the current trajectory of the camera being determined from a sequence of prior camera poses

30. The computer-readable medium of claim 28, wherein the indication of movement direction is provided using at least one of:

a Screen-aligned 2-Dimensional (2D) guidance scheme comprising 2D screen aligned UI elements to provide the indication of movement direction;

or

a Screen-aligned 3-Dimensional (3D) guidance scheme comprising 3D screen aligned UI elements to provide the indication of movement direction;

or

a World aligned 3D guidance scheme, comprising 3D UI elements aligned to real world objects to provide the indication of movement direction or

a World aligned 2.5-Dimensional (2.5-D) guidance scheme, comprising UI elements displayed in 2.5-D, the UI elements aligned to real world objects to provide the indication of movement direction.