EGOMOTION SPEED ESTIMATION ON A MOBILE DEVICE

• US 20100053324A1
• Filed: 08/24/2009
• Published: 03/04/2010
• Est. Priority Date: 09/02/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Marti, Stefan, Kim, Seung Wook, Imai, Francisco
• Granted Patent: US 8,253,795 B2
• Time in Patent Office: Days
• US Class Current: 348/142
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: H04M 2203/359 Augmented reality
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking

Abstract

Linear and rotational speeds of a mobile device are calculated using distance estimates between imaging sensors in the device and objects or scenes in front of the sensors. The distance estimates are used to modify optical flow vectors from the sensors. Shifting and rotational speeds of the mobile device may then be calculated using the modified optical flow vector values. For example, given a configuration where the first imaging sensor and the second imaging sensor face opposite directions on a single axis, a shifting speed is calculated in the following way: multiplying a first optical flow vector and a first distance estimate, thereby deriving a first modified optical flow vector value; multiplying a second optical flow vector and a second distance estimate, thereby deriving a second modified optical flow vector value; the second modified optical flow vector value may then be subtracted from the first modified optical flow vector value, resulting in a measurement of the shifting speed.

Claims

1. A method of estimating motion speed of a mobile device having a first imaging sensor and a second imaging sensor, the method comprising:

measuring a first distance between the first imaging sensor and a first object and a second distance between the second imaging sensor and a second object;

modifying first optical flow vector values created by the first sensor using the first distance and modifying second optical flow vector values created by the second sensor using the second distance;

calculating a shifting speed and a rotational speed of the mobile device using modified first optical flow vector values and modified second optical flow vector values; and

wherein the first imaging sensor and the second imaging sensor have a specific alignment.

2. A method as recited in claim 1 further comprising:

measuring a third distance between a third imaging sensor and a third object; and

modifying third optical flow vector values created by the third sensor using the third distance.

3. A method as recited in claim 2 wherein calculating a shifting speed or a rotational speed further comprises:

using modified third optical flow vector values.

4. A method as recited in claim 1 wherein the specific alignment is the first imaging sensor and the second imaging sensor facing opposite directions on a single axis.

5. A method as recited in claim 2 wherein the third imaging sensor is configured perpendicularly to the first imaging sensor and the second imaging sensor.

6. A method as recited in claim 1 wherein calculating the shifting speed and the rotational speed requires an overall optical flow.

7. A method as recited in claim 2 further comprising:

utilizing an autofocus system in the mobile device for measuring the first distance, the second distance, or the third distance.

8. A method as recited in claim 2 further comprising:

utilizing active distance sensors for measuring the first distance, the second distance, or the third distance.

9. A method as recited in claim 1 wherein calculating a shifting speed further comprises:

multiplying a first optical flow vector value and the first distance, producing a first modified optical flow vector value;

multiplying a second optical flow vector value and the second distance, producing a second modified optical flow vector value; and

subtracting the second modified optical flow vector value from the first modified optical flow vector value.

10. A method as recited in claim 1 wherein calculating a rotational speed further comprises:

multiplying the first optical flow vector value and the first distance, producing a first modified optical flow vector value;

multiplying the second optical flow vector value and the second distance, producing the second modified optical flow vector value; and

adding the second modified optical flow vector value with the first modified optical flow vector value.

11. A mobile device comprising:

one or more processors;

a memory component;

a first imaging sensor for measuring a first optical flow having a first horizontal component and a first vertical component;

a second imaging sensor for measuring a second optical flow having a second horizontal component and a second vertical component;

an object distance estimator component;

an optical flow vector modification module for adjusting a plurality of distance factors; and

a speed calculation module using the plurality of adjusted distance factors.

12. A mobile device as recited in claim 11 wherein the first imaging sensor is one of a one-dimensional optical flow sensor, a two-dimensional optical flow sensor, or a three-dimensional optical flow sensor.

13. A mobile device as recited in claim 11 wherein the second imaging sensor is one of a one-dimensional optical flow sensor, a two-dimensional optical flow sensor, or a three-dimensional optical flow sensor.

14. A mobile device as recited in claim 11 further comprising:

an autofocus system.

15. A mobile device as recited in claim 11 further comprising:

an active distance sensor.

16. A mobile device as recited in claim 11 wherein the speed calculation module further utilizes one or more resolution factors of the first imaging sensor and the second imaging sensor.

17. A mobile device as recited in claim 11 wherein the speed calculation module further utilizes one or more frame rates of the first imaging sensor and the second imaging sensor.

18. A computer-readable medium containing computer instructions for estimating motion speed of a mobile computing device having a first imaging sensor and a second imaging sensor, the computer-readable medium comprising:

computer instructions for measuring a first distance between the first imaging sensor and a first object and a second distance between the second imaging sensor and a second object;

computer instructions for modifying first optical flow vector values created by the first sensor using the first distance and modifying second optical flow vector values created by the second sensor using the second distance; and

computer instructions for calculating a shifting speed and a rotational speed of the mobile device using modified first optical flow vector values and modified second optical flow vector values;

wherein the computer-readable medium is a tangible and persistent data storage device.

19. A computer-readable medium as recited in claim 18 further comprising:

computer instructions for measuring a third distance between a third imaging sensor and a third object; and

computer instructions for modifying third optical flow vector values created by the third sensor using the third distance.

20. A computer-readable medium as recited in claim 18 wherein computer instructions for calculating a shifting speed or a rotational speed further comprises:

computer instructions for using modified third optical flow vector values.

21. A computer-readable medium as recited in claim 18 further comprising:

computer instructions for utilizing an autofocus system in the mobile computing device for measuring the first distance, the second distance, or the third distance.

22. A computer-readable medium as recited in claim 18 further comprising:

computer instructions for utilizing active distance sensors for measuring the first distance, the second distance, or the third distance.

23. A computer-readable medium as recited in claim 18 wherein computer instructions for calculating a shifting speed further comprises:

computer instructions for multiplying a first optical flow vector value and the first distance, producing a first modified optical flow vector value;

computer instructions for multiplying a second optical flow vector value and the second distance, producing a second modified optical flow vector value; and

computer instructions for subtracting the second modified optical flow vector value from the first modified optical flow vector value.

24. A computer-readable medium as recited in claim 23 wherein computer instructions for calculating a rotational speed further comprises:

computer instructions for multiplying the first optical flow vector value and the first distance, producing a first modified optical flow vector value;

computer instructions for multiplying the second optical flow vector value and the second distance, producing the second modified optical flow vector value; and

computer instructions for adding the second modified optical flow vector value with the first modified optical flow vector value.