Method and apparatus for speed or velocity estimation using optical sensor

• US 10,302,669 B2
• Filed: 09/15/2016
• Issued: 05/28/2019
• Est. Priority Date: 11/01/2013
• Status: Active Grant

• Current Assignee: Invensense Incorporated (TDK Corporation)
• Original Assignee: Invensense Incorporated (TDK Corporation)
• Inventors: Al-Hamad, Amr, Kazemipur, Bashir, Georgy, Jacques, Syed, Zainab, Goodall, Christopher
• Primary Examiner(s): Elchanti, Hussein
• Assistant Examiner(s): Castro, Paul A
• Application Number:
  US15/266,241
• Publication Number:
  US 20170004630A1
• Time in Patent Office: 985 Days
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G01P 3/36 Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light G01P3/68 takes precedence; gyrometers using the Sagnac effect, i.e. rotation-induced shifts between counter-rotating electromagnetic beams G01C19/64
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 19/393 Trajectory determination or predictive tracking, e.g. Kalman filtering
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/49 whereby the further system is an inertial position system, e.g. loosely-coupled
• CPC Class Codes: G01S 5/017 Detecting state or type of motion
• CPC Class Codes: G01S 5/018 Involving non-radio wave signals or measurements
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04M 2203/551 Call history

Abstract

Systems and methods are disclosed for estimating speed and/or velocity for a portable device using an optical sensor. The optical sensor may capture a plurality of samples to be processed to estimate speed. The speed or velocity may be estimated based on a determined context and/or usage of the portable device. The estimated speed or velocity may be used to supplement other navigational solutions or may be used independently.

Claims

1. A method for navigating with a portable device comprising:

processing a sequence of samples from an optical sensor of the portable device, wherein processing the sequence of samples from the optical sensor comprises performing an optical flow analysis and wherein the portable device and the optical sensor may assume varying orientations with respect to a user depending at least in part on how the portable device is being used;

determining usage of the portable device based at least in part on the processed sequence of samples; and

estimating a speed for the portable device based at least in part on the processed sequence of samples and the determined usage, wherein one or more techniques for estimating speed are selected from multiple techniques based on the determined usage.

2. The method of claim 1, wherein estimating speed includes determining a context for the portable device.

3. The method of claim 2, further comprising estimating a device angle with respect to a direction of motion.

4. The method of claim 3, further comprising determining the context of the portable device by assessing variance in a plurality of estimated device angles.

5. The method of claim 2, further comprising determining the context of the portable device by calculating an optical flow ratio.

6. The method of claim 2, further comprising determining the context of the portable device by comparing similarity between samples.

7. The method of claim 2, further comprising determining the context of the portable device by identifying a feature in the samples.

8. The method of claim 2, further comprising determining the context corresponds to being static.

9. The method of claim 2, further comprising determining the context corresponds to fidgeting motion.

10. The method of claim 8, further comprising estimating zero speed for the portable device based at least in part on the determination of being static.

11. The method of claim 9, further comprising estimating zero speed for a user corresponding to non-walking motion based at least in part on the determination of fidgeting motion.

12. The method of claim 1, further comprising estimating a device angle with respect to a direction of motion based at least in part on the processed sequence of samples.

13. The method of claim 12, further comprising determining a usage of the portable device based at least in part on the estimated device angle.

14. The method of claim 13, wherein determining the usage of the portable device is also based at least in part on an orientation determination using an inertial sensor.

15. The method of claim 13, wherein determining the usage of the portable device comprise determining if the usage is dangling, texting or calling.

16. The method of claim 1, wherein performing the optical flow analysis comprises aggregating estimations of pixel translations between the samples.

17. The method of claim 16, further comprising determining usage of the portable device corresponds to dangling motion based at least in part on the aggregated estimations of pixel translations.

18. The method of claim 17, wherein estimating the speed of the portable device is based at least in part on the aggregated estimations of pixel translations during half-cycles of arm motion.

19. The method of claim 18, wherein estimating the speed of the portable device is based at least in part on a value for user arm length selected from at least one of a value estimated from user dynamics, a predetermined average value and a user input value.

20. The method of claim 19, wherein the user dynamics are derived from a source of absolute navigation information.

21. The method of claim 17, further comprising determining an angle of the portable device in relation to a direction of movement based at least in part on the determination that usage of the portable device corresponds to dangling motion.

22. The method of claim 21, wherein determining the angle of the portable device in relation to the direction of movement is based at least in part on the aggregated estimations of pixel translations during complementary half-cycles of arm motion.

23. The method of claim 21, further comprising determining a velocity of the portable device based at least in part on the estimated speed and the determined angle.

24. The method of claim 16, further comprising determining usage of the portable device does not correspond to dangling motion based at least in part on the aggregated estimations of pixel translations.

25. The method of claim 24, further comprising determining an angle of the portable device in relation to a direction of movement based at least in part on the aggregated estimations of pixel translations, wherein determining usage of the portable device is based at least in part on an orientation determination using an inertial sensor and the determined angle.

26. The method of claim 25, wherein estimating the speed of the portable device is based at least in part on the aggregated estimations of pixel translations and the determined angle.

27. The method of claim 26, wherein estimating the speed of the portable device is based at least in part on a value for user height selected from at least one of a value estimated from user dynamics, a predetermined average value and a user input value.

28. The method of claim 27, wherein the user dynamics are derived from a source of absolute navigation information.

29. The method of claim 25, wherein estimating the speed of the portable device is based at least in part on an inclination angle of the portable device.

30. The method of claim 25, further comprising determining usage of the portable device corresponds to texting.

31. The method of claim 25, further comprising determining usage of the portable device corresponds to calling.

32. The method of claim 31, further comprising detecting that no turn has occurred.

33. The method of claim 31, further comprising detecting a ground feature in the sequence of samples.

34. The method of claim 25, further comprising determining a velocity of the portable device based at least in part on the estimated speed and the determined angle.

42. The portable device of claim 23, wherein the source of absolute navigation information includes any one or combination of the following:

(i) a global navigation satellite system (GNSS);

(ii) cell-based positioning;

(iii) WiFi-based positioning;

or

(iv) other wireless-based positioning.

35. A portable device comprising:

an optical sensor, wherein the portable device and the optical sensor may assume varying orientations with respect to a user depending at least in part on how the portable device is being used; and

an optical navigation module, wherein the optical navigation module is configured to;

process a sequence of samples from the optical sensor of the portable device, wherein the processing of the sequence of samples from the optical sensor comprises performing an optical flow analysis;

determine usage of the portable device based at least in part on the processed sequence of samples; and

estimate a speed for the portable device based at least in part on the processed sequence of samples and the determined usage, wherein one or more techniques for estimating speed are selected from multiple techniques based on the determined usage.

36. The portable device of claim 35, wherein the optical navigation module is further configured to determine that usage of the portable device corresponds to dangling motion or does not correspond to dangling motion based at least in part on aggregated estimations of pixel translations between samples.

37. The portable device of claim 35, further comprising an inertial sensor, wherein the optical navigation module is further configured to determine an angle of the portable device in relation to a direction of movement based at least in part on aggregated estimations of pixel translations between samples, wherein determining usage of the portable device is based at least in part on an orientation determination using an inertial sensor and the determined angle.

38. The portable device of claim 35, wherein estimating the speed of the portable device is based at least in part on a value estimated from user dynamics.

39. The portable device of claim 38, wherein the value estimated from user dynamics corresponds to user arm length.

40. The portable device of claim 38, wherein the value estimated from user dynamics corresponds to user height.

41. The portable device of claim 38, further comprising a source of absolute navigation information wherein the user dynamics are derived from the source of absolute navigation information.

43. The portable device of claim 35, wherein the optical navigation module is further configured to determine an angle of the portable device in relation to a direction of movement based at least in part on aggregated estimations of pixel translations between samples and to estimate the speed of the portable device is based at least in part on the aggregated estimations of pixel translations and the determined angle.

44. The portable device of claim 35, further comprising an inertial sensor, wherein the optical navigation module is further configured to provide a navigation solution based at least in part on data from the inertial sensor and the estimated speed of the portable device.