METHOD AND APPARATUS FOR USER AND MOVING VEHICLE DETECTION

• US 20170030716A1
• Filed: 07/29/2015
• Published: 02/02/2017
• Est. Priority Date: 07/29/2015
• Status: Active Grant

• Current Assignee: Invensense Incorporated (TDK Corporation)
• Original Assignee: Invensense Incorporated (TDK Corporation)
• Inventors: Housholder, Michael D., Ali, Abdelrahman, Elhoushi, Mostafa, Georgy, Jacques, Goodall, Christopher, Heshmati, Ardalan
• Granted Patent: US 10,726,281 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06F 18/259 Fusion by voting
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An apparatus and method are disclosed for user and moving vehicle detection in which sensor data for a portable device is processed to determine whether the portable device is in a moving vehicle. Following a determination the portable device is in a moving vehicle, the sensor data is to characterize an association between the user and the portable device to determine whether the portable device is connected to the user. If the user is connected to the portable device, it is then determined if the portable device is being held in hand. If the portable device is held in hand, it is then determined if the user is operating the moving vehicle. Output from an image sensor of the portable device may be used in determining if the user is the operator.

Claims

1. A method for user and moving vehicle detection, the method comprising:

obtaining sensor data for a portable device;

processing the sensor data to determine whether the portable device is in a moving vehicle; and

processing the sensor data to determine whether the portable device is connected to a user if the portable device is determined to be in the moving vehicle.

2. The method of claim 1, wherein processing the sensor data to determine whether the portable device is in a moving vehicle comprises applying a machine learning technique.

3. The method of claim 2, further comprising inputting features extracted from the processed sensor data to at least one stored classification model to determine a motion mode of the portable device.

4. The method of claim 3, wherein the at least one stored classification model comprises extracted features developed during a training phase.

5. The method of claim 1, wherein processing the sensor data to determine whether the portable device is in a moving vehicle comprises applying a signal analysis technique.

6. The method of claim 5, wherein the signal analysis technique comprises any one or any combination of the following:

(i) a statistical analysis;

(ii) a frequency domain analysis;

or (iii) a time-domain analysis.

7. The method of claim 5, wherein the signal analysis technique comprises an analysis of at least one signal selected from the group consisting of an angular rotation signal, a signal derived from the angular rotation signal, an acceleration and a signal derived from the acceleration signal.

8. The method of claim 1, wherein the sensor data comprises inertial sensor data.

9. The method of claim 8, further comprising obtaining supplemental sensor data for the portable device and processing the supplemental sensor data with the inertial sensor data.

10. The method of claim 1, further comprising obtaining absolute navigational information for the portable device, wherein determining whether the portable device is in a moving vehicle is based at least in part on the absolute navigational information.

11. The method of claim 1, wherein processing the sensor data to determine whether the portable device is connected to the user comprises applying a machine learning technique.

12. The method of claim 11, further comprising inputting features extracted from the processed inertial sensor data to at least one stored classification model to characterize if the portable device is connected to the user.

13. The method of claim 12, wherein the at least one stored classification model comprises extracted features developed during a training phase.

14. The method of claim 1, wherein processing the sensor data to determine whether the portable device is connected to the user comprises applying a signal analysis technique.

15. The method of claim 14, wherein the signal analysis technique comprises any one or any combination of the following:

(i) a statistical analysis;

(ii) a frequency-domain analysis;

or (iii) a time-domain analysis.

16. The method of claim 14, wherein the signal analysis technique comprises an analysis of at least one signal selected from the group consisting of an angular rotation signal, a signal derived from the angular rotation signal, an acceleration signal, and a signal derived from the acceleration signal.

17. The method of claim 1, further comprising processing the sensor data to determine whether the portable device is hand held if it is determined the portable device is connected to the user.

18. The method of claim 17, wherein determining whether the portable device is hand held comprises determining a device use case for the portable device.

19. The method of claim 17, wherein processing the sensor data to determine whether the portable device is hand held comprises applying a machine learning technique.

20. The method of claim 19, further comprising inputting features extracted from the processed inertial sensor data to at least one stored classification model to determine whether the portable device is hand held.

21. The method of claim 20, wherein the at least one stored classification model comprises extracted features developed during a training phase.

22. The method of claim 17, wherein processing the sensor data to determine whether the portable device is hand held comprises applying a signal analysis technique.

23. The method of claim 22, wherein the signal analysis technique comprises any one or any combination of the following:

(i) a statistical analysis;

(ii) a frequency-domain analysis;

or (iii) a time-domain analysis.

24. The method of claim 22, wherein the signal analysis technique comprises an analysis of at least one signal selected from the group consisting of an angular rotation signal, a signal derived from the angular rotation signal, an acceleration signal, and a signal derived from the acceleration signal.

25. The method of claim 17, wherein the sensor data comprises inertial sensor data.

26. The method of claim 25, further comprising obtaining supplemental sensor data, wherein the determination of whether the portable device is hand held is based at least in part on the supplemental sensor data.

27. The method of claim 26, wherein the supplemental sensor data is obtained from at least one of an ambient light sensor and a proximity sensor.

28. The method of claim 26, further comprising activating a source of the supplemental sensor data from a power save mode if it is determined the portable device is connected to the user.

29. The method of claim 17, further comprising obtaining information regarding the portable device, wherein the determination of whether the portable device is hand held is based at least in part on the information regarding the portable device.

30. The method of claim 29, wherein the information regarding the portable device comprises a status of at least one application running on the portable device.

31. The method of claim 17, further comprising obtaining image sensor data for the portable device if it is determined the portable device is hand held, processing the image sensor data, and determining whether the user of the portable device is operating the moving vehicle based at least in part on the processed image sensor data.

32. The method of claim 31, wherein processing the image sensor data comprises extracting at least one of a feature and an object from the image sensor data.

33. The method of claim 32, further comprising preprocessing the image sensor data before extracting the at least one of a feature and an object.

34. The method of claim 33, wherein preprocessing the image comprises performing an operation selected from the group consisting of generating a full color image, generating a reduced dimension image, generating a grey scale image, generating a resized image, generating a normalized image, and generating a reduced noise image.

35. The method of claim 12, wherein extracting the at least one of a feature and an object comprises a line detection operation.

36. The method of claim 32, wherein extracting the at least one of a feature and an object comprises an edge detection operation.

37. The method of claim 32, wherein extracting the at least one of a feature and an object comprises a circular shape detection operation.

38. The method of claim 32, wherein extracting the at least one of a feature and an object comprises performing a Hough transform.

39. The method of claim 32, wherein processing the image sensor data further comprises recognizing the at least one of a feature and an object.

40. The method of claim 39, wherein the at least one of a feature and an object comprises an object associated with the user.

41. The method of claim 40, wherein the object associated with the user is at least one of a head, a face and shoulders.

42. The method of claim 39, wherein the at least one of a feature and an object comprises an object associated with the vehicle.

43. The method of claim 42, wherein the object associated with the vehicle is at least one of a steering wheel, a mirror, a seat belt, a windshield and a window.

44. The method of claim 39, further comprising recognizing a combination of objects associated with the user and the vehicle.

45. The method of claim 31, wherein obtaining image sensor data further comprises activating an image sensor of the portable device from a power save mode.

46. The method of claim 31, wherein image sensor data is obtained from multiple image sensors of the portable device.

47. The method of claim 31, further comprising obtaining a coarse location for the portable device and determining an expected orientation of a vehicle operator with respect to the vehicle based on the coarse location, wherein determining whether the user of the portable device is operating the moving vehicle is based at least in part on the expected vehicle operator orientation.

48. The method of claim 47, wherein the coarse location is obtained from any one or any combination of the following:

(i) a global navigation satellite system (GNSS);

(ii) cell-based positioning;

(iii) WiFi-based positioning;

or (iv) other wireless-based positioning.

49. A portable device for providing user and moving vehicle detection, the portable device comprising:

a sensor assembly integrated with the portable device, including at least one sensor configured to output data representing motion of the portable device;

a processor configured to process the sensor data; and

a detector configured to;

process the sensor data to determine whether the portable device is in a moving vehicle; and

process the sensor data to determine whether the portable device is connected to a user if it is determined that the portable device is in the moving vehicle.

50. The portable device of claim 49, wherein the detector is further configured to process the sensor data to determine whether the portable device is hand held if it is determined that the portable device is connected to the user.

51. The portable device of claim 50, wherein the sensor assembly comprises an inertial sensor.

52. The portable device of claim 51, further comprising a supplemental sensor selected from the group consisting of an ambient light sensor and a proximity sensor, wherein the detector is further configured to determine whether the portable device is hand held based at least in part on output from the supplemental sensor.

53. The portable device of claim 50, further comprising an image sensor, wherein the detector is further configured to process image sensor data to determine whether the user of the portable device is operating the moving vehicle.

54. The portable device of claim 53, further comprising a location module configured to obtain a coarse location for the portable device, wherein the detector is further configured to determine whether the user of the portable device is operating the moving vehicle based at least in part an expected orientation of a vehicle operator with respect to the vehicle based on the coarse location.

55. The portable device of claim 54, wherein the location module comprises a source of absolute navigational information from any one or any combination of the following:

(i) a global navigation satellite system (GNSS) receiver;

(ii) cell-based positioning receiver;

(iii) WiFi-based positioning receiver;

or (iv) other wireless-based positioning receiver.

56. The portable device of claim 49, wherein the sensor assembly includes an accelerometer and a gyroscope.

57. The device of claim 56, wherein the sensor assembly comprises an inertial sensor implemented as a Micro Electro Mechanical System (MEMS).