Ambient vehicle and pedestrian state detection for device notification

• US 9,487,215 B2
• Filed: 06/30/2015
• Issued: 11/08/2016
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Microsoft Corporation
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Paek, Timothy S., Bahl, Paramvir
• Primary Examiner(s): POPE, DARYL C
• Application Number:
  US14/788,328
• Publication Number:
  US 20150375754A1
• Time in Patent Office: 497 Days
• Field of Search: 340/903, 340/901, 340/435, 340/995, 340/990
• US Class Current: 1/1
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

Described is a technology by which ambient data related to a vehicle is sensed and processed, for use in determining a state change related to external traffic awareness. Based upon the state change, an allowed level of interactivity with a user interface may be changed, and/or a notification may be output. Images and/or depth data may be sensed as part of determining whether a user who is interacting with a device in a stopped vehicle is to be made aware of the changed condition with respect to other vehicles, pedestrians and/or the like.

Claims

1. In a computing environment, a method performed at least in part on at least one processor comprising:

allowing, by the at least one processor, a first level of user interaction with an interactive device while a vehicle associated with the interactive device is stopped;

monitoring ambient activity relative to the vehicle based on data received from a set of sensors, the ambient activity corresponding to external traffic awareness;

based on the data received from the set of sensors, determining a change with respect to the ambient activity indicating the vehicle is moving; and

based on the determined change, allowing, by the at least one processor, a second level of user interaction with the interactive device while the vehicle is moving.

2. The method of claim 1, further comprising:

calibrating a stopped state for the vehicle.

3. The method of claim 1, wherein determining the change with respect to the ambient activity indicating the vehicle is moving further comprises:

sensing an object coming within a field of view of the vehicle, wherein the sensed object comprises a pedestrian or an animal.

4. The method of claim 1, wherein allowing the second level of user interaction with the interactive device further comprises:

changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is allowed at the first level of user interaction, and wherein the user interaction with the at least one program is disallowed at the second level of interaction.

5. The method of claim 1, wherein determining the change with respect to the ambient activity further comprises:

processing the data received from the set of sensors to detect a change in a state of at least one brake light of another vehicle.

6. The method of claim 1, wherein monitoring the ambient activity relative to the vehicle further comprises:

capturing depth data from a depth camera; and

processing the depth data to determine that an object has increased its depth relative to the vehicle over time.

7. The method of claim 1, wherein determining the change with respect to the ambient activity further comprises:

changing the first level of user interaction to the second level of user interaction based on a set of interaction-allowed rules.

8. The method of claim 1, further comprising:

reducing pedestrian sensing sensitivity for a highway situation; and

increasing the pedestrian sensing sensitivity for city driving.

9. The method of claim 1, further comprising:

outputting a notification to a user.

10. The method of claim 1, wherein allowing the second level of user interaction further comprises:

gradually transitioning from an interactive state to a notification state.

11. The method of claim 1, wherein monitoring the ambient activity relative to the vehicle further comprises:

processing traffic data and vehicle-related data.

12. The method of claim 1, wherein determining the change with respect to the ambient activity further comprises:

determining whether the vehicle associated with the interactive device is in one of a stop mode and a park mode, the park mode different from the stop mode.

13. A system comprising:

an interactive device associated with a vehicle, the interactive device configured to provide a first level of interaction while the vehicle is stopped and a second level of user interaction with the interactive device while the vehicle is moving;

a set of sensors configured to detect ambient activity relative to the vehicle and generate data associated with the ambient activity, the ambient activity corresponding to external traffic awareness; and

a rule-based classifier comprising a set of interaction-allowed rules and detection and notification logic, the rule-based classifier configured to receive the generated data from the set of sensors to monitor the ambient activity relative to the vehicle, determine a change with respect to the ambient activity, and provide one of the first level of user interaction or the second level of user interaction based on the determined change.

14. The system of claim 13, wherein the set of interaction-allowed rules comprises:

a first set of rules for a first user and a second set of rules for a second user, the first set of rules is different from the second set of rules.

15. The system of claim 13, wherein the set of interaction-allowed rules comprises user preference data specifying permitted types of interaction and restricted types of interaction.

16. The system of claim 13, wherein the ambient activity comprises at least one sensed object coming within a field of view of the vehicle, the at least one sensed object comprising a pedestrian or an animal.

17. The system of claim 16, wherein the interactive device is configured to allow interaction with at least one program when providing the first level of user interaction, and disallow interaction with the at least one program when providing the second level of user interaction.

18. The system of claim 13, further comprising:

at least one depth camera configured to capture frames of depth data.

19. One or more computer-readable memory devices having computer-executable instructions, which when executed cause at least one processor to:

allow, by the at least one processor, a first level of user interaction with an interactive device while a vehicle associated with the interactive device is stopped;

monitor ambient activity relative to the vehicle based on data received from a set of sensors, the ambient activity corresponding to external traffic awareness;

based on the data received from the set of sensors, determine a change with respect to the ambient activity indicating the vehicle is moving; and

based on the determined change, allow, by the at least one processor, a second level of user interaction with the interactive device while the vehicle is moving.

20. The one or more computer-readable memory devices of claim 19, wherein the computer executable instructions are executed to further cause the at least one process to:

sense an object coming within a field of view of the vehicle, wherein the sensed object comprises a pedestrian or an animal.