Ambient vehicle and pedestrian state detection for device notification

• US 9,919,711 B2
• Filed: 08/15/2016
• Issued: 03/20/2018
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Paek, Timothy S., Bahl, Paramvir
• Primary Examiner(s): Pope, Daryl
• Application Number:
  US15/237,540
• Publication Number:
  US 20160355188A1
• Time in Patent Office: 582 Days
• Field of Search: 340903, 340901, 340435, 340995, 340990
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

Described is a technology by which ambient data related to a vehicle is sensed and processed, for use in determining a state change related to external traffic awareness. Based upon the state change, an allowed level of interactivity with a user interface may be changed, and/or a notification may be output. Images and/or depth data may be sensed as part of determining whether a user who is interacting with a device in a stopped vehicle is to be made aware of the changed condition with respect to other vehicles, pedestrians and/or the like.

Claims

1. A method comprising:

allowing a first level of user interaction with an interactive device while a vehicle associated with the interactive device is moving;

monitoring ambient activity relative to the vehicle based on data received from one or more sensors, the ambient activity corresponding to external traffic awareness;

based on the data received from the one or more sensors, determining a change with respect to the ambient activity indicating the vehicle is not moving; and

based on the determined change, allowing a second level of user interaction with the interactive device while the vehicle is not moving.

2. The method of claim 1, further comprising calibrating a stopped state for the vehicle when the vehicle is not moving.

3. The method of claim 1, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

4. The method of claim 1, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

5. The method of claim 1, wherein determining the change with respect to the ambient activity further comprises processing the data received from the set of sensors to detect a change in a state of at least one brake light of another vehicle.

6. The method of claim 1, wherein monitoring the ambient activity relative to the vehicle further comprises:

capturing depth data from a depth camera; and

processing the depth data to determine that an object has decreased its depth relative to the vehicle over time.

7. The method of claim 1, wherein determining the change with respect to the ambient activity further comprises changing the first level of user interaction to the second level of user interaction based on a set of interaction-allowed rules.

8. The method of claim 1, wherein ambient activity comprises pedestrian sensing, and wherein the method further comprising:

reducing pedestrian sensing sensitivity for a highway situation; and

increasing the pedestrian sensing sensitivity for city driving.

9. The method of claim 1, further comprising allowing a third level of user interaction with the interactive device while the vehicle associated with the interactive device is in park mode, the park mode different from the vehicle being stopped.

10. The method of claim 1, wherein the second level of user interaction and the third level of user interaction are allowed when the vehicle is in the park mode.

11. A system comprising:

an interactive device associated with a vehicle, the interactive device providing a first level of interaction while the vehicle is stopped and a second level of user interaction with the interactive device while the vehicle is moving;

one or more sensors for detecting ambient activity relative to the vehicle; and

a processor programmed to;

allow the second level of user interaction with the interactive device while the vehicle associated with the interactive device is moving;

monitor ambient the activity relative to the vehicle based on data received from the one or more sensors;

based on the data received from the one or more sensors, determine a change with respect to the ambient activity indicating the vehicle is not moving; and

based on the determined change, allow the first level of user interaction with the interactive device while the vehicle is not moving.

12. The system of claim 11, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

13. The system of claim 11, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

14. The system of claim 11, wherein the ambient activity comprises at least one sensed object coming within a field of view of the vehicle, the at least one sensed object comprising a pedestrian or an animal.

15. The system of claim 11, wherein the one or more sensors comprises at least one depth sensor configured to capture depth data.

16. One or more computer-readable storage devices having computer-executable instructions, which when executed cause at least one processor to:

allow a first level of user interaction with the interactive device while the vehicle associated with the interactive device is moving;

monitor ambient activity relative to the vehicle based on data received from one or more sensors;

based on the data received from the one or more sensors, determine a change with respect to the ambient activity indicating the vehicle is not moving; and

based on the determined change, allow a second level of user interaction with the interactive device while the vehicle is not moving.

17. The one or more computer-readable storage devices of claim 16, wherein the computer executable instructions are executed to further cause the at least one processor to sense an object coming within a field of view of the vehicle, wherein the sensed object comprises a pedestrian or an animal.

18. The one or more computer-readable storage devices of claim 16, wherein determining the change with respect to the ambient activity indicating the vehicle is not moving further comprises receiving information from a global positioning system.

19. The one or more computer-readable storage devices of claim 16, wherein allowing the second level of user interaction with the interactive device further comprises changing the first level of user interaction to the second level of user interaction, wherein the user interaction with at least one program is not allowed at the first level of user interaction, and wherein the user interaction with the at least one program is allowed at the second level of interaction.

20. The one or more computer-readable storage devices of claim 16, wherein the one or more sensors comprises at least one depth sensor configured to capture depth data.