Ambient vehicle and pedestrian state detection for device notification

• US 9,165,463 B2
• Filed: 06/26/2012
• Issued: 10/20/2015
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Paek, Timothy S., Bahl, Paramvir
• Primary Examiner(s): POPE, DARYL C
• Application Number:
  US13/533,119
• Publication Number:
  US 20130342363A1
• Time in Patent Office: 1,211 Days
• Field of Search: 340/903, 340/901, 340/435, 340/995, 340/990, 701/117
• US Class Current: 1/1
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

Described is a technology by which ambient data related to a vehicle is sensed and processed, for use in determining a state change related to external traffic awareness. Based upon the state change, an allowed level of interactivity with a user interface may be changed, and/or a notification may be output. Images and/or depth data may be sensed as part of determining whether a user who is interacting with a device in a stopped vehicle is to be made aware of the changed condition with respect to other vehicles, pedestrians and/or the like.

Claims

1. In a computing environment, a method performed at least in part on at least one processor comprising:

allowing one level of user interaction by a user with an interactive mobile device while a vehicle associated with the mobile device is stopped;

sensing ambient state relative to the vehicle;

determining a state change in the ambient state corresponding to a condition associated with external traffic awareness;

outputting a notification; and

responsive to determining from input that the vehicle is moving, reducing user interaction with the mobile device to another level of interaction with the mobile device based upon the ambient state changing to the condition associated with external traffic awareness, reducing the user interaction with the mobile device to the other level of interaction comprising disallowing interaction with at least one program that is allowed in the one level of interaction.

2. The method of claim 1, wherein determining the state change comprises processing sensed data to detect a change in a stoplight'"'"'s state.

3. The method of claim 1, wherein determining the state change comprises processing sensed data to detect a change in at least one brake light'"'"'s state.

4. The method of claim 1, wherein sensing the ambient state relative to the vehicle comprises capturing depth data, and wherein determining the state change comprises processing the depth data to determine that an object has increased its depth relative to the vehicle over time.

5. The method of claim 1, wherein determining the state change comprises:

monitoring for pedestrians near the vehicle;

processing sensed data to detect at least one pedestrian of the pedestrians; and

notifying the user of the at least one pedestrian of the pedestrians.

6. The method of claim 5, wherein sensing the ambient state relative to the vehicle comprises capturing depth data, and wherein determining the state change comprises processing the sensed data and further comprises processing the depth data.

7. The method of claim 1, wherein allowing one level of interaction comprises allowing interaction with at least one program that is not allowed in the other level of interaction.

8. The method of claim 1, wherein outputting the notification comprises outputting feedback comprising graphics, video, animation, one or more special effects, audio, or haptic feedback, or any combination of graphics, video, animation, one or more special effects, audio, or haptic feedback.

9. The method of claim 1, wherein outputting the notification and reducing the interaction to the other level of interaction comprises gradually transitioning from an interactive state to a notification state.

10. The method of claim 1, wherein determining the state change comprises processing GPS data, user preference data, traffic data or vehicle-related data, or any combination of GPS data, user preference data, traffic data or vehicle-related data.

11. The method of claim 1, further comprising calibrating a stopped state for the vehicle.

12. The method of claim 1, wherein determining a state change in the ambient state comprises determining whether the vehicle associated with the mobile device is in one of a stop mode and a park mode, the park mode different from the stop mode.

13. A system comprising:

an automotive user interface of a vehicle;

a sensor set; and

detection and notification logic, the detection and notification logic configured to process sensed data from the sensor set, including to monitor ambient activity near the vehicle, the ambient activity comprising external traffic and pedestrians, and determine a state change with respect to the ambient activity, associated with an external traffic awareness state, the detection and notification logic further configured to output a notification to the automotive user interface upon detection of the state change.

14. The system of claim 13, wherein the detection and notification logic and the automotive user interface are incorporated into a mobile device.

15. The system of claim 14, wherein the sensor set comprises a camera of the mobile device, or a set of one or more sensors coupled to the mobile device, or both a camera of the mobile device and a set of one or more sensors coupled to the mobile device.

16. The system of claim 13, wherein the sensor set comprises a sensor that captures frames of depth data.

17. The system of claim 13, wherein the detection and notification logic is further configured to change an allowed level of user interaction with the automotive user interface based upon the detection of the state change.

18. The system of claim 13, wherein the detection and notification logic is further configured to process GPS data, user preference data, traffic data or vehicle-related data, or any combination of GPS data, user preference data, traffic data or vehicle-related data in association with the state change determination.

19. The system of claim 13, further comprising:

the automotive user interface configured to notify a user of the pedestrians, wherein notification comprises graphics, video, animation, one or more special effects, audio, or haptic feedback, or any combination of graphics, video, animation, one or more special effects, audio, or haptic feedback.

20. In a computing environment, a method performed at least in part on at least one processor comprising:

identifying a vehicle-related parameter;

based on the vehicle-related parameter, determining whether the vehicle associated with the mobile device is in one of a stop mode and a park mode, the park mode different from the stop mode;

on determining that the vehicle associated with the mobile device is in the stop mode, allowing a first level of user interaction with one or more programs; and

on determining that the vehicle associated with the mobile device is in the park mode, allowing a second level of user interaction with the one or more programs, the second level different form the first level.