Ambient Vehicle and Pedestrian State Detection for Device Notification

• US 20130342363A1
• Filed: 06/26/2012
• Published: 12/26/2013
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Paek, Timothy S., Bahl, Paramvir
• Granted Patent: US 9,165,463 B2
• Time in Patent Office: Days
• US Class Current: 340/901
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

Described is a technology by which ambient data related to a vehicle is sensed and processed, for use in determining a state change related to external traffic awareness. Based upon the state change, an allowed level of interactivity with a user interface may be changed, and/or a notification may be output. Images and/or depth data may be sensed as part of determining whether a user who is interacting with a device in a stopped vehicle is to be made aware of the changed condition with respect to other vehicles, pedestrians and/or the like.

Claims

1. In a computing environment, a method performed at least in part on at least one processor comprising:

allowing one level of interaction with an interactive device while a vehicle associated with the device is stopped;

sensing ambient state relative to the vehicle;

determining a state change in the ambient state corresponding to a condition associated with external traffic awareness; and

outputting a notification and reducing interaction to another level of interaction with the interactive device based upon the ambient state changing to the condition associated with external traffic awareness.

2. The method of claim 1 wherein determining the state change comprises processing sensed data to detect a change in a stoplight'"'"'s state.

3. The method of claim 1 wherein determining the state change comprises processing sensed data to detect a change in at least one brake light'"'"'s state.

4. The method of claim 1 wherein sensing the ambient state relative to the vehicle comprises capturing depth data, and wherein determining the state change comprises processing the depth data to determine that an object has increased its depth relative to the vehicle over time.

5. The method of claim 1 wherein determining the state change comprises processing sensed data to detect at least one pedestrian or other object.

6. The method of claim 5 wherein sensing the ambient state relative to the vehicle comprises capturing depth data, and wherein determining the state change comprises processing the sensed data comprises processing the depth data.

7. The method of claim 1 wherein allowing one level of interaction comprises allowing interaction with at least one program that is not allowed in the other level of interaction, and wherein reducing the interaction to the other level of interaction comprises disallowing interaction with at least one program that is allowed in the one level of interaction.

8. The method of claim 1 wherein outputting the notification comprises outputting feedback comprising graphics, video, animation, one or more special effects, audio, or haptic feedback, or any combination of graphics, video, animation, one or more special effects, audio, or haptic feedback.

9. The method of claim 1 wherein outputting the notification and reducing the interaction to the other level of interaction comprises gradually transitioning from an interactive state to a notification state.

10. The method of claim 1 wherein determining the state change comprises processing GPS data, user preference data, traffic data or vehicle-related data, or any combination of GPS data, user preference data, traffic data or vehicle-related data.

11. The method of claim 1 further comprising calibrating a stopped state for the vehicle.

12. A system comprising, an automotive user interface of a vehicle, a sensor set, and detection and notification logic, the detection and notification logic configured to process sensed data from the sensor set, including to determine a state change associated with an external traffic awareness state, the detection and notification logic further configured to output a notification to the automotive user interface upon detection of the state change.

13. The system of claim 12 wherein the detection and notification logic and the automotive user interface are incorporated into a mobile device.

14. The system of claim 13, wherein the sensor set comprises a camera of the mobile device, or a set of one or more sensors coupled to the mobile device, or both a camera of the mobile device and a set of one or more sensors coupled to the mobile device.

15. The system of claim 12 wherein the sensor set comprises a sensor that captures frames of depth data.

16. The system of claim 12 wherein the detection and notification logic is further configured to change an allowed level of user interaction with the automotive user interface based upon the detection of the state change.

17. The system of claim 12 wherein the detection and notification logic is further configured to process GPS data, user preference data, traffic data or vehicle-related data, or any combination of GPS data, user preference data, traffic data or vehicle-related data in association with the state change determination.

18. The system of claim 12 wherein notification comprises graphics, video, animation, one or more special effects, audio, or haptic feedback, or any combination of graphics, video, animation, one or more special effects, audio, or haptic feedback.

19. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

processing depth data to determine a state change related to external traffic awareness; and

changing an allowed level of interactivity with a user interface based upon the state change.

20. The one or more computer-readable media of claim 19 having further computer executable instructions comprising outputting a notification based upon the state change.