User transportable device with hazard monitoring

• US 8,953,841 B1
• Filed: 09/07/2012
• Issued: 02/10/2015
• Est. Priority Date: 09/07/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Leblang, Jonathan A., Mackraz, James D.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Harandi, Siamak
• Application Number:
  US13/607,145
• Time in Patent Office: 886 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

In some examples, a user transportable device may determine, based at least in part on sensor input, that the device is in motion. For example, the device may determine there is a likelihood that a user of the device is walking, running, traveling in a vehicle, or the like. In response, the device may present, on a display, an image obtained from a camera oriented, at least in part, toward a direction of travel. Further, in some examples, one or more images from the camera and/or sensor input from other sensors on the device may be analyzed to detect whether an object, obstruction or other hazard is in a direction of travel of the user of the device. If the device determines that a hazard may be imminently encountered by the user, the device may provide an alert to the user.

Claims

1. One or more non-transitory computer-readable media maintaining instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

determining, based at least in part on information from a sensor, that a user transportable device is in motion;

determining a current orientation of a display of the user transportable device;

based at least in part on determining that the user transportable device is in motion and the current orientation of the display is determined to be an orientation where the display faces away from a direction of motion of the user transportable device, automatically start monitoring for potential hazards, the monitoring for potential hazards comprises;

obtaining an image from a camera oriented, at least in part, in a direction of travel of the user transportable device;

presenting, on the display of the user transportable device, the image from the camera;

determining, based at least in part on the image, that an object in the image is a potential hazard; and

based at least in part on determining that the object in the image is a potential hazard, providing an alert.

2. The one or more non-transitory computer-readable media as recited in claim 1, wherein the determining the object is the potential hazard further comprises recognizing one or more objects in the image that are substantially in the direction of travel.

3. The one or more non-transitory computer-readable media as recited in claim 1, wherein the obtaining the image from the camera is further based, at least in part, on identifying one or more applications or modules presenting information on the display of the user transportable device.

4. The one or more non-transitory computer-readable media as recited in claim 1, wherein the alert includes at least one of:

a visual alert presented on the display of the user transportable device;

an audible alert played over a speaker of the user transportable device;

or

a tactile alert generated by a tactile output component.

5. The one or more non-transitory computer-readable media as recited in claim 1, wherein the information from the sensor comprises information received from at least one of:

an accelerometer;

a compass;

a Global Positioning System device;

a communication interface;

a sonar device;

a microphone;

or

the camera.

6. A method comprising:

under control of one or more processors configured with executable instructions,

determining, based at least in part on information from a sensor, that a user transportable device is in motion;

in response to determining that the user transportable device is in motion, presenting, on a display, an image from a camera oriented, at least in part, in a direction of the motion; and

recognizing one or more objects in the presented image indicating that a user of the user transportable device is driving a vehicle.

7. The method as recited in claim 6, wherein the presenting the image on the display is further based, at least in part, on determining that an orientation of a display of the user transportable device indicates that the display is likely to be viewable by a user.

8. The method as recited in claim 6, wherein the presenting the image on the display is further based, at least in part, on determining that a user has made at least one user input to the user transportable device within a threshold period of time of the determining that the user transportable device is in motion.

9. The method as recited in claim 8, wherein the at least one user input is at least one of a text entry or a touch input.

10. The method as recited in claim 6, wherein the presenting the image on the display is further based, at least in part, on identifying at least one application executing on the user transportable device.

11. The method as recited in claim 6, further comprising determining a likelihood that a user of the user transportable device is in motion based on at least one of:

detected sustained motion of the user transportable device;

or

detection of motion indicative of a gait of the user.

12. The method as recited in claim 6, wherein the image is of a region in a direction of travel of a user of the user transportable device.

13. The method as recited in claim 6, further comprising determining, based at least in part on the image, that an object in the image is a potential hazard.

14. The method as recited in claim 13, wherein the determining that the object is the potential hazard comprises:

recognizing the object in the image; and

determining that the object is the potential hazard based at least in part on at least one of;

the recognizing the object in the image, or

the information from the sensor.

15. The method as recited in claim 13, wherein the determining that the object is the potential hazard comprises:

sending the image to a remote computing device; and

receiving information related to the potential hazard from the remote computing device.

16. The method as recited in claim 13, further comprising providing an alert on the user transportable device in response to the determining that the object is the potential hazard, wherein the alert comprises at least one of:

a visual alert;

an audible alert;

or

a tactile alert.

17. The method as recited in claim 6, further comprising:

detecting that the user transportable device is no longer in motion; and

ceasing the presenting of the image from the camera on the display.

18. The method as recited in claim 6, further comprising:

at least one of;

providing a warning to the user;

deactivating at least one hardware component of the user transportable device;

or

deactivating at least one executable module of the user transportable device.

19. The method as recited in claim 6, wherein the information from the sensor comprises information received from at least one of:

an accelerometer;

a compass;

a Global Positioning System device;

a communication interface;

a sonar device;

a microphone;

or

the camera.

20. The method as recited in claim 6, wherein the image is presented as at least one of:

a partially transparent image overlaid on a user interface;

an image presented under a partially transparent user interface;

a portion of the image received from the camera;

or

an augmented image that visually distinguishes one or more objects in the image.

21. A user transportable device comprising:

one or more processors;

one or more computer-readable media; and

one or more modules maintained on the one or more computer-readable media and executed by the one or more processors to perform acts comprising;

in response to determining that the user transportable device is in motion, monitoring a region in a direction of travel of the user transportable device;

recognizing one or more objects in the monitored region indicating that a user of the user transportable device is driving a vehicle; and

providing an alert on the user transportable device when a potential hazard is determined to be in the region in the direction of travel based at least in part on recognizing the one or more objects in the monitored region.

22. The user transportable device as recited in claim 21, the acts further comprising presenting, on a display of the user transportable device, an image from a camera facing, at least in part, the direction of travel of the user transportable device.

23. The user transportable device as recited in claim 21, the acts further comprising:

receiving an image from a camera facing, at least in part, the direction of travel of the user transportable device; and

determining that the potential hazard is in the region based, at least in part, on a content of the image.

24. The user transportable device as recited in claim 21, the acts further comprising:

receiving an image from a camera facing, at least in part, the direction of travel of the user transportable device;

sending the image to a remote computing device; and

receiving, from the remote computing device, information related to the potential hazard based, at least in part, on a content of the image.

25. The user transportable device as recited in claim 21, wherein the determining that the user transportable device is in motion comprises receiving information from at least one of:

an accelerometer;

a compass;

a Global Positioning System device;

a communication interface;

a sonar device;

a microphone;

or

the camera.

26. The user transportable device as recited in claim 21, wherein the potential hazard is determined to be in the region based at least in part on information received from at least one of:

an accelerometer;

a compass;

a Global Positioning System device;

a communication interface;

a sonar device;

a microphone;

or

the camera.

27. A system comprising:

one or more processors;

one or more computer readable media maintaining one or more instructions executable by the one or more processors to perform operations comprising;

receiving sensor information from a user transportable device;

determining, based at least in part on the sensor information, that the user transportable device is in motion;

receiving orientation information of a display of the user transportable device;

based at least in part on determining that the user transportable device is in motion and the current orientation of the display is an orientation where the display faces away from the direction of motion of the user transportable device, automatically start monitoring for potential hazards, the monitoring for potential hazards comprises;

receiving an image from the user transportable device;

detecting an object in the image; and

sending information to the user transportable device indicating that the object in the image is a potential hazard.

28. The system as recited in claim 27, the operations further comprising receiving sensor information from the user transportable device, wherein the sending the information indicating that the object in the image is the potential hazard is based at least in part on the sensor information.

29. The system as recited in claim 28, wherein:

the potential hazard is determined based, at least in part, on a direction of travel of the user transportable device; and

the direction of travel is determined based at least in part on the sensor information.