Visual route book data sets

• US 10,203,211 B1
• Filed: 12/18/2015
• Issued: 02/12/2019
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Mishra, Pragyana K., Norwood, Jonathan Blair
• Primary Examiner(s): Antonucci, Anne M
• Assistant Examiner(s): Malhotra, Sanjeev
• Application Number:
  US14/975,432
• Time in Patent Office: 1,152 Days
• Field of Search: 701 1, 701 2, 701 41, 701117, 701427, 701439, 701530, 701532, 701533, 701424, 348135, 345501
• CPC Class Codes: G01C 21/3415 Dynamic re-routing, e.g. recalculating the route when the user deviates from calculated route or after detecting real-time traffic data or accidents
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects
• CPC Class Codes: G06Q 10/0838 Historical data

Abstract

Route images that have been acquired along delivery routes may be automatically analyzed for visual saliency (e.g., based on image motion, image content, sensor information, carrier actions, etc.) and stored along with related route information as part of a visual route book data set. The route images may be acquired by carriers utilizing mobile recording devices (e.g., mobile phones, wearable cameras, vehicle mounted cameras, etc.) while travelling along delivery routes for delivering items. For assisting a carrier in navigating along a delivery route, route images and related route information, such as visual cues associated with salient objects or features, may be selected from the visual route book data set and presented to the carrier (e.g., as part of a visual route summary and/or as based on a current location of the carrier).

Claims

1. A computer implemented method for analyzing route images, the method comprising:

under control of one or more computing systems configured with executable instructions,

determining delivery routes for carriers for delivering items to delivery locations;

receiving a plurality of route images from recording devices carried on the delivery routes by the carriers;

receiving, from a velocity sensor, sensor information indicating stops and starts of movement of a carrier along a delivery route;

selecting a first route image from the plurality of route images for inclusion in a visual route book data set, the selecting being based at least in part on the sensor information indicating that the first route image was taken at a stopping location along the delivery route;

identifying a visual cue associated with the delivery route in the first route image;

recording the first route image and associated visual cue in the visual route book data set; and

selecting one or more route images from the visual route book data set for presentation to a carrier to assist the carrier in navigating along the delivery route that the visual cue is associated with, wherein the one or more route images that are selected include the first route image and the visual cue is indicated to the carrier by an emphasis element that is included in the presented first route image.

2. The computer implemented method of claim 1, wherein at least one of the recording devices is worn by a carrier or mounted on a vehicle while traveling on one of the delivery routes.

3. The computer implemented method of claim 1, wherein at least two of the received plurality of route images are included in a video sequence of route images that is received from one of the recording devices.

4. The computer implemented method of claim 1, further comprising not selecting a second route image of the plurality of route images for inclusion in the visual route book data set based at least in part on the sensor information indicating that the second route image was not taken at a stopping location along the delivery route.

5. The computer implemented method of claim 1, wherein the first route image is acquired inside a structure in which a carrier is required to navigate on foot as part of the delivery route and the stopping location along the delivery route where the first route image was taken is inside the structure.

6. The computer implemented method of claim 1, wherein the first route image is of an object at the stopping location along the delivery route.

7. A computer implemented method, comprising:

under control of one or more computing systems configured with executable instructions,

receiving, from a velocity sensor, sensor information indicating stops and starts of movement;

selecting a first route image from a plurality of route images for inclusion in a visual route book data set, the selecting being based at least in part on the sensor information indicating that the first route image was taken at a stopping location along a delivery route;

determining a delivery route for a carrier for delivering an item to a delivery location;

selecting one or more route images from the visual route book data set for presentation to the carrier to assist the carrier in navigating along the delivery route, wherein the selected one or more route images include the first route image;

identifying a visual cue in the first route image based at least in part on the delivery route for the carrier; and

providing the selected one or more route images for presentation to the carrier, wherein the visual cue is indicated to the carrier by an emphasis element that is included in the presented first route image.

8. The computer implemented method of claim 7, further comprising providing an instruction for presentation to the carrier that is related to the first route image, wherein the instruction is determined from metadata that is included in the visual route book data set.

9. The computer implemented method of claim 7, further comprising selecting a second route image from the plurality of route images for inclusion in the visual route book data set based at least in part on an indication that the second route image corresponds to a transition along the delivery route, and the transition occurs due to at least one of a turn, an exit, a ramp, a traffic condition, a detour, a speed limit, or a school zone.

10. The computer implemented method of claim 7, wherein the first route image corresponds to at least one of a parking location, a delivery location, or a placement location.

11. The computer implemented method of claim 7, further comprising providing an instruction for presentation to the carrier that indicates at least one of a shape or a color that is included in the first route image.

12. The computer implemented method of claim 7, wherein the first route image includes at least one of a doorway, stairway, elevator, counter or desk that is located along the delivery route.

13. The computer implemented method of claim 7, wherein the first route image indicates a placement location where the item is to be placed by the carrier for delivering the item.

14. A computing system, comprising:

one or more processors; and

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to at least;

receive a plurality of route images that have been acquired at locations along a delivery route;

receive associated location information that indicates the locations along the delivery route where the route images were acquired;

receive associated sensor information from a velocity sensor that indicates stops and starts of movement along the delivery route;

perform an analysis to select which route images from the plurality of route images will be included in a visual route book data set, wherein the analysis that is performed is based at least in part on the sensor information and includes at least;

selecting a first route image from the plurality of route images for inclusion in the visual route book data set based at least in part on the sensor information indicating that the first route image was taken at a stopping location along the delivery route;

based on the selection of the first route image for inclusion in the visual route book data set, store the first route image as part of the visual route book data set; and

select one or more route images from the visual route book data set for presentation to a carrier to assist the carrier in navigating along the delivery route, wherein the one or more route images that are selected include the first route image.

15. The computing system of claim 14, wherein the analysis to select which route images from the plurality of route images will be included in the visual route book data set is further based at least in part on content analysis, and the content analysis includes utilizing a computer vision algorithm for detecting an object in a route image.

16. The computing system of claim 15, wherein the content analysis includes comparing the first route image to a prior route image that was acquired at the same location along the delivery route, the object that is detected in the first route image represents a change from the prior route image, and the first route image is selected based at least in part on the change.

17. The computing system of claim 14, wherein the analysis to select which route images from the plurality of route images will be included in the visual route book data set is further based at least in part on motion analysis, and the motion analysis includes determining at least one motion vector.

18. The computing system of claim 17, wherein the at least one motion vector is utilized for determining a relative movement which corresponds to at least one of a transition or a stop along the delivery route.

19. The computing system of claim 14, wherein the analysis to select which route images from the plurality of route images will be included in the visual route book data set is further based at least in part on a carrier action that was performed when a route image was acquired.

20. The computing system of claim 14, wherein the memory further includes program instructions that when executed by the one or more processors cause the one or more processors to include a second route image in the visual route book data set that is provided by a user and indicates a placement location where an item is to be placed for delivering the item to the user.