Vision augmented navigation

• US 9,671,243 B2
• Filed: 06/13/2014
• Issued: 06/06/2017
• Est. Priority Date: 06/13/2013
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon
• Primary Examiner(s): Chen, Shelley
• Application Number:
  US14/303,830
• Publication Number:
  US 20140372020A1
• Time in Patent Office: 1,089 Days
• Field of Search: 701438
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects

Abstract

A navigation system may include at least one processing device configured to determine, based on an output of one or more position sensors associated with the navigation system, a current location of at least one component associated with the navigation system and determine a destination location different from the current location. The navigation system may also acquire, from one or more image acquisition devices, a plurality of images representative of an environment of a user of the navigation system and derive, from the plurality of images, visual information associated with at least one object in the environment. The system may also determine one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include at least one reference to the visual information derived from the plurality of images. The system may also deliver to the user the one or more instructions.

Claims

1. A navigation system, comprising:

at least one processing device configured to;

determine, based on an output of one or more position sensors associated with the navigation system, a current location of at least one component associated with the navigation system;

determine a destination location different from the current location;

acquire, from one or more image acquisition devices, while a user of the navigation system is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;

analyze, while the user is navigating from the current location to the destination location, the plurality of images to derive a physical characteristic of at least one object appearing within a field of view of at least one of the image acquisition devices;

determine, based on an appearance of the physical characteristic of the at least one object derived from the plurality of images while the user is navigating, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include a description of the physical characteristic of the at least one object derived from the plurality of images; and

deliver to the user the one or more instructions.

2. The navigation system of claim 1, wherein determination of the destination location is based on user input.

3. The navigation system of claim 2, wherein the user input includes entry of an address corresponding to the destination location.

4. The navigation system of claim 1, wherein the one or more image acquisition devices include a first camera and a second camera, the first camera having a focal length different from a focal length of the second camera.

5. The navigation system of claim 4, wherein the one or more image acquisition devices include a third camera having a focal length different from the focal length of the first camera and the focal length of the second camera.

6. The navigation system of claim 1, wherein the one or more image acquisition devices include a camera having a resolution of at least 5 megapixels.

7. The navigation system of claim 1, wherein the at least one object includes at least one of a vehicle, a building, a sign, a traffic light, a road feature, a structure, or a person.

8. The navigation system of claim 1, wherein the at least one processing device is configured to:

use map data to identify at least one mapped entity;

determine, based on the plurality of images, whether the at least one mapped entity is visible to the user; and

include in the one or more instructions at least one reference to the mapped entity, if the mapped entity is determined to be visible to the user.

9. The navigation system of claim 1, wherein the at least one processing device is configured to:

use map data to identify at least one mapped entity;

determine, based on the plurality of images, whether the at least one mapped entity is visible to the user; and

include in the determined one or more instructions at least one reference to the mapped entity and at least one reference to an object obscuring the mapped entity from the user'"'"'s view, if the mapped entity is determined not to be visible to the user.

10. The navigation system of claim 1, wherein the at least one processing device is configured to:

use map data to identify at least one mapped entity;

determine, based on the plurality of images, whether the at least one mapped entity is visible to the user; and

omit from the determined one or more instructions any references to the mapped entity, if the mapped entity is determined not to be visible to the user.

11. The navigation system of claim 1, wherein the physical characteristic includes a color associated with the at least one object in the environment.

12. The navigation system of claim 1, wherein the physical characteristic includes an object type associated with the at least one object in the environment.

13. The navigation system of claim 1, wherein the one or more instructions are delivered audibly to the user.

14. The navigation system of claim 1, wherein the one or more instructions are delivered to at least one of a display or a speaker associated with a mobile device.

15. The navigation system of claim 1, wherein the one or more instructions are delivered to at least one of a display or a speaker associated with a vehicle.

16. The navigation system of claim 1, wherein the one or more instructions are delivered to at least one of a display or a speaker associated with a wearable device.

17. The navigation system of claim 1, wherein the one or more image acquisition devices include a camera from an Advanced Driver Assistance System.

28. The navigation system of claim 1, wherein the at least one object includes a target vehicle, and the at least one processing device is further configured to:

analyze the plurality of images to derive the physical characteristic associated with a target vehicle located in the environment at a distance of interest from the current location,

wherein the one or more instructions include a reference to the description of the derived physical characteristic associated with the target vehicle.

35. The navigation system of claim 1, wherein the physical characteristic of the at least one object is a color of an awning.

36. The navigation system of claim 1, wherein the at least one object is a mapped entity.

37. The navigation system of claim 36, wherein the mapped entity is a landmark.

38. The system of claim 1, wherein the at least one object includes a plurality of mapped entities, and the plurality of mapped entities are prioritized to determine which of the plurality of mapped entities to use for deriving the physical characteristic.

39. The navigation system of claim 38, wherein the plurality of mapped entities are prioritized based on their locations relative to an intersection or a turn direction.

40. The navigation system of claim 38, wherein the plurality of mapped entities are prioritized based on their visibility and locations.

41. The navigation system of claim 1, wherein the one or more instructions are further determined based on motion characteristics of the at least one object derived from the plurality of images.

42. The navigation system of claim 1, wherein the one or more instructions are further determined based on a visibility condition of the at least one object.

18. A vehicle, comprising:

a GPS sensor configured to provide an output representative of a position of the vehicle;

one or more image acquisition devices configured to acquire a plurality of images of an environment outside of the vehicle;

at least one processing device configured to;

determine, based on an output of one or more position sensors associated with the vehicle, a current location of at least one component associated with the vehicle;

determine a destination location different from the current location;

acquire, from one or more image acquisition devices, while a user of the vehicle is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;

analyze, while the user is navigating from the current location to the destination location, the plurality of images to derive a physical characteristic of at least one object appearing within a field of view of at least one of the image acquisition devices;

determine, based on an appearance of the physical characteristic of the at least one object derived from the plurality of images while the user is navigating, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include a description of the physical characteristic of the at least one object derived from the plurality of images; and

deliver to the user the one or more instructions.

19. The vehicle of claim 18, wherein determination of the destination location is based on user input including an address corresponding to the destination location.

20. The vehicle of claim 18, wherein the one or more image acquisition devices include a first camera and a second camera, the first camera having a focal length different from a focal length of the second camera.

21. The vehicle of claim 18, wherein the one or more image acquisition devices include a camera having a resolution of at least 5 megapixels.

22. The vehicle of claim 18, wherein the at least one object includes at least one of a vehicle, a building, a sign, a traffic light, a road feature, a structure, or a person.

23. The vehicle of claim 18, wherein the at least one processing device is configured to:

use map data to identify at least one mapped entity;

determine, based on the plurality of images, whether the at least one mapped entity is visible to the user; and

include in the determined one or more instructions at least one reference to the mapped entity, if the mapped entity is determined to be visible to the user.

24. The vehicle of claim 23, wherein the at least one processing device is configured to:

include in the determined one or more instructions at least one reference to the mapped entity and at least one reference to an object obscuring the mapped entity from the user'"'"'s view, when the mapped entity is determined not to be visible to the user.

25. The vehicle of claim 18, wherein the physical characteristic includes at least one of a color or a type associated with the at least one object in the environment.

26. The vehicle of claim 18, wherein the one or more instructions are delivered audibly to the user.

27. A method for generating and delivering navigational information, the method comprising:

using one or more processing devices to determine a current location based on an output of one or more position sensors;

determining a destination location different from the current location;

acquiring, from one or more image acquisition devices, while an individual is located in an environment and navigating from the current location to the destination location, a plurality of images representative of the environment;

using the one or more processing devices to analyze, while the user is navigating from the current location to the destination location, the plurality of images to derive a physical characteristic of at least one object appearing within a field of view of at least one of the image acquisition devices;

determining, based on an appearance of the physical characteristic of the at least one object derived from the plurality of images while the user is navigating, one or more instructions for navigating from the current location to the destination location, wherein the one or more instructions include physical characteristic of the at least one object derived from the plurality of images; and

delivering to the individual the one or more instructions.

29. A navigation system comprising:

at least one processing device configured to;

acquire, from one or more image acquisition devices, while a vehicle is located in an environment and navigating from a current location to a destination location, a plurality of images representative of the environment;

analyze, while the vehicle is navigating from the current location to the destination location, the plurality of images to derive a physical characteristic associated with at least one object appearing in a field of view of at least one of the image acquisition devices; and

determine one or more instructions for navigating the vehicle from the current location to the destination location, wherein the one or more instructions include at least one reference to a description of the physical characteristic of the at least one object derived from the plurality of images.

30. The navigation system of claim 29, wherein the at least one object appearing in the field of view of the at least one of the image acquisition devices is at least partially visually obstructing a landmark.

31. The navigation system of claim 29, wherein the at least one object appearing in the field of view of the at least one of the image acquisition devices is another vehicle, and the one or more instructions include at least one reference to the description of the physical characteristic derived from the another vehicle.

32. The navigation system of claim 29, wherein the at least one object appearing in the field of view of the at least one of the image acquisition devices is a pedestrian.

33. The navigation system of claim 29, wherein at least one of the image acquisition devices is mounted behind a windshield of the vehicle.

34. The navigation system of claim 33, wherein the image acquisition devices are part of or are associated with an Advanced Driver Assistance System (“

ADAS”

) or an Autonomous Vehicle (“

AV”

) system.