REAL TIME DRIVING DIFFICULTY CATEGORIZATION

• US 20160207458A1
• Filed: 01/21/2015
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Pillai, Preeti Jayagopi, Yalla, Veeraganesh, Oguchi, Kentaro, Nomoto, Hirokazu
• Granted Patent: US 9,686,451 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

The disclosure includes a system and method for determine a real time driving difficulty category. The method may include determining image feature vector data based on one or more features depicted in a real time image of a road scene. The image feature vector data may describe an image feature vector for an edited version of the real time image. The method may include determining offline road map data for the road scene, which includes a static label for a road included in the road scene and offline road information describing a regulatory speed limit for the road. The method may include selecting, based on the static label, a classifier for analyzing the image feature vector. The method may include executing the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene.

Claims

1. A method comprising:

receiving real time image data describing a real time image depicting a road scene;

determining region of interest data describing a version of the real time image configured to remove redundant features from the real time image

determining dense grid sampling data describing one or more features included in the version of the real time image described by the region of interest data;

determining image feature vector data based on the one or more features included dense grid sampling data, the image feature vector data describing an image feature vector for the version of the real time image described by the region of interest data; and

receiving client location data describing the geographic location of a client device associated with the road scene included in the real time image;

determining offline road map data for the road scene based on the geographic location of the client device, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene; and

selecting, based on the static label, a classifier for analyzing the image feature vector; and

executing, using a special-purpose processor device programmed to perform or control performance of the executing, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector, offline road information and runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

2. A method comprising:

determining image feature vector data based on one or more features depicted in a real time image of a road scene described by real time image data, the image feature vector data describing an image feature vector for a version of the real time image which has been edited to exclude one or more extraneous features included in the real time image; and

determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene; and

selecting, based on the static label, a classifier for analyzing the image feature vector; and

executing, using a processor-based computing device programmed to perform or control performance of the executing, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

3. The method of claim 2, wherein the selected classifier is executed in real time relative to when the real time image depicting the road scene is captured.

4. The method of claim 2, wherein the selected classifier is executed within 5 seconds of the real time image depicting the road scene being captured.

5. The method of claim 2, wherein the real time driving difficulty category is determined based in part on the runtime road description data describing the real time driving conditions of the road scene depicted in the real time image.

6. The method of claim 2, wherein the client device is a mobile device.

7. The method of claim 2, wherein the real time image data is captured by an onboard image system.

8. The method of claim 7, wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

9. The method of claim 7, wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

10. The method of claim 2, wherein the offline road map data is determined during an offline process.

11. The method of claim 2, wherein the real time image data and the runtime road description data are captured during a runtime process.

12. A non-transitory computer-readable medium having computer instructions stored thereon that are executable by a processing device to perform or control performance of operations comprising:

determining image feature vector data based on one or more features depicted in a real time image of a road scene described by real time image data, the image feature vector data describing an image feature vector for a version of the real time image which has been edited to exclude one or more extraneous features included in the real time image; and

determining offline road map data for the road scene based on a geographic location of a client device associated with the road scene included in the real time image, the offline road map data describing (1) a static label for a road included in the road scene and (2) offline road information describing a regulatory speed limit for the road included in the road scene; and

selecting, based on the static label, a classifier analyzing the image feature vector; and

executing, using a processor-based computing device programmed to perform or control performance of the executing, the selected classifier to determine a real time driving difficulty category describing the difficulty for a user of the client device to drive in the road scene depicted in the real time image, wherein the real time driving difficulty category is determined based on the image feature vector and the offline road information.

13. The non-transitory computer-readable medium of claim 12, wherein the selected classifier is executed in real time relative to when the real time image depicting the road scene is captured.

14. The non-transitory computer-readable medium of claim 12, wherein the selected classifier is executed within 5 seconds of the real time image depicting the road scene being captured.

15. The non-transitory computer-readable medium of claim 12, wherein the client device is a robot.

16. The non-transitory computer-readable medium of claim 12, wherein the client device is an IoT device.

17. The non-transitory computer-readable medium of claim 12, wherein the real time image data is captured by an onboard image system.

18. The non-transitory computer-readable medium of claim 17, wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be frontward facing.

19. The non-transitory computer-readable medium of claim 17, wherein the client device is a vehicle and the onboard image system is mounted on the vehicle to be rearward facing.

20. The non-transitory computer-readable medium of claim 17 or 19, wherein:

the offline road map data is determined during an offline process; and

the real time image data and the runtime road description data are captured during a runtime process.