IMAGE RECONGITION SYSTEM

• US 20080056535A1
• Filed: 09/04/2007
• Published: 03/06/2008
• Est. Priority Date: 09/01/2006
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Mohr, Ulrich, Bergmann, Stephan
• Granted Patent: US 8,311,274 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Methods, devices and systems for recognizing an object in an image are provided, in which the object is recognized by evaluation of both image data and digital map information that corresponds to an area represented by the image. Evaluation of the image data and the digital map information may involve various methods of evaluation including cross-checking, in which the digital map information is utilized to verify correct object recognition in the image data; prediction, in which digital map information is utilized to predict a feature of an object to facilitate object recognition in the image data; or modeling in which a generic model of an object is compared with the image data.

Claims

1. A method for recognizing an object in an image, the method comprising:

retrieving image data corresponding to the image;

retrieving digital map information, where the digital map information comprises map information on at least a portion of an area represented by the image and where the digital map information comprises information on the object; and

evaluating both the image data and the digital map information to recognize the object.

2. The method of claim 1, where evaluating both the image data and the digital map information includes evaluating the image data to recognize the object and evaluating the digital map information to verify that the object has been correctly recognized.

3. The method of claim 1, where the information on the object includes information on at least one of a position, a shape, or an orientation of the object, and where evaluating both the image data and the digital map information includes determining at least one of a position, a shape, or an orientation of the object from the image data, and comparing the at least one of a position, a shape, or an orientation of the object determined from the image data with the corresponding digital map information.

4. The method of claim 1, where evaluating both the image data and the digital map information includes evaluating the digital map information to obtain information on the object and utilizing that information in evaluating the image data in order to facilitate recognition of the object in the image data.

5. The method of claim 1, where evaluating both the image data and the digital map information includes utilizing the digital map information to predict at least one of a position, a shape, or an orientation of the object in the image and where evaluating the image data includes assigning a probability, based upon the predicted at least one of a position, a shape, or an orientation of the object, that a structure having a specific position, shape or orientation in the image is the object to be recognized.

6. The method of claim 1, where the information on the object includes information on the position of the object and where evaluating the image data is restricted to a sub-portion of the image data selected based upon the information on the position of the object.

7. The method of claim 6, where the information on the object includes information on either or both of shape and orientation of the object and where evaluating the image data is restricted to identifying objects based upon information on the either or both of shape and orientation of the object.

8. The method of claim 1, where the evaluating the image data and the digital map information includes comparing a model of the object with the image data.

9. The method of claim 8, where the model of the object has at least one variable parameter, and the evaluating the image data and the digital map information includes finding an optimum value for the at least one variable parameter.

10. The method of claim 9, where the finding the optimum value for the at least one variable parameter includes Bayes filtering or particle filtering.

11. The method of claim 1, where the digital map information is retrieved from a navigation system.

12. The method of claim 1, where the method includes determining a current vehicle position and where the retrieving digital map information is based upon the current vehicle position.

13. The method of claim 12, where the method includes determining a current vehicle orientation and where the retrieving digital map information is based on the current vehicle orientation.

14. The method of claim 1, where the method includes recording the image, where the image shows an area in the vicinity of a vehicle.

15. The method of claim 1, where the method includes providing a result of the evaluating of both the image data and the digital map information to a navigation system.

16. The method of claim 15, where the method includes providing driving instructions to a driver, or alerting a driver, or controlling a vehicle based upon a result of evaluating both the image data and the digital map information.

17. The method of claim 1, where the object is a road, a road lane, a lane marker, a road edge, a road intersection, or a road turnoff or a traffic sign.

18. An image recognition device for recognizing an object in an image, comprising:

memory unit that stores image data corresponding to the image and that stores digital map information, where the digital map information includes map information on at least a portion of an area represented by the image and where the digital map information comprises information on the object; and

processing unit coupled to the memory unit, where the processing unit evaluates both the image data and the digital map information to recognize the object.

19. The image recognition device of claim 18, where evaluating both the image data and the digital map information includes evaluating the image data to recognize the object and evaluating the digital map information to verify that the object has been correctly recognized.

20. The image recognition device of claim 18, where the information on the object includes information on at least one of a position, a shape, or an orientation of the object and where the processing unit (i) evaluates the image data to recognize the object, (ii) determines at least one of a position, a shape, or an orientation of the object from the image data, and (iii) compares the at least one of a position, a shape, or an orientation of the object determined from the image data with said the corresponding digital map information.

21. The image recognition device of claim 18, where the processing unit utilizes the digital map information to facilitate evaluating the image data to recognize the object.

22. The image recognition device of claim 18, where the processing unit utilizes the digital map information to predict at least one of a position, a shape, or an orientation of the object in the image and where the processing unit evaluates the image data assigning a probability, based upon the predicted at least one of a position, a shape or an orientation of the object, that a structure having a specific position, shape or orientation in the image is the object to be recognized.

23. The image recognition device of claim 18, where the information on the object includes information on the position of the object and where the processing unit evaluates a sub-portion of the image data selected based upon the information on the position of the object.

24. The image recognition device of claim 22, where the information on the object includes information on either or both of the shape and orientation of the object and where the processing unit evaluates the image data based upon the information on the shape and/or orientation of the object.

28. The image recognition device of claim 18, where the device includes position and orientation determining for determining a current vehicle position and where the position and orientation determining is coupled to the processing unit and the processing unit is adapted to select a portion of the digital map information based upon the current vehicle position.

29. The image recognition device of claim 28, where the device includes:

orientation determining unit for determining a current vehicle orientation, where the orientation determining unit is coupled to the processing unit and the processing unit is adapted to select a portion of the digital map information based upon the current vehicle orientation.

30. The image recognition device of claim 18, where the device includes:

a camera unit for recording the image, the camera unit being coupled to the memory unit for providing the image to the memory unit.

25. The image recognition device of 18, where the processing unit compares a model of the object with the image data to recognize the object.

26. The image recognition device of claim 25, where the model of the object has at least one variable parameter, and the processing unit finds an optimum value for the at least one variable parameter.

27. The image recognition device of claim 26, where the processing unit employs a Bayes filter or a particle filter to find the optimum value for the at least one variable parameter.

31. A navigation system, comprising:

a. an image recognition device comprising;

(i) memory unit for storing image data corresponding to the image and for storing digital map information, where the digital map information comprises map information on at least a portion of an area represented by the image and where the digital map information comprises information on the object; and

(ii) processing unit coupled to the memory unit, where both the image data and the digital map information are evaluated to recognize the object; and

b. a storage unit for storing a digital map, where the image recognition device is coupled to the storage unit to retrieve at least a portion of the digital map therefrom.

32. The navigation system of claim 31, including an optical or acoustical output unit for outputting driving instructions or warning signals to a driver, where the driving instructions or warning signals are dependent upon the object recognized by the image recognition device.

33. The navigation system of 31, which is a vehicle navigation system including a position and orientation determining for determining a current vehicle position, where the position and orientation determining provides information to the processing unit to identify the relevant map portion for recognition of the object and object position and the processing unit further determines the distance of the vehicle from the object.

34. A navigation system and image recognition device assembly, comprising:

a position determining device that determines position of the device; and

an image recognition device configured to evaluate both image data and digital map information to recognize an object in an image represented by the image data, the image recognition device being coupled to the position determining device to provide information on relative position of the object to the device.

35. The navigation system and image recognition assembly of claim 34, where the position determining device outputs information on a current vehicle position to the image recognition device.

36. The navigation system and image recognition device assembly of claim 35, where the position determining device includes a control circuit configured to recalibrate the position determining device based upon the information on the relative position.