Method and device for capturing image of traffic sign

• US 10,325,339 B2
• Filed: 04/26/2016
• Issued: 06/18/2019
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Lee, Hee-Seok, Kim, Kang, Kim, Duck Hoon, Kim, Sungwoong, Hong, Seok-Soo
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Brumfield, Shanika M
• Application Number:
  US15/138,778
• Publication Number:
  US 20170308989A1
• Time in Patent Office: 1,148 Days
• Field of Search: 348148
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G06F 18/2135 based on approximation criteria, e.g. principal component analysis
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/685 performed by mechanical compensation

Abstract

A method for capturing an image of a traffic sign by an image sensor is disclosed. The method may include capturing, by the image sensor, at least one image including the traffic sign, wherein the image sensor is mounted in a vehicle. The method may include detecting, by a processor, the traffic sign in the at least one image. The method may also include in response to detecting the traffic sign, determining, by the processor, at least one direction of the image sensor based on the at least one image and motion of the vehicle. In addition, the method may include adjusting, by the processor, the image sensor to the at least one direction.

Claims

1. A method for capturing an image of a traffic sign by an image sensor, comprising:

capturing, by the image sensor, at least one image comprising the traffic sign, wherein the image sensor is mounted in a vehicle;

detecting, by a processor, the traffic sign in the at least one image;

in response to detecting the traffic sign, determining, by the processor, at least one direction of the image sensor based on a direction normal to a face of the traffic sign in the at least one image, a distance between the traffic sign and the image sensor, and motion of the vehicle; and

adjusting, by the processor, the image sensor to the at least one direction based on the direction normal to the face of the traffic sign, the distance, and the motion of the vehicle, wherein adjusting the image sensor comprises adjusting at least one among yaw, pitch, and roll of the image sensor.

2. The method of claim 1, further comprising:

in response to adjusting the image sensor, capturing, by the image sensor, at least one or more images comprising the traffic sign.

3. The method of claim 1, further comprising adjusting the image sensor to one or more directions based on the motion of the vehicle.

4. The method of claim 1, wherein the image sensor tracks the traffic sign based on the at least one direction.

5. The method of claim 1, further comprising:

determining the direction normal to the face of the traffic sign based on the at least one image; and

determining the distance between the traffic sign and the image sensor.

6. The method of claim 5, wherein the distance is determined based on the at least one image or distance information from a distance sensor.

7. The method of claim 5, wherein determining the direction normal to the face of the traffic sign comprises determining a direction of a road in the at least one image as the normal direction.

8. The method of claim 1, wherein the motion of the vehicle is indicative of information comprising at least one of displacement, elevation, speed, direction, or acceleration of the vehicle.

9. The method of claim 1, wherein detecting the traffic sign in the at least one image is based on at least one of a color, a shape, or a location of the traffic sign.

10. The method of claim 1, wherein adjusting the image sensor comprises:

generating, by the processor, at least one control signal configured to rotate the image sensor to the at least one direction; and

rotating, by at least one motor unit, the image sensor to the at least one direction in response to the at least one control signal to reduce motion blur associated with the motion of the vehicle relative to the traffic sign.

11. An electronic device in a vehicle for capturing an image of a traffic sign, comprising:

an image sensor configured to capture at least one image comprising the traffic sign;

a traffic sign detection unit configured to detect the traffic sign in the at least one image;

a rotation determination unit configured to determine, in response to detecting the traffic sign, at least one direction of the image sensor based on a direction normal to a face of the traffic sign in the at least one image, a distance between the traffic sign and the image sensor, and motion of the vehicle; and

a motor control unit configured to adjust the image sensor to the at least one direction based on the direction normal to the face of the traffic sign, the distance, and the motion of the vehicle, wherein adjusting the image sensor comprises adjusting at least one among yaw, pitch, and roll of the image sensor.

12. The electronic device of claim 11, wherein the image sensor is further configured to capture one or more images comprising the traffic sign in response to determining the at least one direction.

13. The electronic device of claim 11, wherein the motor control unit is further configured to adjust the image sensor to one or more directions based on the motion of the vehicle.

14. The electronic device of claim 11, wherein the image sensor is configured to track the traffic sign based on the at least one direction.

15. The electronic device of claim 11, further comprising:

a normal determination unit configured to determine the direction normal to the face of the traffic sign based on the at least one image; and

a distance determination unit configured to determine the distance between the traffic sign and the image sensor.

16. The electronic device of claim 15, wherein the distance determination unit is configured to determine the distance based on the at least one image or distance information from a distance sensor.

17. The electronic device of claim 15, wherein the normal determination unit is configured to determine a direction of a road in the at least one image as the normal direction.

18. The electronic device of claim 11, wherein the motion of the vehicle is indicative of information comprising at least one of displacement, elevation, speed, direction, or acceleration of the vehicle.

19. The electronic device of claim 11, wherein the traffic sign detection unit is configured to detect the traffic sign based on at least one of a color, a shape, or a location of the traffic sign in the at least one image.

20. The electronic device of claim 11, further comprising at least one motor unit configured to rotate the image sensor,

wherein the motor control unit is further configured to generate at least one control signal adapted to rotate the image sensor to the at least one direction, and

wherein the at least one motor unit is configured to rotate the image sensor to the at least one direction in response to the at least one control signal to reduce motion blur associated with the motion of the vehicle relative to the traffic sign.

21. A non-transitory computer-readable storage medium comprising instructions causing at least one processor of an electronic device to perform operations of:

capturing, by an image sensor, at least one image comprising a traffic sign, wherein the image sensor is mounted in a vehicle;

detecting, by the at least one processor, the traffic sign in the at least one image;

in response to detecting the traffic sign, determining, by the at least one processor, at least one direction of the image sensor based on a direction normal to a face of the traffic sign in the at least one image, a distance between the traffic sign and the image sensor, and motion of the vehicle; and

adjusting, by the at least one processor, the image sensor to the at least one direction based on the direction normal to the face of the traffic sign, the distance, and the motion of the vehicle, wherein adjusting the image sensor comprises adjusting at least one among yaw, pitch, and roll of the image sensor.

22. The non-transitory computer-readable storage medium of claim 21, further comprising instructions that cause the at least one processor to perform an operation of:

in response to adjusting the image sensor, capturing, by the image sensor, at least one or more images comprising the traffic sign.

23. The non-transitory computer-readable storage medium of claim 21, further comprising instructions that cause the at least one processor to perform an operation of adjusting the image sensor to one or more directions based on the motion of the vehicle.

24. The non-transitory computer-readable storage medium of claim 21, wherein determining the at least one direction enables the image sensor to track the traffic sign.

25. The non-transitory computer-readable storage medium of claim 21, further comprising instructions that cause the at least one processor to perforin an operation of determining the direction normal to the face of the traffic sign based on the at least one image and determining the distance between the traffic sign and the image sensor.

26. The non-transitory computer-readable storage medium of claim 25, wherein the distance is determined based on the at least one image or distance information from a distance sensor.

27. The non-transitory computer-readable storage medium of claim 25, wherein determining the direction normal to the face of the traffic sign comprises determining a direction of a road in the at least one image as the normal direction.

28. The non-transitory computer-readable storage medium of claim 21, wherein the motion of the vehicle is indicative of information comprising at least one of displacement, elevation, speed, direction, or acceleration of the vehicle.

29. The non-transitory computer-readable storage medium of claim 21, wherein detecting the traffic sign in the at least one image is based on at least one of a color, a shape, or a location of the traffic sign.

30. The non-transitory computer-readable storage medium of claim 21, wherein adjusting the image sensor comprises:

generating, by the at least one processor, at least one control signal configured to rotate the image sensor to the at least one direction; and

rotating, by at least one motor unit, the image sensor to the at least one direction in response to the at least one control signal to reduce motion blur associated with the motion of the vehicle relative to the traffic sign.