GRAPHIC RECOGNITION DEVICE, GRAPHIC RECOGNITION METHOD, AND GRAPHIC RECOGNITION PROGRAM

• US 20100254595A1
• Filed: 12/26/2006
• Published: 10/07/2010
• Est. Priority Date: 01/17/2006
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Miyamoto, Shinichi
• Granted Patent: US 8,229,162 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A graphic recognition device, method, and recognition program recognize graphics without being influenced by an image shadow area. Image input unit acquires the image of the outside environment of a vehicle using a vehicle mounted camera. A light source location information acquiring unit calculates location of a light source such as the sun using the acquired image. User vehicle shape acquiring unit and other vehicle shape acquiring unit generate shape information for the vehicles indicating the location of points forming vehicle contours. Shadow area calculating unit calculates, on the basis of both vehicles'"'"' shape information, the object shape information and the light source location information, the location coordinates of the shadow area, and converts the location coordinates into two-dimensional coordinates to the shadow/non-shadow area emphasis flag recognizing unit, which recognizes the flag in the image by judging the presence/absence of the recognition object in each shadow and non-shadow area specified.

Claims

1-35. -35. (canceled)

36. A graphic recognition device for recognizing a graphic in an image photographed by a photographic device, the graphic recognition device comprising:

a shadow area calculating unit for calculating an area of a shadow of an object in the image;

an area graphic recognizing unit for determining whether or not a predefined specific graphic exists in the image individually for an inside and an outside of the area of the shadow calculated by the shadow area calculating unit; and

a graphic recognizing unit for determining whether or not the specific graphic exists in the image based on a result determined by the area graphic recognizing unit.

37. The graphic recognition device according to claim 36, wherein the area graphic recognizing unit determines whether or not the specific graphic exists after emphasizing contrast on the area of the shadow in the image.

38. The graphic recognition device according to claim 36, wherein the shadow area calculating unit calculates the area of the shadow based on light source location information indicating a location of a light source, and object location information indicating an actual location of a plurality of points forming a contour of an object forming the shadow.

39. The graphic recognition device according to claim 38, wherein the shadow area calculating unit calculates a point at the intersection of a line passing through one point indicated by the light source location information and one point indicated by the object location information with the ground as a point in the area of the shadow.

40. The graphic recognition device according to claim 36, wherein the area graphic recognizing unit determines whether or not the specific graphic exists individually for the inside and the outside of the area of the shadow by template matching.

41. The graphic recognition device according to claim 36, wherein the area graphic recognizing unit determines whether or not the specific graphic exists individually for the inside and the outside of the area of the shadow based on a statistical learning using neural network.

42. The graphic recognition device according to claim 36, wherein the area graphic recognizing unit judges presence/absence of a periodicity of a specific pattern for the inside and the outside of the area of the shadow if the specific graphic has periodicity.

43. The graphic recognition device according to claim 36, wherein

the area graphic recognizing unit outputs the determination result for the inside and the outside of the area of the shadow in numerical values;

and

the graphic recognizing unit determines whether or not the specific graphic exists in the image based on a magnitude relationship between the determination result for each area output by the area graphic recognizing unit and a predefined threshold value.

44. The graphic recognition device according to claim 43, wherein the graphic recognizing unit determines whether or not the specific graphic exists in the image based on a magnitude relationship between a weighted linear sum of the determination result for each area output by the area graphic recognizing unit and a predefined threshold value.

45. The graphic recognition device according to claim 38, further comprising a light source location position acquiring unit for calculating the location of the light source from the location of the photographic device and a photographing direction, and outputting as the light source location information.

46. The graphic recognition device according to claim 38, wherein the light source location information indicates an actual location of a celestial object.

47. The graphic recognition device according to claim 38, wherein the light source location information indicates an actual location of an artificial light source.

48. The graphic recognition device according to claim 38, wherein the shadow area calculating unit calculates the area of the shadow estimated based on the light source location information, compares the same with a corresponding portion in the image, and specifies the area of the shadow based on the comparison result.

49. The graphic recognition device according to claim 48, wherein the light source location information indicates a location of a virtual light source.

50. The graphic recognition device according to claim 38, further comprising an own vehicle shape acquiring unit for calculating information indicating an actual location of each point forming a contour of an own vehicle based on coordinate data indicating a contour shape of the own vehicle mounted with the photographic device and location information of the photographic device, and outputting as the object location information.

51. The graphic recognition device according to claim 38, further comprising an other vehicle shape acquiring unit for calculating information indicating an actual location of each point forming a contour of other vehicle based on coordinate data indicating a contour shape of the other vehicle near the own vehicle mounted with the photographic device and the location information of the photographic device, and outputting as the object location information.

52. The graphic recognition device according to claim 38, further comprising a map database for storing information indicating the contour shape and the location of a structure as the object location information.

53. A graphic recognition means for recognizing a graphic in an image photographed by a photographic device, the graphic recognition means comprising:

a shadow area calculating means for calculating an area of a shadow of an object in the image;

an area graphic recognizing means for determining whether or not a predefined specific graphic exists in the image individually for an inside and an outside of the area of the shadow calculated by the shadow area calculating means; and

a graphic recognizing means for determining whether or not the specific graphic exists in the image based on a result determined by the area graphic recognizing means.

54. A graphic recognition method for recognizing a graphic in an image photographed by a photographic device, the method comprising:

calculating an area of a shadow of an object in the image;

determining whether or not a predefined specific graphic exists in the image individually for an inside and an outside of the area of the shadow calculated in calculating the area of the shadow of the object in the image; and

determining whether or not a specific graphic exists in the image based on a result determined in determining whether or not the predefined specific graphic exists in the image individually for the inside and the outside of the area of the shadow.

55. The graphic recognition method according to claim 54, wherein in determining whether or not the predefined specific graphic exists in the image, whether or not the specific graphic exists is determined after emphasizing a contrast on the area of the shadow in the image.

56. The graphic recognition method according to claim 54, wherein in calculating the area of the shadow of the object in the image, the area of the shadow is calculated based on light source location information indicating a location of a light source, and object location information indicating an actual location of a plurality of points forming a contour of the body forming the shadow.

57. The graphic recognition method according to claim 56, wherein in calculating the area of the shadow of the object in the image, a point at the intersection of a line passing through one point indicated by the light source location information and one point indicated by the object location information with the ground is calculated as a point in the area of the shadow.

58. The graphic recognition method according to claim 54, wherein in determining whether or not the predefined specific graphic exists in the image, whether or not the specific graphic exists is determined individually for the inside and the outside of the area of the shadow by template matching.

59. The graphic recognition method according to claim 54, wherein in determining whether or not the predefined specific graphic exists in the image, whether or not the specific graphic exists is determined individually for the inside and the outside of the area of the shadow based on a statistical learning using neural network.

60. The graphic recognition method according to claim 54, wherein in determining whether or not the predefined specific graphic exists in the image, presence/absence of a periodicity of a specific pattern is judged for the inside and the outside of the area of the shadow if the specific graphic has periodicity.

61. The graphic recognition method according to claim 54, wherein

in determining whether or not the predefined specific graphic exists in the image, the determination result for the inside and the outside of the area of the shadow is output in numerical values;

and

in determining whether or not a specific graphic exists in the image, whether or not the specific graphic exists in the image is determined based on a magnitude relationship between the determination result for each area output in determining whether or not the predefined specific graphic exists in the image and a predefined threshold value.

62. The graphic recognition method according to claim 61, wherein in determining whether or not a specific graphic exists in the image, whether or not the specific graphic exists in the image is determined based on a magnitude relationship between a weighted linear sum of the determination result for each area output in determining whether or not the predefined specific graphic exists in the image and a predefined threshold value.

63. The graphic recognition method according to claim 56, further comprising calculating the location of the light source from the location of the photographic device and a photographing direction, and having as the light source location information.

64. The graphic recognition method according to claim 56, wherein the light source location information indicates an actual location of a celestial object.

65. The graphic recognition method according to claim 56, wherein the light source location information indicates an actual location of an artificial light source.

66. The graphic recognition method according to claim 56, wherein in calculating an area of a shadow of an object in the image, the area of the shadow estimated based on the light source location information is calculated and compared with a corresponding portion in the image to specify the area of the shadow.

67. The graphic recognition method according to claim 66, wherein the light source location information indicates a location of a virtual light source.

68. The graphic recognition method according to claim 56, further comprising calculating information indicating an actual location of each point forming a contour of an own vehicle based on coordinate data indicating a contour shape of the own vehicle mounted with the photographic device and location information of the photographic device, and having the same as the object location information.

69. The graphic recognition method according to claim 56, further comprising calculating information indicating an actual location of each point forming a contour of other vehicle based on coordinate data indicating a contour shape of the other vehicle near the own vehicle mounted with the photographic device and the location information of the photographic device, and having the same as the object location information.

70. The graphic recognition method according to claim 56, wherein information indicating the contour shape and the location information of a structure are the object location information.

71. A computer readable medium storing a graphic recognition program for recognizing a graphic in an image photographed by a photographic device, the program causing the computer to execute:

shadow area calculating process of calculating an area of a shadow of an object in the image;

area graphic recognition process of determining whether or not a predefined specific graphic exists in the image individually for an inside and an outside of the area of the shadow calculated in the shadow area calculating process; and

graphic recognition process of determining whether or not a specific graphic exists in the image based on a result determined in the area graphic recognition process.