Image processing apparatus, display state determination apparatus, and image processing method

• US 9,971,939 B2
• Filed: 09/30/2015
• Issued: 05/15/2018
• Est. Priority Date: 06/23/2015
• Status: Expired due to Fees

• Current Assignee: Toshiba Tec Corporation (Toshiba Corporation)
• Original Assignee: Toshiba Tec Corporation (Toshiba Corporation)
• Inventors: Sawada, Takayuki
• Primary Examiner(s): Perungavoor, Sathyanaraya V
• Assistant Examiner(s): Young, Patricia I
• Application Number:
  US14/870,704
• Publication Number:
  US 20160379052A1
• Time in Patent Office: 958 Days
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

In accordance with one embodiment, an image processing apparatus which is accessible to at least one storage device, the apparatus includes a decision unit, a generation unit and a determination unit. The decision unit estimates a separation distance from the first camera to the display shelf. The decision unit decides a search distance, based on the separation distance. The generation unit generates a template image by converting a number of pixels of single item image acquired by photographing the product as a single piece and stored in the storage device, with a magnification corresponding to a ratio between the search distance and a known photographing distance. The determination unit determines an area, which is similar to the template image, within a shelf image stored in the storage device.

Claims

1. An image processing apparatus which is accessible to at least one storage device storing a first shelf image acquired by photographing, with a first camera, a display shelf for displaying a product, a second shelf image acquired by photographing the display shelf with a second camera from a direction that is different from that of the first camera, and an single item image acquired by photographing the product as a single piece with a known photographing distance, the apparatus comprising a processor configured to:

receive the first shelf image, the second shelf image, and the single item image,

estimate a separation distance from the first camera to the display shelf, based on the first shelf image and the second shelf image,

determine a search distance, based on the estimated separation distance,

generate a template image by converting a number of pixels of the single item image stored in the storage device, with a magnification corresponding to a ratio between the determined search distance and the known photographing distance,

identify the product and an area of the product within a target area of the first shelf image based on similarity degrees between pixels of the first shelf image and the pixels of the template image.

2. The image processing apparatus according to claim 1, wherein the processor is further configured to:

estimate, with respect to each of pixels constituting the first shelf image, a distance from the first camera to an object which the pixel represents, and

set, when each of differences between distances of neighboring pixels of a plurality of pixels which are consecutive in a horizontal direction over a prescribed number of pixels or more is less than a tolerable difference value, a representative value of the distances of the plurality of pixels to be the separation distance.

3. The image processing apparatus according to claim 2, wherein the processor is further configured to:

generate an equidistance image which indicates a pixel value in the first shelf image, with respect to a pixel of the shelf image, which has a distance that is equal to the search distance, and indicates a preset pixel value with respect to other pixels of the shelf image, and

set an area in the equidistance image, which is similar to the template image as the target area.

4. The image processing apparatus according to claim 1, wherein the processor determines the separation distance that is estimated based on the first shelf image and the second shelf image as the search distance.

5. The image processing apparatus according to claim 1, wherein the processor is further configured to:

set an adjustment value in accordance with an instruction by a user,

wherein the processor estimates the separation distance from the first camera to the display shelf, and determines the search distance by adjusting the separation distance by the set adjustment value.

6. A display state determination apparatus comprising:

a first camera;

a second camera;

at least one storage device storing a first shelf image acquired by photographing, with the first camera, a display shelf for displaying a product, a second shelf image acquired by photographing the display shelf with a second camera from a direction that is different from that of the first camera, and an single item image acquired by photographing the product as a single piece with a known photographing distance; and

a processor configured to;

estimate a separation distance from the first camera to the display shelf, based on the first shelf image and the second shelf image,

determine a search distance, based on the estimated separation distance,

generate a template image by converting a number of pixels of the single item image stored in the storage device, with a magnification corresponding to a ratio between the determined search distance and the known photographing distance, and

identify the product and an area of the product within a target area of the first shelf image based on similarity degrees between pixels of the first shelf image and the pixels of the template image.

7. An image processing method by a computer included in an image processing apparatus which is accessible to at least one storage device storing a first shelf image acquired by photographing, with a first camera, a second shelf image acquired by photographing the display shelf with a second camera from a direction that is different from that of the first camera, a display shelf for displaying a product, and an single item image acquired by photographing the product as a single piece with a known photographing distance, the method comprising:

estimating a separation distance from the first camera to the display shelf, based on the first shelf image and the second shelf image, and determining a search distance, based on the estimated separation distance;

generating a template image by converting a number of pixels of the single item image stored in the storage device, with a magnification corresponding to a ratio between the determined search distance determined by the decision unit and the known photographing distance; and

identifying the product and an area of the product within the first shelf image based on similarity degrees between pixels of the first shelf image and the pixels of the template image.