MERCHANDISE SPECIFICATION SYSTEM

• US 20200134688A1
• Filed: 06/08/2018
• Published: 04/30/2020
• Est. Priority Date: 06/27/2017
• Status: Active Grant

• Current Assignee: Marketvision Company Limited
• Original Assignee: Marketvision Company Limited
• Inventors: TANII, Akiyoshi
• Granted Patent: US 11,640,709 B2
• Time in Patent Office: Days
• CPC Class Codes: A47F 5/00 Show stands, hangers, or shelves characterised by their constructional features
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A merchandise specification system has: a keystone correction processing unit that performs a keystone correction process on photographed image information obtained by photographing the display shelf to generate front-facing image information; and a merchandise identification information specification processing unit that specifies merchandise on a face in the front-facing image information.

Claims

1. A merchandise specification system for specifying merchandise displayed on a display shelf, comprising:

a keystone correction processing unit that performs a keystone correction process on photographed image information obtained by photographing the display shelf to generate front-facing image information; and

a merchandise identification information specification processing unit that specifies merchandise on a face in the front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information, and

the merchandise identification information specification processing unit

compares image information of a face in second front-facing image information with image information of a face within a predetermined range of first front-facing image information corresponding to the face to specify merchandise identification information of merchandise on the face in the second front-facing image information.

2. The merchandise specification system according to claim 1, wherein

the keystone correction processing unit

projects the vertexes used in the keystone correction process on the first photographed image information onto the second photographed image information using similarity between an image feature amount in the first photographed image information and an image feature amount in the second photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate the front-facing image information.

3. The merchandise specification system according to claim 2, wherein

the keystone correction processing unit

compares an image feature amount in a feature amount collection region that is set in the first photographed image information and includes the vertexes used in the keystone correction process on the first photographed image information with an image feature amount in a feature amount collection region that is set in the second photographed image information and is larger than the feature amount collection region in the first photographed image information to specify a pair of image feature amounts with a high similarity,

specifies a function of projection from the first photographed image information onto the second photographed image information based on coordinates of the specified pair of image feature amounts,

projects the vertexes used in the keystone correction process on the first photographed image information onto the second photographed image information using the specified function, and

executes the keystone correction process on the second photographed image information using the projected vertexes.

4. The merchandise specification system according to claim 3, further comprising

a sample image information storage unit that stores sample image information of merchandise, wherein

when, as a result of the comparison between the image information of the face in the second front-facing image information and the image information of the face within the predetermined range in the first front-facing image information corresponding to the face, the similarity does not satisfy a predetermined condition, the merchandise identification information specification processing unit further compares the image information of the face in the second front-facing image information with the sample image information stored in the sample image information storage unit to specify the merchandise identification information of the merchandise on the face in the second front-facing image information.

5. The merchandise specification system according to claim 4, further comprising

a face specification processing unit that specifies the face in the front-facing image information, wherein

the face specification processing unit

specifies a position of the face in the second front-facing image information based on a position of the face in the first front-facing image information.

6. The merchandise specification system according to claim 5, further comprising

a shelf tier position specification processing unit that specifies a position of a shelf tier in the front-facing image information, wherein

the shelf tier position specification processing unit

specifies a position of the shelf tier in the second front-facing image information based on a position of the shelf tier in the first front-facing image information.

7. The merchandise specification system according to claim 6, wherein

when the merchandise specification system is further configured to compare image information of a shelf tier in the second front-facing image information with image information of a shelf tier in the first front-facing image information, and similarity satisfies a predetermined condition, the merchandise identification information specification processing unit specifies merchandise identification information of a face on the shelf tier in the second front-facing image information as merchandise identification information of a face on the shelf tier in the first front-facing image information, without comparing the image information of the face in the second front-facing image information with the image information of the face in the predetermined range of the first front-facing image information corresponding to the face.

8. A merchandise specification system for specifying merchandise on a display shelf, comprising

a keystone correction processing unit that performs a keystone correction process on image information obtained by photographing the display shelf to generate front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information.

9. A merchandise specification system for specifying merchandise on a display shelf, comprising

a merchandise identification information specification processing unit that specifies merchandise on a face in image information obtained by photographing the display shelf, wherein

the merchandise identification information specification processing unit

compares image information of a face in second front-facing image information with image information of a face within a predetermined range of first front-facing image information corresponding to the face to specify merchandise identification information of merchandise on the face in the second front-facing image information.

10. An image processing system that specifies vertexes for performing a keystone correction process on photographed image information, comprising

a keystone correction processing unit that performs the keystone correction process on photographed image information obtained by photographing a target object to generate front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information.

11. The image processing system according to claim 10, wherein

the keystone correction processing unit

projects the vertexes used in the keystone correction process on the first photographed image information onto the second photographed image information using similarity between an image feature amount in the first photographed image information and an image feature amount in the second photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate the front-facing image information.

12. The image processing system according to claim 11, wherein

the keystone correction processing unit

compares an image feature amount in a feature amount collection region that is set in the first photographed image information and includes the vertexes used in the keystone correction process on the first photographed image information with an image feature amount in a feature amount collection region that is set in the second photographed image information and is larger than the feature amount collection region in the first photographed image information to specify a pair of image feature amounts with a high similarity,

specifies a function of projection from the first photographed image information onto the second photographed image information based on coordinates of the specified pair of image feature amounts,

projects the vertexes used in the keystone correction process on the first photographed image information onto the second photographed image information using the specified function, and

executes the keystone correction process on the second photographed image information using the projected vertexes.

13. A merchandise specification program for causing a computer to serve as

a keystone correction processing unit that performs a keystone correction process on photographed image information obtained by photographing a display shelf to generate front-facing image information, and

a merchandise identification information specification processing unit that specifies merchandise on a face in the front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information obtained by photographing the same display shelf as in the first photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information, and

the merchandise identification information specification processing unit

compares the image information of a face in second front-facing image information with the image information of a face within a predetermined range of first front-facing image information corresponding to the face to specify merchandise identification information of merchandise on the face in the second front-facing image information.

14. A merchandise specification program for causing a computer to serve as

a keystone correction processing unit that performs a keystone correction process on image information obtained by photographing a display shelf to generate front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information obtained by photographing the same display shelf as in first photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information.

15. A merchandise specification program for causing a computer to serve as

a merchandise identification information specification processing unit that specifies merchandise on a face in image information obtained by photographing a display shelf, wherein

the merchandise identification information specification processing unit

compares image information of a face in second front-facing image information with image information of a face within a predetermined range of first front-facing image information corresponding to the face to specify merchandise identification information of merchandise on the face in the second front-facing image information.

16. An image processing program for causing a computer to serve as

a keystone correction processing unit that performs a keystone correction process on photographed image information obtained by photographing a target object to generate front-facing image information, wherein

the keystone correction processing unit

projects vertexes used in the keystone correction process on first photographed image information onto second photographed image information, based on correspondence between the first photographed image information and the second photographed image information obtained by photographing the same target object as in the first photographed image information, and

executes the keystone correction process on the second photographed image information based on the projected vertexes to generate front-facing image information.