System, method and program for managing and displaying product information

• US 9,395,195 B2
• Filed: 03/27/2014
• Issued: 07/19/2016
• Est. Priority Date: 03/30/2010
• Status: Active Grant

• Current Assignee: NS Solutions Corporation (Nippon Steel Corporation)
• Original Assignee: NS Solutions Corporation (Nippon Steel Corporation)
• Inventors: Ihara, Noboru, Sasao, Kazuhiro, Yokoyama, Masaru, Sakurai, Arata
• Primary Examiner(s): Beck, Leron
• Application Number:
  US14/228,168
• Publication Number:
  US 20140210859A1
• Time in Patent Office: 845 Days
• Field of Search: 348/116
• US Class Current: 1/1
• CPC Class Codes: B63B 2213/02 using satellite radio beacon positioning systems, e.g. the Global Positioning System GPS
• CPC Class Codes: B63B 43/18 preventing collision or grounding; reducing collision damage reducing pollution by collision B63B25/082
• CPC Class Codes: B63B 49/00 Arrangements of nautical instruments or navigational aids nautical measuring instruments G01C; radio navigation, analogous arrangements using other waves G01S
• CPC Class Codes: G01C 13/008 measuring depth of open water
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/203 Specially adapted for sailing ships
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3407 specially adapted for specific applications
• CPC Class Codes: G01C 21/3685 the POI's being parking facilities
• CPC Class Codes: G01C 21/3889 Transmission of selected map data, e.g. depending on route
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/003 : Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/143 : inside the vehicles
• CPC Class Codes: G08G 3/00 : Traffic control systems for marine craft marking of navigational route B63B22/16, B63B51/00
• CPC Class Codes: G09B 29/007 : using computer methods
• CPC Class Codes: Y02A 90/30 : Assessment of water resources

Abstract

A mobile terminal (200) obtains image data related to a product posted on a flier (300) by imaging and sends the image data related to the product to a shopping management server (100). Upon reception of the image data related to the product from the mobile terminal (200), the shopping management server (100) determines the product by analyzing the image data related to the product, obtains at least one of inventory information and sales information of the product, and sends at least one of the inventory information and the sales information of the product to the mobile terminal (200). The mobile terminal (200) combines computer-graphics image data indicating at least one of the inventory information and the sales information of the product sent from the shopping management server (100) with an image of the flier in real space and presents the combined image, thereby solving the problem.

Claims

1. An information processing system in which an augmented reality presenting device having a transmissive or non-transmissive display and a managing device managing product information related to respective products posted on a flier are connected via a network,

wherein the augmented reality presenting device comprises:

an imaging unit obtaining image data of the flier captured in an angle of view in response to a unit detecting a user of the augmented reality presenting device identifying a product posted on the flier based at least on captured image data initiated by one or more finger shape patterns indicating a direction and distance relationship between a user finger and the product;

a first obtaining unit obtaining image data related to the product; and

a first transmitting unit transmitting the image data related to the product to the managing device,

wherein the managing device comprises;

a flier information storage unit storing image data related to the respective products posted on the flier;

a determining unit receiving the image data related to the product from the first transmitting unit of the augmented reality presenting device, determining the product specified by the user by comparing the image data related to the product with the image data related to the respective products and stored in the flier information storage unit;

a second obtaining unit obtaining the product information related to the product correlated with the user; and

a second transmitting unit transmitting the product information obtained by the second obtaining unit to the augmented reality presenting device, and

wherein the augmented reality presenting device further comprises a presenting unit displaying, on the display, computer-graphics image data indicating the product information transmitted from the second transmitting unit of the managing device by combining with an image in real space.

2. The information processing system according to claim 1,

wherein, upon detecting that the products posted on the flier are associated with markers, the imaging unit of the augmented reality presenting device is configured to obtain image data of a marker associated wit the product, and

wherein the managing device is further configured to determined and store a correlation between the markers and the products posted on the flier, and the determining unit of the managing device is configured to identify the product specified by the user by analyzing the image data of the marker associated with the product received from the augmented reality presenting device.

3. The information processing system according to claim 1, wherein the second obtaining unit of the managing device is configured to obtains updated information related to the product.

4. The information processing system according to claim 1, wherein the second obtaining unit of the managing device is configured to obtains updated product information, and the second transmitting unit of the managing device transmits the product information to the augmented reality presenting device.

5. The information processing system according to claim 1, wherein the second obtaining unit of the managing device is configured to select a product according to personal information of the user and obtain the product information related to the product, and the second transmitting unit of the managing device is configured to transmit the product information related to the product to the augmented reality presenting device.

6. The information processing system according to claim 1, wherein the managing device is configured to obtain address information of a shop selling the product, obtain map information including a position indicated by current location information received from the augmented reality presenting device and a position indicated by the address information, using the map information to determine and generate a route from a current location of the augmented reality presenting device to the shop selling the product based on a predetermined rule.

7. The information processing system according to claim 6, wherein the predetermined rule is selected from a plurality of rules.

8. The information processing system according to claim 7, wherein the plural rules include a rule related to a distance from the current location of the augmented reality presenting device to a shop selling the product and a rule related to inventory of the product.

11. The information processing system according to claim 1, wherein the determining unit is further configured to:

correlate the user and the product in response to determining the image data related to the product matching the image data related to the respective products stored in the flier information storage unit, and

provide the product correlated with the user to the second obtaining unit.

12. The information processing system according to claim 11,

wherein the determining unit of the managing device is further configured to register a correlation of the user and the product in the flier information storage unit, and

when receiving the image data related to the product posted on the flier form the augmented reality presenting device, analyze header information of the image data to obtain user information of the user, analyze the product information and the user information based on the correlation, and obtain the product information related to the product correlated with the user.

13. The information processing system according to claim 12, wherein the second transmitting unit of the managing device is further configured to transmit to the augmented reality presenting device information for notifying that the product posted on the flier is a product registered in the flier information storage unit.

9. An information processing method executed by an information processing system in which an augmented reality presenting device having a transmissive or non-transmissive display and a managing device managing product information related to respective products posted on a flier are connected via a network,

wherein the augmented reality presenting device is configured to:

obtain image data of the flier captured in an angle of view in response to a detecting unit detecting a user of the augmented reality presenting device identifying a product posted on the flier based at least on captured image data initiated by one or more finger shape patterns indicating a direction and distance between relationship between a user finger and the product;

obtain image data related to the product; and

transmit the image data related to the product to the managing device, and

wherein the managing device is configured to;

receive the image data related to the product from the augmented reality resenting device, determined the product specified by the user by comparing the image data related to the product with image data related to the respective products posted on the flier and stored in flier information storage unit;

obtain the product information related to the product correlated with the user; and

transmit the product information to the augmented reality presenting device.

10. A non-transitory computer readable storage medium storing thereon a program causing a computer to execute an information processing method executed by an information processing system in which an augmented reality presenting device having a transmissive or non-transmissive display and a managing device managing product information related to respective products posted on a flier are connected via a network,

wherein a computer of the augmented reality presenting device is configured to:

obtain image data of the flier captured in an angle of view in response to a detecting unit detecting a user of the augmented reality presenting device identifying a product posted on the flier based at least on captured image data initiated by one or more finger shape patterns indicating a direction and distance relationship between a user finger and the product;

obtain image data related to the product; and

transmit the image data related to the product to the managing device, and

wherein a computer of the managing device is configured;

receive the image data related to the product from the augmented reality presenting device, determine the product specified by the user by comparing the image data related to the product with image data related to the respective products posted on the flier and stored in a flier information storage unit;

obtain the product information related to the product correlated with the user; and

transmit the product information to the augmented reality presenting device.