INFORMATION PROCESSING APPARATUS, CONTROL METHOD, AND STORAGE MEDIUM

• US 20170255838A1
• Filed: 09/27/2016
• Published: 09/07/2017
• Est. Priority Date: 03/04/2016
• Status: Abandoned Application

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: NORIEDA, Shin, YOSHIMOTO, Makoto, SAIDA, Yoshinori, KOSUGE, Hiroki
• Application Number:
  US15/276,953
• Publication Number:
  US 20170255838A1
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/68 Food, e.g. fruit or vegetables

Abstract

According to an embodiment, an information processing system includes a display device and an information processing apparatus. The information processing apparatus includes a product recognition unit, a display control unit, and an operation recognition unit. The product recognition unit recognizes product. The display control unit acquires product information regarding the product recognized by the product recognition unit. Further, the display control unit displays an operation screen including the acquired product information on a display device. The operation recognition unit recognizes an input operation with respect to the operation screen on the basis of the position or movement of an operation body in a captured image.

Claims

1. An information processing apparatus to be coupled with a display device, the apparatus comprising a hardware processor configured to:

extract a partial region of a captured image as an object recognition region;

recognize an object in the object recognition region;

acquire object information regarding the recognized object;

control the display device to display an operation screen and the acquired object information;

recognize an operation body in the captured image; and

recognize an input operation with respect to the operation screen on the basis of at least one of a position and a movement of the operation body.

2. The information processing apparatus of claim 1,

wherein the hardware processor further configured to recognize the object by at least one of:

extracting an image of the object from the captured image;

or

reading an identifier of the object from a symbol that is attached to the object.

3. The information processing apparatus of claim 1,

wherein the object recognition region is determined based on a position of a marker in the captured image.

4. The information processing apparatus of claim 1,

wherein the object recognition region is predetermined.

5. The information processing apparatus of claim 1,

wherein the hardware processor is further configured to control the display device to display the operation screen when the marker is included in the captured image, and

wherein the hardware processor is further configured not to control the display device to display the operation screen when the marker is not included in the captured image.

6. The information processing apparatus of claim 5,

wherein the hardware processor is further configured to control the display to display the operation screen at a position determined on the basis of a position of the marker that is included in the captured image.

7. The information processing apparatus of claim 3,

wherein the marker is an image that is displayed on a display unit of a device worn by the user.

8. The information processing apparatus of claim 1,

wherein the hardware processor is further configured to transmit the operation screen or the captured image on which the operation screen is superimposed through a communication line.

9. The information processing apparatus of claim 1 to be further coupled with a sensor that is worn by a user of the apparatus,

wherein the hardware processor is further configured to:

detect a position of the operation body in the captured image at a timing determined by a detection result of the sensor or detect a movement of the operation body in the captured image at a period including the timing; and

recognize an input operation on the basis of the detected position or movement.

10. A control method to be executed by a computer to be coupled with a display device, the method comprising:

extracting a partial region of a captured image as an object recognition region;

recognizing an object in the object recognition region;

acquiring object information regarding the recognized object;

controlling the display device to display an operation screen and the acquired object information;

recognizing an operation body in the captured image; and

recognizing an input operation with respect to the operation screen on the basis of at least one of a position and a movement of the operation body.

11. A non-transitory computer-readable storage medium storing a program that causes a computer, which is to be coupled with a display device, to:

extract a partial region of a captured image as an object recognition region;

recognize an object in the object recognition region;

acquire object information regarding the recognized object;

control the display device to display an operation screen and the acquired object information;

recognize an operation body in the captured image; and

recognize an input operation with respect to the operation screen on the basis of at least one of a position and a movement of the operation body.