Task assistance system, task assistance method, and program

• US 9,690,983 B2
• Filed: 02/28/2013
• Issued: 06/27/2017
• Est. Priority Date: 05/17/2012
• Status: Active Grant

• Current Assignee: NEC Corporation, NEC Solution Innovators, Ltd (NEC Corporation)
• Original Assignee: NEC Corporation, Nec Solution Innovators
• Inventors: Harada, Hiroo, Takasugi, Teppei, Nagai, Katsuyuki, Kaneko, Kenichi
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/401,663
• Publication Number:
  US 20150139489A1
• Time in Patent Office: 1,580 Days
• Field of Search: 382100, 382103, 382106-107, 382141, 382162, 382168, 382173, 382181, 382190, 382199, 382203, 382209, 382219, 382224, 382232, 382254, 382274, 382276, 382287-291, 382305, 382312, 382318, 3405681, 705 2642, 701 23, 345633
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A task assistance system includes: a location detection unit (101) for setting a sensing plane in a first face side of a display means capable of displaying a plurality of articles and calculating location information of a hand located on the sensing plane or location information of an object different from the hand; a region setting unit (102) for setting a plurality of sensing regions on the sensing plane using the location information of the hand or the location information of the object calculated by the location detection unit (101); and a pass-through region determination unit (104) for determining which one of the plurality of sensing regions a hand extended to the display means from the first face side has passed through using the location information of the hand calculated by the location detection unit (101), after the region setting unit (102) sets the plurality of sensing regions.

Claims

1. A task assistance system comprising:

a memory that stores a set of instructions and;

a hardware processor configured to execute the set of instructions to;

receive, from a user, an input of information for identifying a setting start location in a voice form;

determine a sensing plane in a first face side of a display capable of displaying a plurality of articles;

determine, for identifying the setting start location, first location information of a first hand located on the sensing plane or second location information of an object different from the first hand;

generate a plurality of sensing regions on the sensing plane based on the first location information of the first hand or the second location information of the object; and

determine which one of the plurality of sensing regions a second hand extended to the display from the first face side has passed through based on third location information of the second hand, after determining the plurality of sensing regions.

2. The task assistance system according to claim 1, wherein the hardware processor is further configured to execute the set of instructions to:

determine a moving path of the first hand or the object on the sensing plane based on the first location information of the first hand or the second location information of the object; and

a generate the plurality of sensing regions on the sensing plane based on the identified moving path of the first hand or the object on the sensing plane.

3. The task assistance system according to claim 2, wherein the hardware processor is further configured to execute the set of instructions to:

determine the setting start location and a setting end location from the identified moving path of the first hand or the object on the sensing plane; and

determine a region surrounded by the moving path of the first hand or the object on the sensing plane from the setting start location to the setting end location as the sensing region.

4. The task assistance system according to claim 3, wherein the hardware processor is further configured to execute the set of instructions to:

determine, as the setting start location when the input of information is received, a location of the first hand or the object on the sensing plane.

5. The task assistance system according to claim 3, wherein the hardware processor is further configured to execute the set of instructions to:

determine, when the input information is received, a location of the first hand or the object on the sensing plane as the setting end location.

6. The task assistance system according to claim 3, wherein the hardware processor is further configured to execute the set of instructions to:

determine a circle having the setting start location as a center on the sensing plane; and

determine, as the setting end location, a location of the moving path of the first hand or the object on the sensing plane immediately after the moving path of the first hand or the object passing through the setting start location returns inside the circle after moving outside the circle.

7. The task assistance system according to claim 3, wherein the hardware processor is further configured to execute the set of instructions to:

determine, as the sensing region, a region surrounded by a line connecting the setting start location and the setting end location based on a straight line or based on a curved line when locations of the setting start location and the setting end location on the sensing plane differ from each other, and further based on the moving path of the first hand or the object on the sensing plane from the setting start location to the setting end location.

8. The task assistance system according to claim 3, wherein the hardware processor is further configured to execute the set of instructions to:

determine an intersection of the moving path of the first hand or the object on the sensing plane as the setting start location and the setting end location.

9. The task assistance system according to claim 2, wherein the hardware processor is further configured to execute the set of instructions to:

determine a conversion moving path where the moving path of the first hand or the object on the sensing plane is partially converted into either a movement to a height direction of the display unit or a movement to a width direction thereof; and

generate the plurality of sensing regions on the sensing plane based on the conversion moving path instead of the moving path of the first hand or on the object on the sensing plane.

10. The task assistance system according to claim 9, wherein the hardware processor is further configured to execute the set of instructions to:

receive, from a user, an input of information for identifying a timing of switching from a movement to a height direction of the display to a movement to a width direction of the display and a timing of switching from the movement to the width direction to the movement to the height direction; and

perform switching from the movement to the height direction to the movement to the width direction or from the movement to the width direction to the movement to the height direction at a location when receiving the input of information for identifying the timing of switching in the moving path of the first hand or the object on the sensing plane, after determining the conversion moving path from the moving path of the first hand or the object on the sensing plane.

11. The task assistance system according to claim 10, wherein the hardware processor is further configured to execute the set of instructions to:

receive the input of information identifying the timing of switching by using a voice.

12. The task assistance system according to claim 9, wherein the hardware processor is further configured to execute the set of instructions to:

determine a plurality of grid lines extending in the height direction and the width direction of the display on the sensing plane after determining the conversion moving path from the moving path of the first hand or the object on the sensing plane; and

determine the conversion moving path based on the moving path of the first hand or the object on the sensing plane into a line along the grid lines.

13. The task assistance system according to claim 1, wherein the hardware processor is further configured to execute the set of instructions to:

determine a pass-through location on the sensing plane of the first hand or the object passing through the sensing plane within a predetermined period of time; and

determine a region including the identified pass-through location as the sensing region.

14. The task assistance system according to claim 1, wherein the hardware processor is further configured to execute the set of instructions to:

determine a pass-through location based on the sensing plane of the first hand or the object passing through the sensing plane within a predetermined period of time, and a moving path of the first hand or the object moving in the sensing plane; and

determine a region including the identified pass-through location and the identified moving path as the sensing region.

15. The task assistance system according to claim 1, wherein the hardware processor is further configured to execute the set of instructions to:

adjust the sensing regions so as not to overlap with each other when a sensing region overlapping with another sensing region among a plurality of the set sensing regions.

16. A task assistance method performed by a hardware processor of a computer, the task assistance method including:

receiving, from a user, an input of information for identifying a setting start location in a voice form;

determining a sensing plane in a first face side of a display unit capable of displaying a plurality of articles;

determining, for identifying the setting start location, first location information of a first hand located on the sensing plane or second location information of an object different from the first hand;

generating a plurality of sensing regions on the sensing plane based on the first location information of the first hand or the second location information of the object; and

determining which one of the plurality of sensing regions a second hand extended to the display unit from the first face side has passed through based on third location information of the second hand, after setting the plurality of sensing regions.

17. A non-transitory computer readable medium having stored therein instructions that, when executed by a computer, causes the computer to perform a task assistance method, the method comprising:

receive, from a user, an input of information for identifying a setting start location in a voice form;

determining a sensing plane in a first face side of a display unit capable of displaying a plurality of articles;

determining, for identifying the setting start location, first location information of a first hand located on the sensing plane or second location information of an object different from the first hand;

generating a plurality of sensing regions on the sensing plane based on the first location information of the first hand or the second location information of the object; and

determining which one of the plurality of sensing regions a second hand extended to the display unit from the first face side has passed through based on third location information of the second hand, after determining the plurality of sensing regions.

18. A task assistance system comprising:

storage means for storing a set of instructions and;

processing means for executing the set of instructions for;

receiving, from a user, an input of information for identifying a setting start location in a voice form;

determining a sensing plane in a first face side of a display capable of displaying a plurality of articles;

determining, for identifying the setting start location, first location information of a first hand located on the sensing plane or second location information of an object different from the first hand;

generating a plurality of sensing regions on the sensing plane based on the first location information of the first hand or the second location information of the object; and

determining which one of the plurality of sensing regions a second hand extended to the display from the first face side has passed through based on third location information of the second hand, after determining the plurality of sensing regions.