System to provide a guide display based on a predicted action of a subject

• US 9,465,984 B2
• Filed: 12/11/2013
• Issued: 10/11/2016
• Est. Priority Date: 12/21/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Watanabe, Kazuhiro, Sako, Yoichiro, Tange, Akira, Sakoda, Kazuyuki, Asada, Kohei, Nakamura, Takatoshi, Takehara, Mitsuru, Kamada, Yasunori, Kon, Takayasu, Hayashi, Kazunori, Hanaya, Hiroyuki, Koga, Yuki, Onuma, Tomoya
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): PRINGLE-PARKER, JASON A
• Application Number:
  US14/103,032
• Publication Number:
  US 20140176599A1
• Time in Patent Office: 1,035 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

There is provided a display control system including a plurality of display units, an imaging unit configured to capture a subject, a predictor configured to predict an action of the subject according to a captured image captured by the imaging unit, a guide image generator configured to generate a guide image that guides the subject according to a prediction result from the predictor, and a display controller configured to, on the basis of the prediction result from the predictor, select a display unit capable of displaying an image at a position corresponding to the subject from the plurality of display units, and to control the selected display unit to display the guide image at the position corresponding to the subject.

Claims

1. A display control system comprising:

a plurality of display units;

an imaging unit configured to capture an image of a subject;

an action history information acquirer configured to acquire action history information of the subject;

a predictor configured to predict an action of the subject based on the action history information acquired by the action history information acquirer;

a guide image generator configured to generate a guide image that guides the subject to a first goal according to a prediction result from the predictor; and

a display controller configured to, based on the prediction result from the predictor, select a display unit capable of displaying an image at a position corresponding to the subject from the plurality of display units, and to control the selected display unit to display the guide image at the position corresponding to the subject,

in which the action history information acquirer is configured to recognize a reaction of the subject to the guide image displayed by the selected display unit based on a captured image from the imaging unit, and to acquire a recognized reaction as the action history information,

in which the reaction of the subject is one of walking, stopping, a hand gesture, or a path selection and the action history information of the subject includes at least one of the walking, the stopping, the hand gesture, or the path selection, and

in which the predictor is configured to provide an updated prediction result of the subject based on the recognized reaction acquired as the action history information and the guide image generator is configured to generate an updated guide image according to the updated prediction result to guide the subject to an updated goal which is different from the first goal so as to provide an interactive guide arrangement which enables a goal to be updated based on the recognized reaction.

2. The display control system according to claim 1, wherein

the guide image generator is configured to generate the guide image according a physical characteristic of the subject.

3. The display control system according to claim 2, wherein

the physical characteristic of the subject is a height of the subject, and

the guide image generator is configured to generate a large guide image in proportion to the height of the subject.

4. The display control system according to claim 1, wherein

the display controller is configured to control the selected display unit to display the guide image at a position according to a physical characteristic of the subject.

5. The display control system according to claim 4, wherein

the physical characteristic of the subject is a height of the subject, and

the display controller is configured to control the selected display unit to display the guide image at a position distanced in proportion to the height of the subject.

6. The display control system according to claim 1, wherein

the display controller is configured to control the selected display unit to display the guide image at a position distanced in proportion to a movement speed of the subject.

7. The display control system according to claim 1, wherein

the display controller is configured to select, from the plurality of display units, a display unit capable of displaying an image near a position of the subject.

8. The display control system according to claim 1, wherein

the display controller is configured to select, from the plurality of display units, a display unit capable of displaying an image in a direction in which the subject faces.

9. The display control system according to claim 1, wherein

the display controller, in a case in which another person is present near the subject, is configured to control the selected display unit to display the guide image for the subject distinguishably from another guide image displayed for the other person.

10. The display control system according to claim 1, further comprising:

a learning unit configured to learn an action pattern of the subject from the action history information,

wherein the predictor is configured to predict the reaction of the subject based on the action pattern learned by the learning unit.

11. The display control system according to claim 10, wherein

the guide image includes a display suggesting an action to the subject according to a prediction reaction based on the action pattern from the predictor.

12. A non-transitory computer-readable recording medium having a program recorded thereon, the program causing a computer to execute:

capturing an image of a subject;

acquiring action history information of the subject;

predicting an action of the subject based on acquired action history information;

selecting, from a plurality of display units based on a prediction result, a display unit capable of displaying an image at a position corresponding to the subject;

generating a guide image that guides the subject to a first goal according to the prediction result; and

controlling the selected display unit to display the guide image at the position corresponding to the subject,

in which the acquiring of the action history information of the subject includes recognizing a reaction of the subject to a displayed guide image based on a captured image and acquiring a recognized reaction as the action history information,

in which the reaction of the subject is one of walking, stopping, a hand gesture, or a path selection and the action history information of the subject includes at least one of the walking, the stopping, the hand gesture, or the path selection, and

in which the predicting includes providing an updated prediction result of the subject based on the recognized reaction acquired as the action history information and the generating includes generating an updated guide image according to the updated prediction result to guide the subject to an updated goal which is different from the first goal so as to provide an interactive guide arrangement which enables a goal to be updated based on the recognized reaction.