VIEWING STATE DETECTION DEVICE, VIEWING STATE DETECTION SYSTEM AND VIEWING STATE DETECTION METHOD

• US 20180242898A1
• Filed: 08/08/2016
• Published: 08/30/2018
• Est. Priority Date: 08/17/2015
• Status: Abandoned Application

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: MATSUO, Masatoshi, NAKAMURA, Tsuyoshi, TEZUKA, Tadanori
• Application Number:
  US15/747,651
• Publication Number:
  US 20180242898A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 2503/12 Healthy persons not otherwise provided for, e.g. subjects of a marketing survey
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/16 Devices for psychotechnics using teaching or educational appliances G09B1/00 - G09B7/00; Testing reaction times ; Devices for evaluating the psychological state
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A viewing state detection device (6) is configured to include an image input unit (11) to which temporally consecutive captured images including an audience and information on the captured time of the captured images are input, an area detector (12) that detects a skin area of the audience from the captured images, a vital information extractor (13) that extracts vital information of the audience based on time-series data of the skin area, a viewing state determination unit (17) that determines the viewing state of the audience based on the vital information, a content information input unit (14) to which content information including at least temporal information of the content is input, and a content viewing state storage unit (19) that stores the viewing state in association with the temporal information of the content.

Claims

1. A viewing state detection device that detects a viewing state of an audience from images including the audience viewing a content, the device comprising:

an image input unit to which temporally consecutive captured images including the audience and information on the captured time of the captured images are input;

an area detector that detects a skin area of the audience from the captured images;

a vital information extractor that extracts vital information of the audience based on time-series data of the skin area;

a viewing state determination unit that determines the viewing state of the audience based on the extracted vital information;

a content information input unit to which content information including at least temporal information of the content is input; and

a viewing state storage unit that stores the viewing state in association with the temporal information of the content.

2. The viewing state detection device of claim 1,

wherein the viewing state includes at least one of the degree of concentration and drowsiness of the audience.

3. The viewing state detection device according to claim 1, further comprising:

an information output unit that outputs information on the viewing state stored in the viewing state storage unit to an external display device.

4. The viewing state detection device of claim 3,

wherein the information output unit outputs information on the viewing state as a viewing state of each audience when there are a plurality of audiences.

5. The viewing state detection device of claim 3,

wherein the information output unit outputs information on the viewing state as information on viewing state of all or a part of the plurality of people in a case where there are the plurality of audiences.

6. A viewing state detection system comprising:

the viewing state detection device according to claim 1;

an imaging device that inputs captured images to the viewing state detection device; and

a content information input device that inputs content information including at least temporal information of a content to the viewing state detection device.

7. The viewing state detection system of claim 6, further comprising:

a display device that displays information on viewing state output from the viewing state detection device.

8. A viewing state detection method for detecting a viewing state of an audience from images including the audience viewing a content, the method comprising:

an image input step of inputting temporally consecutive captured images including the audience and information on the captured time of the captured images;

an area detection step of detecting a skin area of the audience from the captured images;

a vital information extraction step of extracting vital information of the audience based on time-series data of the skin area;

a viewing state determination step of determining the viewing state of the audience based on the extracted vital information;

a content information input step of inputting content information including at least temporal information of the content; and

a viewing state storage step of storing information on the viewing state in association with temporal information of the content.