Video display device and manipulation detection method used therefor

• US 10,459,577 B2
• Filed: 10/07/2015
• Issued: 10/29/2019
• Est. Priority Date: 10/07/2015
• Status: Active Grant

• Current Assignee: Maxell, Ltd.
• Original Assignee: Maxell, Ltd.
• Inventors: Niikura, Takehiro, Matsubara, Takashi, Mori, Naoki
• Primary Examiner(s): Xie, Kwin
• Application Number:
  US15/766,617
• Publication Number:
  US 20190065004A1
• Time in Patent Office: 1,483 Days
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 25/531 by controlling rolling shutters in CMOS SSIS

Abstract

A video display device includes first and second illuminators irradiating a manipulation surface with illumination light for manipulation detection in different directions, an imaging unit employing an imaging element of a rolling shutter system that images the irradiated manipulation surface together with a manipulation object of a user, and a contact point detecting unit detecting a contact point at which the manipulation object of the user touches the manipulation surface based on a captured image of the imaging unit. Captured images of shadows of the manipulation object of the user formed by repeatedly lighting up the first illuminator and the second illuminator alternately are used in a contact point detection process of the contact point detecting unit, and in the alternate lighting of the first illuminator and the second illuminator, a lighting period of one illuminator is longer than an imaging cycle of one frame of the imaging unit.

Claims

1. A video display device that displays a video on a video display surface and is able to detect a manipulation performed on a manipulation surface superimposed on the video display surface by a manipulation object of a user, comprising:

first and second illuminators that irradiate the manipulation surface with illumination light for manipulation detection in different directions;

an imaging unit that employs an imaging element of a rolling shutter system that images the manipulation surface irradiated with the illumination light together with the manipulation object of the user; and

a contact point detecting unit that detects a contact point at which the manipulation object of the user touches the manipulation surface on the basis of a captured image of the imaging unit,

wherein captured images of shadows of the manipulation object of the user formed by repeatedly lighting up the first illuminator and the second illuminator alternately are used in a contact point detection process of the contact point detecting unit, and

in the alternate lighting of the first illuminator and the second illuminator, a lighting period of one illuminator is longer than an imaging cycle of one frame of the imaging unit,

wherein, in a case in which there is a non-manipulation region which is not manipulated by the manipulation object of the user in an imaging region by the imaging unit on the video display surface,

an imaging position at which lighting switching between the first illuminator and the second illuminator is completed during imaging by the imaging unit is arranged in the non-manipulation region,

wherein the first illuminator, the second illuminator, and the imaging unit are installed on an upper side of the video display surface, and

in a case in which the non-manipulation region by the user exists on a lower side of the video display surface,

the imaging element of the imaging unit is rotated, and an imaging start position is set on the lower side of the video display surface.

2. The video display device according to claim 1, wherein image capturing completion of each frame of the imaging unit is detected, and the lighting of the first illuminator and the second illuminator is switched each time imaging of two frames is completed, and

in the contact point detection process, an image of a frame unit irradiated with only the illumination light of one of the first illuminator and the second illuminator is used, but an image of a frame being imaged when the lighting of the illuminator is switched is not used.

3. The video display device according to claim 1, wherein image capturing completion of each frame of the imaging unit is detected, and the lighting of the first illuminator and the second illuminator is switched each time imaging of k (k is an integer of 3 or more) frames is completed, and

in the contact point detection process, an image of a frame unit irradiated with only the illumination light of one of the first illuminator and the second illuminator is used, but images of a plurality of frames being imaged before illuminator switching is completed after the illuminator switching starts with detection of the image capturing completion are not used.

4. The video display device according to claim 1, wherein image capturing completion of each frame of the imaging unit is detected, and the lighting of the first illuminator and the second illuminator is switched each time imaging of k (k is an integer of 3 or more) frames is completed, and

in the contact point detection process, images of a plurality of frames imaged before illuminator switching is completed after the illuminator switching starts with detection of the image capturing completion are used, and an image of a frame being when the illuminator switching is completed is not used.

5. The video display device according to claim 2, wherein the imaging unit causes an exposure time for an image of a frame to be used in the contact point detection process to be different from an exposure time for an image of a frame not to be used, and sets an average frame rate to a predetermined value.