Search assisting apparatus, search assisting method, and computer readable medium

• US 10,176,394 B2
• Filed: 03/17/2017
• Issued: 01/08/2019
• Est. Priority Date: 03/18/2016
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Subaru Corp.
• Inventors: Onomura, Yoichi
• Primary Examiner(s): Gilliard, Delomia L
• Application Number:
  US15/461,749
• Publication Number:
  US 20170270383A1
• Time in Patent Office: 662 Days
• Field of Search: None
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A search assisting apparatus includes a line-of-sight trajectory detector, a gaze region extractor, and a target presence determiner. The line-of-sight trajectory detector that detects a line-of-sight trajectory of the operator on an image displayed on a display. The gaze region extractor that extracts, on a basis of the line-of-sight trajectory detected by the line-of-sight trajectory detector, a gaze region gazed by the operator in the image displayed on the display. The target presence determiner that determines, on a basis of a distance from a predetermined position, presence of a predetermined detection target in the gaze region.

Claims

1. A search assisting apparatus comprising:

a controller; and

a memory storing instructions that, when executed by the controller, cause the controller to;

detect a line-of-sight trajectory of an operator on an image displayed on a display;

extracting, on a basis of the line-of-sight trajectory detected, a gaze region gazed by the operator in the image displayed on the display; and

determine, on a basis of a distance from a predetermined position, a presence of a predetermined detection target in the gaze region,

wherein extracting the gaze region comprises extracting, from the image displayed on the display, a circular region as the gaze region, the circular region being a region in which an average position of the line-of-sight trajectory is defined as a center and in which probability of presence of the line-of-sight trajectory takes a predetermined value.

2. The search assisting apparatus according to claim 1, further comprising a stereo camera obtaining information on the distance from the predetermined.

3. The search assisting apparatus according to claim 2, wherein determining the gaze region comprises extracting, from the image displayed on the display, a circular region as the gaze region, the circular region being a region in which an average position of the line-of-sight trajectory is defined as a center and in which probability of presence of the line-of-sight trajectory takes a predetermined value.

4. The search assisting apparatus according to claim 1, wherein

the presence of the predetermined detection target in the gaze region is determined when a difference between an average value of the distance in the gaze region from a capturing unit and a distance nearest in the gaze region from the capturing unit is equal to or greater than a predetermined value, and

orientation of the detection target is performed, by defining a position of the distance that is the nearest in the gaze region from the capturing unit as a position of the detection target.

5. The search assisting apparatus according to claim 2, wherein

the presence of the predetermined detection target in the gaze region is determined when a difference between an average value of the distance in the gaze region from the stereo camera and a distance nearest in the gaze region from the stereo camera is equal to or greater than a predetermined value, and

performs orientation of the detection target, by defining a position of the distance that is the nearest in the gaze region from the stereo camera as a position of the detection target.

6. The search assisting apparatus according to claim 2, wherein the stereo camera is mounted on an aircraft.

7. A search assisting method comprising:

detecting a line-of-sight trajectory of an operator on an image displayed on a display;

extracting, on a basis of the line-of-sight trajectory detected by the detecting of the line-of-sight trajectory, a gaze region gazed by the operator in the image displayed on the display; and

determining, on a basis of a distance from a predetermined position measured by the measuring of the distance, a presence of a predetermined detection target in the gaze region,

wherein extracting the gaze region comprises extracting, from the image displayed on the display, a circular region as the gaze region, the circular region being a region in which an average position of the line-of-sight trajectory is defined as a center and in which probability of presence of the line-of-sight trajectory takes a predetermined value.

8. A non-transitory computer readable medium having a search assisting program, the search assisting program causing, when executed by a controller of a search assisting apparatus, the search assisting apparatus to:

detect a line-of-sight trajectory of an operator on an image displayed on a display that displays an image captured by a capturing unit;

extract, on a basis of the detected line-of-sight trajectory, a gaze region gazed by the operator in the image displayed on the display; and

determine, on a basis of a distance in the gaze region from a predetermined position measured by a ranging unit that measures a distance from the capturing unit, a presence of the detection target in the gaze region,

wherein extracting the gaze region comprises extracting, from the image displayed on the display, a circular region as the gaze region, the circular region being a region in which an average position of the line-of-sight trajectory is defined as a center and in which probability of presence of the line-of-sight trajectory takes a predetermined value.