Pose estimation apparatus and vacuum cleaner system

• US 10,375,376 B2
• Filed: 11/17/2016
• Issued: 08/06/2019
• Est. Priority Date: 11/17/2015
• Status: Active Grant

• Current Assignee: Toshiba Lifestyle Products & Services Corporation (Midea Group Co., Ltd), Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Yamamoto, Takuma, Mishima, Nao
• Primary Examiner(s): Tucker, Wesley J
• Application Number:
  US15/354,378
• Publication Number:
  US 20170140540A1
• Time in Patent Office: 992 Days
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/211 using temporal multiplexing

Abstract

According to an embodiment, a pose estimation apparatus includes first and second imagers and an estimator. The first imager generates first and second standard images respectively captured at first and second times. The second imager generates first and second reference images respectively associated with the first and second times. The estimator estimates, based on the first and second standard images and the first and second reference images, (a) a pose of the first imager at the first time, with a pose of the first imager at the second time being regarded as a standard and (b) a positional shift of the second imager capturing the first reference image from an estimated position of the second imager at the first time.

Claims

1. A pose estimation apparatus comprising:

a first camera configured to generate a first standard image captured at a first time and a second standard image captured at a second time;

a second camera configured to generate a first reference image associated with the first time and a second reference image associated with the second time; and

a processor configured to estimate, based on the first standard image, the second standard image, the first reference image and the second reference image, (a) a pose of the first camera at the first time, with a pose of the first camera at the second time being regarded as a standard and (b) a positional shift of a position of the second camera when capturing the first reference image from an estimated ideal position of the second camera at the first time, wherein the estimated ideal position of the second camera at the first time is calculated by shifting the position of the first camera at the first time in accordance with a positional relationship between the first camera and the second camera,

wherein the processor is further configured to;

search for corresponding feature points in the first standard image, the second standard image, the first reference image and the second reference image,

restore positions of the feature points based on the second standard image and the second reference image, and

estimate (a) and (b), such that an error between re-projected points obtained by projecting restored positions of the feature points on the first camera and the second camera at the first time and the feature points included in the first standard image and the first reference image is minimized.

2. The apparatus according to claim 1, wherein the processor is configured to regard the positional shift of the second camera capturing the first reference image from the estimated position of the second camera at the first time as being caused by a rotation motion.

3. The apparatus according to claim 1, wherein the processor is further configured to estimate a positional shift of a position of the second camera when capturing the second reference image from an estimated ideal position of the second camera at the second time, wherein the estimated ideal position of the second camera at the second time is calculated by shifting the position of the first camera at the second time in accordance with the positional relationship between the first camera and the second camera.

4. The apparatus according to claim 1, wherein the processor is further configured to estimate (c) the pose of the first camera at the first time, with the pose of the first camera at the second time being regarded as the standard, as translation components of two degrees of freedom and a rotation component of one degree of freedom and (d) the positional shift of the second camera capturing the first reference image from the estimated position of the second camera at the first time as a rotation component of one degree of freedom.

5. A vacuum cleaner system comprising the apparatus according to claim 1.

6. The system according to claim 5, further comprising a drive wheel that permits a vacuum cleaner to travel, and wherein

a rotating shaft of the drive wheel is substantially parallel to a straight line connecting the first camera and the second camera.

7. A non-transitory computer readable storage medium storing instructions of a computer program which when executed by a computer results in performance of stops comprising cause the computer to carry out pose estimation of a first camera of a stereo camera having a first and second camera whose imaging timings are not synchronous with each other, causing the computer to carry out the steps of:

controlling the first camera that generates a first standard image captured at a first time and a second standard image captured at a second time;

controlling the second camera that generates a first reference image associated with the first time and a second reference image associated with the second time; and

estimating, based on the first standard image, the second standard image, the first reference image and the second reference image, (a) a pose of the first camera at the first time, with a pose of the first camera at the second time being regarded as a standard and (b) a positional shift of a position of the second camera when capturing the first reference image from an estimated ideal position of the second camera at the first time, wherein the estimated ideal position of the second camera at the first time is calculated by shifting the position of the first camera at the first time in accordance with a positional relationship between the first camera and the second camera, the estimation step comprising;

searching for corresponding feature points in the first standard image, the second standard image, the first reference image and the second reference image,

restoring positions of the feature points based on the second standard image and the second reference image, and

estimating (a) and (b), such that an error between re-projected points obtained by projecting restored positions of the feature points on the first camera and the second camera at the first time and the feature points included in the first standard image and the first reference image is minimized.