Apparatus, method, and system for identifying laser spot

• US 8,599,134 B2
• Filed: 02/04/2011
• Issued: 12/03/2013
• Est. Priority Date: 03/11/2010
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Zhang, Wenbo
• Primary Examiner(s): Nguyen, Jimmy H
• Application Number:
  US13/021,046
• Publication Number:
  US 20110221919A1
• Time in Patent Office: 1,033 Days
• Field of Search: 345156-178
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0386 for light pen
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Disclosed are an apparatus, a method, and a system for identifying a laser spot and locating coordinates of the laser spot. The apparatus comprises a foreground calculation unit used to calculate a foreground image in an input image captured by a camera device; a threshold value estimation unit used to automatically estimate, at an initialization stage, pixel brightness threshold values, pixel degree-of-saturation threshold values, and a pixel color-tone threshold value range based on the foreground image; a laser spot detection unit used to detect, at a processing stage, the laser spot based on the foreground image as well as the pixel brightness threshold values, the pixel degree-of-saturation threshold values, and the pixel color-tone threshold value range; and a coordinate locating unit used to calculate the coordinates of the laser spot in the input image based on the laser spot detected by the laser spot detection unit.

Claims

1. An apparatus for identifying a laser spot and locating coordinates of the laser spot, comprising:

a foreground calculation unit used to calculate a foreground image in an input image captured by a camera device;

a threshold value estimation unit used to automatically estimate, at an initialization stage of the apparatus, a first pixel brightness threshold value, a second pixel brightness threshold value, a first pixel degree-of-saturation threshold value, a second pixel degree-of-saturation threshold value, and a pixel color-tone threshold value range based on the foreground image calculated by the foreground calculation unit;

a laser spot detection unit used to detect, at a processing stage of the apparatus, the laser spot based on the foreground image calculated by the foreground calculation unit as well as the first pixel brightness threshold value, the second pixel brightness threshold value, the first pixel degree-of-saturation threshold value, the second pixel degree-of-saturation threshold value, and the pixel color-tone threshold value range; and

a coordinate locating unit used to calculate the coordinates of the laser spot in the input image based on the laser spot detected by the laser spot detection unit.

2. The apparatus according to claim 1, wherein:

the foreground calculation unit first calculates a background image in the input image, and then calculates the foreground image by subtracting the background image from the input image.

3. The apparatus according to claim 1, wherein:

the threshold value estimation unit analyzes the foreground image, and then estimates the first pixel brightness threshold value, the second pixel brightness threshold value, the first pixel degree-of-saturation threshold value, the second pixel degree-of-saturation threshold value, and the pixel color-tone threshold value range based on brightness values, degree-of-saturation values, and color-tone values of pixels in the foreground image.

4. The apparatus according to claim 1, wherein:

the laser spot detection unit classifies, based on brightness values, degree-of-saturation values, and color-tone values of pixels in the foreground image, the pixels in the foreground image, and detects, based on the classified results, the laser spot, wherein, the classified results are obtained as follows;

if a pixel in the foreground image satisfies following relationships

B(p)>

T1(b)

and

S(p)<

T1(s),

then the pixel in the foreground image is classified as belonging to a first category;

here p refers to the pixel in the foreground image, B(p) refers to a brightness value of the pixel in the foreground image, T1(b) refers to the first pixel brightness threshold value, S(p) refers to a degree-of-saturation value of the pixel in the foreground image, and T1(s) refers to the first pixel degree-of-saturation threshold value,

if a pixel in the foreground image satisfies following relationships

H(p)ε

T(h),

B (p)>

T2(b),

and

S (p)>

T2(s),

then the pixel in the foreground image is classified as belonging to a second category;

here H(p) refers to a color-tone value of the pixel in the foreground image, T(h) refers to the pixel color-tone threshold value range, B(p) refers to a brightness value of the pixel in the foreground image, T2(b) refers to the second pixel brightness threshold value, S(p) refers to a degree-of-saturation value of the pixel in the foreground image, and T2(s) refers to the second pixel degree-of-saturation threshold value, and

if a pixel in the foreground image does not belong to the first category and the second category, then the pixel in the foreground image is classified as belonging to a third category.

5. The apparatus according to claim 4, wherein, the laser spot detection unit detects the laser spot by carrying out operations of:

calculating one or more combination areas RA and one or more combination areas RB composed of the pixels belonging to the first category and the second category, respectively, and removing all the pixels belonging to the third category;

removing the combination areas RA in which the numbers of the pixels are greater than a predetermined threshold value;

combining the remaining combination areas RA, in which a distance between any two remaining combination areas RA is less than a predetermined threshold value, so as to generate one or more new combination areas MA;

removing the new combination areas MA which are not adjacent to the combination areas RB;

combining the combination areas RB which are adjacent to the new combination areas MA so as to generate one or more new combination areas MB;

calculating overlapping degrees of the remaining new combination areas MA and the new combination areas MB, and removing the remaining new combination areas MA and the new combination areas MB whose overlapping degrees are less than a predetermined threshold value;

calculating outlines of the remaining new combination areas MA; and

letting the new combination area MA whose outline is an approximate circle be the laser spot.

6. The apparatus according to claim 1, wherein:

the coordinate locating unit calculates coordinates of the center of the laser spot to serve as the coordinates of the laser spot.

7. A method of identifying a laser spot and locating coordinates of the laser spot, comprising:

a foreground calculation step of calculating a foreground image in an input image captured by a camera device;

a threshold value estimation step of automatically estimating, at an initialization stage, a first pixel brightness threshold value, a second pixel brightness threshold value, a first pixel degree-of-saturation threshold value, a second pixel degree-of-saturation threshold value, and a pixel color-tone threshold value range based on the foreground image calculated by the foreground calculation step;

a laser spot detection step of detecting, at a processing stage, the laser spot based on the foreground image calculated by the foreground calculation step as well as the first pixel brightness threshold value, the second pixel brightness threshold value, the first pixel degree-of-saturation threshold value, the second pixel degree-of-saturation threshold value, and the pixel color-tone threshold value range; and

a coordinate locating step of calculating the coordinates of the laser spot in the input image based on the laser spot detected by the laser spot detection step.

8. The method according to claim 7, wherein, the laser spot detection step comprises:

a pixel classification step of classifying, based on brightness values, degree-of-saturation values, and color-tone values of pixels in the foreground image, the pixels in the foreground image, and detects, based on the classified results, the laser spot, wherein, the classified results are obtained as follows;

if a pixel in the foreground image satisfies following relationships

B(p)>

T1(b)

and

S(p)<

T1(s),

then the pixel in the foreground image is classified as belonging to a first category;

here p refers to the pixel in the foreground image, B(p) refers to a brightness value of the pixel in the foreground image, T1(b) refers to the first pixel brightness threshold value, S(p) refers to a degree-of-saturation value of the pixel in the foreground image, and T1(s) refers to the first pixel degree-of-saturation threshold value,

if a pixel in the foreground image satisfies following relationships

H(p)ε

T(h),

B(p)>

T2(b),

and

S(p)>

T2(s),

then the pixel in the foreground image is classified as belonging to a second category;

here H(p) refers to a color-tone value of the pixel in the foreground image, T(h) refers to the pixel color-tone threshold value range, B(p) refers to a brightness value of the pixel in the foreground image, T2(b) refers to the second pixel brightness threshold value, S(p) refers to a degree-of-saturation value of the pixel in the foreground image, and T2(s) refers to the second pixel degree-of-saturation threshold value, and

if a pixel in the foreground image does not belong to the first category and the second category, then the pixel in the foreground image is classified as belonging to a third category.

9. The method according to claim 8, wherein, the laser spot detection step further comprises steps of:

calculating one or more combination areas RA and one or more combination areas RB composed of the pixels belonging to the first category and the second category, respectively, and removing all the pixels belonging to the third category;

removing the combination areas RA in which the numbers of the pixels are greater than a predetermined threshold value;

combining the remaining combination areas RA, in which a distance between any two remaining combination areas RA is less than a predetermined threshold value, so as to generate one or more new combination areas MA;

removing the new combination areas MA which are not adjacent to the combination areas RB;

combining the combination areas RB which are adjacent to the new combination areas MA so as to generate one or more new combination areas MB;

calculating overlapping degrees of the remaining new combination areas MA and the new combination areas MB, and removing the remaining new combination areas MA and the new combination areas MB whose overlapping degrees are less than a predetermined threshold value;

calculating outlines of the remaining new combination areas MA; and

letting the new combination area MA whose outline is an approximate circle be the laser spot.

10. A virtual whiteboard system comprising:

a projection device used to project a visible image onto a screen;

a laser pointer used to project a laser spot onto the screen;

a camera device used to capture the image projected onto the screen; and

an apparatus for identifying the laser spot and locating coordinates of the laser spot, including;

a foreground calculation unit used to calculate a foreground image in the image captured by the camera device;

a threshold value estimation unit used to automatically estimate, at an initialization stage of the apparatus, a first pixel brightness threshold value, a second pixel brightness threshold value, a first pixel degree-of-saturation threshold value, a second pixel degree-of-saturation threshold value, and a pixel color-tone threshold value range based on the foreground image calculated by the foreground calculation unit;

a laser spot detection unit used to detect, at a processing stage of the apparatus, the laser spot based on the foreground image calculated by the foreground calculation unit as well as the first pixel brightness threshold value, the second pixel brightness threshold value, the first pixel degree-of-saturation threshold value, the second pixel degree-of-saturation threshold value, and the pixel color-tone threshold value range; and

a coordinate locating unit used to calculate the coordinates of the laser spot in the image based on the laser spot detected by the laser spot detection unit.