SCENE SEGMENTATION USING PRE-CAPTURE IMAGE MOTION

• US 20130272609A1
• Filed: 12/12/2011
• Published: 10/17/2013
• Est. Priority Date: 12/12/2011
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Sun, Wei, Del Pasqua, Kieran, Haussecker, Horst
• Application Number:
  US13/977,185
• Publication Number:
  US 20130272609A1
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Systems, devices and methods are described including using object motion appearing in pre-capture images to perform 3D reconstruction of a scene. Objects may be segmented and tracked within the pre-capture images using image processing techniques such as image segmentation and/or object recognition. The image processing results may then be used to automatically tag subsequently captured images. Further, the image processing results may also be used to interactively control an imaging device'"'"'s focusing mechanism prior to image capture.

Claims

1-30. -30. (canceled)

31. A computer-implemented method, comprising:

receiving a plurality of pre-capture images of a scene, the pre-capture images having been generated by an imaging device, the pre-capture images exhibiting object motion; and

performing image segmentation based on the object motion until a shutter mechanism of the imaging device is engaged to capture an image of the scene.

32. The method of claim 31, further comprising:

using results of the image segmentation to recognize and automatically tag objects appearing in the captured image.

33. The method of claim 32, wherein performing image segmentation comprises generating object masks corresponding to objects identified in the plurality of pre-capture images, and wherein using results of the image segmentation to recognize and automatically tag objects comprises:

storing the image and the object masks;

using the object masks to perform object recognition on the image; and

tagging objects appearing in the image using results of the object recognition.

34. The method of claim 33, wherein tagging objects appearing in the image comprises storing metadata associated with the image.

35. The method of claim 31, further comprising:

using results of the image segmentation to interactively control a focus mechanism of the imaging device.

36. The method of claim 35, wherein performing image segmentation includes segmenting and tracking a plurality of objects in the scene, and wherein using results of the image segmentation to interactively control the focus mechanism comprises:

setting the focus mechanism to focus on a first object of the plurality of objects; and

resetting the focus mechanism to focus on a second object of the plurality of objects.

37. The method of claim 36, wherein resetting the focus mechanism to focus on the second object comprises resetting the focus mechanism in response to user input.

38. The method of claim 36, wherein setting the focus mechanism to focus on the first object comprises highlighting the first object relative to other objects of the plurality of objects in the scene.

39. The method of claim 38, wherein highlighting the first object relative to the other objects comprises displaying the first object as sharp while displaying the other objects as blurred.

40. The method of claim 31, wherein performing image segmentation includes segmenting and tracking objects in the scene by:

performing image segmentation on at least a first image of the plurality of pre-capture images to generate a first plurality of object results;

storing the first plurality of object results;

performing image segmentation on a second image of the plurality of pre-capture images to generate a second plurality of object results; and

comparing the second plurality of object results to the first plurality of object results.

41. The method of claim 40, wherein the first plurality of object results comprises a first plurality of identified objects, wherein the second plurality of object results comprises a second plurality of identified objects, and wherein comparing the second plurality of object results to the first plurality of object results comprises:

increasing a confidence value of each object of the first plurality of identified objects that is included in the second plurality of identified objects; and

decreasing a confidence value of each object of the first plurality of identified objects that is not included in the second plurality of identified objects.

42. An article comprising a computer program product having stored therein instructions that, if executed, result in:

receiving a plurality of pre-capture images of a scene, the pre-capture images having been generated by an imaging device, the pre-capture images exhibiting object motion; and

performing image segmentation based on the object motion until a shutter mechanism of the imaging device is engaged to capture an image of the scene.

43. The article of claim 42, the computer program product having stored therein further instructions that, if executed, result in:

using results of the image segmentation to recognize and automatically tag objects appearing in the captured image.

44. The article of claim 42, the computer program product having stored therein further instructions that, if executed, result in:

using results of the image segmentation to interactively control a focus mechanism of the imaging device.

45. The article of claim 44, wherein performing image segmentation includes segmenting and tracking a plurality of objects in the scene, and wherein using results of the image segmentation to interactively control the focus mechanism comprises:

setting the focus mechanism to focus on a first object of the plurality of objects; and

resetting the focus mechanism to focus on a second object of the plurality of objects.

46. A device, comprising:

a processor configured to;

receive data corresponding to a plurality of pre-capture images of a scene, the pre-capture images having been generated by an imaging device, the pre-capture images exhibiting object motion; and

perform image segmentation based on the object motion until a shutter mechanism of the imaging device is engaged to capture an image of the scene.

47. The device of claim 46, wherein the processor is configured to:

use results of the image segmentation to recognize and automatically tag objects appearing in the captured image.

48. The device of claim 46, wherein the processor is configured to:

use results of the image segmentation to interactively control a focus mechanism of the imaging device.

49. The device of claim 48, wherein performing image segmentation includes segmenting and tracking a plurality of objects in the scene, and wherein using results of the image segmentation to interactively control the focus mechanism comprises:

setting the focus mechanism to focus on a first object of the plurality of objects; and

resetting the focus mechanism to focus on a second object of the plurality of objects.

50. A system comprising:

an imaging device to obtain a plurality of pre-capture images of a scene, the pre-capture images exhibiting object motion; and

an image processing module to receive the plurality of pre-capture images, and to perform image segmentation based on the object motion until a shutter mechanism of the imaging device is engaged to capture an image of the scene.

51. The system of claim 50, wherein the image processing module is configured to:

use results of the image segmentation to recognize and automatically tag objects appearing in the captured image.

52. The system of claim 50, wherein the image processing module is configured to:

use results of the image segmentation to interactively control a focus mechanism of the imaging device.

53. The system of claim 52, wherein performing image segmentation includes segmenting and tracking a plurality of objects in the scene, and wherein using results of the image segmentation to interactively control the focus mechanism comprises:

setting the focus mechanism to focus on a first object of the plurality of objects; and

resetting the focus mechanism to focus on a second object of the plurality of objects.