System and method for object extraction

• US 9,595,108 B2
• Filed: 04/13/2016
• Issued: 03/14/2017
• Est. Priority Date: 08/04/2009
• Status: Expired due to Fees

• Current Assignee: EyeCue Vision Technologies LTD
• Original Assignee: EyeCue Vision Technologies LTD
• Inventors: Horovitz, Ronen, Kaftory, Ran, Feder, Shai
• Primary Examiner(s): MIKESKA, NEIL R
• Application Number:
  US15/097,950
• Publication Number:
  US 20160225137A1
• Time in Patent Office: 335 Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

A system and method for capturing images from various angles of an object that is resting on an orientation indication background, using a camera of a mobile device and visible light. The images are processed to construct a three dimensional image model and a representation of the three dimensional image model are displayed on an electronic display screen and manipulated or embedded into a video game.

Claims

1. A method of extracting images of a real-world object and displaying a representation of a three-dimensional model of said object in an electronic display, the method comprising:

capturing, using an image capture device and visible light, a plurality of two-dimensional images of said real-world object, said object at a fixed position relative to an orientation indicator visible in at least some of said captured two-dimensional images, a first image of said plurality of said captured two-dimensional images captured from a first orientation of said object to the image capture device capturing said plurality of images, and a second image of said plurality of said captured two-dimensional images captured from a second orientation of said object to said image capture device;

calculating said first orientation of said object from a position of said orientation indicator in said first image, and calculating said second orientation of said object from a position of said orientation indicator in said second image;

constructing a three-dimensional image model of said object from said plurality of captured images;

deleting data of said orientation indicator prior to said constructing;

identifying the object using a predefined database of objects;

based on said identifying, classifying said object as a model type;

embedding a representation of said three-dimensional image model into an electronic display of said object; and

digitally animating said three-dimensional image model in said display, said animating being based on said type.

2. The method as in claim 1, comprising constructing using a three-dimensional printer, a real-world three-dimensional object from said three-dimensional image model.

3. The method as in claim 1, wherein said capturing comprises rotating said mobile imager around said real-world object during said capturing.

4. The method as in claim 1, wherein said capturing comprises rotating said real-world object in front of said imager during said capturing.

5. The method as in claim 1, wherein said constructing is performed by a processor in a housing that includes said imager.

6. The method as in claim 1, wherein said deleting of said image of said orientation indicator from said first image is performed prior to said capture of said second image.

7. The method as in claim 1, comprising:

detecting control points in said first image and said second image; and

determining a perspective transformation of at least one of said control points in said first image and said second image.

8. The method as in claim 1, comprising manipulating on said electronic display said representation of said three-dimensional image model.

9. The method as in claim 1, wherein said representation of said three-dimensional image model is embedded in a view of a video game displayed on said electronic display.

10. The method as in claim 1, comprising capturing said plurality of two-dimensional images with a mobile imager.

19. The method of claim 1 wherein said model type is selected from:

humanoid, four-legged animal, and car.

20. The method of claim 1 wherein said digitally animating of said three-dimensional image model uses one or more predefined animations.

21. The method of claim 20 wherein said one or more predefined animations are based on said model type.

22. The method of claim 21 wherein said one or more predefined animations are selected from:

walking, jumping, and flying.

23. The method of claim 1 wherein a smart rig or skeleton is associated with the model type, the method further comprising:

associating one or more vertices of the three-dimensional image model with one or more rig or skeleton parts.

11. A system for extracting images of a real-world object and displaying a representation of a three-dimensional model of said object in such images in an electronic display, the system comprising:

an image capture device, a memory;

a processor;

a mat having visible thereon an orientation indicator, and configured to support said object;

wherein said image capture device is configured to capture a plurality of images of said object using visible light, a first image of said plurality of images from a first orientation relative to said object, and a second image of said plurality of images from a second orientation relative to said object; and

said processor is configured to;

calculate said first orientation of said object from a position of said orientation indicator in said first image, and calculate said second orientation of said object from a position of said orientation indicator in said second image;

construct a three-dimensional image model of said object from said plurality of captured images;

delete data of said orientation indicator prior to said construction;

identify the object using a predefined database of objects;

based on said identifying of said object, classify said object as a model type;

embed a representation of said three-dimensional image model into an electronic display of said object; and

digitally animate said three-dimensional image model in said display, said animating being based on said model type.

12. The system as in claim 11, comprising a three-dimensional printer, and wherein said processor is configured to transmit data of said three-dimensional image model to said three-dimensional printer, and said three-dimensional printer is to construct a real-world three-dimensional object using said data.

13. The system as in claim 11, wherein said image capture device is stored in a single housing with said processor.

14. The system as in claim 11, wherein said processor is configured to delete image data of said orientation indicator prior to said capture of said second image.

15. The system as in claim 11 wherein said processor is configured to:

match a first of a plurality of control points of said real world object that appears in said first image to a first of said plurality of control points of said real-world object that appears in said second image; and

determine a perspective transformation of at least one of said control points between said first image and said second image.

16. The system as in claim 11, comprising an electronic display, and wherein said processor is configured to issue signals to manipulate on said electronic display said representation of said three-dimensional image model.

17. The system as in claim 11, wherein said processor is configured to embed said representation of said three-dimensional image model in a view of a video game displayed on said electronic display.

18. The system as in claim 11, wherein said image capture device is a mobile imager.

24. The system of claim 11 wherein said model type is selected from:

humanoid, four-legged animal, and car.

25. The system of claim 11 wherein said one or more predefined animations are based on said model type.

26. The system of claim 25 wherein said one or more predefined animations are selected from:

walking, jumping, and flying.

27. The system of claim 11 wherein a smart rig or skeleton is associated with the model type, the method further comprising:

associating one or more vertices of the three-dimensional image model with one or more rig or skeleton parts.

28. A method of extracting images of a real-world object and displaying a representation of a three-dimensional model of said object in an electronic display, the method comprising:

(A) capturing, using an image capture device and visible light, a plurality of two-dimensional images of said real-world object, said object at a fixed position relative to an orientation indicator visible in at least some of said captured two-dimensional images, a first image of said plurality of said captured two-dimensional images captured from a first orientation of said object to the image capture device capturing said plurality of images, and a second image of said plurality of said captured two-dimensional images captured from a second orientation of said object to said image capture device;

(B) calculating said first orientation of said object from a position of said orientation indicator in said first image, and calculating said second orientation of said object from a position of said orientation indicator in said second image;

(C) constructing a three-dimensional image model of said object from said plurality of captured images, wherein data of said orientation indicator is deleted prior to said constructing;

(D) classifying said object as a particular model type of a plurality of model types, said classifying being based on identifying the object using a predefined database of objects, wherein each model type of said plurality of model types has a rig or skeleton associated therewith;

(E) associating one or more vertices of the three-dimensional image model with one or more rig or skeleton parts associated with the particular model type;

(F) digitally animating a representation of said three-dimensional image model in an electronic display, said animating being based on said particular model type, wherein said digitally animating of said three-dimensional image model uses one or more predefined animations based on said particular model type.

29. The method of claim 28 wherein said model type is selected from:

humanoid, four-legged animal, and car.

30. The method of claim 28 wherein said one or more predefined animations are selected from:

walking, jumping, and flying.