Selective interactive mapping of real-world objects to create interactive virtual-world objects

• US 8,902,227 B2
• Filed: 09/10/2007
• Issued: 12/02/2014
• Est. Priority Date: 09/10/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.), Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Harrison, Phil
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US11/852,970
• Publication Number:
  US 20090066690A1
• Time in Patent Office: 2,640 Days
• Field of Search: 382/107, 715/707
• US Class Current: 345/420
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering

Abstract

A method for interactively defining a virtual-world space based on real-world objects in a real-world space is disclosed. In one operation, one or more real-world objects in the real-world space is captured to define the virtual-world space. In another operation, one of the real-world objects is identified, the identified object is to be characterized into a virtual-world object. In yet another operation, a user is prompted for user identification of one or more object locations to enable extraction of parameters for real-world object, and the object locations are identified relative to an identifiable reference plane in the real-world space. In another operation, the extracted parameters of the real-world object may be stored in memory. The virtual-world object can then be generated in the virtual world space from the stored extracted parameters of the real-world object.

Claims

1. A method for interactively defining a virtual-world space based on real-world objects in a real-world space, comprising:

(a) receiving an input identifying one or more reference points in the real-world space to define the virtual-world space;

(b) receiving an input identifying one of the real-world objects to characterize into a virtual-world object, the real-world object being other than a user;

(c) prompting, via an on-screen display, the user to select one or more object part locations on the real-world object, the object part locations being identified relative to an identifiable reference plane in the real-world space, the identifiable reference plane defined based on the reference points;

(d) receiving an input identifying the object part locations selected by the user in response to said prompting;

(e) extracting one or more parameters of the real-world object from the input identifying the object part locations; and

(f) generating a virtual-world object in the virtual-world space using the extracted parameters of the real-world object.

2. The method as recited in claim 1, further comprising:

repeating (c)-(e), so as to extract additional parameters of the real-world object to further define the virtual-world object.

3. The method as recited in claim 1, further comprising:

repeating (b)-(f), so as to generate a different virtual-world object in the virtual-world space from a second real-world object.

4. The method as recited in claim 3, further comprising repeating (c)-(e), so as to extract additional parameters of the second real-world object to further define the different virtual-world object.

5. The method as recited in claim 1, wherein the virtual-world object, upon being generated, is displayed on a display screen along with other virtual-world objects, and the other virtual-world objects are able to interact with the generated virtual-world object, the interaction detected when dimensional parameters of the other virtual objects at least partially intersect with the generated virtual-world object defined by the extracted parameters of the real-world object.

6. The method as recited in claim 5, wherein when intersection between the dimensional parameters of the other virtual-world objects and the generated virtual-world object is detected, performing one of:

(a) occluding the other virtual-world objects by the generated virtual-world object;

(b) occluding the generated virtual-world object by the other virtual-world objects;

(c) obstructing motion or placement of the other virtual-world objects with the generated virtual-world object;

(d) deforming the other virtual-world objects so as to conform the other virtual-world objects to the generated virtual-world object;

or

(e) enabling interaction between the other virtual-world objects and the generated virtual-world object.

7. The method as recited in claim 1, wherein the reference points are digitally captured by obtaining one or more image frames of the real-world space.

8. The method as recited in claim 1, further comprising:

looking up an object template that substantially matches geometric attributes of the real-world object; and

implementing the object template to render a representation of the real-world object as the virtual-world object.

9. The method as recited in claim 8, wherein the object template defines the object part locations used to prompt user identification.

10. The method as recited in claim 9, wherein the object part locations represent one or more of:

(a) substantially vertical surfaces on the real-world object;

(b) substantially horizontal surfaces of the real-world object;

or

(c) edge boundaries of the real-world object.

11. The method as recited in claim 1, wherein the parameters include one or more dimensions of the object part locations.

12. The method as recited in claim 1, wherein (c) includes prompting, via the on-screen display, the user to touch the object part locations, and

wherein (d) includes receiving the input identifying the object part locations touched by the user.

13. A method for interactively defining a virtual-world space based on real-world objects in a real-world space, comprising:

(a) receiving an input identifying one or more reference points in the real-world space to define the virtual-world space;

(b) receiving an input identifying one of the real-world objects to characterize into a virtual-world object, the real-world object being other than a user;

(c) prompting, via an on-screen display, the user to select one or more transition boundaries of the real-world object, the transition boundaries being identified relative to an identifiable reference plane in the real-world space, the identifiable reference plane defined based on the reference points;

(d) receiving an input identifying the transition boundaries selected by the user in response to said prompting;

(e) determining geometric aspects of the real-world object from the input identifying the transition boundaries; and

(f) generating the virtual-world object in the virtual-world space using the geometric aspects of the real-world object.

14. The method as recited in claim 13, wherein the transition boundaries include vertices, edges, filleted vertices, or filleted edges.

15. The method as recited in claim 13, further comprising:

repeating (c)-(e), so as to determine additional geometric aspects of the real-world object to further define the virtual-world object.

16. The method as recited in claim 13, further comprising:

repeating (b)-(f), so as to generate a different virtual-world object in the virtual-world space from a second real-world object.

17. The method as recited in claim 13, wherein the virtual-world object, upon being generated, is displayed on a display screen along with other virtual-world objects, and the other virtual-world objects are able to interact with the generated virtual-world object, the interaction detected when dimensional parameters of the other virtual objects at least partially intersect with the generated virtual-world object defined by the geometric aspects of the real-world object.

18. The method as recited in claim 17, wherein when intersection between the dimensional parameters of the other virtual-world objects and the generated virtual-world object is detected, performing one of:

(a) occluding the other virtual-world objects by the generated virtual-world object;

(b) occluding the generated virtual-world object by the other virtual-world objects;

(c) obstructing motion or placement of the other virtual-world objects with the generated virtual-world object;

(d) deforming the other virtual-world object so as to conform the other virtual-world objects to the generated virtual-world object; and

(e) enabling interaction between the other virtual-world objects and the generated virtual-world object.

19. The method of claim 13, wherein the transition boundaries include corners of the real-world object.

20. The method of claim 13, wherein (c) includes prompting, via the on-screen display, the user to touch the transition boundaries of the real-world object, and

wherein (d) includes receiving the input identifying the transition boundaries touched by the user in response to said prompting.

21. A method for interactively defining a virtual-world space based on real-world objects in a real-world space, comprising:

(a) receiving an input identifying one or more reference points in the real-world space to define the virtual-world space;

(b) receiving an input identifying one of the real-world objects to characterize into a virtual-world object, the real-world object being other than a user;

(c) prompting, via an on-screen display, the user to select one or more surface locations on the real-world object, the surface locations being identified relative to an identifiable reference plane in the real-world space, the identifiable reference plane defined based on the reference points;

(d) receiving an input identifying the surface locations selected by the user in response to said prompting;

(e) extracting one or more geometric aspects of the real-world object from the input identifying the surface locations; and

(f) generating the virtual-world object in the virtual-world space using the geometric aspects of the real-world object.

22. The method as recited in claim 21, further comprising:

repeating (c)-(e), so as to extract additional geometric aspects of the real-world object to further define the virtual-world object.

23. The method as recited in claim 21, wherein the virtual-world object, upon being generated, is displayed on a display screen along with other virtual-world objects, and the other virtual-world objects are able to interact with the generated virtual-world object, the interaction detected when dimensional parameters of the other virtual objects at least partially intersect with the generated virtual-world object defined by the geometric aspects of the real-world object.

24. The method as recited in claim 23, wherein when intersection between the dimensional parameters of the other virtual-world objects and the generated virtual-world object is detected, performing one of:

(a) occluding the other virtual-world objects by the generated virtual-world object;

(b) occluding the generated virtual-world object by the other virtual-world objects;

(c) obstructing motion or placement of the other virtual-world objects with the generated virtual-world object;

(d) deforming the other virtual-world objects so as to conform the other virtual-world objects to the generated virtual-world object; and

(e) enabling interaction between the other virtual-world objects and the generated virtual-world object.

25. The method of claim 21, wherein (c) includes prompting, via the on-screen display, the user to touch the surface locations on the real-world object,

wherein (d) includes receiving the input identifying the surface locations touched by the user in response to said prompting.

26. A computer for creating virtual objects based on real-world objects, comprising:

a processor, a memory, and a display screen, the memory coupled to the processor, the display screen coupled to the processor;

wherein the processor is configured to;

receive an input identifying one or more reference points in a real-world space to define a virtual-world space;

receive an input identifying one of the real-world objects to characterize into a virtual-world object, the real-world object being other than a user;

wherein the display screen is configured to prompt the user to select one or more object part locations on the real-world object, the object part locations being identified relative to an identifiable reference plane in the real-world space;

wherein the processor is further configured to;

receive an input identifying the object part locations selected by the user in response to the prompt;

extract one or more parameters of the real-world object from the input identifying the object part locations;

wherein the processor is configured to generate the virtual-world object in the virtual-world space using the extracted parameters of the real-world object.

27. The computer system as recited in claim 26, wherein the processor is further configured to manipulate other virtual objects, the other virtual objects able to interact with the generated virtual-world object, the interaction of the other virtual-world objects restricted to conform to geometric aspects of the generated virtual-world object.

28. The computer system as recited in claim 26, wherein the memory is configured to store the generated virtual-world object.

29. The computer system as recited in claim 26, wherein the display screen is configured to prompt the user to touch the object part locations on the real-world object,

wherein the processor is further configured to receive the input identifying the object part locations touched by the user in response to the prompt.