OBJECT CREATION USING BODY GESTURES

• US 20160027199A1
• Filed: 03/01/2013
• Published: 01/28/2016
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Cao, Xiang, Liu, Yang, Tong, Xin, Han, Teng, Shiratori, Takaaki, Umetani, Nobuyuki, Zhang, Yupeng, Ren, Zhimin
• Granted Patent: US 9,928,634 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/20 Indexing scheme for editing of 3D models

Abstract

An intuitive interface may allow users of a computing device (e.g., children, etc.) to create imaginary three dimensional (3D) objects of any shape using body gestures performed by the users as a primary or only input. A user may make motions while in front of an imaging device that senses movement of the user. The interface may allow first-person and/or third person interaction during creation of objects, which may map a body of a user to a body of an object presented by a display. In an example process, the user may start by scanning an arbitrary body gesture into an initial shape of an object. Next, the user may perform various gestures using his body, which may result in various edits to the object. After the object is completed, the object may be animated, possibly based on movements of the user.

Claims

1. A method comprising:

creating, using an imaging device, a three dimensional (3D) object from an initial pose of a user;

editing the 3D object by;

determining an edit operation to be performed on the 3D object;

performing the edit operation on the 3D object based at least in part on a body gesture of the user that is sensed by the imaging device; and

causing presentation of the 3D object to the user; and

animating the 3D object based on additional body gestures of the user.

2. The method as recited in claim 1, wherein the performing the edit operation includes:

selecting at least a portion of a surface of the 3D object based on a first part of the body gesture of the user; and

modifying the at least a portion of the surface of the 3D object based on a second part of the body gesture of the user.

3. The method as recited in claim 1, wherein the editing the 3D object is performed by using a mapping between the user and the 3D object, the mapping being at least one of:

a body reference mapping where the user is mapped to a similar location as the 3D object to allow the user to interact with the 3D object in a first-person editing perspective, or

a spatial mapping where the user is mapped to a location outside of the 3D object to allow the user to interact with the 3D object in a third-person editing perspective.

4. The method as recited in claim 3, further comprising determining the mapping prior to performing the edit operation.

5. The method as recited in claim 1, wherein the 3D object is an avatar.

6. The method as recited in claim 1, wherein the edit operation is at least one of a push operation, a drag operation, a sketch operation, a sculpt operation, a cut operation, or a paint operation.

7. The method as recited in claim 1, further comprising mapping features of the 3D object to body parts of the user sensed by the imaging device, and wherein the animating the 3D object includes moving the features of the 3D object based on movement of the body parts of the user.

8. The method as recited in claim 1, further comprising storing the 3D object for access by another computing device.

9. The method as recited in claim 1, further comprising:

receiving a selection of an existing 3D object, and

adding the existing 3D object to the 3D object.

10. The method as recited in claim 9, further comprising obtaining the existing 3D object by scanning a real world object using the imaging device.

11. The method as recited in claim 1, wherein the determining the edit operation includes identifying a tool based on object recognition or based on the body gesture.

12. The method as recited in claim 1, further comprising receiving a voice command using a microphone, and wherein the determining the edit operation or the performing the edit operation is based at least partly on the voice command.

13. A system comprising:

an imaging device to detect body gestures of a user;

one or more processors; and

memory to store computer readable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

identifying the body gesture of the user based on information received from the imaging device;

determining an edit operation to be performed on a three dimensional (3D) object based at least in part on the body gesture; and

modifying the 3D object using the edit operation.

14. The system as recited in claim 13, wherein the edit operation modifies a location of at least a portion of a surface of the 3D object based on movement of the user during the body gesture.

15. The system as recited in claim 13, wherein the determining the edit operation is based at least in part on identification of a tool held by the user and detected by the imaging device.

16. The system as recited in claim 13, further comprising creating the 3D object using at least one of voxel modeling or implicit surface modeling.

17. The system as recited in claim 13, wherein the 3D object is an avatar, and wherein the acts further comprise animating the avatar based on movements of the user.

18. One or more memory storing computer-executable instructions that, when executed on one or more processors, causes the one or more processors to perform acts comprising:

obtaining a three dimensional (3D) object from an initial shape;

determining an edit operation to be performed on the 3D object;

performing the edit operation on the 3D object based at least in part on a body gesture of a user that is sensed by a imaging device; and

causing presentation of the 3D object.

19. The one or more memory as recited in claim 18, wherein the edit operation modifies a location of at least a portion of a surface of the 3D object based on movement of the user during the body gesture.

20. The one or more memory as recited in claim 18, further comprising animating the 3D object.