System and method for texturing in virtual reality and mixed reality environments

• US 10,001,901 B2
• Filed: 06/14/2017
• Issued: 06/19/2018
• Est. Priority Date: 06/14/2016
• Status: Active Grant

• Current Assignee: Unity IPR ApS (Unity Software, Inc.)
• Original Assignee: Unity IPR ApS (Unity Software, Inc.)
• Inventors: Palmaro, Gregory Lionel Xavier Jean
• Primary Examiner(s): Lowenberger, Aaron
• Application Number:
  US15/623,114
• Publication Number:
  US 20170357407A1
• Time in Patent Office: 370 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes

Abstract

A system includes hardware processors and a texturing module configured to display a texturing tray within a virtual reality environment or a mixed reality environment, the texturing tray including a first plurality of texturing mode objects, detect that a user has grabbed a first texturing mode object of the first plurality of texturing mode objects from the texturing tray with a virtual hand of the user, based on motion of the virtual hand after the grabbing, moving the first texturing mode object into a position adjacent to a surface of a target object, detect that the user has applied a texturing agent toward the first texturing mode object, and apply a texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the first texturing mode object, the texture effect altering the appearance of the surface of the target object.

Claims

1. A system comprising:

one or more hardware processors; and

a texturing module, executable by the one or more hardware processors, configured to perform operations for applying a texture effect to a target object that is to be included in a setting of a computer game for subsequent rendering of the target object to a player of the computer game, the operations comprising;

displaying a texturing tray within an environment, the environment being one of a virtual reality environment and a mixed reality environment;

displaying a texturing menu in association with the texturing tray, the texturing menu including a plurality of texture mode selectors, the plurality of texture mode selectors corresponding to a plurality of editing modes;

based on a detecting that a user has selected the first texture mode selector from the texturing menu with a virtual hand of the user, presenting a first plurality of texture mode objects in the texturing tray and selecting a first editing mode of the plurality of editing modes, the first editing mode being a transferring mode;

based on a detecting that the user grabbed a texturing mode object of the first plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the transferring mode, the operations for implementing the transferring mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the first plurality of texturing mode objects into a position adjacent to a surface of a target object;

detecting that the user has applied a texturing agent in relation to the texturing mode object of the first plurality of texturing mode objects;

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the first plurality of texture mode objects, the texture effect altering the surface of the target object by transferring a texture pattern corresponding to the texturing mode object of the first plurality of texturing mode objects onto the target object; and

including the target object in the setting of a computer game for the subsequent rendering of the target object to the player of the computer game.

2. The system of claim 1, wherein the texturing mode object of the first plurality of texturing objects includes a frame having a particular shape and the texture pattern fills the frame.

3. The system of claim 1, wherein the texturing tray includes a display wall, wherein the plurality of texturing mode objects of the first plurality of texturing mode objects are attached to the display wall.

4. The system of claim 1, the operations further comprising:

based on a detecting that a user has selected a second texture mode selector of the plurality of texture mode selectors from the texturing menu with the virtual hand of the user,

removing the first plurality of texturing mode objects from the texturing tray, presenting a second plurality of texturing mode objects on the texturing tray, and selecting a second editing mode of the plurality of editing modes, the second editing mode being one of a stencil mode, a mask mode, and a stamp mode.

5. The system of claim 4, wherein the second editing mode is the mask mode and the operations further comprise:

based on a detecting that the user grabbed a texturing mode object of the second plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the mask mode, the operations for implementing the mask mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the second plurality of texturing mode objects into a position adjacent to a surface of a target object, the moving including simulating a magnetic attachment of the texturing mode object of the second plurality of texturing mode objects to the target object;

detecting that the user has applied a texturing agent in relation to the texturing mode object of the second plurality of texturing mode objects; and

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the second plurality of texture mode objects, the texture effect altering the surface of the target object by applying a color to a portion of the surface of the target object that is not covered by a mask shape corresponding to the texturing object of the second plurality of texturing mode objects.

6. The system of claim 5, wherein the simulating of the magnetic attachment-includes:

detecting that the texturing mode object of the second plurality of texture mode objects is within a predetermined distance of the surface of the target object; and

removably attaching the texturing mode object of the second plurality of texture mode objects to the surface of the target object.

7. The system of claim 4, wherein the second editing mode is the stencil mode and the operations further comprise:

based on a detecting that the user grabbed a texturing mode object of the second plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the mask mode, the operations for implementing the mask mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the second plurality of texturing mode objects into a position adjacent to a surface of a target object, the moving including simulating a magnetic attachment of the texturing mode object of the second plurality of texturing mode objects to the target object;

detecting that the user has applied a texturing agent in relation to the texturing mode object of the second plurality of texturing mode objects; and

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the second plurality of texture mode objects, the texture effect altering the surface of the target object by applying a color to portions of the surface of the target object that are not covered by a pattern of a stencil corresponding to the texturing object of the second plurality of texturing mode objects.

8. The system of claim 4, wherein the second editing mode is the stamp mode and the operations further comprise:

based on a detecting that the user grabbed a texturing mode object of the second plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the mask mode, the operations for implementing the mask mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the second plurality of texturing mode objects into a position adjacent to a surface of a target object;

detecting that the user has applied a texturing agent in relation to the texturing mode object of the second plurality of texturing mode objects; and

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the second plurality of texture mode objects, the texture effect altering the surface of the target object by applying a stamp pattern to the surface of the target object, the stamp pattern corresponding to the texturing object of the second plurality of texturing mode objects.

9. A computer-implemented method comprising:

performing operations for applying a texture effect to a target object that is to be included in a setting of a computer game for subsequent rendering of the target object to a player of the computer game, the operations comprising;

displaying a texturing tray within an environment, the environment being one of a virtual reality environment and a mixed reality environment;

displaying a texturing menu in association with the texturing tray, the texturing menu including a plurality of texture mode selectors, the plurality of texture mode selectors corresponding to a plurality of editing modes;

based on a detecting that a user has selected the first texture mode selector from the texturing menu with a virtual hand of the user, presenting a first plurality of texture mode objects in the texturing tray and selecting a first editing mode of the plurality of editing modes, the first editing mode being a transferring mode;

based on a detecting that the user grabbed a texturing mode object of the first plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the transferring mode, the operations for implementing the transferring mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the first plurality of texturing mode objects into a position adjacent to a surface of a target object;

detecting that the user has applied a texturing agent in relation to texturing mode object of the first plurality of texturing mode objects;

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the first plurality of texture mode objects, the texture effect altering the surface of the target object by transferring a texture pattern corresponding to the texturing mode object of the first plurality of texturing mode objects onto the target object; and

including the target object in the setting of a computer game for the subsequent rendering of the target object to the player of the computer game.

10. The method of claim 9 wherein the texturing mode object of the first plurality of texturing objects includes a frame having a particular shape and the texture pattern fills the frame.

11. The method of claim 9, wherein the texturing tray includes a display wall, wherein the plurality of texturing mode objects of the first plurality of texturing mode objects are attached to the display wall.

12. The method of claim 9, the operations further comprising:

based on a detecting that a user has selected a second texture mode selector of the plurality of texture mode selectors from the texturing menu with the virtual hand of the user,

removing the first plurality of texturing mode objects from the texturing tray, presenting a second plurality of texturing mode objects on the texturing tray, and selecting a second editing mode of the plurality of editing modes, the second editing mode being one of a stencil mode, a mask mode, and a stamp mode.

13. A non-transitory machine-readable medium storing processor-executable instructions which, when executed by a processor, cause the processor to perform operations for applying a texture effect to a target object that is to be included in a setting of a computer game for subsequent rendering of the target object to a player of the computer game, the operations comprising:

displaying a texturing tray within an environment, the environment being one of a virtual reality environment and a mixed reality environment;

displaying a texturing menu in association with the texturing tray, the texturing menu including a plurality of texture mode selectors, the plurality of texture mode selectors corresponding to a plurality of editing modes;

based on a detecting that a user has selected the first texture mode selector from the texturing menu with a virtual hand of the user, presenting a first plurality of texture mode objects in the texturing tray and selecting a first editing mode of the plurality of editing modes, the first editing mode being a transferring mode;

based on a detecting that the user grabbed a texturing mode object of the first plurality of texturing mode objects from the texturing tray with the virtual hand of the user, performing operations for implementing the transferring mode, the operations for implementing the transferring mode comprising;

based on motion of the virtual hand after the grabbing, moving the texturing mode object of the first plurality of texturing mode objects into a position adjacent to a surface of a target object;

detecting that the user has applied a texturing agent in relation to the texturing mode object of the first plurality of texturing mode objects;

applying the texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the texturing mode object of the first plurality of texture mode objects, the texture effect altering the surface of the target object by transferring a texture pattern corresponding to the texturing mode object of the first plurality of texturing mode objects onto the target object; and

including the target object in the setting of a computer game for the subsequent rendering of the target object to the player of the computer game.

14. The machine-readable medium of claim 13, wherein the texturing mode object of the first plurality of texturing objects includes a frame having a particular shape and the texture pattern fills the frame.

15. The machine-readable medium of claim 13, wherein the texturing tray includes a display wall, wherein the plurality of texturing mode objects of the first plurality of texturing mode objects are attached to the display wall.

16. The machine-readable medium of claim 13, the operations further comprising:

based on a detecting that a user has selected a second texture mode selector of the plurality of texture mode selectors from the texturing menu with the virtual hand of the user,

removing the first plurality of texturing mode objects from the texturing tray, presenting a second plurality of texturing mode objects on the texturing tray, and selecting a second editing mode of the plurality of editing modes, the second editing mode being one of a stencil mode, a mask mode, and a stamp mode.