Augmented reality design system

• US 9,384,578 B2
• Filed: 12/16/2014
• Issued: 07/05/2016
• Est. Priority Date: 02/22/2010
• Status: Active Grant

• Current Assignee: Nike, Inc.
• Original Assignee: Nike, Inc.
• Inventors: Friesen, David Y.
• Primary Examiner(s): NGUYEN, ANH TUAN V
• Application Number:
  US14/571,443
• Publication Number:
  US 20150109322A1
• Time in Patent Office: 567 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A43B 3/0078 provided with logos, letters, signatures or the like decoration A43B23/24 takes precedence
• CPC Class Codes: G06F 2111/02 CAD in a network environment, e.g. collaborative CAD or distributed simulation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/16 Cloth

Abstract

An augmented reality design system is disclosed. The augmented reality design system allows a user to create a design for an article in real time using a proxy. The system can be configured using a head mounted display for displaying at least one virtual design element over a proxy located in a real-world environment. The system can also be configured using a projector that projects at least one virtual design element onto a proxy located in the real world.

Claims

1. A method of designing an article, comprising the steps of:

viewing a proxy for the article on a display device, the proxy existing in a real-world environment, the proxy having a three-dimensional irregular shape, the proxy including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device, and wherein a portion of the proxy, as viewed on the display device, has a first color;

wherein the sensors disposed in the proxy substantially cover any surface of the proxy to be interacted with the interacting device;

viewing, on the display device, a virtual menu in a location in the real-world environment;

physically placing the interacting device in the location to select a color from a color palette, wherein the placing the interacting device in the location has been detected by an imaging device;

contacting the portion of the proxy in the real-world environment with the interacting device at a contact point, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy; and

viewing, during the step of contacting, the proxy on the display device, wherein the portion of the proxy, as viewed on the display device, has a second color that is different from the first color, and wherein the portion has been displayed, by the display device, as the second color based on the contacting being detected by the sensor of the sensors disposed in the proxy and based on the physically placing the interacting device in the location being detected by the imaging device.

2. The method according to claim 1, wherein the step of viewing during the step of contacting comprises a step of displaying a virtual design element at a location on the proxy corresponding to the portion of the proxy;

and

wherein the virtual design element has the second color.

3. The method according to claim 1, wherein the proxy has the approximate shape of a human body.

4. The method according to claim 3, wherein the article is an article of clothing.

5. The method according to claim 3, wherein the article is a hat.

6. The method according to claim 1, wherein the article is an article of footwear.

7. A method of designing an article, comprising the steps of:

selecting a color from a virtual color palette associated with a proxy for the article, the selecting the color from the virtual color palette having been detected by an imaging device, the proxy existing in a real-world environment and having a first color, the proxy having a three-dimensional irregular shape, and the proxy including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device, wherein a portion of the proxy has a first color;

wherein the sensors disposed in the proxy substantially cover any surface of the proxy to be interacted with the interacting device;

contacting the portion of a proxy in the real-world environment with the interacting device at a contact point, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy; and

viewing a projection of a second color on the portion of the proxy including the contact point substantially immediately after the interacting device has contacted the portion of the proxy in the real-world environment, wherein the second color is different from the first color, and wherein the projection of the second color has been based on the contacting being detected by the sensor of the sensors disposed in the proxy and based on the selected color from the virtual color palette being detected by the imaging device.

8. The method according to claim 7, wherein the proxy has the approximate shape of a foot.

9. The method according to claim 8, wherein the article is an article of footwear.

10. The method according to claim 7, wherein the interacting device is a stylus.

11. The method according to claim 7, wherein the article is an article of clothing.

12. The method according to claim 7, wherein the portion of the proxy corresponds to an upper of an article of footwear.

13. An augmented reality design system for designing an article, comprising:

an imaging device for receiving images from a real-world environment;

a proxy for the article existing in the real world environment, the proxy having a three dimensional irregular shape;

an interacting device for interacting with the proxy in the real world environment, wherein sensors disposed in the proxy are configured to detect interactions between portions of the proxy with the interacting device;

wherein the sensors disposed in the proxy substantially cover any surface of the proxy to be interacted with the interacting device;

a computing device capable of generating virtual images, at least one of the virtual images being based on at least one detection of a contact point by a sensor of the sensors disposed in the proxy;

a display device, wherein the at least one virtual image is superimposed onto an image, received by the imaging device, of the proxy;

a virtual menu that may be displayed on the display device, wherein the virtual menu is associated with a location in the real world environment and wherein a user can interact with the virtual menu by placing the interacting device in the location and wherein the interaction with the virtual menu is detected by the imaging device;

wherein the virtual menu can be used to apply a virtual design element to the proxy at a location on the proxy corresponding with a contact point of the interacting device with the proxy in the real world environment; and

wherein the sensors disposed in the proxy detect the location on the proxy corresponding with the contact point of the interacting device with a surface of the proxy in the real-world environment.

14. The design system according to claim 13, wherein the virtual menu includes a color palette.

15. The design system according to claim 13, wherein the virtual menu includes a drawing tool.

16. The design system according to claim 13, wherein the virtual menu includes a letter tool.

17. The design system according to claim 13, wherein the virtual menu includes a virtual graphic.

18. The design system according to claim 13, wherein the virtual menu includes an eraser tool.

19. The design system according to claim 13, wherein the virtual menu includes a selection tool.

20. An augmented reality design system for designing an article, comprising:

an imaging device for receiving images from a real-world environment;

a proxy for the article existing in a real world environment, the proxy having a three dimensional irregular shape;

an interacting device for interacting with the proxy in the real world environment, wherein sensors disposed in the proxy are configured to detect interactions between portions of the proxy with the interacting device;

wherein the sensors disposed in the proxy substantially cover any surface of the proxy to be interacted with the interacting device;

a computing device capable of generating virtual images, at least one of the virtual images being based on at least one detection of a contact point by a sensor of the sensors disposed in the proxy;

a projecting device, wherein the at least one virtual image is projected onto the proxy;

a virtual menu that may be projected onto a location in the real-world environment, wherein a user can interact with the virtual menu by placing the interacting device in the location and wherein the interaction with the virtual menu is detected by the imaging device;

wherein the virtual menu can be used to apply a virtual design element to the proxy at a location on the proxy corresponding with a contact point of the interacting device with the proxy in the real world environment; and

wherein the sensors disposed in the proxy detect the location on the proxy corresponding with the contact point of the interacting device with a surface of the proxy in the real world environment.

21. The design system according to claim 20, wherein the virtual design element is a virtual line.

22. The design system according to claim 20, wherein the virtual design element corresponds to a color.

23. The design system according to claim 20, wherein the virtual design element is a virtual graphic.