Augmented reality design system

• US 8,947,455 B2
• Filed: 02/22/2010
• Issued: 02/03/2015
• Est. Priority Date: 02/22/2010
• Status: Active Grant

• Current Assignee: Nike, Inc.
• Original Assignee: Nike, Inc.
• Inventors: Friesen, David Y.
• Primary Examiner(s): Richer, Aaron M
• Assistant Examiner(s): NGUYEN, ANH TUAN V
• Application Number:
  US12/709,636
• Publication Number:
  US 20110205242A1
• Time in Patent Office: 1,807 Days
• Field of Search: None
• US Class Current: 345/633
• CPC Class Codes: A43B 3/0078 provided with logos, letters, signatures or the like decoration A43B23/24 takes precedence
• CPC Class Codes: G06F 2111/02 CAD in a network environment, e.g. collaborative CAD or distributed simulation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/16 Cloth

Abstract

An augmented reality design system is disclosed. The augmented reality design system allows a user to create a design for an article in real time using a proxy. The system can be configured using a head mounted display for displaying at least one virtual design element over a proxy located in a real-world environment. The system can also be configured using a projector that projects at least one virtual design element onto a proxy located in the real world.

Claims

1. A method of designing an article of footwear, comprising the steps of:

viewing a proxy on a display device, the proxy existing in a real-world environment, the proxy including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device, and the proxy having a shape of the article of footwear;

contacting the proxy in the real-world environment with the interacting device along a surface on the proxy to create a mark associated with a virtual design for the article of footwear, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy;

physically rotating, during the step of contacting, the proxy in the real-world environment while maintaining contact between the interacting device and the proxy during the rotation of the proxy;

viewing, on the display device and during the step of physically rotating, a virtual line in the virtual design corresponding to the contacting of the proxy with the interacting device along the surface on the proxy during the step of physically rotating the proxy;

wherein the virtual design is displayed on the display device so as to appear to be a part of the proxy;

wherein orientation and position of the virtual design is displayed on the display device so as to appear to be a part of the proxy as the proxy is rotated in the real-world environment; and

wherein the virtual line extends across the proxy between a medial side and a lateral side while the proxy is being rotated.

2. The method according to claim 1, wherein rotating the proxy further comprises rotating the proxy from one of the medial side or the lateral side of the article of footwear to the opposite lateral side or medial side.

3. The method according to claim 2, wherein the proxy is a footwear last having the shape of the article of footwear.

4. The method according to claim 2, wherein the proxy is an article of footwear.

5. The method according to claim 1, wherein the display device is a head mounted display that is worn by a user.

6. The method according to claim 1, further comprising:

viewing, on the display device, a virtual menu in a location in the real-world environment;

physically placing the interacting device in the location to select a virtual design element;

wherein the virtual line has been applied in the virtual design based on the step of physically placing the interacting device in the location; and

wherein the step of viewing the virtual line has been based on the virtual line being applied in the virtual design.

7. The method according to claim 1, wherein the method further includes a step of manufacturing the article of footwear using the virtual design.

8. The method according to claim 1, wherein the proxy includes at least one marking indicating a location of a feature of the article of footwear.

9. The method according to claim 8, wherein the at least one marking indicating the location of a feature of the article of footwear includes one or more of a lacing region and a sole of the article of footwear.

10. The method according to claim 1, wherein the sensors are disposed in the medial side of the proxy and the sensors are disposed on the lateral side of the proxy.

11. A method of designing an article of footwear, comprising the steps of:

providing a proxy existing in a real-world environment, the proxy having a shape of the article of footwear and the proxy including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device;

contacting a portion of the proxy in the real-world environment with the interacting device at at least one contact point on the proxy to create at least one mark associated with a virtual design for the article of footwear, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy;

physically rotating, during the contacting, the proxy in the real-world environment while maintaining contact between the interacting device and the proxy during the rotation of the proxy;

viewing, during the step of physically rotating, a projection of the at least one mark associated with the virtual design on the proxy at one or more locations that correspond to the at least one contact point on the proxy;

wherein the virtual design is projected onto the proxy so as to appear to be a part of the proxy;

wherein orientation and position of the virtual design is projected onto the proxy as the proxy is rotated in the real-world environment; and

wherein the virtual design extends between a medial side and a lateral side of the article of footwear.

12. The method according to claim 11, wherein a projecting device is used to directly project the virtual design onto the proxy.

13. The method according to claim 12, further comprising:

viewing, on the projection, another projection, or a combination thereof, a virtual menu in a location in the real-world environment;

physically placing the interacting device in the location to select a virtual design element; and

wherein a property of the at least one mark has been varied based on the selected virtual design element.

14. The method according to claim 11, wherein the proxy is a footwear last having the shape of the article of footwear.

15. The method according to claim 14, wherein the rotating the proxy further comprises rotating the proxy from one of the medial side or the lateral side of the article of footwear to the opposite lateral side or medial side.

16. The method according to claim 15, wherein the sensors are disposed in the medial side of the proxy and the sensors are disposed on the lateral side of the proxy.

17. The method according to claim 11, wherein the proxy is an article of footwear.

18. The method according to claim 11, wherein the method further includes a step of manufacturing the article of footwear using the virtual design.

19. The method according to claim 11, wherein the proxy includes at least one marking indicating a location of a feature of the article of footwear.

20. The method according to claim 19, wherein the at least one marking indicating the location of a feature of the article of footwear includes one or more of a lacing region and a sole of the article of footwear.

21. A method of designing an article of footwear, comprising the steps of:

viewing a proxy on a display device, the proxy existing in a real-world environment, the proxy including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device, and the proxy having a shape of the article of footwear;

contacting the proxy in the real-world environment with the interacting device along a surface of the proxy, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy;

physically rotating, during the contacting, the proxy in the real-world environment while maintaining contact between the interacting device and the proxy during the rotation of the proxy;

viewing a virtual line on the display device after contacting the proxy in the real-world environment with the interacting device, the virtual line corresponding to the contacting of the proxy with the interacting device along the surface on the proxy during the step of physically rotating the proxy;

wherein the virtual line is displayed on the display device so as to appear to be a part of the proxy;

wherein orientation and position of the virtual line is displayed on the display device so as to appear to be a part of the proxy as the proxy is rotated in the real-world environment; and

wherein the virtual line extends across the proxy between a medial side and a lateral side while the proxy is being rotated.

22. The method according to claim 21, wherein the virtual line is applied directly to the surface of the proxy.

23. The method according to claim 21, wherein the virtual line is a curved line that corresponds to the contours of the proxy.

24. The method according to claim 21, further comprising:

viewing, on the display device, a virtual menu in a location in the real-world environment;

physically placing the interacting device in the location to select a virtual design element;

wherein the virtual line has been applied in the virtual design based on the selection of the virtual design element; and

wherein the step of viewing the virtual line has been based on the virtual line being applied in the virtual design.

25. The method according to claim 21, wherein a thickness of the virtual line can be modified.

26. The method according to claim 21, wherein two or more virtual lines are applied to the proxy.

27. The method according to claim 21, wherein a virtual graphic is applied to the proxy using the interacting device.

28. The method according to claim 21, wherein the sensors are disposed in the medial side of the proxy and the sensors are disposed on the lateral side of the proxy.

29. The method according to claim 21, wherein the proxy includes at least one marking indicating the location of one or more of a lacing region and a sole of the article of footwear.

30. A method of designing an article of footwear, comprising the steps of:

contacting a proxy in a real-world environment with an interacting device along a surface of the proxy, the proxy having a shape of the article of footwear and including sensors disposed in the proxy that are configured to detect interactions between portions of the proxy with an interacting device, wherein the contacting has been detected by a sensor of the sensors disposed in the proxy;

physically rotating, during the contacting, the proxy in the real-world environment while maintaining contact between the interacting device and the proxy during the rotation of the proxy;

viewing a projection of a virtual line on the proxy after the interacting device has contacted the proxy in the real-world environment;

wherein a location of the virtual line corresponds with movement of the interacting device along the surface of the proxy;

wherein the virtual line is projected onto the proxy so as to appear to be a part of the proxy;

wherein orientation and position of the virtual line is projected onto the proxy as the proxy is rotated in the real-world environment; and

wherein the virtual line extends across the proxy between a medial side and a lateral side while the proxy is being rotated.

31. The method according to claim 30, wherein the virtual line is projected directly onto the surface of the proxy.

32. The method according to claim 30, wherein the virtual line has a shape that corresponds to a shape of the proxy.

33. The method according to claim 30, wherein the virtual line has a curved shape.

34. The method according to claim 30, further comprising:

viewing, on the projection, another projection, or a combination thereof, a virtual menu in the real-world environment;

physically moving the interacting device to intersect with the virtual menu in the real-world environment;

wherein the virtual line has been applied in the virtual design based on the step of physically moving the interacting device to intersect with the virtual menu in the real-world environment; and

wherein the step of viewing the projection of the virtual line has been based on the virtual line being applied in the virtual design.

35. The method according to claim 30, wherein the virtual line has a nonlinear shape.

36. The method according to claim 30, wherein multiple virtual lines are projected onto to the proxy.

37. The method according to claim 30, wherein the sensors are disposed in the medial side of the proxy and the sensors are disposed on the lateral side of the proxy.

38. The method according to claim 30, wherein the proxy includes at least one marking indicating the location of one or more of a lacing region and a sole of the article of footwear.

39. An augmented reality design system, comprising:

an imaging device for receiving images from a real-world environment;

a proxy existing in the real world environment, the proxy having a shape of an article of footwear and the proxy including sensors disposed in the proxy, wherein the sensors are disposed in a medial side of the proxy and the sensors are disposed on a lateral side of the proxy;

an interacting device for interacting with the proxy in the real world environment, wherein the sensors disposed in the proxy are configured to detect interactions between portions of the proxy with the interacting device;

a computing device capable of generating virtual images, at least one of the virtual images being based on at least one detection of a contact point by a sensor of the sensors disposed in the proxy;

a display device, wherein the at least one virtual image is superimposed onto an image, received by the imaging device, of the proxy;

a virtual menu that may be displayed on the display device, wherein the virtual menu is associated with a location in the real world environment and wherein a user can interact with the virtual menu by placing the interacting device in the location;

wherein the virtual menu can be used to apply a virtual design element to the proxy at a location on the proxy corresponding with a contact point of the interacting device with the proxy in the real world environment;

wherein the sensors disposed in the proxy detect the location on the proxy corresponding with the contact point of the interacting device with a surface of the proxy in the real-world environment;

wherein the proxy is physically rotated in the real-world environment while contact between the interacting device and the proxy is maintained; and

wherein the at least one virtual image extends across the proxy between the medial side and the lateral side while the proxy is being rotated.

40. The design system according to claim 39, wherein the virtual menu includes a color palette.

41. The design system according to claim 39, wherein the virtual menu includes a drawing tool.

42. The design system according to claim 39, wherein the virtual menu includes a letter tool.

43. The design system according to claim 39, wherein the virtual menu includes a virtual graphic.

44. The design system according to claim 39, wherein the virtual menu includes an eraser tool.

45. The design system according to claim 39, wherein the virtual menu includes a selection tool.

46. The design system according to claim 39, wherein the proxy further includes sensors disposed in the proxy that are configured to detect the location and orientation of the proxy as the proxy is rotated within the real world environment.

47. An augmented reality design system, comprising:

an imaging device for receiving images from a real-world environment;

a proxy existing in a real world environment, the proxy having a shape of an article of footwear and the proxy including sensors disposed in the proxy, wherein the sensors are disposed in a medial side of the proxy and the sensors are disposed on a lateral side of the proxy;

an interacting device for interacting with the proxy in the real world environment, wherein the sensors disposed in the proxy are configured to detect interactions between portions of the proxy with the interacting device;

a computing device capable of generating virtual images, at least one of the virtual images being based on at least one detection of a contact point by the sensors disposed in the proxy;

a projecting device, wherein the at least one virtual image is projected onto the proxy;

a virtual menu that may be projected onto a location in the real-world environment, wherein a user can interact with the virtual menu by placing the interacting device in the location;

wherein the virtual menu can be used to apply a virtual design element to the proxy at a location on the proxy corresponding with a contact point of the interacting device with the proxy in the real world environment;

wherein the sensors detect the location on the proxy corresponding with the contact point of the interacting device with the proxy in the real world environment;

wherein the proxy is physically rotated in the real-world environment while contact between the interacting device and the proxy is maintained; and

wherein the at least one virtual image extends across the proxy between the medial side and the lateral side while the proxy is being rotated.

48. The design system according to claim 47, wherein the virtual design element is a virtual line.

49. The design system according to claim 47, wherein the virtual design element corresponds to a color.

50. The design system according to claim 47, wherein the virtual design element is a virtual graphic.

51. The design system according to claim 47, wherein the proxy further includes sensors disposed in the proxy that are configured to detect the location and orientation of the proxy as the proxy is rotated within the real world environment.