Visualizing a custom product in situ

• US 9,436,963 B2
• Filed: 02/14/2014
• Issued: 09/06/2016
• Est. Priority Date: 08/31/2011
• Status: Active Grant

• Current Assignee: Zazzle, Inc.
• Original Assignee: Zazzle, Inc.
• Inventors: Beaver, Robert Irven III, Harvill, Leslie Young, Bean, Richard Harold
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Ge, Jin
• Application Number:
  US14/181,544
• Publication Number:
  US 20140160118A1
• Time in Patent Office: 935 Days
• Field of Search: 345/419
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0621 Item configuration or customization
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2210/04 Architectural design, interior design

Abstract

Techniques are described for visualizing a product at the actual location in the environment at which the product is to be used or displayed. An embodiment of the approaches described herein may be used in the context of a computer-based system that can receive and store digital images, receive a request to manufacture a custom framed product including an identification of an image to be framed and a type of mat and/or frame, and display a preview image of the custom framed product that simulates the actual appearance of the product as closely as possible. With such a system, the preview image may be highly realistic under idealized lighting and display conditions. A digital asset visualizing the custom product may be built using a data that maps nodes of a reference connectivity graph of a marker to nodes of a candidate connectivity graph.

Claims

1. A method for visualizing a custom product in situ, the method comprising:

storing first data that represents a reference connectivity graph of a marker;

obtaining a digital image of at least the marker;

analyzing the digital image to generate second data that represents a candidate connectivity graph;

based on the first data and the second data, determining whether for each first node, in the candidate connectivity graph, a first count of first polytopes that are adjacent to a first node in the candidate connectivity graph matches a second count of second polytopes that are adjacent to a second node, corresponding to the first node, in the reference connectivity graph, and if so;

generating third data that comprises at least a transform that maps coordinates of nodes of the reference connectivity graph to coordinates of nodes of the candidate connectivity graph;

using at least the third data to build a digital asset that visualizes the custom product in the digital image;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the first data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

3. The method of claim 1, wherein the second data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

4. The method of claim 1, wherein the marker comprises one or more colored open spaces for aiding a digital image analysis technique applied to the digital image in detecting lighting in an environment in which the marker was photographed.

5. The method of claim 4, wherein at least one of the one or more colored open spaces is colored in a medium tone gray or a pastel color tone for aiding the digital image analysis technique in detecting color bias of lighting in the environment in which the marker was photographed.

6. The method of claim 1, further comprising applying a thresholded bandpass filter or an edge filter to the digital image to produce a digital image that comprises linear features in a size range of lines of the marker as black on a white background.

7. The method of claim 6, further comprising using a rule-based cellular automata with thresholded neighborhood inputs to thin at least one of the linear features.

8. The method of claim 1, further comprising determining an orientation or position of the marker in the digital image using the third data.

9. The method of claim 8, further comprising using a point mapping technique involving singular value decomposition to determine the third data.

10. The method of claim 1, further comprising using the third data to transform coordinates of a light sampling point in a coordinate space of the marker to an equivalent point in a coordinate space of the digital image.

11. The method of claim 10, further comprising sampling pixel values at the equivalent point to determine a baseline white value.

12. The method of claim 1, wherein building the digital asset comprises placing a custom product reference into the digital image using the third data for placement.

13. The method of claim 1, further comprising:

analyzing the digital image to detect one or more light sampling points and one or more dark sampling points;

determining a lighting gradient that exists between certain sampling points;

modifying luminance of the custom product to match the lighting gradient.

14. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause performance of a method for visualizing a custom product in situ, the method comprising:

storing first data that represents a reference connectivity graph of a marker;

obtaining a digital image of at least the marker;

analyzing the digital image to generate second data that represents a candidate connectivity graph;

based on the first data and the second data, determining whether for each first node, in the candidate connectivity graph, a first count of first polytopes that are adjacent to a first node in the candidate connectivity graph matches a second count of second polytopes that are adjacent to a second node, corresponding to the first node, in the reference connectivity graph, and if so;

generating third data that comprises at least a transform that maps coordinates of nodes of the reference connectivity graph to coordinates of nodes of the candidate connectivity graph;

using at least the third data to build a digital asset that visualizes the custom product in the digital image;

wherein the method is performed by one or more computing devices.

15. The one or more non-transitory computer-readable media of claim 14, wherein the first data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

16. The method of claim 14, wherein the second data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

17. The one or more non-transitory computer-readable media of claim 14, wherein the marker comprises one or more colored open spaces for aiding a digital image analysis technique applied to the digital image in detecting lighting in an environment in which the marker was photographed.

18. The one or more non-transitory computer-readable media of claim 17, wherein at least one of the one or more colored open spaces is colored in a medium tone gray or a pastel color tone for aiding the digital image analysis technique in detecting color bias of lighting in the environment in which the marker was photographed.

19. The one or more non-transitory computer-readable media of claim 14, the method further comprising applying a thresholded bandpass filter or an edge filter to the digital image to produce a digital image that comprises linear features in a size range of lines of the marker as black on a white background.

20. The one or more non-transitory computer-readable media of claim 19, the method further comprising using a rule-based cellular automata with thresholded neighborhood inputs to thin at least one of the linear features.

21. The one or more non-transitory computer-readable media of claim 14, the method further comprising determining an orientation or position of the marker in the digital image using the third data.

22. The method of claim 21, the method further comprising using a point mapping technique involving singular value decomposition to determine the third data.

23. The one or more non-transitory computer-readable media of claim 14, the method further comprising using the third data to transform coordinates of a light sampling point in a coordinate space of the marker to an equivalent point in a coordinate space of the digital image.

24. The one or more non-transitory computer-readable media of claim 23, the method further comprising sampling pixel values at the equivalent point to determine a baseline white value.

25. The one or more non-transitory computer-readable media of claim 14, wherein building the digital asset comprises placing a custom product reference into the digital image using the third data for placement.

26. The one or more non-transitory computer-readable media of claim 14, the method further comprising:

analyzing the digital image to detect one or more light sampling points and one or more dark sampling points;

determining a lighting gradient that exists between certain sampling points;

modifying luminance of the custom product to match the lighting gradient.