Visualizing a custom product in situ

• US 9,147,213 B2
• Filed: 07/02/2012
• Issued: 09/29/2015
• Est. Priority Date: 10/26/2007
• Status: Active Grant

• Current Assignee: Zazzle, Inc.
• Original Assignee: Zazzle, Inc.
• Inventors: Beaver, Robert Irven III, Harvill, Leslie Young, Bean, Richard Harold
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Ge, Jin
• Application Number:
  US13/539,788
• Publication Number:
  US 20130050218A1
• Time in Patent Office: 1,184 Days
• Field of Search: 345/440
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0621 Item configuration or customization
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2210/04 Architectural design, interior design

Abstract

Techniques are described for visualizing a product at the actual location in the environment at which the product is to be used or displayed. An embodiment of the approaches described herein may be used in the context of a computer-based system that can receive and store digital images, receive a request to manufacture a custom framed product including an identification of an image to be framed and a type of mat and/or frame, and display a preview image of the custom framed product that simulates the actual appearance of the product as closely as possible. With such a system, the preview image may be highly realistic under idealized lighting and display conditions.

Claims

1. A method for visualizing a custom product in situ, the method comprising:

analyzing a first digital image of at least a marker comprising non-uniform corners and the marker having one particular corner defined by two perpendicular lines in which one of the lines crosses another of the lines to detect at least one light sampling point and at least one dark sampling point within the first digital image;

determining a lighting gradient that exists between the at least one light sampling point and the at least one dark sampling point within the first digital image;

wherein the lighting gradient comprises a set of values representing a range of magnitudes of reflected light across the site;

modifying luminance of a second digital image of the custom product to match the lighting gradient that exists between the at least one light sampling point and the at least one dark sampling point within the first digital image;

determining an orientation of the marker by analyzing a visualization of the particular corner of the non-uniform corners of the marker and determining whether one line of the two perpendicular lines crosses another line of the two perpendicular lines at the particular corner;

building a visual asset that visualizes the custom product in the first digital image according to the determined orientation of the marker;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising:

storing first data that represents a reference connectivity graph of the marker;

analyzing the first digital image to generate second data that represents a candidate connectivity graph;

based at least in part upon first data and the second data, determining whether the candidate connectivity graph matches the reference connectivity graph;

in response to determining that the candidate connectivity graph matches the reference connectivity graph, generating third data that at least maps nodes of the reference connectivity graph to nodes of the candidate connectivity graph;

building the visual asset that visualizes the custom product in the first digital image using at least the third data.

3. The method of claim 2, wherein the second data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

4. The method of claim 2, wherein the marker comprises one or more colored open spaces for aiding a digital image analysis technique applied to the first digital image in detecting lighting in the environment in which the marker was photographed.

5. The method of claim 4, wherein at least one of the one or more colored open spaces is colored in a medium tone gray or a pastel color tone for aiding the digital image analysis technique in detecting color bias of lighting in the environment in which the marker was photographed.

6. The method of claim 1, further comprising applying a thresholded bandpass filter or an edge filter to the first digital image to produce a digital image that comprises linear features in a size range of lines of the marker as black on a white background.

7. The method of claim 6, further comprising using a rule-based cellular automata with thresholded neighborhood inputs to thin at least one of the linear features.

8. The method of claim 2, wherein generating the third data comprises calculating a marker transform that maps nodes of the reference connectivity graph to nodes of the candidate connectivity graph.

9. The method of claim 8, further comprising determining an orientation or position of the marker in the first digital image using the marker transform.

10. The method of claim 9, further comprising using a point mapping technique involving singular value decomposition to determine the marker transform.

11. The method of claim 8, further comprising using the marker transform to transform coordinates of a light sampling point in a coordinate space of the marker to an equivalent point in a coordinate space of the first digital image.

12. The method of claim 11, further comprising sampling pixel values at the equivalent point to determine a baseline white value.

13. The method of claim 8, wherein building the visual asset comprises placing a custom product reference into the first digital image using the marker transform for placement.

14. The method of claim 1, wherein the custom product is a custom framed product.

15. The method of claim 1, wherein the non-uniform corners of the marker are non-uniform with respect to a manner of arc intersection.

16. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause performance of a method for visualizing a custom product in situ, the method comprising:

analyzing a first digital image of at least a marker comprising non-uniform corners and the marker having one particular corner defined by two perpendicular lines in which one of the lines crosses another of the lines to detect at least one light sampling point and at least one dark sampling point within the first digital image;

determining a lighting gradient that exists between the at least one light sampling point and the at least one dark sampling point within the first digital image;

wherein the lighting gradient comprises a set of values representing a range of magnitudes of reflected light across the site;

modifying luminance of a second digital image of the custom product to match the lighting gradient that exists between the at least one light sampling point and the at least one dark sampling point within the first digital image;

determining an orientation of the marker by analyzing a visualization of the particular corner of the non-uniform corners of the marker and determining whether one line of the two perpendicular lines crosses another line of the two perpendicular lines at the particular corner;

building a visual asset that visualizes the custom product in the first digital image according to the determined orientation of the marker.

17. The one or more non-transitory computer-readable media of claim 16, the method further comprising:

storing first data that represents a reference connectivity graph of the marker;

analyzing the first digital image to generate second data that represents a candidate connectivity graph;

based at least in part upon first data and the second data, determining whether the candidate connectivity graph matches the reference connectivity graph;

in response to determining that the candidate connectivity graph matches the reference connectivity graph, generating third data that at least maps nodes of the reference connectivity graph to nodes of the candidate connectivity graph;

building the visual asset that visualizes the custom product in the first digital image using at least the third data.

18. The one or more non-transitory computer-readable media of claim 16, wherein the custom product is a custom framed product.

19. The one or more non-transitory computer-readable media of claim 17, wherein the first data indicates one or more of:

a count of the nodes of the reference connectivity graph,

a count of arcs connecting at a particular node of the reference connectivity graph,

a count of lines or arcs of the reference connectivity graph,

a count of polytopes of the reference connectivity graph, or

a count of nodes of a particular polytope of the reference connectivity graph.

20. The one or more non-transitory computer-readable media of claim 16, wherein the marker comprises one or more colored open spaces for aiding a digital image analysis technique applied to the first digital image in detecting lighting in the environment in which the marker was photographed.

21. The one or more non-transitory computer-readable media of claim 20, wherein at least one of the one or more colored open spaces is colored in a medium tone gray or a pastel color tone for aiding the digital image analysis technique in detecting color bias of lighting in the environment in which the marker was photographed.

22. The one or more non-transitory computer-readable media of claim 16, the method further comprising applying a thresholded bandpass filter or an edge filter to the first digital image to produce a digital image that comprises linear features in a size range of lines of the marker as black on a white background.

23. The one or more non-transitory computer-readable media of claim 22, the method further comprising using a rule-based cellular automata with thresholded neighborhood inputs to thin at least one of the linear features.

24. The one or more non-transitory computer-readable media of claim 17, wherein generating the third data comprises calculating a marker transform that maps nodes of the reference connectivity graph to nodes of the candidate connectivity graph.

25. The one or more non-transitory computer-readable media of claim 24, the method further comprising determining an orientation or position of the marker in the first digital image using the marker transform.

26. The one or more non-transitory computer-readable media of claim 25, the method further comprising using a point mapping technique involving singular value decomposition to determine the marker transform.

27. The one or more non-transitory computer-readable media of claim 26, the method further comprising using the marker transform to transform coordinates of a light sampling point in a coordinate space of the marker to an equivalent point in a coordinate space of the first digital image.

28. The one or more non-transitory computer-readable media of claim 27, the method further comprising sampling pixel values at the equivalent point to determine a baseline white value.

29. The one or more non-transitory computer-readable media of claim 24, wherein building the visual asset comprises placing a custom product reference into the first digital image using the marker transform for placement.

30. The one or more non-transitory computer-readable media of claim 16, wherein the non-uniform corners of the marker are non-uniform with respect to a manner of arc intersection.