Method and apparatus for simulating the appearance of paving stone on an existing driveway

• US 6,975,334 B1
• Filed: 03/27/2003
• Issued: 12/13/2005
• Est. Priority Date: 03/27/2003
• Status: Expired due to Fees

• Current Assignee: System Planning Corporation
• Original Assignee: System Planning Corporation
• Inventors: Barrus, John W.
• Primary Examiner(s): Tung, Kee M.
• Assistant Examiner(s): Richer, Aaron M.
• Application Number:
  US10/402,284
• Time in Patent Office: 992 Days
• Field of Search: 345/633, 345/581, 345/582, 345/632, 345/420, 382/173, 382/175
• US Class Current: 345/632
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/50 Lighting effects

Abstract

Using a digital camera, special markers, and a computer system, an image is taken of an existing surface that is usually a driveway or a walkway. The boundary of the surface to be modified is manually or automatically selected and the markers are also either manually or automatically selected. The user selects a simulation material and, using the boundary and marker geometry, a new surface is calculated to cover the original surface in the image to give the old surface the appearance of being made of the new chosen material.

Claims

1. A method for simulating a portion of a surface in an image, the method comprising the steps of:

placing at least one marker on the surface wherein the marker has a first side and a second side, the first side of the marker having a first color and the second side having a second color, so that the marker will have contrast with the surface depending on which side is visible;

capturing an image of the surface including the marker;

determining a boundary that defines a field for the portion of the image;

determining a location of the marker;

calculating the geometry for the field;

selecting a material and texture for a simulated surface; and

generating a new image comprising the captured image and the simulated surface in the field.

2. The method of claim 1, wherein the step of placing at least one marker on the surface includes placing a plurality of markers on the surface.

3. The method of claim 1, wherein the step of placing at least one marker on the surface includes placing at least three markers on the surface.

4. The method of claim 1, wherein the marker has a first member and a second member both of a predefined length and the first member has an orthogonal position relative to the second member.

5. The method of claim 1, wherein the marker has a color that contrasts with a color of the surface.

6. The method of claim 1, wherein the marker has an elliptical shape.

7. The method of claim 1, wherein the step of capturing an image of the surface includes capturing an image of the surface with a digital camera.

8. The method of claim 1, wherein the step of capturing an image of the surface includes obtaining a photograph of the surface and the marker, and scanning the photograph with a digital scanner.

9. The method of claim 1, wherein the step of determining the boundary includes receiving input from a user to identify a plurality of segments in the image of the surface.

10. The method of claim 1, wherein the step of determining the boundary includes using computer vision techniques to automatically identify a plurality of segments based on edges or texture changes in the image of the surface that comprise the boundary.

11. The method of claim 1, wherein the step of determining the location of the marker includes the step of receiving input from a user to identify a first end point, a second end point and a knee-point of the marker.

12. The method of claim 1, wherein the step of determining the location of the marker includes using computer vision techniques to automatically identify the marker in the image of the surface.

13. The method of claim 1, wherein the step of determining the boundary further comprises the step of receiving additional input modifying the location of the segments in the image of the surface that comprise the boundary and wherein the step of determining the location of the marker further comprises the step of receiving additional input modifying the position of the points that identify the marker.

14. The method of claim 1, further comprising the step of automatically determining portions of the boundary that comprise a border.

15. The method of claim 1, further comprising the step of receiving input indicating portions of the boundary that comprise a border.

16. The method of claim 15, further comprising the step of generating the shape and geometry for the border.

17. The method of claim 1, further comprising the steps of:

receiving input indicating an object on top of the surface; and

rendering the indicated objects as an overlay upon the simulated surface portion of the new image.

18. The method of claim 1, further comprising the step of calculating a three-dimensional surface geometry for lighting.

19. The method of claim 1, further comprising the steps of:

receiving input indicating a position for shadows on top of the surface; and

rendering shadows as a transparent overlay upon the simulated surface portion of the new image.

20. The method of claim 1, further comprising the step of storing the new image to an image buffer.

21. The method of claim 1, further comprising the step of storing the new image on a non-volatile storage medium.

22. A method for simulating a portion of a surface in an image, the method comprising the steps of:

placing at least one marker on the surface;

capturing an image of the surface including the marker;

determining a boundary that defines a field for the portion of the image;

determining a location of the marker;

calculating a geometry for the field by

creating a region that contains the boundary;

creating a Delaunay triangulation of the region using the marker as a vertex;

calculating derivatives of direction vectors for the field;

determining a spacing grid for the field from the derivatives of direction; and

calculating perspective and inverse transforms for the field and spacing grid

selecting a material and texture for a simulated surface; and

generating a new image comprising the captured image and the simulated surface in the field.

26. The method of claim 25, further comprising the step of repeating the steps of claim 22 for an additional point on the spacing grid.

23. A method for simulating a portion of a surface in an image, the method comprising the steps of:

placing at least one marker on the surface;

capturing an image of the surface including the marker;

receiving input indicating portions of the boundary that comprise a border;

determining a boundary that defines a field for the portion of the image;

generating the shape and geometry for the border including

projecting the border from the image onto a spacing grid for the field;

creating an offset border curve offset a predetermined distance inward from the border;

creating at least one border polygon between the border and the offset border curve;

assigning a consistent texture value to each border polygon; and

transforming the border polygon into image space from grid space

determining a location of the marker;

calculating a geometry for the field;

selecting a material and texture for a simulated surface; and

generating a new image comprising the captured image and the simulated surface in the field.

24. The method of claim 23, wherein the step of generating the shape and geometry for the border further comprises the steps of:

dividing the border polygons into smaller polygons that are contained within a unit square of the spacing grid;

assigning a consistent texture value to each smaller polygon; and

wherein the step of transforming the border polygon, transforms the smaller polygons to image space from grid space.

25. A method for simulating a portion of a surface in an image, the method comprising the steps of:

receiving input indicating an object on top of the surface;

placing at least one marker on the surface;

capturing an image of the surface including the marker;

determining a boundary that defines a field for the portion of the image;

determining a location of the marker;

calculating a geometry for the field;

selecting a material and texture for a simulated surface;

calculating a three-dimensional surface geometry for lighting by

estimating a three-dimensional position of a point on a spacing grid based on a size of the marker;

creating a unit sphere having a center at a selected point on the spacing grid;

creating a three-dimensional projection line from an image viewpoint to an adjacent point on the spacing grid;

calculating an intersection of the projection line and the unit sphere to create a three-dimensional line segment;

calculating an orthogonal line segment based on another adjacent grid point; and

determining a surface normal at the selected point from a cross product of the three-dimensional line segment and the orthogonal line segment

generating a new image comprising the captured image and the simulated surface in the field; and

rendering the indicated objects as an overlay upon the simulated surface portion of the new image.