Previewing a graphic in an environment

• US 9,013,507 B2
• Filed: 03/04/2011
• Issued: 04/21/2015
• Est. Priority Date: 03/04/2011
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Plante, Jean-Frederic, Edmondson, David, Wiesner, Eric G
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US13/040,599
• Publication Number:
  US 20120223961A1
• Time in Patent Office: 1,509 Days
• Field of Search: 345/632, 345/633
• US Class Current: 345/632
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04N 1/00411 the display also being used for user input, e.g. touch screen
• CPC Class Codes: H04N 1/0044 for image preview or review, e.g. to help the user position a sheet
• CPC Class Codes: H04N 1/00469 with enlargement of a selected area of the displayed information
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3261 of multimedia information, e.g. a sound signal
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A method includes defining a surface within a first captured image of an environment. The defined surface is identified in a second captured image of the environment. A graphic is overlaid on the surface identified in the second captured image. The second captured image is caused to be displayed to preview the graphic in the environment.

Claims

1. A non-transitory computer readable medium storing computer executable instructions that when executed cause a system including a processor to:

define a wall within a first captured image of an environment using a feature detection model, wherein the feature detection model detects lines in the first captured image and calculates intersections between the detected lines to define the wall in the first captured image, the first captured image captured by an image capture device at a first time;

identify the defined wall in a second captured image of the environment, the second captured image captured by the image capture device at a second time different from the first time;

overlay a graphic on the identified wall in the second captured image, wherein the graphic includes a predefined zone associated with an action;

cause a display of the second captured image with the graphic overlaying the identified wall to allow the graphic to be previewed in the environment; and

trigger the action in response to detecting user interaction with a corresponding zone in the displayed second captured image that corresponds to the predefined zone.

2. The medium of claim 1, wherein detecting the user interaction comprises determining that at least a portion of the corresponding zone falls within a predetermined position within the second captured image.

3. The medium of claim 1, wherein the overlaying includes:

identifying a geometry of the identified wall in the second captured image;

modifying the graphic to match the geometry; and

overlaying the modified graphic on the identified wall in the second captured image.

4. The medium of claim 1, wherein the second captured image is one of a series of captured images of a video of the environment captured by the image capture device, wherein the instructions when executed cause the system to, following capture of each respective image of the series:

identify the defined wall in the respective image and overlay the graphic on the wall identified in the respective image; and

cause a display of the respective image that includes the graphic overlaying the wall identified in the respective image.

5. The medium of claim 4, wherein the instructions when executed cause the system to adjust an opacity of the graphic and wherein overlaying the graphic on the wall identified in the respective image comprises overlaying the graphic such that at least a portion of the wall identified in the respective image bleeds through the graphic and is visible when the respective image is displayed.

6. The medium of claim 4, wherein overlaying the graphic on the wall identified in the respective image comprises identifying a geometry of the wall identified in the respective image, modifying the graphic to match the geometry, and overlaying the modified graphic on the wall identified in the respective image.

7. The medium of claim 4, wherein the captured images in the series have different views of the environment, and wherein overlaying the graphic in each of the captured images in the series allows for previews of the graphic in the different views of the environment.

8. The medium of claim 1, wherein the instructions are for execution by the image capture device that comprises the system.

9. The medium of claim 1, wherein the first captured image is captured by the image capture device set in a still image capture mode, and the second captured image is captured by the image capture device set in a video mode.

10. A system, comprising:

a surface engine to;

define, using a feature detection process, a surface within a first captured image of an environment captured by an image capture device at a first time, wherein the feature detection process detects lines in the first captured image and calculates intersections between the detected lines to define the surface in the first captured image, and

identify the defined surface in a second captured image of the environment captured by the image capture device at a second time different from the first time;

a graphic engine to overlay a graphic on the identified surface in the second captured image, wherein the graphic includes a predefined zone associated with an action; and

a display engine to cause a display of the second captured image with the graphic overlaying the identified surface to allow the graphic to be previewed in the environment;

a zone engine to detect user interaction with a zone in the displayed second captured image that corresponds to the predefined zone; and

an action engine to trigger the action associated with the predefined zone following the zone engine detecting the user interaction.

11. The system of claim 10, wherein the zone engine is to detect the user interaction by determining that at least a portion of the zone falls within a predetermined position within the second captured image.

12. The system of claim 10, wherein the graphic engine is to adjust an opacity of the graphic and overlay the graphic on the identified surface in the second captured image such that at least a portion of the identified surface bleeds through the graphic and is visible when the second captured image is displayed.

13. The system of claim 10, wherein the second captured image is one of a series of captured images of a video captured by the image capture device, wherein, following capture of each respective image of the series:

the surface engine is to identify the defined surface in the respective image;

the graphic engine is to overlay the graphic on the surface identified in the respective image; and

the display engine is to cause a display of the respective image that includes the graphic overlaying the surface identified in the respective image.

14. The system of claim 13, wherein the graphic engine is, for each respective image of the series, to identify a geometry of the surface identified in the respective image, modify the graphic to match the geometry, and overlay the modified graphic on the surface identified in the respective image.

15. The system of claim 13, wherein the captured images in the series have different views of the environment, and wherein overlaying the graphic in each of the captured images in the series allows for previews of the graphic in the different views of the environment.

16. The system of claim 10, wherein the first captured image is captured by the image capture device set in a still image capture mode, and the second captured image is captured by the image capture device set in a video mode.

17. A method comprising:

receiving a first image of an environment captured by an image capture device at a first time;

defining, by a system including a processor, a surface within the first image of the environment;

receiving a second image of the environment captured by the image capture device at a second time different from the first time;

identifying the defined surface in the second image of the environment;

overlaying, by the system, a graphic on the identified surface in the second image, wherein the graphic includes a predefined zone associated with an action; and

causing, by the system, display of the second image with the graphic overlaying the identified surface to allow the graphic to be previewed in the environment;

detecting user interaction with a zone in the displayed second image that corresponds to the predefined zone; and

triggering the action associated with the predefined zone in response to detecting the user interaction.

18. The method of claim 17 wherein the first image is a still image captured by the image capture device in a still image capture mode, and the second image is one of a series of images of a video captured by the image capture device in a video mode.

19. The method of claim 18, further comprising:

for each respective image in the series of images, identifying the defined surface in the respective image, and overlaying the graphic on the surface identified in the respective image; and

causing sequential display of the series of images each with the graphic overlaying the surface identified within that image.

20. The method of claim 19, wherein the images in the series have different views of the environment, and wherein overlaying the graphic in each of the images in the series allows for previews of the graphic in the different views of the environment.

21. The method of claim 17, wherein the first captured image is captured by the image capture device set in a still image capture mode, and the second captured image is captured by the image capture device set in a video mode.