Real-time image capture and manipulation based on streaming data

• US 8,085,318 B2
• Filed: 10/11/2005
• Issued: 12/27/2011
• Est. Priority Date: 10/11/2005
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ciudad, Jean-Pierre, Bellandi, Luke, Christie, Gregory N., Dellinger, Richard, Forstall, Scott, Matas, Mike, Misaki, Erika, Tiene, Kevin, van Os, Marcel M W A, Lemay, Steve
• Primary Examiner(s): SELBY, GEVELL V
• Application Number:
  US11/248,630
• Publication Number:
  US 20070081740A1
• Time in Patent Office: 2,268 Days
• Field of Search: 348/239, 348/333.05, 348/333.11, 715/810
• US Class Current: 348/239
• CPC Class Codes: H04N 1/00286 with studio circuitry, devices or equipment, e.g. television cameras television studio circuitry, devices or equipment per se H04N5/222
• CPC Class Codes: H04N 1/00289 in a video photo booth or the like
• CPC Class Codes: H04N 1/00442 Simultaneous viewing of a plurality of images, e.g. using a mosaic display arrangement of thumbnails
• CPC Class Codes: H04N 1/00448 horizontally
• CPC Class Codes: H04N 1/00453 arranged in a two dimensional array
• CPC Class Codes: H04N 1/00458 Sequential viewing of a plurality of images, e.g. browsing or scrolling
• CPC Class Codes: H04N 1/00461 marking or otherwise tagging one or more displayed image, e.g. for selective reproduction
• CPC Class Codes: H04N 1/6013 by simulating several colour corrected versions of the same image simultaneously on the same picture reproducer

Abstract

Systems and techniques to provide image capture and manipulation. In general, in one implementation, the technique includes receiving an input stream including image data from a source, displaying the input stream in real-time including displaying a plurality of instantiations of the stream at a same time, each stream different, the step of displaying including applying a filter to each instantiation of the input stream, and receiving a prompt to select one of the instantiations of the stream.

Claims

1. A computer-implemented method, comprising:

providing for display a mosaic having a first cell and a second cell, the first cell displaying a video with a first visual effect, the second cell displaying the video with a second visual effect, the second visual effect being different from the first visual effect;

receiving a selection of the first cell and the second cell, the selection including a drag and drop input that drags the first cell and drops the first cell onto the second cell;

creating a third visual effect, the third visual effect including both the first visual effect and the second visual effect; and

capturing at least one frame of the video; and

applying the third visual effect to the captured frame of the video.

2. The method of claim 1, comprising:

illuminating a subject of the video using a flash of a user interface displaying the mosaic.

3. The method of claim 2, wherein applying the third visual to the captured frame comprises:

selecting the frame from the video, the frame substantially coinciding in time with the flash of the user interface; and

applying the third visual effect to the selected frame.

4. The method of claim 1, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulation, blurring, sharpening, distorting, geometric transforming, and pattern generating.

5. The method of claim 1, comprising providing for display controls operable to send a captured frame of the video as input to an application program.

6. The method of claim 5, wherein the application program is an instant messaging application and the captured frame is used as a representation of a participant in an instant messaging session.

7. The method of claim 1, comprising providing for display an image strip displaying a sequence of captured frames, the image strip operable to display different portions of the sequence of captured frames.

8. The method of claim 1, wherein the video includes at least one of a digital image or a video feed.

9. The method of claim 8, wherein the video feed is a live video feed.

10. The method of claim 9, wherein:

the mosaic is displayed on a display of a computing device; and

the live video feed is captured by a camera device coupled to or embedded in the computing device, the camera device being oriented in such a way that a user of the computing device is in a front of the camera device.

11. A computer-readable medium having stored thereon instructions, which, when executed by a processor, causes the processor to perform operations comprising:

providing for display a mosaic having a first cell and a second cell, the first cell displaying a portion of a live video feed of a subject with a first visual effect, the second cell displaying the portion of the live video feed with a second visual effect, the second visual effect being different from the first visual effect;

receiving a selection of the first cell and the second cell, the selection including a drag and drop input that drags the first cell and drops the first cell onto the second cell;

creating a third visual effect, the third visual effect including both the first visual effect and the second visual effect;

receiving an image capture input;

in response to the image capture input, capturing from the live video feed a digital image;

applying the third visual effect to the digital image; and

providing for display the captured digital image with the third visual effect.

12. The computer-readable medium of claim 11, the operations comprising:

illuminating the subject using a flash of a user interface in which the mosaic is displayed.

13. The computer-readable medium of claim 12, wherein applying the third visual effect to the digital image comprises:

selecting a frame from the portion of the video feed, the frame substantially coinciding in time with the flash of the user interface; and

applying the third visual effect to the selected frame.

14. The computer-readable medium of claim 11, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulating, blurring, sharpening, distorting, geometric transforming, and pattern generating.

15. The computer-readable medium of claim 11, comprising:

providing for display controls operable to send one or more captured digital images as input to an application program.

16. The computer-readable medium of claim 15, wherein the application program is an instant messaging application and the captured digital image includes a participant in an instant messaging session.

17. The computer-readable medium of claim 11, comprising:

providing for display an image strip displaying a sequence of captured digital images, the image strip operable to display different portions of the sequence of captured digital images.

18. The computer-readable medium of claim 11, wherein:

the mosaic is displayed on a display device coupled to the processor;

the live video feed is captured by a camera device coupled to the processor; and

the subject is a user, where the camera device is configured such that the user is in a front of the camera device.

19. A system comprising:

a digital image capture device; and

a processor operatively coupled to the digital capture device, the processor configurable to perform operations comprising;

providing for display a mosaic having a first cell and a second cell, the first cell displaying a video with a first visual effect, the second cell displaying the video with a second visual effect, the second visual effect being different from the first visual effect;

receiving a selection of the first cell and the second cell, the selection including a drag and drop input that drags the first cell and drops the first cell onto the second cell;

creating a third visual effect, the third visual effect including both the first visual effect and the second visual effect;

capturing from the video a digital image; and

applying the third visual effect to the digital image.

20. The system of claim 19, the operations comprising:

illuminating a subject of the video using a flash of a user interface displaying the mosaic.

21. The system of claim 20, wherein applying the third visual effect to the digital image comprises:

selecting a frame from the video, the frame substantially coinciding in time with the flash of the user interface; and

applying the third visual effect to the selected frame.

22. The system of claim 19, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulating, blurring, sharpening, distorting, geometric transforming, and pattern generating.

23. The system of claim 19, the operations comprising:

providing for display controls operable to send one or more captured digital images as input to an application program.

24. The system of claim 23, wherein the application program is an instant messaging application and the captured digital image includes a participant in an instant messaging session.

25. The system of claim 19, the operations comprising:

providing for display an image strip displaying a sequence of captured digital images, the image strip operable to display different portions of the sequence of captured digital images.

26. The system of claim 19, wherein the video includes at least one of a still image and a video feed.

27. The system of claim 26, wherein the video feed is a live video feed.

28. The system of claim 27, wherein:

the mosaic is displayed on a display device of the system;

the live video feed is captured by the digital image capture device; and

a subject of the video feed includes a user of the system.

29. A method executed by a computing device, comprising:

receiving at least a portion of a video feed;

providing for display a mosaic in a single user interface of a single display device, the mosaic including a first cell a second cell, and a third cell, the first cell displaying the portion of the video with a first visual effect, the second cell displaying the portion of the video feed with a second visual effect that differs from the first visual effect, the third cell displaying the portion of the video feed without a visual effect or with a null effect;

receiving a selection input from the single user interface, the selection input selecting at least one of the first cell or second cell in the mosaic; and

upon receiving the selection input;

switching the mosaic to a display of the video feed, including applying the visual effect of the selected cell to the video feed; and

capturing from the video feed a digital image, including applying the identified visual effect to the digital image.

30. The method of claim 29, where the mosaic includes a plurality of rows of cells and a plurality of columns of cells that display the portion of the video feed.

31. The method of claim 30, where the mosaic includes a square grid of nine cells and a center cell of the square grid displays the portion of the video feed without a visual effect or with the null effect.

32. The method of claim 29, comprising:

providing for display, in the single user interface, a control that is operable for selecting a mosaic from a set of mosaics to be displayed in the single user interface, where each mosaic is associated with a different set of visual effects.

33. The method of claim 29, where the video feed is a live video feed.

34. The method of claim 29, comprising illuminating a subject of the video using a flash of a user interface displaying the mosaic.

35. The method of claim 29, comprising providing for display a control operable to send one or more captured digital images as input to an application program.

36. The method of claim 35, wherein the application program is an instant messaging application and the captured digital image includes a participant in an instant messaging session.

37. The method of claim 29, comprising providing for display an image strip displaying a sequence of captured digital images, the image strip operable to display different portions of the sequence of captured digital images.

38. The method of claim 29, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulating, blurring, sharpening, distorting, geometric transforming, and pattern generating.

39. A computer-readable medium having stored thereon instructions, which, when executed by a processor, causes the processor to perform operations comprising:

receiving at least a portion of a video feed;

providing for display a mosaic in a single user interface of a single display device, the mosaic including a first cell, a second cell, and a third cell, the first cell displaying the portion of the video with a first visual effect, the second cell displaying the portion of the video feed with a second visual effect that differs from the first visual effect, the third cell displaying the portion of the video feed without a visual effect or with a null effect;

receiving a selection input from the single user interface, the selection input selecting at least one of the first cell or second cell in the mosaic; and

upon receiving the selection input;

switching the mosaic to a display of the video feed, including applying the visual effect of the selected cell to the video feed; and

capturing from the video feed a digital image, including applying the identified visual effect to the digital image.

40. The medium of claim 39, where the mosaic includes a plurality of rows of cells and a plurality of columns of cells that display the portion of the video feed.

41. The medium of claim 40, where the mosaic includes a square grid of nine cells and a center cell of the square grid displays the portion of the video feed without a visual effect or with the null effect.

42. The medium of claim 39, the operations comprising:

providing for display, in the single user interface, a control that is operable for selecting a mosaic from a set of mosaics to be displayed in the single user interface, where each mosaic is associated with a different set of visual effects.

43. The medium of claim 39, where the video feed is a live video feed.

44. The medium of claim 39, the operations comprising illuminating a subject of the video using a flash of a user interface displaying the mosaic.

45. The medium of claim 39, the operations comprising providing for display a control operable to send one or more captured digital images as input to an application program.

46. The medium of claim 45, wherein the application program is an instant messaging application and the captured digital image includes a participant in an instant messaging session.

47. The medium of claim 39, the operations comprising providing for display an image strip displaying a sequence of captured digital images, the image strip operable to display different portions of the sequence of captured digital images.

48. The medium of claim 39, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulating, blurring, sharpening, distorting, geometric transforming, and pattern generating.

49. A system, comprising:

one or more computers configured to perform operations comprising;

receiving at least a portion of a video feed;

providing for display a mosaic in a single user interface of a single display device, the mosaic including a first cell, a second cell, and a third cell, the first cell displaying the portion of the video with a first visual effect, the second cell displaying the portion of the video feed with a second visual effect that differs from the first visual effect, the third cell displaying the portion of the video feed without a visual effect or with a null effect;

receiving a selection input from the single user interface, the selection input selecting at least one of the first cell or second cell in the mosaic; and

upon receiving the selection input;

switching the mosaic to a display of the video feed, including applying the visual effect of the selected cell to the video feed; and

capturing from the video feed a digital image, including applying the identified visual effect to the digital image.

50. The system of claim 49, where the mosaic includes a plurality of rows of cells and a plurality of columns of cells that display the portion of the video feed.

51. The system of claim 50, where the mosaic includes a square grid of nine cells and a center cell of the square grid displays the portion of the video feed without a visual effect or with the null effect.

52. The system of claim 49, the operations comprising:

providing for display, in the single user interface, a control that is operable for selecting a mosaic from a set of mosaics to be displayed in the single user interface, where each mosaic is associated with a different set of visual effects.

53. The system of claim 49, where the video feed is a live video feed.

54. The system of claim 49, the operations comprising illuminating a subject of the video using a flash of a user interface displaying the mosaic.

55. The system of claim 49, the operations comprising providing for display a control operable to send one or more captured digital images as input to an application program.

56. The system of claim 55, wherein the application program is an instant messaging application and the captured digital image includes a participant in an instant messaging session.

57. The system of claim 49, the operations comprising providing for display an image strip displaying a sequence of captured digital images, the image strip operable to display different portions of the sequence of captured digital images.

58. The system of claim 49, wherein the first visual effect and the second visual effect include at least one of:

cropping, flipping, adding one or more borders, adding one or more backdrops, scaling, color manipulating, blurring, sharpening, distorting, geometric transforming, and pattern generating.