Gesture mapping for image filter input parameters

• US 10,191,636 B2
• Filed: 12/01/2016
• Issued: 01/29/2019
• Est. Priority Date: 01/11/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hayward, David, Zhang, Chendi, Naaman, Alexandre, Dellinger, Richard R., Murthy, Giridhar S
• Primary Examiner(s): McDowell, Jr., Maurice L.
• Application Number:
  US15/366,469
• Publication Number:
  US 20170083218A1
• Time in Patent Office: 789 Days
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

This disclosure pertains to systems, methods, and computer readable medium for mapping particular user interactions, e.g., gestures, to the input parameters of various image processing routines, e.g., image filters, in a way that provides a seamless, dynamic, and intuitive experience for both the user and the software developer. Such techniques may handle the processing of both “relative” gestures, i.e., those gestures having values dependent on how much an input to the device has changed relative to a previous value of the input, and “absolute” gestures, i.e., those gestures having values dependent only on the instant value of the input to the device. Additionally, inputs to the device beyond user-input gestures may be utilized as input parameters to one or more image processing routines. For example, the device'"'"'s orientation, acceleration, and/or position in three-dimensional space may be used as inputs to particular image processing routines.

Claims

1. A method, comprising:

obtain, by an electronic device, an image;

detecting, by the electronic device, a representation of a human face within the image;

selecting, by the electronic device, a focus area within the image corresponding to the representation of the human face;

receiving, by the electronic device, a selection of a first filter to apply to the image;

receiving input from one or more sensors integrated with the electronic device;

determining a value of a first input parameter for the first filter from the received input; and

applying the first filter to the focus area of the image to generate a first filtered image, the first input parameter having the determined value.

2. The method of claim 1, further comprising:

receiving input selecting a location within the image corresponding to the representation of the human face; and

in response to receiving input selecting the location, selecting a particular area of the image corresponding to the representation of the human face as the focus area for applying image filters.

3. The method of claim 1, further comprising:

storing the image within a machine-readable storage device communicatively coupled to the electronic device; and

storing, separate from the image, metadata including the first filter and the first input parameter used to generate the first filtered image.

4. The method of claim 3, wherein applying the first filter to the image to generate the first filtered image generates a low resolution output;

and

wherein the method further comprises displaying, on a display device coupled to the electronic device, the low resolution output.

5. The method of claim 1, wherein receiving input from one or more sensors in communication with the electronic device comprises receiving gesture input from an electronic device having a touch-sensitive display.

6. The method of claim 1, wherein receiving a selection of a first filter to apply to an image comprises receiving a selection of a first filter to apply to an image frame of a video stream of images.

7. The method of claim 1, wherein determining the value of the first input parameter for the first filter from the received input comprises determining the value from analysis of a relative gesture.

8. The method of claim 1, wherein determining the first input parameter for the first filter from the received input comprises determining the value from analysis of an absolute gesture.

9. The method of claim 1, wherein the act of determining the value of the first input parameter from the received input comprises applying a translation to the received input, wherein the translation applied is based on the received input.

10. The method of claim 9, wherein the electronic device comprises a plurality of cameras, and wherein the translation applied to the received input is based on the camera used by the electronic device to capture the image.

11. The method of claim 1, wherein the image comprises a high-resolution image and a low-resolution image, and wherein applying the first filter to the image comprises applying the first filter to both the high-resolution and low-resolution images, wherein applying the first filter to the high-resolution image is performed by a CPU (central processing unit) of the electronic device, and wherein applying the first filter to the low-resolution image is performed by a GPU (graphics processing unit) of the electronic device.

12. An apparatus comprising:

an image sensor for capturing an image representative of a scene;

a display;

a memory in communication with the image sensor; and

a programmable control device communicatively coupled to the image sensor, the display, and the memory, wherein the memory includes instructions for causing the programmable control device to perform operations comprising;

detecting a representation of a human face within the image;

selecting a focus area within the image corresponding to the representation of the human face;

receiving a selection of a first filter to apply to the image;

receiving input from one or more sensors in communication with the apparatus;

associating a first input parameter for the first filter with the received input;

determining a value of the first input parameter based on analysis of the received input; and

applying the first filter to the focus area of the image to generate a first filtered image, the first input parameter having the determined value.

13. The apparatus of claim 12, wherein the operations include:

receiving input selecting a location within the image corresponding to the representation of the human face; and

in response to receiving input selecting the location, selecting a particular area of the image corresponding to the representation of the human face as the focus area for applying image filters.

14. The apparatus of claim 12, wherein the display comprises a touch-sensitive display and wherein the received input comprises gesture input.

15. The apparatus of claim 14, further comprising a storage device in communication with the programmable control device, and wherein the operations further comprise:

storing the first filtered image within the storage device; and

storing, separate from the image, metadata corresponding to the first filtered image, the meta data including the first filter and the first input parameter used to generate the first filtered image.

16. The apparatus of claim 12, wherein the programmable control device further comprises a CPU (central processing unit) and a GPU (graphics processing unit);

wherein the operation for receiving the image includes receiving a high-resolution image and a low-resolution image;

wherein the operation for applying the first filter to the image comprises applying the first filter to at least one of the high-resolution image and the low-resolution image; and

wherein an operation applying the first filter to the high-resolution image is performed by the CPU, and an operation applying the first filter to the low-resolution image is performed by the GPU.

17. The apparatus of claim 12 further comprising a plurality of cameras;

wherein the operation of determining the value of the first input parameter from the received input comprises applying a translation to the received input, wherein the translation applied is based on the received input; and

wherein the translation applied to the received input is based on the camera of the plurality of cameras used by the electronic device to capture the image.

18. A non-transitory computer readable storage medium including computer readable program code stored thereon, wherein the computer readable program code, when executed by an electronic device having one or more processors, causes the electronic device to perform operations comprising:

obtain, by an electronic device, an image;

detecting, by the electronic device, a representation of a human face within the image;

selecting, by the electronic device, a focus area within the image corresponding to the representation of the human face;

receiving a selection of a first filter to apply to the image;

receiving input from one or more sensors integrated with the electronic device;

determining a value of a first input parameter for the first filter from the received input; and

applying the first filter to the image to generate a first filtered image, the first input parameter having the determined value.

19. The computer readable storage medium of claim 18, wherein the operations further comprise:

storing the image within a machine-readable storage device communicatively coupled to the electronic device; and

storing, separate from the image, metadata including the first filter and the first input parameter used to generate the first filtered image.

20. The computer readable storage medium of claim 18, wherein the operations comprise:

receiving input selecting a location within the image corresponding to the representation of the human face; and

in response to receiving input selecting the location, selecting a particular area of the image corresponding to the representation of the human face as the focus area for applying image filters.