Gesture mapping for image filter input parameters

• US 10,788,967 B2
• Filed: 01/29/2019
• Issued: 09/29/2020
• Est. Priority Date: 01/11/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hayward, David, Zhang, Chendi, Naaman, Alexandre, Dellinger, Richard R., Murthy, Giridhar S.
• Primary Examiner(s): McDowell, Jr., Maurice L.
• Application Number:
  US16/260,525
• Publication Number:
  US 20190310768A1
• Time in Patent Office: 609 Days
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

This disclosure pertains to systems, methods, and computer readable medium for mapping particular user interactions, e.g., gestures, to the input parameters of various image processing routines, e.g., image filters, in a way that provides a seamless, dynamic, and intuitive experience for both the user and the software developer. Such techniques may handle the processing of both “relative” gestures, i.e., those gestures having values dependent on how much an input to the device has changed relative to a previous value of the input, and “absolute” gestures, i.e., those gestures having values dependent only on the instant value of the input to the device. Additionally, inputs to the device beyond user-input gestures may be utilized as input parameters to one or more image processing routines. For example, the device'"'"'s orientation, acceleration, and/or position in three-dimensional space may be used as inputs to particular image processing routines.

Claims

1. An image processing method, comprising:

obtaining, by an electronic device, an image using a camera;

displaying, by the electronic device, the image to a touchscreen display;

receiving, by the electronic device, a selection of a particular location of the image using the touchscreen display;

adjusting, by the electronic device, a focus of the image to coincide with the particular location to produce an adjusted focus image; and

storing, by the electronic device, the adjusted focus image in a memory.

2. The method of claim 1, further comprising:

receiving, by the electronic device, a selection of a first filter to apply to the adjusted focus image;

receiving, by the electronic device, movement input via the touchscreen display;

determining, by the electronic device, a value of a first input parameter for the first filter from the movement input; and

applying, by the electronic device, the first filter with the first input parameter having the value to the adjusted focus image to generate a first filtered image.

3. The method of claim 2, further comprising:

receiving, by the electronic device, input selecting a location within the adjusted focus image corresponding to representation of a human face; and

in response to receiving the input selecting the location, selecting, by the electronic device, a particular area of the adjusted focus image corresponding to the representation of the human face as a focus area for applying the first filter.

4. The method of claim 2, further comprising:

storing, by the electronic device, the first filtered image in the memory; and

storing, by the electronic device separate from the first filtered image, metadata including the first filter and the first input parameter used to generate the first filtered image.

5. The method of claim 2, wherein the image obtained by the camera is a video stream of images, and wherein receiving the selection of the first filter to apply to the adjusted focus image further comprises receiving, by the electronic device, a selection of an image frame of the video stream of images.

6. The method of claim 2, wherein determining the value of the first input parameter for the first filter comprises analyzing, by the electronic device, a gesture indicated by the movement input to determine the value of the first input parameter.

7. An apparatus comprising:

an image sensor for capturing an image;

a touchscreen display;

a memory in communication with the image sensor; and

a programmable control device communicatively coupled to the image sensor, the touchscreen display, and the memory, wherein the memory includes instructions for causing the programmable control device to perform operations comprising;

obtaining an image using the image sensor;

displaying the image to the touchscreen display;

receiving, via the touchscreen display, a selection of a particular location of the image;

adjusting a focus of the image to coincide with the particular location to produce an adjusted focus image; and

storing the adjusted focus image in the memory.

8. The apparatus of claim 7, wherein the operations include:

receiving a selection of a first filter to apply to the adjusted focus image;

receiving movement input via the touchscreen display;

determining a value of a first input parameter for the first filter from the movement input; and

applying the first filter with the first input parameter having the value to the adjusted focus image to generate a first filtered image.

9. The apparatus of claim 8, wherein the operations include:

receiving input selecting a location within the adjusted focus image corresponding to representation of a human face; and

in response to receiving the input selecting the location, selecting a particular area of the adjusted focus image corresponding to the representation of the human face as a focus area for applying the first filter.

10. The apparatus of claim 8, wherein the operations include:

storing the first filtered image in the memory; and

storing, separate from the first filtered image, metadata including the first filter and the first input parameter used to generate the first filtered image.

11. The apparatus of claim 8, wherein the image obtained by the image sensor is a video stream of images, and wherein receiving the selection of the first filter to apply to the adjusted focus image includes receiving a selection of an image frame of the video stream of images.

12. The apparatus of claim 8, wherein determining the value of the first input parameter for the first filter comprises analyzing a gesture indicated by the movement input to determine the value of the first input parameter.

13. A non-transitory computer readable storage medium including computer readable program code stored thereon, wherein the computer readable program code, when executed by an electronic device having one or more processors, causes the electronic device to perform operations comprising:

obtaining, by an electronic device, an image using a camera;

displaying, by the electronic device, the image to a touchscreen display;

receiving, by the electronic device, a selection of a particular location of the image using the touchscreen display;

adjusting, by the electronic device, a focus of the image to coincide with the particular location to produce an adjusted focus image; and

storing, by the electronic device, the adjusted focus image in a memory.

14. The non-transitory computer readable storage medium of claim 13, wherein the operations further comprise:

receiving, by the electronic device, a selection of a first filter to apply to the adjusted focus image;

receiving, by the electronic device, movement input via the touchscreen display;

determining, by the electronic device, a value of a first input parameter for the first filter from the movement input; and

applying, by the electronic device, the first filter with the first input parameter having the value to the adjusted focus image to generate a first filtered image.

15. The non-transitory computer readable storage medium of claim 14, wherein the operations further comprise:

receiving, by the electronic device, input selecting a location within the adjusted focus image corresponding to representation of a human face; and

in response to receiving the input selecting the location, selecting, by the electronic device, a particular area of the adjusted focus image corresponding to the representation of the human face as a focus area for applying the first filter.

16. The non-transitory computer readable storage medium of claim 14, wherein the operations further comprise:

storing, by the electronic device, the first filtered image in the memory; and

storing, by the electronic device separate from the first filtered image, metadata including the first filter and the first input parameter used to generate the first filtered image.

17. The non-transitory computer readable storage medium of claim 14, wherein the image obtained by the camera is a video stream of images, and wherein receiving the selection of the first filter to apply to the adjusted focus image further comprises receiving, by the electronic device, a selection of an image frame of the video stream of images.

18. The non-transitory computer readable storage medium of claim 14, wherein determining the value of the first input parameter for the first filter comprises analyzing, by the electronic device, a gesture indicated by the movement input to determine the value of the first input parameter.