Gesture mapping for image filter input parameters

• US 8,717,381 B2
• Filed: 03/21/2011
• Issued: 05/06/2014
• Est. Priority Date: 01/11/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hayward, David, Zhang, Chendi, Naaman, Alexandre, Dellinger, Richard R., Murthy, Giridhar S.
• Primary Examiner(s): McDowell, Jr., Maurice L
• Application Number:
  US13/052,994
• Publication Number:
  US 20120176401A1
• Time in Patent Office: 1,142 Days
• Field of Search: 345/619, 382/260, 715/863
• US Class Current: 345/619
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

This disclosure pertains to systems, methods, and computer readable medium for mapping particular user interactions, e.g., gestures, to the input parameters of various image processing routines, e.g., image filters, in a way that provides a seamless, dynamic, and intuitive experience for both the user and the software developer. Such techniques may handle the processing of both “relative” gestures, i.e., those gestures having values dependent on how much an input to the device has changed relative to a previous value of the input, and “absolute” gestures, i.e., those gestures having values dependent only on the instant value of the input to the device. Additionally, inputs to the device beyond user-input gestures may be utilized as input parameters to one or more image processing routines. For example, the device'"'"'s orientation, acceleration, and/or position in three-dimensional space may be used as inputs to particular image processing routines.

Claims

1. An image processing method, comprising:

receiving, at an electronic device, a selection of a first filter to apply to an image;

receiving input from one or more sensors in communication with the electronic device;

associating a first input parameter for the first filter with the received input;

assigning a value to the first input parameter based on the received input;

applying the first filter to the image to generate a first filtered image, the first input parameter having the assigned value; and

storing the first filtered image in a memory.

2. The method of claim 1, wherein the first filter comprises a filter selected from the group consisting of:

Thermal, Mirror, X-Ray, Kaleidoscope, Normal, Light Tunnel, Squeeze, Twirl, and Stretch.

3. The method of claim 1, wherein the act of receiving input from one or more sensors in communication with the electronic device comprises receiving gesture input from an electronic device having a touch-sensitive display.

4. The method of claim 3, wherein the act of receiving gesture input comprises receiving gesture input corresponding to a single point of contact with the touch-sensitive display.

5. The method of claim 3, wherein the act of assigning a value to the first input parameter comprises mapping the gesture input to a value, wherein the value is limited to a predetermined range, and wherein the predetermined range is based on the input parameter.

6. The method of claim 1, further comprising the act of displaying the first filtered image on a display.

7. The method of claim 1, wherein the act of receiving a selection of a first filter to apply to an image comprises receiving a selection of a first filter to apply to a previously stored image.

8. The method of claim 1, wherein the act of receiving a selection of a first filter to apply to an image comprises receiving a selection of a first filter to apply to an image frame of a video stream of images.

9. The method of claim 8, wherein the act of storing the first filtered image comprises storing an unfiltered version of the image and corresponding video gesture track information separately in the memory.

10. The method of claim 1, wherein the act of associating a first input parameter for the first filter with the received input comprises associating a first input parameter with a relative gesture.

11. The method of claim 1, wherein the act of associating a first input parameter for the first filter with the received input comprises associating a first input parameter with an absolute gesture.

12. The method of claim 1, wherein the received input from the one or more sensors in communication with the electronic device is indicative of a position of the device.

13. The method of claim 11, wherein the one or more sensors comprises a sensor selected from the group consisting of:

an image sensor, an orientation sensor, an accelerometer, a Global Positioning System (GPS), and a gyrometer.

14. The method of claim 11, wherein the act of receiving input comprises receiving input corresponding to an orientation of the electronic device in three-dimensional space.

15. The method of claim 1, wherein the act of assigning a value to the first input parameter based on the received input comprises applying a translation to the received input, wherein the translation applied is based on the received input.

16. The method of claim 15, wherein the received input is indicative of a position of the device.

17. The method of claim 15, wherein the electronic device comprises a plurality of cameras, and wherein the translation applied to the received input is based on the camera used by the electronic device to capture the image.

18. The method of claim 1, wherein the image comprises a high-resolution image and a low-resolution image, and wherein the act of applying the first filter to the image comprises applying the first filter to both the high-resolution and low-resolution images, wherein the act of applying the first filter to the high-resolution image is performed by a CPU (central processing unit) of the electronic device, and wherein the act of applying the first filter to the low-resolution image is performed by a GPU (graphics processing unit) of the electronic device.

19. The method of claim 1, further comprising resetting the value of the first input parameter to a default value in response to a user performing a specified gesture on a touch-sensitive display of the electronic device.

20. A non-transitory computer readable medium having computer readable program code stored thereon, wherein the computer readable program code, when executed by an electronic device having one or more processors, causes the electronic device to perform the method of claim 1.

21. An image processing method, comprising:

receiving, at an electronic device, a selection of a first filter to apply to an image;

receiving input from one or more sensors in communication with the electronic device;

associating a first input parameter for the first filter with the received input;

assigning a value to the first input parameter based on the received input;

applying the first filter to the image to generate a first image overlay preview grid, the first input parameter having the assigned value; and

storing the first image overlay preview grid in a memory.

22. The method of claim 21, wherein the first filter comprises a filter selected from the group consisting of:

Thermal, Mirror, X-Ray, Kaleidoscope, Normal, Light Tunnel, Squeeze, Twirl, and Stretch.

23. The method of claim 21, wherein the act of receiving input from one or more sensors in communication with the electronic device comprises receiving gesture input from an electronic device having a touch-sensitive display.

24. The method of claim 23, wherein the act of assigning a value to the first input parameter comprises mapping the gesture input to a value, wherein the value is limited to a predetermined range, and wherein the predetermined range is based on the input parameter.

25. The method of claim 21, further comprising the act of displaying the first filtered image and the first image overlay preview grid on a display in communication with the electronic device.

26. The method of claim 21, wherein the act of associating a first input parameter for the first filter with the received input comprises associating a first input parameter with a relative gesture.

27. The method of claim 21, wherein the act of associating a first input parameter for the first filter with the received input comprises associating a first input parameter with an absolute gesture.

28. The method of claim 21, wherein the received input from the one or more sensors in communication with the electronic device is indicative of a position of the device.

29. The method of claim 28, wherein the one or more sensors comprises a sensor selected from the group consisting of:

an image sensor, an orientation sensor, an accelerometer, a Global Positioning System (GPS), and a gyrometer.

30. The method of claim 28, wherein the act of receiving input comprises receiving input corresponding to an orientation of the electronic device in three-dimensional space.

31. The method of claim 21, wherein the act of assigning a value to the first input parameter based on the received input comprises applying a translation to the received input, wherein the translation applied is based on the received input.

32. The method of claim 31, wherein the received input is indicative of a position of the device.

33. The method of claim 21, wherein the image comprises a high-resolution image and a low-resolution image, and wherein the act of applying the first filter to the image comprises applying the first filter to both the high-resolution and low-resolution images, wherein the act of applying the first filter to the high-resolution image is performed by a CPU (central processing unit) of the electronic device, and wherein the act of applying the first filter to the low-resolution image is performed by a GPU (graphics processing unit) of the electronic device.

34. An image processing method, comprising:

receiving, at an electronic device, a selection of a first filter to apply to an image;

receiving at the electronic device input indicative of a selection of a face in the image;

associating a first input parameter for the first filter with the received input indicative of the selection of a face in the image;

assigning a value to the first input parameter based on the received input indicative of the selection of a face in the image;

applying the first filter to the image to generate a first filtered image, the first input parameter having the assigned value; and

storing the first filtered image in a memory.

35. The method of claim 34, wherein the act of receiving input indicative of a selection of a face in the image comprises receiving gesture input from an electronic device having a touch-sensitive display.

36. The method of claim 35, wherein the act of receiving gesture input comprises receiving gesture input corresponding to a single point of contact with the touch-sensitive display.

37. The method of claim 34, wherein the selected face remains selected until the electronic device receives further input indicative of a selection of a new face in the image.

38. The method of claim 34, wherein the selected face remains selected until it is no longer present in the image.

39. An apparatus comprising:

an image sensor for capturing an image representative of a scene;

a display;

a memory in communication with the image sensor; and

a programmable control device communicatively coupled to the image sensor, the display, and the memory, wherein the memory includes instructions for causing the programmable control device to perform a method comprising;

receiving a selection of a first filter to apply to the image;

receiving input from one or more sensors in communication with the apparatus;

associating a first input parameter for the first filter with the received input;

assigning a value to the first input parameter based on the received input;

applying the first filter to the image to generate a first filtered image, the first input parameter having the assigned value;

storing the first filtered image in the memory; and

displaying the filtered image on the display.

40. The apparatus of claim 39, wherein the display comprises a touch-sensitive display.

41. The apparatus of claim 40, wherein the received input comprises gesture input.

42. The apparatus of claim 41, wherein the received gesture input comprises gesture input corresponding to a single point of contact with the touch-sensitive display.

43. The apparatus of claim 41, wherein the programmed act of assigning a value to the first input parameter comprises mapping the gesture input to a value, wherein the value is limited to a predetermined range, and wherein the predetermined range is based on the input parameter.