Message user interfaces for capture and transmittal of media and location content

• US 9,185,062 B1
• Filed: 09/30/2014
• Issued: 11/10/2015
• Est. Priority Date: 05/31/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Garcia, Roberto, Wood, Justin, Dellinger, Richard R., Chaudhri, Imran, Yang, Lawrence Y., Lindeman, Kevin J., Macomber, Kyle S.
• Primary Examiner(s): Kujundzic, Dino
• Application Number:
  US14/503,376
• Publication Number:
  US 20150350130A1
• Time in Patent Office: 406 Days
• Field of Search: 715/752
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: H04L 51/04 : Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/046 : Interoperability with other network applications or services
• CPC Class Codes: H04L 51/08 : Annexed information, e.g. attachments
• CPC Class Codes: H04L 51/10 : Multimedia information
• CPC Class Codes: H04L 51/222 : using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
• CPC Class Codes: H04L 51/234 : for tracking messages
• CPC Class Codes: H04L 51/42 : Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04M 1/72433 : for voice messaging, e.g. dictaphones for answering incoming calls H04M1/64
• CPC Class Codes: H04M 1/72439 : for image or video messaging
• CPC Class Codes: H04M 1/72454 : according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/62 : Control of parameters via user interfaces
• CPC Class Codes: H04N 23/631 : Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 : for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04W 4/02 : Services making use of location information

Abstract

A device provides user interfaces for capturing and sending media, such as audio, video, or images, from within a message application. The device detects a movement of the device and in response, plays or records an audio message. The device sends the recorded audio message in response to detecting a movement of the device. The device removes messages from a conversation based on expiration criteria. The device shares a location with one or more message participants in a conversation.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a touch-sensitive surface, cause the device to:

display, on the display, a media affordance and a message region for displaying messages sent between participants in a message conversation;

detect a gesture on the touch-sensitive surface that includes detecting a contact at a location on the touch-sensitive surface that corresponds to a location of the media affordance on the display;

in response to detecting the contact;

display a send affordance on the display;

detect a continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface;

determine whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the send affordance; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance, and in response to detecting the liftoff, send a new media item captured in response to at least a portion of the gesture to a participant in the message conversation, wherein capturing the new media item in response to at least the portion of the gesture includes creating the new media item based on data detected by one or more sensors of the device during the portion of the gesture.

2. The non-transitory computer-readable storage medium according to claim 1:

wherein the instructions to cause the device to detect the gesture on the touch-sensitive surface comprise instructions to cause the device to detect a single contact, and

wherein the instruction to cause the device to detect the continuation of the gesture comprise instructions to cause the device to detect a continuation of the single contact.

3. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

in response to detecting the contact, display a cancel affordance on the display;

determine whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the cancel affordance; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the cancel affordance, followed by the liftoff, forgo sending the captured new media item to the participant.

4. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

after detecting the liftoff, remove the send affordance from the display.

5. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

capture the new media item, and

send the captured new media item to the participant.

6. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

in response to detecting the contact, begin to capture the media; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

cease to capture the new media item, and

send the captured new media item to the participant.

7. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

in response to detecting the contact, begin to capture the new media item;

in response to detecting the movement of the contact, cease to capture the new media item; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff while the contact is at the location on the touch-sensitive surface that corresponds to the send affordance, send the captured new media item to the participant.

8. The non-transitory computer-readable storage medium according to claim 1:

wherein the new media item represents audio data received from a microphone,

wherein the instructions to cause the device to detect the continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface comprises instructions to cause the device to detect liftoff at the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and

wherein the non-transitory computer-readable storage medium further comprises instructions to cause the device to;

in response to detecting the contact, begin to capture the audio data; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff, cease to capture the audio data and send the audio data to the participant.

9. The non-transitory computer-readable storage medium according to claim 3, further comprising instructions to cause the device to:

display a message compose field; and

in accordance with a determination that the contact did not move to the location on the touch-sensitive surface that corresponds to the location of the send affordance nor to the location on the touch-sensitive surface that corresponds to the location the cancel affordance, and in response to detecting the liftoff;

display an image in the message compose field, the image having a waveform representing the captured new media item.

10. The non-transitory computer-readable storage medium according to claim 9, further comprising instructions to cause the device to:

in response to detecting the liftoff, replace display of the media affordance with display of a send button for sending the captured new media item to the participant.

11. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

in response to detecting the contact on the touch-sensitive surface, display a media preview on the display; and

after detecting the liftoff, cease to display the media preview on the display.

12. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

while the contact is maintained on the media affordance;

display a visually distinguished region, wherein the visually distinguished region encloses the media affordance and the send affordance; and

in response to detecting the movement of the contact, expand the visually distinguished region.

13. The non-transitory computer-readable storage medium according to claim 12, further comprising instructions to cause the device to:

display a virtual keyboard for entering a message to the participant; and

display the media affordance along a particular edge of the display, above the virtual keyboard, wherein the displayed visually distinguished region is partially bounded by the particular edge.

14. The non-transitory computer-readable storage medium according to claim 12, further comprising instructions to cause the device to:

display the media affordance at a corner of the display, wherein the displayed visually distinguished region is partially bounded by two different edges of the display.

15. The non-transitory computer-readable storage medium according to claim 12, wherein the touch-sensitive surface is a touch-sensitive display, and further comprising instructions to cause the device to:

assign a hit region to the media affordance, wherein the hit region is an area of the display responsive to touch input, and wherein the hit region has a larger size when a virtual keyboard is displayed than when the virtual keyboard is not displayed.

16. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

determine a duration of the contact on the media affordance; and

in accordance with a determination that the duration of the contact is less than a threshold duration, display a media selector user interface for selecting an existing media item to send to the participant.

17. The non-transitory computer-readable storage medium according to claim 1, further comprising instructions to cause the device to:

receive a user designation of whether to prevent sending a set of meta-data of the captured new media item to the participant.

18. The non-transitory computer-readable storage medium according to claim 17, wherein the meta-data comprises a representation of the location of the electronic device.

19. The non-transitory computer-readable storage medium according to claim 1, wherein the media affordance is a first media affordance, and further comprising instructions to cause the device to:

display a second media affordance on the display.

20. The non-transitory computer-readable storage medium according to claim 19, wherein the first media affordance is an audio capture affordance for capturing audio data and the second media affordance is a visual-media capture affordance for capturing video or images.

21. The non-transitory computer-readable storage medium according to claim 1, wherein the new media item that is sent to the participant expires based on expiration criteria.

22. A method, comprising:

at an electronic device with a touch-sensitive surface and a display;

displaying, on the display, a media affordance and a message region for displaying messages sent between participants in a message conversation;

detecting a gesture on the touch-sensitive surface that includes detecting a contact at a location on the touch-sensitive surface that corresponds to a location of the media affordance on the display;

in response to detecting the contact;

displaying a send affordance on the display;

detecting a continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface;

determining whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the send affordance; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance, and in response to detecting the liftoff, sending a new media item captured in response to at least a portion of the gesture to a participant in the message conversation, wherein capturing the new media item in response to at least the portion of the gesture includes creating the new media item based on data detected by one or more sensors of the device during the portion of the gesture.

23. The method according to claim 22,

wherein detecting the gesture on the touch-sensitive surface comprises detecting a single contact, and

wherein detecting the continuation of the gesture comprises detecting a continuation of the single contact.

24. The method according to claim 22, further comprising:

in response to detecting the contact, displaying a cancel affordance on the display;

determining whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the cancel affordance; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the cancel affordance, followed by the liftoff, forgoing sending the captured new media item to the participant.

25. The method according to claim 24, further comprising:

displaying a message compose field; and

in accordance with a determination that the contact did not move to the location on the touch-sensitive surface that corresponds to the location of the send affordance nor to the location on the touch-sensitive surface that corresponds to the location the cancel affordance, and in response to detecting the liftoff;

displaying an image in the message compose field, the image having a waveform representing the captured new media item.

26. The method according to claim 25, further comprising:

in response to detecting the liftoff, replacing display of the media affordance with display of a send button for sending the captured new media item to the participant.

27. The method according to claim 22, further comprising:

after detecting the liftoff, removing the send affordance from the display.

28. The method according to claim 22, further comprising:

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

capturing the new media item, and

sending the captured new media item to the participant.

29. The method according to claim 22, further comprising:

in response to detecting the contact, beginning to capture the media; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

ceasing to capture the new media item, and

sending the captured new media item to the participant.

30. The method according to claim 22, further comprising:

in response to detecting the contact, beginning to capture the new media item;

in response to detecting the movement of the contact, ceasing to capture the new media item; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff while the contact is at the location on the touch-sensitive surface that corresponds to the send affordance, sending the captured new media item to the participant.

31. The method according to claim 22,

wherein the new media item represents audio data received from a microphone,

wherein detecting the continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface comprises detecting liftoff at the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and

the method further comprising:

in response to detecting the contact, beginning to capture the audio data; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff, ceasing to capture the audio data and sending the audio data to the participant.

32. The method according to claim 22, further comprising:

in response to detecting the contact on the touch-sensitive surface, displaying a media preview on the display; and

after detecting the liftoff, ceasing to display the media preview on the display.

33. The method according to claim 22, further comprising:

while the contact is maintained on the media affordance;

displaying a visually distinguished region, wherein the visually distinguished region encloses the media affordance and the send affordance; and

in response to detecting the movement of the contact, expanding the visually distinguished region.

34. The method according to claim 33, further comprising:

displaying a virtual keyboard for entering a message to the participant; and

displaying the media affordance along a particular edge of the display, above the virtual keyboard, wherein the displayed visually distinguished region is partially bounded by the particular edge.

35. The method according to claim 33, further comprising:

displaying the media affordance at a corner of the display, wherein the displayed visually distinguished region is partially bounded by two different edges of the display.

36. The method according to claim 33, wherein the touch-sensitive surface is a touch- sensitive display, and the method further comprises:

assigning a hit region to the media affordance, wherein the hit region is an area of the display responsive to touch input, and wherein the hit region has a larger size when a virtual keyboard is displayed than when the virtual keyboard is not displayed.

37. The method according to claim 22, further comprising:

determining a duration of the contact on the media affordance; and

in accordance with a determination that the duration of the contact is less than a threshold duration, displaying a media selector user interface for selecting an existing media item to send to the participant.

38. The method according to claim 22, further comprising:

receiving a user designation of whether to prevent sending a set of meta-data of the captured new media item to the participant.

39. The method according to claim 38, wherein the meta-data comprises a representation of the location of the electronic device.

40. The method according to claim 22, wherein the media affordance is a first media affordance, and the method further comprises:

displaying a second media affordance on the display.

41. The method according to claim 40, wherein the first media affordance is an audio capture affordance for capturing audio data and the second media affordance is a visual-media capture affordance for capturing video or images.

42. The method according to claim 22, wherein the new media item that is sent to the participant expires based on expiration criteria.

43. An electronic device, comprising:

a display;

a touch-sensitive surface;

a memory; and

a processor coupled to the display, the touch-sensitive surface, and the memory, the processor configured to;

display, on the display, a media affordance and a message region for displaying messages sent between participants in a message conversation;

detect a gesture on the touch-sensitive surface that includes detecting a contact at a location on the touch-sensitive surface that corresponds to a location of the media affordance on the display;

in response to detecting the contact;

display a send affordance on the display;

detect a continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface;

determine whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the send affordance; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance, and in response to detecting the liftoff, send a new media item captured in response to at least a portion of the gesture to a participant in the message conversation, wherein capturing the new media item in response to at least the portion of the gesture includes creating the new media item based on data detected by one or more sensors of the device during the portion of the gesture.

44. The electronic device according to claim 43,

wherein detecting the gesture on the touch-sensitive surface comprises detecting a single contact, and

wherein detecting the continuation of the gesture comprises detecting a continuation of the single contact.

45. The electronic device according to claim 43, wherein the processor is further configured to:

in response to detecting the contact, display a cancel affordance on the display;

determine whether the contact moved to a location on the touch-sensitive surface that corresponds to a location of the cancel affordance; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the cancel affordance, followed by the liftoff, forgo sending the captured new media item to the participant.

46. The electronic device according to claim 45, wherein the processor is further configured to:

display a message compose field; and

in accordance with a determination that the contact did not move to the location on the touch-sensitive surface that corresponds to the location of the send affordance nor to the location on the touch-sensitive surface that corresponds to the location the cancel affordance, and in response to detecting the liftoff;

display an image in the message compose field, the image having a waveform representing the captured new media item.

47. The electronic device according to claim 46, wherein the processor is further configured to:

in response to detecting the liftoff, replace display of the media affordance with display of a send button for sending the captured new media item to the participant.

48. The electronic device according to claim 43, wherein the processor is further configured to:

after detecting the liftoff, remove the send affordance from the display.

49. The electronic device according to claim 43, wherein the processor is further configured to:

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

capture the new media item, and

send the captured new media item to the participant.

50. The electronic device according to claim 43, wherein the processor is further configured to:

in response to detecting the contact, begin to capture the media; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff;

cease to capture the new media item, and

send the captured new media item to the participant.

51. The electronic device according to claim 43, wherein the processor is further configured to:

in response to detecting the contact, begin to capture the new media item;

in response to detecting the movement of the contact, cease to capture the new media item; and

in accordance with a determination that the contact moved to the location on the touch- sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff while the contact is at the location on the touch-sensitive surface that corresponds to the send affordance, send the captured new media item to the participant.

52. The electronic device according to claim 43:

wherein the new media item represents audio data received from a microphone,

wherein detecting the continuation of the gesture that includes detecting movement of the contact across the touch-sensitive surface followed by detecting liftoff of the contact from the touch-sensitive surface comprises detecting liftoff at the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and

wherein the processor is further configured to;

in response to detecting the contact, begin to capture the audio data; and

in accordance with a determination that the contact moved to the location on the touch-sensitive surface that corresponds to the location of the send affordance on the display, and in response to detecting the liftoff, cease to capture the audio data and send the audio data to the participant.

53. The electronic device according to claim 43, wherein the processor is further configured to:

in response to detecting the contact on the touch-sensitive surface, display a media preview on the display; and

after detecting the liftoff, cease to display the media preview on the display.

54. The electronic device according to claim 43, wherein the processor is further configured to:

while the contact is maintained on the media affordance;

display a visually distinguished region, wherein the visually distinguished region encloses the media affordance and the send affordance; and

in response to detecting the movement of the contact, expand the visually distinguished region.

55. The electronic device according to claim 54, wherein the processor is further configured to:

display a virtual keyboard for entering a message to the participant; and

display the media affordance along a particular edge of the display, above the virtual keyboard, wherein the displayed visually distinguished region is partially bounded by the particular edge.

56. The electronic device according to claim 54, wherein the processor is further configured to:

display the media affordance at a corner of the display, wherein the displayed visually distinguished region is partially bounded by two different edges of the display.

57. The electronic device according to claim 54, wherein the touch-sensitive surface is a touch-sensitive display, and wherein the processor is further configured to:

assign a hit region to the media affordance, wherein the hit region is an area of the display responsive to touch input, and wherein the hit region has a larger size when a virtual keyboard is displayed than when the virtual keyboard is not displayed.

58. The electronic device according to claim 43, wherein the processor is further configured to:

determine a duration of the contact on the media affordance; and

in accordance with a determination that the duration of the contact is less than a threshold duration, display a media selector user interface for selecting an existing media item to send to the participant.

59. The electronic device according to claim 43, wherein the processor is further configured to:

receive a user designation of whether to prevent sending a set of meta-data of the captured new media item to the participant.

60. The electronic device according to claim 59, wherein the meta-data comprises a representation of the location of the electronic device.

61. The electronic device according to claim 43, wherein the media affordance is a first media affordance, and wherein the processor is further configured to:

display a second media affordance on the display.

62. The electronic device according to claim 61, wherein the first media affordance is an audio capture affordance for capturing audio data and the second media affordance is a visual-media capture affordance for capturing video or images.

63. The electronic device according to claim 43, wherein the new media item that is sent to the participant expires based on expiration criteria.