Method and device for reproducing content

• US 10,108,869 B2
• Filed: 05/12/2017
• Issued: 10/23/2018
• Est. Priority Date: 05/23/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yang, Pil-seung, Park, Da-hye, Won, Seol-hye, Yun, In-kuk, Park, Yong-gook
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US15/594,224
• Publication Number:
  US 20170249519A1
• Time in Patent Office: 529 Days
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04807 Pen manipulated menu
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 30/2272 with lexical matching

Abstract

Provided is a device including: a display unit configured to display handwritten content based on an analog handwritten input of a user; a user input unit that receives a user input of selecting a portion of the handwritten content displayed on the display unit; and a control unit reproduces a segment of multimedia content, which corresponds to the portion of the handwritten content, from the multimedia content synchronized with the handwritten content.

Claims

1. A device comprising:

a display;

a memory storing one or more instructions;

a touchscreen configured to receive a handwritten input, wherein the handwritten input is received while audio/video content is displayed on the display; and

a processor configured to execute the one or more instructions to;

associate a time of input of the handwritten input to the touchscreen with a reproduction time of portions of the audio/video content displayed on the display,

store handwritten content based on the handwritten input, and the time of input of the handwritten input associated with the reproduction time of portions of the audio/video content in the memory, and

control the display to display the handwritten content, and in response to detecting a touch input on the touchscreen selecting a portion of the handwritten content being displayed on the display, reproduce a portion of the audio/video content associated with a time of input to the touchscreen of handwritten input corresponding to the portion of the handwritten content stored in the memory.

2. The device of claim 1, wherein the processor is further configured to execute the one or more instructions to reproduce the portion of the audio/video content by obtaining time information of the audio/video content based on coordinate information of the portion of the handwritten content.

3. The device of claim 2, wherein the processor is further configured to execute the one or more instructions to control the display to display the portion of the handwritten content while the portion of the audio/video content is being reproduced.

4. The device of claim 3, wherein the processor is further configured to execute the one or more instructions to control the display to display the portion of the handwritten content in a temporal manner that is the same as a temporal manner in which the handwritten input corresponding to the portion of the handwritten content was input to the touchscreen.

5. The device of claim 3, wherein the processor is further configured to execute the one or more instructions to extract strokes included in the portion of the handwritten content, and determine an order of the strokes by using time stamp information corresponding to the strokes.

6. The device of claim 5, wherein the processor is further configured to execute the one or more instructions to determine a first stroke that is input first from among the strokes as a start position and a last stroke that is input last from among the strokes as an end position.

7. The device of claim 1, wherein the portion of handwritten content comprises a plurality of portions of the handwritten content, and

wherein the processor is further configured to execute the One or more instructions to reproduce a plurality of portions of the audio/video content associated with times of input to the touchscreen of handwritten input corresponding to the plurality of portions of the handwritten content.

8. The device of claim 1, wherein, when the portion of the handwritten content is selected, the processor is further configured to execute the one or more instructions to control the display to display a control panel comprising a reproduce function button within a certain distance from the portion of the handwritten content, and receive an input of activating the reproduce function button through the control panel.

9. The device of claim 1, wherein the processor is further configured to execute the one or more instructions to edit the handwritten content by using time information of strokes included in the portion of the handwritten content.

10. The device of claim 1, wherein the audio/video content comprises video content.

11. The device of claim 1, wherein the audio/video content comprises audio content.

12. The device of claim 1, wherein the audio/video content comprises audio content and video content.

13. The device of claim 1, wherein the audio/video content comprises audio content and still image content.

14. The device of claim 1, wherein the association of the time of input of the handwritten input to the touchscreen with the reproduction time of the portions of the audio/video content displayed on the display while the handwritten input is input and the audio/video content is output comprises a synchronization of the time of input of the handwritten input to the touchscreen with the reproduction time of the portions of the audio/video content displayed on the display while the handwritten input is input and the audio/video content is output.

15. A method of reproducing content, by a device, the method comprising:

receiving a handwritten input on a touchscreen of the device while audio/video content is displayed on a display;

associating a time of input of the handwritten input to the touchscreen with a reproduction time of portions of the audio/video content displayed on the display;

storing handwritten content based on the handwritten input, and the time of input of the handwritten input associated with the reproduction time of portions of the audio/video content in a memory; and

in response to detecting a touch input to the touchscreen selecting a portion of the handwritten content being displayed on the display of the device, reproducing a portion of the audio/video content associated with a time of input to the touchscreen of handwritten input corresponding to the portion of the handwritten content stored in the memory.

16. The method of claim 15, wherein the reproducing comprises reproducing the portion of the audio/video content by obtaining time information of the audio/video content based on coordinate information of the portion of the handwritten content.

17. The method of claim 15, wherein the reproducing comprises displaying the portion of the handwritten content on the display of the device while the portion of the audio/video content is being reproduced.

18. The method of claim 17, wherein the reproducing comprises displaying the portion of the handwritten content in a temporal manner that is the same as a temporal manner in which the handwritten input corresponding to the portion of the handwritten content was input to the touchscreen.

19. The method of claim 17, wherein the reproducing comprises:

extracting strokes included in the portion of the handwritten content; and

determining an order of the strokes by using time stamp information corresponding to the strokes.

20. The method of claim 19, wherein the determining of the order comprises determining a first stroke that is input first from among the strokes as a start position and a last stroke that is input last from among the strokes as an end position.

21. The method of claim 15, wherein the portion of handwritten content comprises a plurality of portions of the handwritten content, and

wherein the method further comprises:

receiving a selection of the plurality of portions of the handwritten content displayed on the display; and

reproducing a plurality of portions of the audio/video content associated with times of input to the touchscreen of handwritten input corresponding to the plurality of portions of the handwritten content.

22. The method of claim 15, further comprising:

displaying a control panel comprising a reproduce function button within a certain distance from the portion of the handwritten content; and

receiving an input of activating the reproduce function button through the control panel.

23. The method of claim 15, further comprising editing the handwritten content by using time information of strokes included in the portion of the handwritten content.

24. A non-transitory computer-readable recording medium having recorded thereon a program, which when executed by a device causes the device to execute a method of reproducing content, the method comprising:

receiving a handwritten input on a touchscreen of the device while audio/video content is displayed on a display;

associating a time of input of the handwritten input to the touchscreen with a reproduction time of portions of the audio/video content displayed on the display;

storing handwritten content based on the handwritten input, and the time of input of the handwritten input associated with the reproduction time of portions of the audio/video content in a memory; and

in response to detecting a touch input to the touchscreen selecting a portion of the handwritten content being displayed on the display of the device, reproducing a portion of the audio/video content associated with a time of input to the touchscreen of handwritten input corresponding to the portion of the handwritten content stored in the memory.

25. The non-transitory computer-readable recording medium of the 24, wherein the reproducing comprises reproducing the portion of the audio/video content by obtaining time information of the audio/video content based on coordinate information of the portion of the handwritten content.

26. The non-transitory computer-readable recording medium of the 24, wherein the reproducing comprises displaying the portion of the handwritten content on the display of the device while the portion of the audio/video content is being reproduced.

27. The non-transitory computer-readable recording medium of the 26, wherein the reproducing comprises displaying the portion of the handwritten content in a temporal manner that is the same as a temporal manner in which the handwritten input corresponding to the portion of the handwritten content was input to the touchscreen.

28. The non-transitory computer-readable recording medium of the 26, wherein the reproducing comprises:

extracting strokes included in the portion of the handwritten content; and

determining an order of the strokes by using time stamp information corresponding to the strokes.

29. The non-transitory computer-readable recording medium of the 28, wherein the determining of the order comprises determining a first stroke that is input first from among the strokes as a start position and a last stroke that is input last from among the strokes as an end position.