INFORMATION SEARCH METHOD AND DEVICE AND COMPUTER READABLE RECORDING MEDIUM THEREOF

• US 20140172831A1
• Filed: 12/13/2013
• Published: 06/19/2014
• Est. Priority Date: 12/14/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KWEON, Ji-hyeon, JIN, Sun-mi
• Granted Patent: US 10,614,120 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

An intuitive information search method and device based on a displayed image and a computer readable recording medium thereof. The information search method based on an image displayed on a device includes recognizing a first input indicating a selection related to a plurality of objects included in the image, recognizing a second input indicating a search relationship between the selected plurality of objects, and searching for information based on the first input and the second input; and outputting found information through the device.

Claims

1. An information search method based on an image displayed on a device, the method comprising:

recognizing a first input indicating a selection related to a plurality of objects included in the image;

recognizing a second input indicating a search relationship between the selected plurality of objects; and

searching for information based on the first input and the second input; and

outputting found information through the device.

2. The method of claim 1, wherein the search relationship between the selected plurality of objects comprises one of a same layer relationship indicating that a relationship between the plurality of objects have a same level when searching for the information and a high-low layer relationship indicating that the relationship between the plurality of objects have a high-low level when searching for the information.

3. The method of claim 2, wherein, when the search relationship between the selected plurality of objects is the same layer relationship, the second input comprises one of a touch based gesture indicating gathering the selected plurality of objects and a touch based gesture indicating moving the selected plurality of objects to at least one location other than display locations of the selected plurality of objects.

4. The method of claim 2, wherein, when the search relationship between the selected plurality of objects is the high-low layer relationship, the second input comprises one of a touch based gesture indicating moving the one or more selected objects excluding one object among the selected plurality of objects to a display location of the excluded one object and a touch based gesture indicating moving at least one of the selected plurality of objects to a display location of one of the selected plurality of objects.

5. The method of claim 4, wherein the touch based gesture comprises a touch and drag.

6. The method of claim 2, wherein the searching for the information comprises:

when the search relationship between the selected plurality of objects is the same layer relationship, searching for information including all of the selected plurality of objects; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, searching for information according to the high-low layer relationship between the selected plurality of objects.

7. The method of claim 2, wherein the searching for the information comprises:

when the search relationship between the selected plurality of objects is the same layer relationship, searching for information regarding at least one of the selected plurality of objects; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, searching for information according to the high-low layer relationship between the selected plurality of objects.

8. The method of claim 2, wherein:

when the search relationship between the selected plurality of objects is the same layer relationship, the searching for the information comprises;

displaying information used to select one of an information search item comprising at least one of the selected plurality of objects and an information search item comprising all of the selected plurality of objects, and

searching for the information according to the information search item selected based on the displayed information; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, the searching for the information comprises;

searching for the information according to the high-low layer relationship between the selected plurality of objects.

9. The method of claim 1, wherein the first input comprises one of a touch based gesture indicating a long touch on the selected plurality of objects or each object and a touch based gesture indicating a closed region on the selected plurality of objects or the each object.

10. The method of claim 1, wherein the searching for the information comprises:

displaying at least one search item determined according to the first input and the second input; and

searching for the information based on a search item selected from the displayed at least one search item.

11. The method of claim 1, further comprising:

displaying information indicating recognition related to the first input by the device.

12. The method of claim 11, wherein the information indicating the recognition related to the first input by the device comprises one of information indicating closed regions based on outlines of the selected plurality of objects or each object, information based on a cut image based on the outlines of the selected plurality of objects or the each object, and a multi-screen based on the cut image.

13. The method of claim 1, further comprising:

displaying information indicating recognition related to the second input by the device.

14. The method of claim 1, wherein the searching for the information is performed with respect to at least one of the device and at least one external device connected to the device.

15. The method of claim 1, wherein the found information comprises at least one of image based information and text based information.

32. A non-transitory computer readable recording medium storing one or more programs comprising a command language to execute the image based information search method of claim 1.

16. A device comprising:

a storage unit to store at least one piece of information and at least one program;

a touch screen to display an image, receiving a first input indicating a selection related to a plurality of objects included in the image, to receive a second input indicating a search relationship between the selected plurality of objects, and to display information searched for based on the first input and the second input; and

a processor to provide a user interface based on the touch screen, to recognize the first input and the second input received through the touch screen, to search for information from the storage unit according to the first input and the second input, and to output found information on the touch screen.

17. The device of claim 16, wherein the search relationship between the selected plurality of objects comprises one of a same layer relationship indicating the plurality of objects have a same level when searching for the information and a high-low layer relationship indicating the plurality of objects have a high-low level when searching for the information.

18. The device of claim 17, wherein, when the search relationship between the selected plurality of objects is the same layer relationship, the second input comprises one of a touch based gesture indicating gathering the selected plurality of objects and a touch based gesture indicating moving the selected plurality of objects to at least one location other than display locations of the selected plurality of objects.

19. The device of claim 17, wherein, when the search relationship between the selected plurality of objects is the high-low layer relationship, the second input comprises one of a touch based gesture for moving the remainder of selected objects excluding one object among the selected plurality of objects to a display location of the excluded one object and a touch based gesture for moving at least one of the selected plurality of objects to a display location of one of the selected plurality of objects.

20. The device of claim 19, wherein the touch based gesture comprises a touch and drag.

21. The device of claim 17, wherein:

when the search relationship between the selected plurality of objects is the same layer relationship, the processor searches for information including all of the selected plurality of objects; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, the processor searches for information according to the high-low layer relationship between the selected plurality of objects.

22. The device of claim 17, wherein:

when the search relationship between the selected plurality of objects is the same layer relationship, the processor searches for information including at least one of the selected plurality of objects; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, the processor searches for information according to the high-low layer relationship between the selected plurality of objects.

23. The device of claim 17, wherein:

when the search relationship between the selected plurality of objects is the same layer relationship, the processor displays information used to select an information search item comprising at least one of the selected plurality of objects and an information search item comprising all of the selected plurality of objects, searches for the information according to the information search item selected based on the displayed information, and,

when the search relationship between the selected plurality of objects is the high-low layer relationship, the processor searches for the information according to the high-low layer relationship between the selected plurality of objects.

24. The device of claim 16, wherein the first input comprises at least one of a touch based gesture indicating a long touch on the selected plurality of objects or each object and a touch based gesture indicating a closed region on the selected plurality of objects or the each object.

25. The device of claim 16, wherein the processor displays at least one search item determined according to the first input and the second input on the touch screen, and searches for the information based on a search item selected from the displayed at least one search item through the touch screen.

26. The device of claim 16, wherein the processor displays information indicating recognition related to the first input by the device.

27. The device of claim 16, wherein the information indicating recognition related to the first input comprises one of information indicating closed regions based on outlines of the selected plurality of objects or each object, information based on a cut image based on the outlines of the selected plurality of objects or the each object, and a multi-screen based on the cut image.

28. The device of claim 16, 26, or 27, wherein the processor displays information indicating recognition related to the second input on the device on the touch screen.

29. The device of claim 16, wherein the processor searches for the information with respect to at least one of the storage unit and at least one external device connected to the device.

30. The device of claim 17, wherein the found information comprises at least one of image based information and text based information.

31. A device comprising:

a touch screen to display an image and receiving a user input based on the displayed image;

a processor to provide a user interface based on the touch screen; and

a storage unit to store at least one program configured to be executed by the processor and information searchable based on the image,

wherein the program includes one or more commands to recognize a first input indicating a selection related to a plurality of objects included in the image, to recognize a second input indicating a search relationship between the selected plurality of objects, to search for information searched for based on the first input and the second input, and to output found information on the touch screen.

33. A service providing method based on a displayed image, the method comprising:

recognizing a first input indicating a selection related to a plurality of objects included in the image;

recognizing a second input indicating a relationship between the selected plurality of objects; and

displaying at least one service item provided based on the first input and the second input; and

executing a service according to a service item selected based on the displayed at least one service item.

34. The method of claim 33, wherein the relationship between the selected plurality of objects comprises one of a same layer relationship indicating the plurality of objects have a same level and a high-low layer relationship indicating the plurality of objects have a high-low level.

35. The method of claim 34, wherein the displaying of the at least one service item comprises:

when the search relationship between the selected plurality of objects is the same layer relationship, the displayed at least one service item comprises service items provided to each of the selected plurality of objects; and

when the search relationship between the selected plurality of objects is the high-low layer relationship, the displayed at least one service item comprises service items provided according to the high-low layer relationship between the selected plurality of objects.

36. The method of claim 33, wherein the first input comprises at least one of a touch based gesture indicating a long touch on the selected plurality of objects or each object and a touch based gesture indicating a closed region on the selected plurality of objects or the each object.

37. A device to perform an information search method, the apparatus comprising:

a display panel to display an image including one or more objects on a screen thereof; and

a processor configured to control the display panel to receive selection of the objects from the image of the display panel as a first input, to receive a movement of at least one position of the selected objects with respect to the other one of the selected objects as a second input, and to display one or more images searched according to the first input and the second input.

38. The device of claim 37, wherein the display panel differently displays the selected objects and non-selected objects according to existence of a user input thereon.

39. The device of claim 37, wherein the display panel displays the at least one position of the selected objects at a different position.

40. The device of claim 37, wherein the screen of the display panel comprises:

a first screen to display the image including the objects;

a second screen to display the selection of the objects;

a third screen to display an indication of the movement; and

a fourth screen to display the one or more searched images.

41. The device of claim 40, wherein:

the display panel displays the second screen and the third screen according to a user interaction with the display panel; and

the display panel displays the first screen and the fourth screen according to operations of the processor without the user interaction with the display panel.

42. The device of claim 40, wherein the second screen includes the first screen.

43. The device of claim 40, wherein the second screen is different from the first screen.

44. The device of claim 40, wherein the third screen includes the first screen and the second screen.

45. The device of claim 40, wherein the third screen is different from the first screen and the second screen.

46. An information search method based on a displayed image, the method comprising:

displaying an image including one or more objects on a screen thereof;

receiving selection of the objects from the image of the display panel as a first input;

receiving a movement of at least one position of the selected objects with respect to the other one of the selected objects as a second input; and

displaying one or more images searched according to the first user input and the second user input.