Methods and systems for personalized, interactive and intelligent searches

• US 10,616,199 B2
• Filed: 12/01/2016
• Issued: 04/07/2020
• Est. Priority Date: 12/01/2015
• Status: Active Grant

• Current Assignee: Integem Incorporated
• Original Assignee: Integem Incorporated
• Inventors: Du, Eliza Y.
• Primary Examiner(s): Pyzocha, Michael
• Application Number:
  US15/367,124
• Publication Number:
  US 20170155631A1
• Time in Patent Office: 1,223 Days
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32

Abstract

Existing search methods/systems are often generic and sometimes offer no user specific information. Disclosed herein are methods and systems for providing personalized, interactive, and intelligent search information. In particular, intelligent analysis for better interpreting and understanding user input and interactive user feedback concerning both search query quality and search result quality are provided to improve search quality and user experience, especially for accurate and intelligent searches in an interactive system (e.g., in an AR system).

Claims

1. A method for providing interactive and intelligent searches to a user, comprising:

identifying, at a user device, a search query based on a user image input, the user image input collected in real-time of the user using a camera associated with the user device, said user image input including at least a portion of the user'"'"'s body separate from the user'"'"'s face;

attempting to authenticate the user using biometric information;

granting access to private information based on a successful authentication result;

limiting access to the private information based on a failed authentication result;

modifying the search query based on information accessible by the user device, as determined by the authentication result, using artificial intelligence to render a modified query;

communicating the modified query from the user device to a remote server;

receiving search results from the remote server; and

checking for user input suggesting to end the search.

2. The method of claim 1, wherein the information accessible by the user device comprises augmented reality data information, user biometric data, user personal data, other people'"'"'s information that the user has permission to access, existing search records, information stored in the user device, system information of the user device, information that is extracted through real-time network communication at the time when the search query is modified, or any combinations thereof.

3. The method of claim 2, wherein the augmented reality data information including augmented reality input data information, the output augmented reality data information where the extracted real-life data from the input image was immersed into the input augmented reality data, or a combination thereof.

4. The method of claim 1, wherein the user can provide instant feedback at any query processing stage if a search query is unsatisfactory.

5. The method of claim 1, wherein the user device comprises a smart phone, a laptop, a tablet, a headset, a glass set, a goggle set, a gaming system, a computer with video and audio input capacity, or combinations thereof.

6. The method of claim 1, further comprising:

receiving, at the user device, the user input, wherein the user input comprises a file input, a voice input, a gesture input, an eye movement, a body movement, a text input, a facial expression, or combinations thereof.

7. The method of claim 1, further comprising:

modifying, at the user device, the modified query based on user feedback to render a further modified query.

8. The method of claim 7, further comprising:

receiving, at the user device, the user feedback.

9. The method of claim 7, further comprising:

communicating the further modified query from the user device to a remote server.

10. The method of claim 1, wherein the search results are processed, at the remote server or the user device, to enhance graphic elements from the search results for immersive user experience.

11. The method of claim 1, wherein the search results are processed, at the remote server or the user device, to remove restricted contents for child users.

12. The method of claim 1, wherein accessibility by a user to the information accessible by the user device requires authentication, said authentication comprising learned biometric data unique to the user and locally stored on the user device.

13. The method of claim 1, wherein the user device can automatically perform continuous recognition of the user in the background.

14. A computer system comprising:

one or more processors; and

a non-transitory computer readable medium including one or more sequences of instructions that, when executed by the one or more processors, cause the processors to;

receive and identify a search query based on a user image input, the user image input collected in real-time of the user using a camera associated with the user device, said user image input including at least a portion of the user'"'"'s body separate from the user'"'"'s face;

attempt to authenticate the user using biometric information;

grant access to private information based on a successful authentication result;

limit access to the private information based on a failed authentication result;

modify the search query based on information accessible by the user device, as determined by the authentication result, using artificial intelligence to render a modified query;

communicate the modified-query from the user device to a remote server;

receive search results from the remote server; and

check for user input suggesting to end the search.

15. The system of claim 14, wherein the information accessible by the user device comprises user augmented reality data information, biometric data, user personal data, other people'"'"'s information that the user has permission to access, existing search records, information stored in the user device, system information of the user device, information that is extracted through real-time network communication at the time when the search query is modified, or any combinations thereof.

16. The system of claim 15, wherein the augmented reality data information including augmented reality input data information, the output augmented reality data information where the extracted real-life data from the input image was immersed into the input augmented reality data, or a combination thereof.

17. The system of claim 14, wherein the search results are processed, at the remote server or the user device, to enhance graphic elements from the search results for immersive user experience.

18. The system of claim 14, wherein the search results are processed, at the remote server or the user device, to remove restricted contents for child users.

19. The method of claim 14, wherein accessibility by a user to the information accessible by the user device requires authentication, said authentication comprising learned biometric data unique to the user and locally stored on the user device.

20. The system of claim 14, wherein the user device can receive instant feedback at any query processing stage if a search query is unsatisfactory.

21. The system of claim 14, wherein the user device can automatically perform continuous recognition of the user in the background.

22. A non-transitory computer-readable storage medium having stored thereon program code instructions that, when executed by a processor of a message management service, cause the message management service to perform a method comprising:

identifying, at a user device, a search query based on a user image input, the user image input collected in real-time of the user using a camera associated with the user device, said user image input including at least a portion of the user'"'"'s body separate from the user'"'"'s face;

attempting to authenticate the user using biometric information;

granting access to private information based on a successful authentication result;

limiting access to the private information based on a failed authentication result;

modifying the search query based on information accessible by the user device, as determined by the authentication result, using artificial intelligence to render a modified query;

communicating the modified query from the user device to a remote server;

receiving search results from the remote server; and

checking for user input suggesting to end the search.

23. The non-transitory computer-readable storage medium of claim 22, wherein the information accessible by the remote server comprises augmented reality data information, user biometric data, user personal data, other people'"'"'s information that the user has permission to access, existing search records, information stored in the user device, system information of the user device, information stored in the remote server, information that is extracted through real-time network communication at the time when the search query is modified, or any combinations thereof.

24. The non-transitory computer-readable storage medium of claim 23, wherein the augmented reality data information including augmented reality input data information, the output augmented reality data information where the extracted real-life data from the input image was immersed into the input augmented reality data, or a combination thereof.

25. The non-transitory computer-readable storage medium of claim 22, wherein the search results are processed, at the remote server or the user device, to enhance graphic elements from the search results for immersive user experience.

26. The non-transitory computer-readable storage medium of claim 22, wherein the search results are processed, at the remote server or the user device, to remove restricted contents for child users.

27. The non-transitory computer-readable storage medium of claim 22, wherein accessibility by a user to the information accessible by the user device requires authentication, said authentication comprising learned biometric data unique to the user and locally stored on the user device.

28. The non-transitory computer-readable storage medium of claim 22, wherein the user device can process feedback from a user at any query processing stage if a search query is unsatisfactory.

29. The non-transitory computer-readable storage medium of claim 22, wherein the user device can automatically perform continuous recognition of the user in the background.