Method, system and computer storage medium for visual searching based on cloud service

• US 9,411,849 B2
• Filed: 04/09/2013
• Issued: 08/09/2016
• Est. Priority Date: 04/25/2012
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Liu, Hailong, Hou, Jie, Xiong, Pengfei, Chen, Bo, Zhou, Xiaobo, Rao, Feng
• Primary Examiner(s): Ly, Anh
• Application Number:
  US14/241,863
• Publication Number:
  US 20150046483A1
• Time in Patent Office: 1,218 Days
• Field of Search: 707/758, 707/706, 382/156, 382/190, 382/209
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A method, system and computer storage medium for visual searching based on cloud service is disclosed. The method includes: receiving, from a client, an image recognition request of cloud service, the request containing image data; forwarding, according to a set classified forwarding rule, the image data to a corresponding classified visual search service; recognizing, by the respective corresponding classified visual search services, corresponding classified type information in the image data, and determining a corresponding name of the image data in accordance with the respective classified type information, and obtaining a classified visual search result; summarizing and sending, to a client, the classified visual search result of the corresponding classified visual search service. By detection and recognition of the classified type information of the image data, the comprehensive feature information of a picture is obtained, based on which further applications are allowed, and thus the user experience is improved.

Claims

1. A method for visual searching based on cloud service, comprising:

receiving, from a client, an image recognition request of cloud service, the request containing image data;

forwarding, according to a set classified forwarding rule, the image data to a corresponding classified visual search service, wherein the forwarding comprises;

detecting classified type information contained in the image data, and forwarding the image data to the classified visual search service corresponding to the detected image data;

detecting, by the respective corresponding classified visual search service, whether the image data contains the corresponding classified type information;

if yes, detect the position information of the classified type information in the image data, and recognize according to the position information a corresponding name;

recognizing, by the respective corresponding classified visual search services, the corresponding classified type information in the image data, and determining the corresponding name of the image data in accordance with the respective classified type information;

obtaining a classified visual search result, the result including the classified type information and the corresponding name; and

summarizing and sending, to the client, the classified visual search result of the corresponding classified visual search service.

2. The method of claim 1, further comprising, before the forwarding, the step of:

recognizing the respective classified type information contained in the request, and the position information of the respective classified type information in the image data; and

setting a classified forwarding rule according to the recognized respective classified type information and the position information, and forwarding, to the classified visual search service in accordance with the recognized classified type information, the image data and the corresponding position information.

3. The method of claim 2, wherein the setting comprises:

if the image recognition request of cloud service contains the classified type information, then forward the image data to the respective classified visual search service in accordance with the classified type information; and

if the image recognition request of cloud service does not contain the classified type information, then forward the image data to all the visual search services.

4. The method of claim 1, further comprising:

after the receiving, the step of;

preprocessing the image recognition request of cloud service, the preprocessing comprising operation of security verification.

5. The method of claim 1, further comprising, after the determining and before the summarizing and sending, the step of:

retrieving, from the respective classified visual search service, detailed information relevant to the specified type according to the recognized classified type information and the name;

the classified visual search result comprising the retrieved detailed information.

6. The method of claim 1, wherein the classified visual search service comprises one or more of:

object visual search service, text visual search service, face visual search service, barcode visual search service, scenery visual search service, Logo visual search service, book visual search service, and compact disc (CD) visual search service, and wherein the classified type information comprises one or more of;

object, text, face, barcode, scenery, Logo, books, and CD.

7. A visual search system based on cloud service, comprising:

a server, comprising;

a cloud service access device, configured to;

receive, from a client, an image recognition request of cloud service containing image data, and

forward, according to a set classified forwarding rule, the image data to a corresponding classified visual search service in a classified visual service system;

a classified visual service system, configured to;

detect, with the respective classified visual search service, whether the image data contains the corresponding classified type information;

if yes, detect the position information of the classified information in the image data, and recognize according to the position information a corresponding name;

recognize, with the respective corresponding classified visual search service, the corresponding classified type information in the image data,

determine the corresponding name of the image data in accordance with the respective classified type information to obtain a classified visual search result, the result containing the classified type information and the corresponding name;

a visual search results summarization device, configured to;

send, upon summarization, the respective corresponding classified visual search results to a client; and

an image recognition service device arranged in the client, configured to;

detect the respective classified type information contained in the image and the position information of the respective classified type information in the image data, and

send, to the server, the image recognition request of cloud service according to the detected classified type information, the request comprising the recognized classified type information, the corresponding position information and the image data of the image.

8. The system of claim 7, further comprising:

an image recognition service device connected between the cloud service access device and the classified visual service system, configured to;

recognize the respective classified type information contained in the image data of the image recognition request of cloud service and the position information of the respective classified type information in the image data, and

set a classified forwarding rule according to the recognized respective classified type information and the position information, and forward, to the corresponding classified visual search service, the image data and the position information in accordance with the recognized classified type information.

9. The system of claim 8, wherein the setting comprises:

if the image recognition request of cloud service contains the classified type information, then forward the image data to the respective classified visual search service in accordance with the classified type information; and

if the image recognition request of cloud service does not contain the classified type information, then forward the image data to all the visual search services.

10. The system of claim 7, wherein the cloud service access device is further configured to perform preprocessing operation to the image recognition request of cloud service, the preprocessing comprising operation of security verification.

11. The system of claim 7, wherein the classified visual service system is further configured to:

retrieve, from the respective classified visual search service, detailed information relevant to the specified type according to the recognized classified type information and the name;

the classified visual search result comprising the retrieved detailed information.

12. The system of claim 7, wherein the classified visual search service comprises any one or more of the followings:

object visual search service, text visual search service, face visual search service, barcode visual search service, scenery visual search service, Logo visual search service, book visual search service, and compact disc (CD) visual search service, and the classified type information comprises any one or more of the followings;

object, text, face, barcode, scenery, Logo, books, and CD.

13. The system of claim 7, further comprising the client.

14. A non-transitory computer-readable storage medium comprising computer-executable instructions for visual searching based on cloud service, the computer-executable instructions comprising instructions for:

receiving, from a client, an image recognition request of cloud service, the request containing image data;

forwarding, according to set classified forwarding rule, the image data to a corresponding classified visual search service, wherein the forwarding comprises;

detecting the classified type information contained in the image data, and forwarding the image data to the classified visual search service corresponding to the detected image data;

detecting, by the respective corresponding classified visual search service, whether the image data contains the corresponding classified type information;

if yes, detect the position information of the classified type information in the image data;

recognizing, according to the position information, a corresponding name;

recognizing, by the respective corresponding classified visual search services, the corresponding classified type information in the image data, and determining the name of the image data in accordance with the respective classified type information, and obtaining a classified visual search result, the result including the classified type information and the corresponding name; and

summarizing and sending, to a client, the classified visual search result of the corresponding classified visual search service.

15. The non-transitory computer-readable storage medium of claim 14, the computer-executable instructions further comprising instructions for:

before the forwarding, recognizing the respective classified type information contained in the request, and the position information of the respective classified type information in the image data;

setting a classified forwarding rule according to the recognized respective classified type information and the position information; and

forwarding, to the classified visual search service in accordance with the recognized classified type information, the image data the corresponding position information.

16. The non-transitory computer-readable storage medium of claim 15, wherein the setting comprises:

if the image recognition request of cloud service contains the classified type information, then forward the image data to the respective classified visual search service in accordance with the classified type information; and

if the image recognition request of cloud service does not contain the classified type information, then forward the image data to all the visual search services.

17. The non-transitory computer-readable storage medium of claim 14, the computer-executable instructions further comprising instructions for:

after the receiving;

preprocessing the image recognition request of cloud service, the preprocessing comprising operation of security verification.

18. The non-transitory computer-readable storage medium of claim 14, the computer-executable instructions further comprising instructions for:

after the determining and before the summarizing and sending;

retrieving, from the respective classified visual search service, detailed information relevant to the specified type according to the recognized classified type information and the name;

the classified visual search result comprising the retrieved detailed information.