Image recognition method and apparatus

• US 10,902,234 B2
• Filed: 06/04/2020
• Issued: 01/26/2021
• Est. Priority Date: 03/22/2018
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Feng, Zhou, Wu, Zhengshan
• Primary Examiner(s): Haupt, Kristy A
• Application Number:
  US16/893,300
• Publication Number:
  US 20200356742A1
• Time in Patent Office: 236 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06K 7/1417 2D bar codes
• CPC Class Codes: G06K 7/1443 locating of the code in an image
• CPC Class Codes: G06K 7/1447 extracting optical codes from image or text carrying said optical code
• CPC Class Codes: G06K 7/1465 using several successive scans of the optical code
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

An image recognition method of a terminal is provided. An image sensor is controlled to capture an image stream in response to receiving an image recognition trigger. A first image is extracted from the image stream. A determination is made as to whether an image type of the first image is one of an object image and a code image. One of a plurality of recognition processes is performed on a second image in the image stream according to the determined image type.

Claims

1. An image recognition method of a terminal, the method comprising:

controlling, by processing circuitry of the terminal, an image sensor to capture an image stream in response to receiving an image recognition trigger;

extracting, by the processing circuitry, a first image from the image stream;

determining, by the processing circuitry, whether an image type of the first image is one of an object image and a code image; and

performing, by the processing circuitry, one of a plurality of recognition processes on a second image in the image stream according to the determined image type, wherein

a first recognition process of the plurality of recognition processes is performed based on the image type of the first image being the object image, and

a second recognition process of the plurality of recognition processes is performed based on the image type of the first image being the code image.

2. The method according to claim 1, wherein the performing one of the plurality of recognition processes on the second image comprises:

extracting the second image from the image stream in a case that the image type of the first image is the object image;

transmitting a recognition request to a recognition server that is configured to recognize an object in the second image; and

receiving a first recognition result from the recognition server.

3. The method according to claim 2, further comprising:

recognizing a graphic code in the second image or a third image in the image stream in a case that the first recognition result indicates that the recognition server fails to recognize the object in the second image.

4. The method according to claim 2, wherein the extracting the second image from the image stream comprises:

presenting an image view-finding interface, the image view-finding interface being configured to display a real-time image obtained by the image sensor;

determining a focusing parameter according to a focusing operation performed via the image view-finding interface; and

extracting the second image captured by the image sensor according to the focusing parameter.

5. The method according to claim 2, wherein the extracting the second image from the image stream comprises:

obtaining, from the image stream, a target image satisfying a preset extraction condition as the second image.

6. The method according to claim 5, wherein the preset extraction condition comprises at least one of:

a difference value between the target image and a reference image in the image stream is less than a preset difference threshold, the reference image being located at a preset location that is relative to the target image;

ambient light luminance at a capture time corresponding to the target image is greater than a preset luminance threshold;

terminal acceleration at the capture time corresponding to the target image is less than a preset acceleration threshold;

or

the capture time corresponding to the target image is after a first time corresponding to a time the first image is captured, and a duration between the capture time and the first time is greater than a preset duration threshold.

7. The method according to claim 2, further comprising:

extracting a third image from the image stream at a preset sampling time interval; and

recognizing a graphic code in the third image.

8. The method according to claim 2, wherein the performing one of the plurality of recognition processes on the second image comprises:

performing recognition of a graphic code in a plurality of images in the image stream in order of capture time in a case that the image type of the first image is the code image.

9. The method according to claim 8, further comprising:

extracting a third image from the image stream in response to failing to recognize the graphic code within a time period of a preset length after an initial recognition time;

determining whether an image type of the third image is one of the object image and the code image; and

performing recognition on a fourth image in the image stream according to the determined image type of the third image.

10. The method according to claim 1, wherein the determining the image type of the first image comprises:

obtaining a probability that the first image includes the code image;

determining that the first image includes the code image in a case that the probability is greater than a preset probability threshold; and

determining that the image type of the first image is the object image in a case that the probability is less than or equal to the preset probability threshold.

11. The method according to claim 10, wherein the obtaining the probability that the first image includes the code image comprises:

inputting the first image into an image classification model, to obtain the probability that the first image includes the code image, the image classification model being a machine learning model that is trained with a sample image pre-labeled as a code image.

12. An image recognition apparatus, comprising:

processing circuitry configured to

control an image sensor to capture an image stream in response to receiving an image recognition trigger;

extract a first image from the image stream;

determine whether an image type of the first image is one of an object image and a code image; and

perform one of a plurality of recognition processes on a second image in the image stream according to the determined image type, wherein

a first recognition process of the plurality of recognition processes is performed based on the image type of the first image being the object image, and

a second recognition process of the plurality of recognition processes is performed based on the image type of the first image being the code image.

13. The image recognition apparatus according to claim 12, wherein the processing circuitry is configured to

extract the second image from the image stream in a case that the image type of the first image is the object image;

transmit a recognition request to a recognition server that is configured to recognize an object in the second image; and

receive a first recognition result from the recognition server.

14. The image recognition apparatus according to claim 13, wherein the processing circuitry is configured to

recognize a graphic code in the second image or a third image in the image stream in a case that the first recognition result indicates that the recognition server fails to recognize the object in the second image.

15. The image recognition apparatus according to claim 13, wherein the processing circuitry is configured to

present an image view-finding interface, the image view-finding interface being configured to display a real-time image obtained by the image sensor;

determine a focusing parameter according to a focusing operation performed via the image view-finding interface; and

extract the second image captured by the image sensor according to the focusing parameter.

16. The image recognition apparatus according to claim 13, wherein the processing circuitry is configured to

obtain, from the image stream, a target image satisfying a preset extraction condition as the second image.

17. The image recognition apparatus according to claim 16, wherein the preset extraction condition comprises at least one of:

a difference value between the target image and a reference image in the image stream is less than a preset difference threshold, the reference image being located at a preset location that is relative to the target image;

ambient light luminance at a capture time corresponding to the target image is greater than a preset luminance threshold;

terminal acceleration at the capture time corresponding to the target image is less than a preset acceleration threshold;

or

the capture time corresponding to the target image is after a first time corresponding to a time the first image is captured, and a duration between the capture time and the first time is greater than a preset duration threshold.

18. The image recognition apparatus according to claim 13, wherein the processing circuitry is configured to

extract a third image from the image stream at a preset sampling time interval;

and

recognize a graphic code in the third image.

19. The image recognition apparatus according to claim 13, wherein the processing circuitry is configured to

perform recognition of a graphic code in a plurality of images in the image stream in order of capture time in a case that the image type of the first image is the code image.

20. A non-transitory computer-readable storage medium storing instructions which when executed by a processor cause the processor to perform an image recognition method, the method comprising:

controlling an image sensor to capture an image stream in response to receiving an image recognition trigger;

extracting a first image from the image stream;

determining whether an image type of the first image is one of an object image and a code image; and

performing one of a plurality of recognition processes on a second image in the image stream according to the determined image type, wherein

a first recognition process of the plurality of recognition processes is performed based on the image type of the first image being the object image, and

a second recognition process of the plurality of recognition processes is performed based on the image type of the first image being the code image.