Prioritized device actions triggered by device scan data

• US 10,678,849 B1
• Filed: 09/14/2016
• Issued: 06/09/2020
• Est. Priority Date: 09/15/2015
• Status: Active Grant

• Current Assignee: Snap, Inc.
• Original Assignee: Snap, Inc.
• Inventors: Ouimet, Kirk, Yan, Rong, Zheng, Yantao
• Primary Examiner(s): Thai, Hanh B
• Application Number:
  US15/265,672
• Time in Patent Office: 1,364 Days
• Field of Search: 707769
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9038 Presentation of query results
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

Systems, methods, devices, server computers, storage media, and instructions for prioritized device action triggered by device scan data are described. In one embodiment, a mobile device performs a method that involves executing a messaging application with an image capture interface and a scanning input. An associated scanning mode comprises capture of scan data from a plurality of input/output modules of the first client device, analyzes the scan data to identify one or more scan data patterns by matching at least a portion of the scan data against a set of data patterns, and selects a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns. In some embodiments, the priority system action is selected based on a priority ranking for identified scan data types.

Claims

1. A processor implemented method of prioritized device action triggered by device scan data comprising:

executing, via one or more processors, a messaging application on a first client device, the messaging application comprising an image capture interface;

initiating, via a scanning input of the image capture interface, a scanning mode, wherein the scanning mode comprises capture of scan data from a plurality of input/output modules of the first client device;

analyzing, using the one or more processors, the scan data to identify one or more scan data patterns by matching at least a portion of the scan data against a set of data patterns;

identifying, using the one or more processors, a plurality of scan data patterns from the scan data, where each scan data patterns of the plurality of scan data patterns matches at least one data pattern of the set of data patterns; and

selecting, using the one or more processors, a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns;

wherein selecting the priority system action comprises selecting a first scan data pattern of the plurality of scan data patterns based on a pattern type ranking.

2. The method of claim 1 further comprising:

presenting, on a display of the first client device, a priority action interface in response to the selection of the priority system action.

3. The method of claim 1 wherein the pattern type ranking comprises a first level ranking for a scan code pattern;

wherein the scan data comprises an image captured using a camera coupled to the one or more processors; and

wherein the image comprises a first scan code.

4. The method of claim 3 wherein the pattern type ranking further comprises ranking levels for audio signatures, logos, and registered keywords.

5. The method of claim 3 wherein the plurality of scan data patterns comprises two or more pattern types of a set of pattern types, the set of pattern types comprising:

the scan code pattern;

an audio signature pattern;

a logo pattern;

a pattern associated with a third party registered keyword; and

a pattern associated with a system keyword.

6. The method of claim 1 wherein the plurality of input/output modules comprises:

a camera device;

a position sensor;

a temperature sensor;

a motion sensor;

a microphone; and

one or more wireless communication modules.

7. The method of claim 1 wherein the scan data comprises two or more data types selected from a set of data types comprising:

wireless access point name data;

image data;

video data;

audio data;

location data;

temperature data;

motion data.

8. The method of claim 1 further comprising:

communicating a second portion of the scan data to a third party search server when a match of the portion of the scan data and the set of pattern data is not found;

wherein the priority system action is associated with a set of search results from the third party search server.

9. The method of claim 1 wherein analyzing, using the one or more processors, the scan data to identify the one or more scan data patterns by matching at least the portion of the scan data against the set of data patterns comprises processing one or more images of the scan data using a neural network implemented on the first client device to identify keywords associated with one or more objects from the one or more images.

10. The method of claim 9 wherein the priority system action comprises performing an image search using the keywords associated with the one or more objects;

and

presenting an image search interface on a display of the first client device with images from a local image gallery of the first client device.

11. The method of claim 10 wherein the priority system action further comprises communicating an image search request to a networked server computer using the keywords;

receiving a plurality of search results from the networked server computer in response to the image search request; and

presenting the image search interface on the display with the plurality of search results from the networked server computer.

12. A client device configured for prioritized device action triggered by device scan data, the client device comprising:

a memory; and

one or more processors coupled to the memory and configured to;

execute a messaging application, the messaging application comprising an image capture interface;

initiate, in response to a scanning input command received via of the image capture interface, a scanning mode, wherein the scanning mode comprises capture of scan data from a plurality of input/output modules of the first client device;

automatically analyze the scan data to identify one or more scan data patterns by matching at least a portion of the scan data against a set of data patterns;

identify a plurality of scan data patterns, where each scan data patterns of the plurality of scan data patterns matches at least one data pattern of the set of data patterns; and

selecting, using the one or more processors, a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns; and

automatically select a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns;

wherein selecting the priority system action comprises selecting a first scan data pattern of the plurality of scan data patterns based on a pattern type ranking.

13. The client device of claim 12 further comprising:

display coupled to the one or more processors and configured to output a priority action interface in response to the selection of the priority system action.

14. The client device of claim 12 further comprising:

a network interface coupled to the one or more processors and configured to communicate with one or more server computers;

wherein the one or more processors are further configured to;

initiate communication of a second portion of the scan data to a third party search server using the network interface when a match of the portion of the scan data and the set of pattern data is not found;

wherein the priority system action is associated with a set of search results from the third party search server.

15. A processors readable storage medium comprising instructions that, when executed by one or more processors of a client device, cause the client device to perform operations comprising:

initiating a scanning mode, wherein the scanning mode comprises capture of scan data from a plurality of input/output modules of the first client device;

analyzing, using the one or more processors, the scan data to identify one or more scan data patterns by matching at least a portion of the scan data against a set of data patterns;

identifying, using the one or more processors, a plurality of scan data patterns, where each scan data patterns of the plurality of scan data patterns matches at least one data pattern of the set of data patterns;

identifying, using the one or more processors, a plurality of scan data patterns, where each scan data patterns of the plurality of scan data patterns matches at least one data pattern of the set of data patterns; and

selecting, using the one or more processors, a priority system action based on the results of the matching of the portion of the scan data against the set of data patterns based on a pattern type ranking;

wherein selecting the priority system action comprises selecting a first scan data pattern of the plurality of scan data patterns based on a pattern type ranking.

16. The processor readable storage medium of claim 15 wherein the plurality of scan data patterns comprises two or more pattern types of a set of pattern types each associated with a pattern type ranking, the set of pattern types two or more of:

a code pattern;

an audio signature pattern;

a logo pattern;

a pattern associated with a third party registered keyword; and

a pattern associated with a system keyword.

17. The processor readable storage medium of claim 16 wherein the scan data comprises:

wireless access point name data;

image data;

video data;

audio data;

location data;

temperature data; and

motion data.

18. The processor readable storage medium of claim 15 wherein analyzing, using the one or more processors, the scan data to identify the one or more scan data patterns by matching at least the portion of the scan data against the set of data patterns comprises processing one or more images of the scan data using a neural network implemented on the first client device to identify keywords associated with one or more objects from the one or more images;

wherein the priority system action comprises;

performing an image search using the keywords associated with the one or more objects to identify images from a local image gallery of the first client device associated with the keywords;

communicating an image search request to a networked server computer using the keywords;

receiving a plurality of search results from the networked server computer in response to the image search request; and

presenting an image search interface on the display with at least a portion of the images and at least a portion of the plurality of search results from the networked server computer.