Data capture and identification system and process

• US 10,509,821 B2
• Filed: 10/08/2018
• Issued: 12/17/2019
• Est. Priority Date: 11/06/2000
• Status: Expired due to Term

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Boncyk, Wayne C., Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US16/154,547
• Publication Number:
  US 20190042599A1
• Time in Patent Office: 435 Days
• Field of Search: 382103, 382195, 382305
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70
• CPC Class Codes: H04M 1/0202 Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets constructional features of telephone transmitters or receivers, e.g. of speakers or microphones H04M1/03

Abstract

A mobile system that performs object recognition, identifying real-world objects in a scene from captured digital data of the scene. The mobile system identifies the real-world objects based on derived salient characteristics from the digital data, and retrieves stored object information corresponding to the identified object such that a CPU can execute a software process based on the retrieved object information about the real-world object.

Claims

1. A mobile system that performs object recognition, comprising:

at least one data capture device;

a computer system programmed to;

store reference characteristics corresponding to a plurality of target objects and object information associated with each of the plurality of target objects;

receive, from the at least one data capture device, digital data of a scene captured by the at least one data capture device, the digital data comprising at least one of image data and position data related to a digital representation of at least one real-world object within the scene;

derive salient characteristics for the at least one real-world object based on the digital representation including the position data;

identify the at least one real-world object as at least one target object among the plurality of target objects based on the derived salient characteristics and the stored reference characteristics;

retrieve the object information associated with the identified at least one target object; and

execute a software process based on the object information.

2. The mobile system that performs object recognition of claim 1, wherein the computer system is further programmed to determine the position and orientation of the at least one real-world object relative to the at least one data capture device based on the object information.

3. The mobile system that performs object recognition of claim 2, wherein the at least one real-world object comprises at least one of a person or a vehicle.

4. The mobile system that performs object recognition of claim 1, wherein the at least one real-world object comprises a machine and the computer system is further programmed to establish a communication link with the machine based on the object information.

5. The mobile system that performs object recognition of claim 4, wherein the computer system is further programmed to enable a user to interact with the machine via a display screen.

6. The mobile system that performs object recognition of claim 1, wherein the at least one data capture device comprises at least two data capture devices.

7. The mobile system that performs object recognition of claim 1, wherein the digital representation of the at least one real-world object comprises a depiction of at least a portion of the at least one real-world object within the image data.

8. The mobile system that performs object recognition of claim 7, wherein the image data comprises video data.

9. The mobile system that performs object recognition of claim 1, wherein the at least one data capture device, and the computer system are integral to a single device.

10. The mobile system that performs object recognition of claim 1, wherein the computer system is further programmed to present, via a display, information about the real-world object based on the object information.

11. A method for object recognition by a mobile system, comprising:

storing, by a computer system, reference characteristics corresponding to a plurality of target objects and object information associated with each of the plurality of target objects;

capturing, by at least one data capture device, digital data of a scene, the digital data comprising at least one of image data and position data related to a digital representation of at least one real-world object within the scene;

receiving, by the computer system and from the at least one data capture device, the digital data of the scene;

deriving, by the computer system, salient characteristics for the at least one real-world object based on the digital representation including the position data;

identifying, by the computer system, the at least one real-world object as at least one target object among the plurality of target objects based on the derived salient characteristics and the stored reference characteristics;

retrieving, by the computer system, the object information associated with the identified at least one target object;

executing, by the computer system, a software process based on the object information.

12. The method of claim 11, wherein the step of executing the software process comprises determining, by the computer system, the position and orientation of the at least one real-world object relative to the at least one data capture device based on the object information.

13. The method of claim 12, wherein the at least one real-world object comprises at least one of a person or a vehicle.

14. The method of claim 11, wherein the at least one real-world object comprises a machine and the step of executing the software process comprises establishing, by the computer system, a communication link with the machine based on the object information.

15. The method of claim 14, wherein the step of executing the software process further comprises enabling, by the computer system, a user to interact with the machine via a display screen.

16. The method of claim 11, wherein the at least one data capture device comprises at least two data capture devices.

17. The method of claim 11, wherein the digital representation of the at least one real-world object comprises a depiction of at least a portion of the at least one real-world object within the image data.

18. The method of claim 17, wherein the image data comprises video data.

19. The method of claim 11, wherein the at least one data capture device and the computer system are integral to a single device.

20. The method of claim 11, wherein the step of executing the software process comprises the presenting, by the computer system and via a display, information about the real-world object based on the object information.

21. A non-transitory computer readable medium comprising instructions executable by a computer processor to execute processing comprising:

storing, by a computer system, reference characteristics corresponding to a plurality of target objects and object information associated with each of the plurality of target objects;

capturing, by at least one data capture device, digital data of a scene, the digital data comprising at least one of image data and position data related to a digital representation of at least one real-world object within the scene;

receiving, by the computer system and from the at least one data capture device, the digital data of the scene;

deriving, by the computer system, salient characteristics for the at least one real-world object based on the digital representation including the position data;

identifying, by the computer system, the at least one real-world object as at least one target object among the plurality of target objects based on the derived salient characteristics and the stored reference characteristics;

retrieving, by the computer system, the object information associated with the identified at least one target object;

executing, by the computer system, a software process based on the object information.