Mobile Wireless Object Recognition and Control

• US 20130335611A1
• Filed: 06/16/2012
• Published: 12/19/2013
• Est. Priority Date: 06/16/2012
• Status: Active Grant

• Current Assignee: John Livacich, Kendyl A. Roman
• Original Assignee: Kendyl A. Roman
• Inventors: Romn, Kendyl A., Livacich, John
• Granted Patent: US 9,336,238 B2
• Time in Patent Office: Days
• US Class Current: 348/333.01
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06Q 10/063114 Status monitoring or status determination for a person or group
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass

Abstract

A system, handheld devices, and methods for object tracking and control. The system tracks and controls items, people, and their interactions. The handheld mobile wireless device provides item specific information, instruction, and supervision. A plurality of handheld devices is configured to be used by different people, and communicate with a central database. A handheld device comprises a camera for scanning an image of an object, a display for displaying information, and a wireless network connection to the database for sending and receiving data. The system recognizes the object and identifies a person. The database includes information regarding people and their roles, permissions, and authorized or assigned tasks; objects and tools; object specific instructions; and the handheld devices. The instructions include documents, checklists, pictures, audio, and video. An observer, or the system, can monitor performance of the tasks, and provide corrective feedback.

Claims

1. A system for object recognition and control, the system comprising:

a) a plurality of handheld devices configured to be used by a plurality of persons, and

b) a database configured to communicate with the plurality of handheld devices,

wherein each handheld device comprises;

i) a camera configured to scan an image of an object,

ii) a display for displaying information, and

iii) a wireless network connection to the database for sending outgoing database information to the database and for receiving incoming database information from the database,

wherein the system is further configured to recognize the object by interpreting a coded label,

wherein the handheld device is further configured to identify a person using the handheld device, by scanning one of a face, an iris, a fingerprint, or a scan label of the person,

wherein the database is configured to store data relating to the object, the handheld devices, and the plurality of persons,

wherein the database data includes role information for the person based on the object, wherein the role is one of the group of maintenance, use, and observation,

wherein the database contains object specific instructions for each role,

wherein the object specific instructions comprise one or more of the group of;

i) checklist,

ii) pictures,

iii) audio, and

iv) video;

whereby when a first one of the plurality of handheld devices scans the object, the system recognizes the object and determines the role of the person, the database sends the object specific instructions to the first handheld device, the object specific instructions are displayed on the first handheld device, and the database records data about object recognized, the first handheld device, and the person.

2. The system of claim 1, wherein the role of the person is maintenance,

wherein the object specific instructions include a list of parts and tools required, if any, and at least one video showing the maintenance steps.

3. The system of claim 1, wherein the role of the person is use,

wherein the object specific instructions include at least one video showing how to perform a task using the object.

4. The system of claim 1, wherein the role of the person is observer,

wherein the observer is notified of the start of maintenance or use of the object by another person.

5. A system for object recognition and control, the system comprising:

a) a plurality of handheld devices configured to be used by a plurality of persons, and

b) a database configured to communicate with the plurality of handheld devices,

wherein each handheld device comprises;

i) a camera configured to scan an image of an object,

ii) a display for displaying information, and

iii) a wireless network connection to the database for sending outgoing database information to the database and for receiving incoming database information from the database,

wherein the database is configured to store data relating to the object, the handheld devices, and the plurality of persons,

whereby when a first one of the plurality of handheld devices scans the object, the system recognizes the object, the database sends object specific instructions to the first handheld device, the object specific instructions are displayed on the first handheld device, and the database records data about object recognized and the first handheld device.

6. The system of claim 5 wherein the system is further configured to recognize the object by interpreting a coded label.

7. The system of claim 5 wherein the handheld device is further configured to identify a person using the handheld device.

8. The system of claim 7 wherein the person is identified by scanning a scan label associated with the person.

9. The system of claim 7 wherein the person is identified by scanning one of a face, an iris, or a fingerprint.

10. The system of claim 7 wherein the database is configured to maintain role information for the person based on the object,

wherein the role is one of the group of maintenance, use, and observation.

11. The system of claim 10 wherein the object specific instructions are based on the role of the person.

12. The system of claim 10 wherein the object specific instructions comprise one or more of the group of:

i) checklist,

ii) pictures,

iii) audio, and

iv) video.

13. The system of claim 7 wherein the database is configured to maintain role information for the person based on the object,

wherein the role of the person is maintenance,

wherein the object specific instructions include a list of parts and tools required, if any, and at least one video showing the maintenance steps.

14. The system of claim 7 wherein the database is configured to maintain role information for the person based on the object,

wherein the role of the person is use,

wherein the object specific instructions include at least one video showing how to perform a task using the object.

15. The system of claim 7 wherein the database is configured to maintain role information for the person based on the object,

wherein the role of the person is observer,

wherein the observer is notified of the start of maintenance or use of the object by another person.

16. A handheld device for object recognition and control to be operated by a person having a predetermined role, the handheld device comprising:

a) a camera configured to scan an image of an object,

b) a display for displaying information, and

c) a wireless network connection, configured to communicate to a database, for sending outgoing database information to the database and for receiving incoming database information from the database,

wherein the handheld device interacts with the database to recognize the object,

wherein the handheld device identifies the person using the handheld device,

wherein the predetermined role is one of the group of maintenance, use, and observation,

wherein the handheld device sends and receives data relating to the object, the handheld device, and the person,

wherein the data received contains object specific instructions for the predetermined role,

wherein the object specific instructions comprise one or more of the group of;

i) checklist,

ii) pictures,

iii) audio, and

iv) video;

whereby the object specific instructions are displayed on the display.

17. A method for object recognition and control using the handheld device of claim 16, the method comprising the steps of:

a) capturing an image with the camera,

b) interpreting the image,

c) determining an object,

d) determining an identity of the person,

e) determining if the person is authorized for the object,

f) if not authorized, indicating no authorized access,

g) otherwise, based on the predetermined role of the person allowing and tracking one or more of the group of;

i) performing maintenance,

ii) performing use, and

iii) performing observation.

18. The method for object recognition and control of claim 17 further comprising steps for performing maintenance of the object:

a) identifying a part associated with the maintenance of the object,

b) identifying zero or more tools that may be required for the current part,

c) displaying object specific instructions for the current maintenance task,

d) confirming completion of the current maintenance task associated with the part,

e) sending completion information to the database for recording in the database,

f) if more tasks are required, performing steps a) through e) for each remaining maintenance task.

19. The method for object recognition and control of claim 17 further comprising steps for performing use of the object:

a) if tools are required for the use of the object, identifying one or more tools,

b) providing object specific instructions for use of the object,

c) confirming completion of the current use task,

d) sending completion information to the database for recording in the database,

e) if more tasks are required, performing steps a) through d) for each remaining use task.

20. The method for object recognition and control of claim 17 further comprising steps for performing observation of the object:

a) if instruction is needed, providing object specific instructions,

b) sending observation start information to the database for recording in the database,

c) sending observation finish information to the database for recording in the database,

d) if more tasks are required, performing steps a) through c) for each remaining observation task.

21. The method for object recognition and control of claim 17 wherein for each confirming completion of the current maintenance task step, the current part and current state of the object being maintained is determined by object recognition, said object recognition comprising the steps of:

a) scanning a current image of the object with the camera,

b) comparing the current image to an current expected image from a plurality of expected images stored in the database and associated with the object,

c) determining whether or not the current image substantially matches the current expected image.