Artificially intelligent systems, devices, and methods for learning and/or using visual surrounding for autonomous object operation

• US 10,223,621 B1
• Filed: 12/04/2017
• Issued: 03/05/2019
• Est. Priority Date: 08/23/2016
• Status: Active Grant

• Current Assignee: Autonomous Devices LLC
• Original Assignee: Jasmin Cosic
• Inventors: Cosic, Jasmin
• Primary Examiner(s): Park, Soo Jin
• Application Number:
  US15/829,984
• Time in Patent Office: 456 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Aspects of the disclosure generally relate to computing devices and/or systems, and may be generally directed to devices, systems, methods, and/or applications for learning operation of an application or an object of an application in various visual surroundings, storing this knowledge in a knowledgebase (i.e. neural network, graph, sequences, etc.), and enabling autonomous operation of the application or the object of the application.

Claims

1. A system comprising:

one or more processor circuits;

a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first application program;

a renderer configured to render digital pictures; and

an artificial intelligence unit configured to;

receive a new one or more digital pictures from the renderer;

anticipate the first one or more instruction sets for operating the first application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipate includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage; and

cause the one or more processor circuits to execute the first one or more instruction sets for operating the first application program, wherein the cause is performed in response to the anticipate, and wherein the first application program or a second application program autonomously performs one or more operations defined by the first one or more instruction sets for operating the first application program.

2. The system of claim 1, wherein the first one or more instruction sets for operating the first application program include one or more instruction sets for operating an object of the first application program, and wherein the first application program or the second application program autonomously performing the one or more operations defined by the first one or more instruction sets for operating the first application program includes the object or another object of the first application program or an object of the second application program autonomously performing one or more operations defined by the one or more instruction sets for operating the object of the first application program.

3. The system of claim 1, wherein the first one or more digital pictures depict at least a portion of a surrounding of an object of the first application program, and wherein the new one or more digital pictures depict:

at least a portion of a surrounding of the object of the first application program, at least a portion of a surrounding of another object of the first application program, or at least a portion of a surrounding of an object of the second application program.

4. The system of claim 1, wherein the first application program includes:

a computer game, a video game, a simulation program, a program including text processing, a program including number processing, a program including picture processing, a program including object processing, or a program, and wherein the second application program includes;

a computer game, a video game, a simulation program, a program including text processing, a program including number processing, a program including picture processing, a program including object processing, or a program, and wherein the memory is further configured to store at least a second one or more digital pictures correlated with a second one or more instruction sets for operating the first application program.

5. The system of claim 1, wherein the cause the one or more processor circuits to execute the first one or more instruction sets for operating the first application program includes:

instrumenting at least a portion of the first application program with the first one or more instruction sets for operating the first application program, or

instrumenting at least a portion of the second application program with the first one or more instruction sets for operating the first application program.

6. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a view of the first application program, and wherein the first one or more instruction sets for operating the first application program are applied to the first application program so that the first application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first application program.

7. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are applied to the second application program so that the second application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first application program.

8. The system of claim 1, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

9. The system of claim 1, wherein the first one or more digital pictures include one or more still digital pictures or one or more motion digital pictures, and wherein the new one or more digital pictures include one or more still digital pictures or one or more motion digital pictures, and wherein the first application program includes an object of the first application program, and wherein the second application program includes an object of the second application program.

10. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are modified and applied to the second application program so that the second application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first application program.

11. The system of claim 1, wherein the portions of the new one or more digital pictures include portions of the new one or more digital pictures that depict recognized objects, and wherein the portions of the first one or more digital pictures include portions of the first one or more digital pictures that depict recognized objects.

12. The system of claim 1, wherein the artificial intelligence unit includes at least one of:

a hardware element that is included in the one or more processor circuits, a hardware element that is included in another one or more processor circuits, an application operating on the one or more processor circuits, an application operating on another one or more processor circuits, or an element coupled to the one or more processor circuits, and wherein the renderer includes at least one of;

a hardware element that is included in the one or more processor circuits, a hardware element that is included in another one or more processor circuits, an application operating on the one or more processor circuits, an application operating on another one or more processor circuits, an element that is included in the artificial intelligence unit, or an element coupled to the artificial intelligence unit, and wherein the memory is further configured to store at least a second one or more digital pictures correlated with a second one or more instruction sets for operating the second application program.

13. A non-transitory machine readable medium having stored thereon instructions that when executed by one or more processor circuits cause the one or more processor circuits to perform operations comprising:

accessing a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first application program;

receiving a new one or more digital pictures from a renderer;

anticipating the first one or more instruction sets for operating the first application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipating includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage; and

causing the one or more processor circuits or another one or more processor circuits to execute the first one or more instruction sets for operating the first application program, wherein the causing is performed in response to the anticipating, and wherein the first application program or a second application program autonomously performs one or more operations defined by the first one or more instruction sets for operating the first application program.

14. The non-transitory machine readable medium of claim 13, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

15. The non-transitory machine readable medium of claim 13, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are applied to the second application program so that the second application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first application program.

16. The non-transitory machine readable medium of claim 13, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are modified and applied to the second application program so that the second application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first application program.

17. A method comprising:

(a) accessing a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first application program, the accessing of (a) performed by one or more processor circuits;

(b) receiving a new one or more digital pictures from a renderer, the receiving of (b) performed by the one or more processor circuits;

(c) anticipating the first one or more instruction sets for operating the first application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipating of (c) includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage, the anticipating of (c) performed by the one or more processor circuits;

(d) executing the first one or more instruction sets for operating the first application program, the executing of (d) performed by the one or more processor circuits or by another one or more processor circuits in response to the anticipating of (c); and

(e) autonomously performing, by the first application program or by a second application program, one or more operations defined by the first one or more instruction sets for operating the first application program.

18. The method of claim 17, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

19. The method of claim 17, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are applied to the second application program so that the second application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first application program.

20. The method claim 17, wherein the new one or more digital pictures depict at least a portion of a view of the second application program, and wherein the first one or more instruction sets for operating the first application program are modified and applied to the second application program so that the second application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first application program.