Artificially intelligent systems, devices, and methods for learning and/or using visual surrounding for autonomous object operation

• US 10,210,434 B1
• Filed: 12/07/2017
• Issued: 02/19/2019
• Est. Priority Date: 08/23/2016
• Status: Active Grant

• Current Assignee: Autonomous Devices LLC
• Original Assignee: Jasmin Cosic
• Inventors: Cosic, Jasmin
• Primary Examiner(s): Park, Soo Jin
• Application Number:
  US15/835,434
• Time in Patent Office: 439 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Aspects of the disclosure generally relate to computing devices and/or systems, and may be generally directed to devices, systems, methods, and/or applications for learning operation of an application or an object of an application in various visual surroundings, storing this knowledge in a knowledgebase (i.e. neural network, graph, sequences, etc.), and enabling autonomous operation of the application or the object of the application.

Claims

1. A system comprising:

one or more processor circuits;

a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first avatar of an application program;

a renderer configured to render digital pictures; and

an artificial intelligence unit configured to;

receive a new one or more digital pictures from the renderer;

anticipate the first one or more instruction sets for operating the first avatar of the application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipate includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage; and

cause the one or more processor circuits to execute the first one or more instruction sets for operating the first avatar of the application program, wherein the cause is performed in response to the anticipate, and wherein the first avatar of the application program or a second avatar of the application program autonomously performs one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

2. The system of claim 1, wherein the application program includes one or more versions, one or more upgrades, one or more sequels, one or more instances, or one or more variations of the application program.

3. The system of claim 1, wherein the application program includes:

a computer game, a video game, a simulation program, a program including text processing, a program including number processing, a program including picture processing, a program including object processing, or a program, and wherein the first avatar of the application program is a first object of the application program, and wherein the second avatar of the application program is a second object of the application program.

4. The system of claim 1, wherein the cause the one or more processor circuits to execute the first one or more instruction sets for operating the first avatar of the application program includes:

instrumenting the first avatar of the application program with the first one or more instruction sets for operating the first avatar of the application program,

instrumenting the second avatar of the application program with the first one or more instruction sets for operating the first avatar of the application program, or

instrumenting the application program with the first one or more instruction sets for operating the first avatar of the application program.

5. The system of claim 1, wherein the memory is further configured to store at least a second one or more digital pictures correlated with a second one or more instruction sets for operating a first avatar of another application program.

6. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a surrounding of the first avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are applied to the first avatar of the application program so that the first avatar of the application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

7. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

8. The system of claim 1, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first avatar of the application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first avatar of the application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

9. The system of claim 1, wherein the first one or more digital pictures include one or more still digital pictures or one or more motion digital pictures, and the new one or more digital pictures include one or more still digital pictures or one or more motion digital pictures, and wherein the memory is further configured to store at least a second one or more digital pictures correlated with a second one or more instruction sets for operating a third avatar of the application program, and wherein the third avatar of the application program is a third object of the application program.

10. The system of claim 1, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are modified and applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first avatar of the application program.

11. The system of claim 1, wherein the portions of the new one or more digital pictures include portions of the new one or more digital pictures that depict recognized objects, and wherein the portions of the first one or more digital pictures include portions of the first one or more digital pictures that depict recognized objects.

12. The system of claim 1, wherein the artificial intelligence unit includes at least one of:

a hardware element that is included in the one or more processor circuits, a hardware element that is included in another one or more processor circuits, an application operating on the one or more processor circuits, an application operating on another one or more processor circuits, or an element coupled to the one or more processor circuits, and wherein the renderer includes at least one of;

a hardware element that is included in the one or more processor circuits, a hardware element that is included in another one or more processor circuits, an application operating on the one or more processor circuits, an application operating on another one or more processor circuits, an element that is included in the artificial intelligence unit, or an element coupled to the artificial intelligence unit.

13. A non-transitory machine readable medium having stored thereon instructions that when executed by one or more processor circuits cause the one or more processor circuits to perform operations comprising:

accessing a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first avatar of an application program;

receiving a new one or more digital pictures from a renderer;

anticipating the first one or more instruction sets for operating the first avatar of the application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipating includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage; and

causing the one or more processor circuits or another one or more processor circuits to execute the first one or more instruction sets for operating the first avatar of the application program, wherein the causing is performed in response to the anticipating, and wherein the first avatar of the application program or a second avatar of the application program autonomously performs one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

14. The non-transitory machine readable medium of claim 13, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first avatar of the application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first avatar of the application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

15. The non-transitory machine readable medium of claim 13, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

16. The non-transitory machine readable medium of claim 13, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are modified and applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first avatar of the application program.

17. A method comprising:

(a) accessing a memory configured to store at least a first one or more digital pictures correlated with a first one or more instruction sets for operating a first avatar of an application program, the accessing of (a) performed by one or more processor circuits;

(b) receiving a new one or more digital pictures from a renderer, the receiving of (b) performed by the one or more processor circuits;

(c) anticipating the first one or more instruction sets for operating the first avatar of the application program based on at least partial match between the new one or more digital pictures and the first one or more digital pictures, wherein the anticipating of (c) includes at least one of;

determining that a number of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold number, or determining that a percentage of at least partially matching portions of the new one or more digital pictures and portions of the first one or more digital pictures exceeds a threshold percentage, the anticipating of (c) performed by the one or more processor circuits;

(d) executing the first one or more instruction sets for operating the first avatar of the application program, the executing of (d) performed by the one or more processor circuits or by another one or more processor circuits in response to the anticipating of (c); and

(e) autonomously performing, by the first avatar of the application program or by a second avatar of the application program, one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

18. The method of claim 17, wherein the memory is further configured to store at least a knowledge structure that includes:

a first knowledge cell including the first one or more digital pictures correlated with the first one or more instruction sets for operating the first avatar of the application program, and

a second knowledge cell including a second one or more digital pictures correlated with a second one or more instruction sets for operating the first avatar of the application program, and wherein the first knowledge cell is connected with the second knowledge cell by a directed connection that indicates a temporal order of the first knowledge cell and the second knowledge cell.

19. The method of claim 17, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the first one or more instruction sets for operating the first avatar of the application program.

20. The method of claim 17, wherein the new one or more digital pictures depict at least a portion of a surrounding of the second avatar of the application program, and wherein the first one or more instruction sets for operating the first avatar of the application program are modified and applied to the second avatar of the application program so that the second avatar of the application program autonomously performs the one or more operations defined by the modified first one or more instruction sets for operating the first avatar of the application program.