Artificially intelligent systems, devices, and methods for learning and/or using visual surrounding for autonomous object operation

• US 9,864,933 B1
• Filed: 08/23/2016
• Issued: 01/09/2018
• Est. Priority Date: 08/23/2016
• Status: Active Grant

• Current Assignee: Autonomous Devices LLC
• Original Assignee: Jasmin Cosic
• Inventors: Cosic, Jasmin
• Primary Examiner(s): Park, Soo
• Application Number:
  US15/245,046
• Time in Patent Office: 504 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

Aspects of the disclosure generally relate to computing devices and/or systems, and may be generally directed to devices, systems, methods, and/or applications for learning operation of an application or an object of an application in various visual surroundings, storing this knowledge in a knowledgebase (i.e. neural network, graph, sequences, etc.), and enabling autonomous operation of the application or the object of the application.

Claims

1. A system for learning and using a visual surrounding for autonomous avatar operating, the system implemented at least in part on one or more computing devices, the system comprising:

a processor circuit configured to execute instruction sets of an application program, the application program including an avatar;

a memory unit configured to store data;

a renderer configured to render streams of digital pictures of a surrounding of the avatar of the application program; and

an artificial intelligence unit configured to;

receive a first stream of digital pictures from the renderer;

receive one or more instruction sets for operating the avatar of the application program;

learn the first stream of digital pictures correlated with the one or more instruction sets for operating the avatar of the application program;

receive a new stream of digital pictures from the renderer;

anticipate the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures based on at least partial match between the new stream of digital pictures and the first stream of digital pictures, wherein the anticipating includes determining at least partial match between at least a threshold number or a threshold percentage of portions of the new stream of digital pictures and portions of the first stream of digital pictures; and

cause the processor circuit to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures, the causing performed in response to the anticipating of the artificial intelligence unit, wherein the avatar of the application program performs one or more operations defined by the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures.

2. The system of claim 1, wherein the receiving the one or more instruction sets for operating the avatar of the application program includes receiving the one or more instruction sets for operating the avatar of the application program at the avatar'"'"'s of the application program runtime.

3. The system of claim 1, wherein the receiving the one or more instruction sets for operating the avatar of the application program includes an instrumentation or a metaprogramming of the application program or an instrumentation or a metaprogramming of the avatar of the application program.

4. The system of claim 1, wherein the receiving the one or more instruction sets for operating the avatar of the application program includes a tracing of the application program or a tracing of the avatar of the application program.

5. The system of claim 1, wherein the causing the processor circuit to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures includes causing the processor circuit to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures at the avatar'"'"'s of the application program runtime.

6. The system of claim 1, wherein the causing the processor circuit to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures includes an instrumentation or a metaprogramming of the application program or an instrumentation or a metaprogramming of the avatar of the application program.

7. The system of claim 1, wherein the receiving the one or more instruction sets for operating the avatar of the application program includes receiving the one or more instruction sets for operating the avatar of the application program from the application program or from the avatar of the application program.

8. The system of claim 1, wherein the causing the processor circuit to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures includes causing the application program or the avatar of the application program to execute the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures.

9. The system of claim 1, wherein the receiving the one or more instruction sets for operating the avatar of the application program includes receiving the one or more instruction sets executed in operating the avatar of the application program.

10. The system of claim 1, wherein the first stream of digital pictures includes one or more digital pictures and the new stream of digital pictures includes one or more digital pictures.

11. The system of claim 1, wherein at least one of:

the memory unit, the renderer, or the artificial intelligence unit are included in a remote computing device or a remote computing system, the remote computing device or the remote computing system coupled to the processor circuit via a network or an interface.

12. The system of claim 1, wherein the streams of digital pictures of a surrounding of the avatar of the application program include a stream of digital pictures depicting a first-person view of a surrounding of the avatar of the application program or a stream of digital pictures depicting a third-person view of a surrounding of the avatar of the application program.

13. The system of claim 1, wherein at least one of:

the processor circuit, the memory unit, the renderer, or the artificial intelligence unit are included in a single device.

14. The system of claim 1, wherein the artificial intelligence unit is:

a hardware element that is included in the processor circuit, a hardware element that is included in another processor circuit, an application operating on the processor circuit, an application operating on another processor circuit, or an element coupled to the processor circuit, and the renderer is;

a hardware element that is included in the processor circuit, a hardware element that is included in another processor circuit, an application operating on the processor circuit, an application operating on another processor circuit, or an element coupled to the processor circuit.

15. A non-transitory computer storage medium having a computer program stored thereon, the program including instructions that when executed by one or more processor circuits cause the one or more processor circuits to perform operations comprising:

receiving a first stream of digital pictures from a renderer, the renderer configured to render streams of digital pictures of a surrounding of an avatar of an application program;

receiving one or more instruction sets for operating the avatar of the application program;

learning the first stream of digital pictures correlated with the one or more instruction sets for operating the avatar of the application program;

receiving a new stream of digital pictures from the renderer;

anticipating the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures based on at least partial match between the new stream of digital pictures and the first stream of digital pictures, wherein the anticipating includes determining at least partial match between at least a threshold number or a threshold percentage of portions of the new stream of digital pictures and portions of the first stream of digital pictures; and

causing an execution of the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures, the causing performed in response to the anticipating, wherein the avatar of the application program performs one or more operations defined by the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures.

16. The non-transitory computer storage medium of claim 15, wherein the execution of the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures is performed by the one or more processor circuits or by another one or more processor circuits.

17. The non-transitory computer storage medium of claim 15, wherein the first stream of digital pictures includes one or more digital pictures and the new stream of digital pictures includes one or more digital pictures.

18. A method comprising:

(a) receiving a first stream of digital pictures from a renderer by a processor circuit, the renderer configured to render streams of digital pictures of a surrounding of an avatar of an application program;

(b) receiving, by the processor circuit, one or more instruction sets for operating the avatar of the application program;

(c) learning the first stream of digital pictures correlated with the one or more instruction sets for operating the avatar of the application program, the learning of (c) performed by the processor circuit;

(d) receiving a new stream of digital pictures from the renderer by the processor circuit;

(e) anticipating the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures based on at least partial match between the new stream of digital pictures and the first stream of digital pictures, wherein the anticipating of (e) includes determining at least partial match between at least a threshold number or a threshold percentage of portions of the new stream of digital pictures and portions of the first stream of digital pictures, the anticipating of (e) performed by the processor circuit;

(f) executing the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures, the executing of (f) performed in response to the anticipating of (e); and

(g) performing, by the avatar of the application program, one or more operations defined by the one or more instruction sets for operating the avatar of the application program correlated with the first stream of digital pictures.

19. The method of claim 18, wherein the executing of (f) is performed by the processor circuit or by another processor circuit.

20. The method of claim 18, wherein the first stream of digital pictures includes one or more digital pictures and the new stream of digital pictures includes one or more digital pictures.