Camera-to-camera interactions, systems and methods

• US 10,382,533 B2
• Filed: 07/20/2018
• Issued: 08/13/2019
• Est. Priority Date: 08/19/2013
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Soon-Shiong, Patrick
• Primary Examiner(s): Le, Hung D
• Application Number:
  US16/041,704
• Publication Number:
  US 20180352021A1
• Time in Patent Office: 389 Days
• Field of Search: None
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4227 Providing Remote input by a user located remotely from the client device, e.g. at work

Abstract

Systems and methods of delegating media capturing functionality from one device to another are presented. A first device configured with an object recognition engine captures a media representation of an environment and identifies an object within that environment. Then based on matched object traits from a database, the engine selects a delegation rules set, and delegates certain media capturing functionality to a second device according to the selected delegation rules set.

Claims

1. A coordinated multi-device system, the system comprising:

an object database storing object traits;

a rules database storing at least one delegation rule set indexed according to at least some of the object traits; and

a first media capturing device coupled with the object database and the rules database, and comprising an object recognition engine that is configured to;

obtain media representative of an augmented reality environment;

identify at least one object in the augmented reality environment as represented in the media;

obtain object traits associated with the at least one object from the object database;

select a delegation rules set as a function of the object traits associated with the at least one object from the object database; and

delegate at least some media capturing functionality to a second media capturing device according to the delegation rules set,

wherein the first media capturing device is further configured to select the delegation rules set based on a determination of whether the first media capturing device is located within a predetermined geo-fenced area.

2. The system of claim 1, wherein the object database comprises at least one of the following:

a file system, a remote database, and a local database.

3. The system of claim 1, wherein the first media capturing device is configured to access the object database via a wireless connection.

4. The system of claim 1, wherein the rules database comprises at least one of the following:

a file system, a remote database, and a local database.

5. The system of claim 1, wherein the object traits associated with the at least one object from the object database comprise at least one of the following:

a physical trait, an emotional trait, a mental trait, a visual trait, a virtual trait, an audible trait, a behavior trait, a thermal trait, a textural trait, a scent trait, a taste trait, a chemical trait, a radioactive trait, a toxic trait, an energy trait, a biological trait, a biometric trait, a medical/health trait, a derived trait, and a geographic trait.

6. The system of claim 1, wherein the delegation rules set comprises at least one of the following:

a performance command, a performance adjustment command, a stop command, a device setting adjustment, the second media capturing device'"'"'s identity, a time shift command, a configuration parameter, a timing command, a triggering command, a play command, a relay command, an acknowledgement protocol, a negative acknowledgement protocol, and an authentication key.

7. The system of claim 1, wherein the first media capturing device comprises at least one of the following:

a camera, a video camera, a webcam, an audio recorder, a scanner, a mobile phone, a telescope, binoculars, eye glasses, a smart phone, a computer, a printer, a tablet, a medical device, a kiosk, a game console, a vehicle, and a phablet.

8. The system of claim 1, wherein the second media capturing device comprises the same type of device as the first media capturing device.

9. The system of claim 1, wherein the media representative of the augmented reality environment comprises at least one of the following data modalities:

image data, video data, audio data, speech data, motion data, acceleration data, temperature data, text data, biometric data, game data, radio data, webpage data, telecommunication data, satellite data, sensor data, ultrasonic data, x-ray data, MRI data, and CT scan data.

10. The system of claim 1, wherein the augmented reality environment comprises a real-world environment, wherein the real-world environment comprises at least one of the following:

a nature setting, a building, a room, an operating room, a street, a stage, a vehicle, a road, an operating room, a school, a classroom, an arena, a stadium, a warehouse, a mall, a plaza, a neighborhood, an office, an ecosystem, infrastructure, a habitat, and a scene.

11. The system of claim 1, wherein the augmented reality environment comprises a virtual environment, wherein the virtual environment comprises at least one of the following:

a virtual world, an augmented reality, a game, on-screen content, broadcast programming, and a mixed reality.

12. The system of claim 1, wherein the at least one object comprises at least one of the following:

a tangible entity, an intangible entity, a virtual entity, an inanimate entity, and an animate entity.

13. The system of claim 1, wherein the object traits associated with the at least one object from the object database comprise a heterogeneous set of traits.

14. The system of claim 13, wherein the first media capturing device is further configured to select the delegation rules set based on the heterogeneous set of traits.

15. The system of claim 1, wherein the first media capturing device is further configured to select the delegation rules set based on a user selection of one or more delegation rules.

16. The system of claim 1, wherein the first media capturing device is further configured to select the delegation rules set based on a geographic location or geographic zone.

17. The system of claim 1, wherein the first media capturing device is further configured to select the delegation rules set based on a time duration that the first media capturing device captures media representative of the object.

18. The system of claim 1, wherein the first media capturing device is further configured to select the delegation rules set based on the type of device comprising the first media capturing device or the type of device comprising the second media capturing device.

19. The system of claim 1, wherein the media capturing functionality comprises at least one of the following:

capturing an image, capturing a video, capturing audio, directing orientation for media capture, adjusting a media capturing setting, instructing document creation based upon second media capture, and requiring a media file format.

20. The system of claim 1, wherein the first media capturing device is further configured to receive an acknowledgement protocol or negative acknowledgement protocol.

21. The system of claim 1, wherein the object database is further configured to store object-specific links to delegation rules sets.

22. The system of claim 21, further comprising a network fabric comprising the first and the second media capturing device.

23. The system of claim 22, wherein the network fabric comprises at least one of the following types of networks:

an ad hoc network, a peer-to-peer network, a personal area network, a wireless network, a local area network, a wide area network, metropolitan area network, server area network, system area network, campus area network, or desk area network.

24. The system of claim 1, further comprising the second media capturing device.

25. A coordinated multi-device method, the method comprising:

storing object traits;

storing at least one delegation rule set indexed according to at least some of the object traits;

obtaining media representative of an augmented reality environment;

identifying at least one object in the augmented reality environment as represented in the media;

obtaining object traits associated with the at least one object;

selecting a delegation rules set as a function of the object traits associated with the at least one object; and

delegating at least some media capturing functionality to a second media capturing device according to the delegation rules set,

wherein the first media capturing device is further configured to select the delegation rules set based on a determination of whether the first media capturing device is located within a predetermined geo-fenced area.

26. A computer program product embedded in a non-transitory computer-readable medium comprising instructions for coordinated multi-device operation, which, when executed, configure one or more computer processors to perform a method comprising:

storing object traits;

storing at least one delegation rule set indexed according to at least some of the object traits;

obtaining media representative of an augmented reality environment;

identifying at least one object in the augmented reality environment as represented in the media;

obtaining object traits associated with the at least one object;

selecting a delegation rules set as a function of the object traits associated with the at least one object; and

delegating at least some media capturing functionality to a second media capturing device according to the delegation rules set,

wherein the first media capturing device is further configured to select the delegation rules set based on a determination of whether the first media capturing device is located within a predetermined geo-fenced area.

27. A coordinated multi-device system comprising:

an object database storing object traits;

a rules database storing at least one delegation rule set indexed according to at least some of the object traits; and

a first media capturing device coupled with the object database and the rules database, and comprising an object recognition engine that is configured to;

obtain media comprising a motion data modality representative of an environment;

identify at least one object in the environment as represented in the media;

obtain object traits associated with the at least one object from the object database;

select a delegation rules set as a function of the object traits associated with the at least one object from the object database; and

delegate at least some media capturing functionality to a second media capturing device according to the delegation rules set,

wherein the first media capturing device is further configured to select the delegation rules set based on a determination of whether the first media capturing device is located within a predetermined geo-fenced area.