Devices, methods and systems for multi-user capable visual imaging arrays

• US 9,866,881 B2
• Filed: 11/24/2015
• Issued: 01/09/2018
• Est. Priority Date: 11/18/2014
• Status: Expired due to Fees

• Current Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Brav, Ehren, Hannigan, Russell, Johanson, 3ric, Rutschman, Phillip
• Primary Examiner(s): Cutler, Albert
• Application Number:
  US14/951,348
• Publication Number:
  US 20160277661A1
• Time in Patent Office: 777 Days
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2183 Cache memory caches in web servers or browsers G06F16/9574; intermediate storage and caching in data networks H04L67/568
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Computationally implemented methods and systems include accepting input of a request for a particular image of a scene that is larger than the particular image, transmitting the request for the particular image to an image sensor array that includes more than one image sensor and that is configured to capture the scene and retain a subset of the scene that includes the request for the particular image of the scene, receiving only the particular image from the image sensor array, wherein the particular image represents a subset of the scene, and wherein a size characteristic of the particular image is at least partially based on a property of a requestor, and presenting the received particular image to the requestor. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A system comprising:

circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data;

circuitry configured for generating an expanded request for the particular image data that includes a request for image data associated with a first portion of the scene at a first resolution, a request for image data associated with a second portion of the scene bordering the first portion at a second resolution that is less than the first resolution, and a request for image data associated with a third portion of the scene bordering the second portion at a third resolution that is less than or equal to the second resolution;

circuitry configured for transmitting the expanded request for the particular image data to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene;

circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution; and

circuitry configured for outputting at least some of the particular image data to a requestor for presentation.

2. The system of claim 1, wherein said circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for accepting from an automated component an input of a request for particular image data that is part of a scene that is larger than the particular image data.

3. The system of claim 1, wherein said circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for accepting an input of a request for particular image data associated with a tracked object that is part of a scene that is larger than the particular image data.

4. The system of claim 1, wherein said circuitry configured for transmitting the expanded request for the particular image data to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene comprises:

circuitry configured for transmitting the expanded request for the particular image data to a server that is configured to relay the expanded request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene.

5. The system of claim 4, wherein said circuitry configured for transmitting the expanded request for the particular image data to a server that is configured to relay the expanded request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene comprises:

circuitry configured for transmitting the expanded request for the particular image data to a server that is configured to package multiple requests and relay the multiple requests to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene.

6. The system of claim 5, wherein said circuitry configured for transmitting the expanded request for the particular image data to a server that is configured to package multiple requests and relay the multiple requests to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene comprises:

circuitry configured for transmitting the expanded request for the particular image data to a server that is configured to combine multiple requests and relay the multiple requests to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to selectively retain a subset of the scene.

7. The system of claim 1, wherein said circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution comprises:

circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution, wherein the image sensor array device discarded portions of the scene other than the particular image data.

8. The system of claim 1, wherein said circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution comprises:

circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution, wherein a size characteristic of the particular image data is at least partially based on a feature of the requestor.

9. The system of claim 1, wherein said circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution comprises:

circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution, wherein a resolution of the particular image data is at least partially based on a screen size of the requestor.

10. The system of claim 1, wherein said circuitry configured for outputting at least some of the particular image data to a requestor for presentation comprises:

circuitry configured for outputting the image data associated with the first portion of the scene at the first resolution to a requestor for presentation.

11. The system of claim 10, further comprising:

circuitry configured for storing the image data associated with the second portion of the scene and the image data associated with the third portion of the scene.

12. The system of claim 11, wherein said circuitry configured for storing the image data associated with the second portion of the scene and the image data associated with the third portion of the scene comprises:

circuitry configured for storing the image data associated with the second portion of the scene and the image data associated with the third portion of the scene to be used as cached image data for one or more subsequent requests.

13. The system of claim 1, wherein the circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for receiving a request for particular image data that is part of a scene that is larger than the particular image data.

14. The system of claim 1, wherein the circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for receiving an input via an interface of the requestor of a request for particular image data that is part of a scene that is larger than the particular image data.

15. The system of claim 1, wherein the circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for receiving an input from at least one of the following types of requestors;

smartphone, television, laptop computer, desktop computer, tablet, camera, wearable computer, or augmented reality device.

16. The system of claim 1, wherein the circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for accepting an input of a request for particular image data that is part of at least one of the following types of scenes;

sporting event, street view, tourist destination, highway, or inside of a home.

17. The system of claim 1, wherein the circuitry configured for accepting an input of a request for particular image data that is part of a scene that is larger than the particular image data comprises:

circuitry configured for accepting an input of a request for particular video data that is part of a scene that is larger than the particular video data.

18. The system of claim 1, wherein the circuitry configured for generating an expanded request for the particular image data that includes a request for image data associated with a first portion of the scene at a first resolution, a request for image data associated with a second portion of the scene bordering the first portion at a second resolution that is less than the first resolution, and a request for image data associated with a third portion of the scene bordering the second portion at a third resolution that is less than or equal to the second resolution comprises:

circuitry configured for generating an expanded request for the particular image data that includes a request for image data associated with a first portion of the scene at a first resolution, a request for image data associated with a second portion of the scene adjacent to the first portion at a second resolution that is less than the first resolution, and a request for image data associated with a third portion of the scene adjacent to the second portion at a third resolution that is less than or equal to the second resolution.

19. The system of claim 1, wherein the circuitry configured for generating an expanded request for the particular image data that includes a request for image data associated with a first portion of the scene at a first resolution, a request for image data associated with a second portion of the scene bordering the first portion at a second resolution that is less than the first resolution, and a request for image data associated with a third portion of the scene bordering the second portion at a third resolution that is less than or equal to the second resolution comprises:

circuitry configured for generating an expanded request for the particular image data that includes a request for image data associated with a first portion of the scene at a first resolution, a request for image data associated with a second portion of the scene bordering the first portion on all sides at a second resolution that is less than the first resolution, and a request for image data associated with a third portion of the scene bordering the second portion on all sides at a third resolution that is less than or equal to the second resolution.

20. The system of claim 1, wherein the circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution comprises:

circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution, wherein a data size of the particular image data is at least partially based on an available bandwidth.

21. The system of claim 1, wherein the circuitry configured for receiving from the image sensor array device the particular image data that includes the image data associated with the first portion of the scene at the first resolution, the image data associated with the second portion of the scene at the second resolution, and the image data associated with the third portion of the scene at the third resolution comprises:

circuitry configured for receiving from the image sensor array device the particular image data that includes only one or more portions of the scene that have changed.

22. The system of claim 1, wherein the circuitry configured for outputting at least some of the particular image data to a requestor for presentation comprises:

circuitry configured for outputting at least some of the particular image data that includes inserted contextual information to a requestor for presentation.

23. The system of claim 1, further comprising:

the image sensory array device including at least;

more than one image sensor configured to capture the scene;

a computer processor configured to process image data of the scene prior to communication;

memory; and

at least one communication interface.

24. The system of claim 23, wherein the a computer processor configured to process image data of the scene prior to communication comprises:

a computer processor configured to process image data of the scene prior to communication including consolidating one or more requested overlapping portions.

25. The system of claim 23, wherein the a computer processor configured to process image data of the scene prior to communication comprises:

a computer processor configured to process image data of the scene prior to communication including removing one or more unchanged portions based on one or more comparisons with previous data.

26. A computationally-implemented method, comprising:

accepting input of a request for a particular image of a scene that is larger than the particular image;

generating an expanded request for the particular image that includes the request for the particular image at a first resolution, a request for a border image that borders the particular image at a second resolution less than the first resolution, and a request for a secondary border image that borders the border image at a third resolution less than or equal to the second resolution;

transmitting the expanded request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and retain a subset of the scene that includes the particular image, the border image, and the secondary border image;

receiving the particular image, the border image, and the secondary border image from the image sensor array device; and

transmitting to a requestor device at least the particular image for presentation.

27. The method of claim 26, wherein the generating an expanded request for the particular image that includes the request for the particular image at a first resolution, a request for a border image that borders the particular image at a second resolution less than the first resolution, and a request for a secondary border image that borders the border image at a third resolution less than or equal to the second resolution comprises:

generating an expanded request for the particular image that includes the request for the particular image at a first resolution, a request for a border image that borders the particular image on each side at a second resolution less than the first resolution, and a request for a secondary border image that borders the border image at a third resolution less than or equal to the second resolution.

28. The method of claim 26, wherein the receiving the particular image, the border image, and the secondary border image from the image sensor array device comprises:

receiving the particular image, the border image, and the secondary border image from the image sensor array device, wherein a resolution of at least the particular image is at least partially based on a property of the requestor device.