Devices, methods and systems for multi-user capable visual imaging arrays

• US 9,942,583 B2
• Filed: 11/18/2015
• Issued: 04/10/2018
• Est. Priority Date: 11/18/2014
• Status: Expired due to Fees

• Current Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Inventors: Brav, Ehren, Hannigan, Russell, Hyde, Roderick A., Ishikawa, Muriel Y., Johanson, 3ric, Kare, Jordin T., Pan, Tony S, Rutschman, Phillip, Tegreene, Clarence T., Whitmer, Charles, Wood, Jr., Lowell L., Wood, Victoria Y. H.
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US14/945,342
• Publication Number:
  US 20160309207A1
• Time in Patent Office: 874 Days
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2183 Cache memory caches in web servers or browsers G06F16/9574; intermediate storage and caching in data networks H04L67/568
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Computationally implemented methods and systems include accepting input of a request for a particular image of a scene that is larger than the particular image, transmitting the request for the particular image to an image sensor array that includes more than one image sensor and that is configured to capture the scene and retain a subset of the scene that includes the request for the particular image of the scene, receiving only the particular image from the image sensor array, wherein the particular image represents a subset of the scene, and wherein a size characteristic of the particular image is at least partially based on a property of a requestor, and presenting the received particular image to the requestor. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A computationally-implemented method, comprising:

accepting input of a request for a particular image of a scene that is larger than the particular image;

comparing one or more previously received images that are determined to be similar to the particular image to determine a changed portion;

identifying a portion of the particular image that corresponds to the changed portion;

determining an updated request that identifies the portion of the particular image that corresponds to the changed portion as targeted for updating; and

transmitting the updated request that identifies the portion of the particular image to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and retain a subset of the scene that includes the portion of the particular image;

receiving the portion of the particular image from the image sensor array device; and

transmitting to a requestor device at least the portion of the particular image for presentation of the particular image of the scene.

2. A system comprising:

circuitry configured for accepting an input of a request for a particular image that is part of a scene that is larger than the particular image;

circuitry configured for comparing one or more previously received images determined to be similar;

circuitry configured for modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating based on a comparison of the one or more previously received images determined to be similar;

circuitry configured for transmitting the updated request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to retain the portion of the scene targeted for updating;

circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating; and

circuitry configured for outputting at least the portion of the scene targeted for updating for presentation at a requestor device.

3. The system of claim 2, wherein said circuitry configured for accepting an input of a request for a particular image that is part of a scene that is larger than the particular image comprises:

circuitry configured for receiving from an automated component an input of a request for a particular image that is part of a scene that is larger than the particular image.

4. The system of claim 3, wherein said circuitry configured for receiving from an automated component an input of a request for a particular image that is part of a scene that is larger than the particular image comprises:

circuitry configured for receiving from an object tracking component an input of a request for a particular image of a tracked object, which particular image is part of a scene that is larger than the particular image.

5. The system of claim 2, wherein said circuitry configured for transmitting the updated request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to retain the portion of the scene targeted for updating comprises:

circuitry configured for transmitting the updated request via a relay to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to retain the portion of the scene targeted for updating.

6. The system of claim 2, further comprising:

circuitry configured for combining the updated request with at least one other request.

7. The system of claim 6, wherein said circuitry configured for combining the updated request with at least one other request comprises:

circuitry configured for combining the updated request with at least one other request such that one or more redundant portions are removed.

8. The system of claim 2, wherein the circuitry configured for comparing one or more previously received images determined to be similar comprises:

circuitry configured for comparing one or more previously received images determined to be similar to the part of the scene associated with the request for the particular image.

9. The system of claim 2, wherein said circuitry configured for comparing one or more previously received images determined to be similar comprises:

circuitry configured for comparing one or more previously received images determined to be similar to identify the portion of the scene targeted for updating.

10. The system of claim 2, wherein said circuitry configured for comparing one or more previously received images determined to be similar comprises:

circuitry configured for comparing one or more previously received images determined to be similar including at least comparing a first previously received image with a second previously received image to determine a changed portion; and

circuitry configured for identifying the portion of the scene targeted for updating that corresponds to the changed portion.

11. The system of claim 2, further comprising:

circuitry configured for generating an expanded request.

12. The system of claim 11, wherein said circuitry configured for generating an expanded request comprises:

circuitry configured for generating an expanded request that includes at least a request for a border image.

13. The system of claim 12, wherein said circuitry configured for generating an expanded request that includes at least a request for a border image comprises:

circuitry configured for generating an expanded request that includes at least a request for a border image that borders the particular image on one or more sides.

14. The system of claim 11, wherein said circuitry configured for generating an expanded request comprises:

circuitry configured for determining an image projected to be subject to a future request; and

circuitry configured for generating an expanded request that identifies a portion of the scene corresponding to the image.

15. The system of claim 14, wherein said circuitry configured for determining an image projected to be subject to a future request comprises:

circuitry configured for determining an image projected to be subject to a future request based at least partly on a direction in which a requestor device is moving.

16. The system of claim 11, wherein said circuitry configured for generating an expanded request comprises:

circuitry configured for generating an expanded request that includes at least a request for a border image that borders the particular image and a request for a secondary border image that borders the border image.

17. The system of claim 16, wherein said circuitry configured for generating an expanded request that includes at least a request for a border image that borders the particular image and a request for a secondary border image that borders the border image comprises:

circuitry configured for generating an expanded request that includes at least the request for the particular image at a first resolution, a request for a border image that borders the particular image at a second resolution that is less than the first resolution, and a request for a secondary border image that borders the border image at a third resolution that is less than the second resolution.

18. The system of claim 2, wherein said circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating comprises:

circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating, wherein the image sensor array device discarded portions of the scene other than the portion of the scene targeted for updating.

19. The system of claim 2, wherein said circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating comprises:

circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating, wherein a size characteristic of the portion of the scene targeted for updating is at least partially based on a property of the requestor device.

20. The system of claim 2, further comprising:

circuitry configured for modifying the portion of the scene received from the image sensor array device.

21. The system of claim 20, wherein said circuitry configured for modifying the portion of the scene received from the image sensor array device comprises:

circuitry configured for combining the portion of the scene received from the image sensor array device with existing image data.

22. The system of claim 2, further comprising:

circuitry configured for storing at least some of the portion of the scene received from the image sensor array device to be used as cached image data for at least partly satisfying a subsequent image request.

23. The system of claim 2, wherein the circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating comprises:

circuitry configured for receiving from the image sensor array device only the portion of the scene targeted for updating.

24. The system of claim 2, wherein the circuitry configured for accepting an input of a request for a particular image that is part of a scene that is larger than the particular image comprises:

circuitry configured for accepting an input of a request for a particular video that is part of a scene that is larger than the particular video.

25. The system of claim 2, further comprising:

the image sensor array device that includes more than one image sensor.

26. The system of claim 2, further comprising:

the requestor device, wherein the requestor device includes at least one of a smartphone or tablet computer.

27. The system of claim 2, wherein the circuitry configured for comparing one or more previously received images determined to be similar comprises:

circuitry configured for comparing one or more previously received images determined to be similar to identify one or more portions that are likely to have changed.

28. The system of claim 27, wherein the circuitry configured for modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating based on a comparison of the one or more previously received images determined to be similar comprises:

circuitry configured for modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating that corresponds to the one or more portions that are likely to have changed and one or more specified areas around the one or more portions that are likely to have changed.

29. The system of claim 2, wherein the circuitry configured for accepting an input of a request for a particular image that is part of a scene that is larger than the particular image comprises:

circuitry configured for accepting from the requestor device a user input of a request for a particular image that is part of a scene that is larger than the particular image.

30. The system of claim 2, wherein the circuitry configured for accepting an input of a request for a particular image that is part of a scene that is larger than the particular image comprises:

circuitry configured for accepting an input of a request for a particular image associated with at least one of an object, target, field of view, pan, zoom, or scroll.

31. The system of claim 2, further comprising:

circuitry configured for receiving during one or more off-peak periods imagery of one or more static objects associated with the scene.

32. The system of claim 2, wherein the circuitry configured for outputting at least the portion of the scene targeted for updating for presentation at a requestor device comprises:

circuitry configured for outputting at least the portion of the scene targeted for updating for presentation at a requestor device, the at least the portion of the scene including inserted alphanumeric or pictorial content.

33. The system of claim 2, wherein the circuitry configured for modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating based on a comparison of the one or more previously received images determined to be similar comprises:

circuitry configured for modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating based on a comparison of the one or more previously received images determined to be similar because at least one of content, field of view, location, or a property.

34. The system of claim 2, wherein the circuitry configured for transmitting the updated request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to retain the portion of the scene targeted for updating comprises:

circuitry configured for transmitting the updated request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and temporarily retain the portion of the scene targeted for updating.

35. The system of claim 2, wherein the circuitry configured for outputting at least the portion of the scene targeted for updating for presentation at a requestor device comprises:

circuitry configured for transmitting at least the portion of the scene targeted for updating for presentation at a requestor device.

36. The system of claim 2, wherein the circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating comprises:

circuitry configured for receiving from the image sensor array device the portion of the scene targeted for updating at a pixel resolution that is bounded by a screen resolution capability of the requestor device.

37. A system comprising:

one or more computer readable memory bearing one or more executable program instructions; and

at least one computer processor configured by the one or more executable program instructions to perform operations including at least;

accepting an input of a request for a particular image that is part of a scene that is larger than the particular image;

comparing one or more previously received images determined to be similar;

modifying the request for the particular image into an updated request that identifies a portion of the scene targeted for updating based on a comparison of the one or more previously received images determined to be similar;

transmitting the updated request to an image sensor array device that includes more than one image sensor and that is configured to capture the scene and to retain the portion of the scene targeted for updating;

receiving from the image sensor array device the portion of the scene targeted for updating; and

outputting at least the portion of the scene targeted for updating for presentation at a requestor device.