Methods and systems for extracting still frames from a compressed video

• US 8,374,421 B1
• Filed: 10/18/2011
• Issued: 02/12/2013
• Est. Priority Date: 10/18/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hickman, Ryan
• Primary Examiner(s): Johns, Andrew W
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US13/276,049
• Time in Patent Office: 483 Days
• Field of Search: None
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1697 Vision controlled systems

Abstract

Methods and systems for robot and cloud communication are described. A robot may interact with the cloud to perform any number of actions using video captured from a point-of-view or in the vicinity of the robot. The cloud may be configured to extract still frames from compressed video received from the robot at a frame rate determined based on a number of factors, including the robot'"'"'s surrounding environment, the available bandwidth, or actions being performed. The cloud may be configured to request that a compressed video with higher frame rate be sent so that the cloud can extract still frames at a higher frame rate. Further, the cloud may be configured to request that a second compressed video from a second perspective be sent to provide additional environment information.

Claims

1. A method comprising:

receiving a compressed video from a robotic device in an environment, the compressed video captured by a video capturing device attached to the robotic device;

determining, by a computing system having a processor and a memory, a frame rate at which to extract still frames from the compressed video based on device information associated with the robotic device, the device information indicating whether the robotic device has previously traversed the environment, wherein determining the frame rate based on device information indicating whether the robotic device has previously traversed the environment comprises determining a location of the robotic device; and

extracting still frames from the compressed video at the determined frame rate.

2. The method of claim 1, further comprising sending a message to the robotic device indicating the frame rate at which to send the compressed video.

3. The method of claim 1, further comprising sending a message to the robotic device indicating a chromatic quality at which to send the compressed video.

4. The method of claim 1, wherein the computing device receiving the compressed video comprises the computing device receiving a stream of video from the robotic device.

5. The method of claim 4, wherein receiving the stream of video from the robotic device comprises accessing a video stream feed received at a second computing device from the robotic device.

6. The method of claim 5, wherein the video stream feed received at the second computing device comprises an intern&

video chat, the intern&

video chat comprising video content associated with a real-time interaction between a party at the second computing device and another party.

7. The method of claim 1, wherein the compressed video includes video from a first perspective of the robotic device, and wherein the method further comprises sending a message to the robotic device requesting video from a second perspective of the robotic device.

8. The method of claim 7, further comprising:

extracting still frames from the video from the second perspective of the robotic device; and

processing the still frames of the compressed video from the first perspective of the robotic device and the still frames from the video from the second perspective of the robotic device in a stereoscopic manner.

9. The method of claim 1, further comprising sending a message to the robotic device requesting a second video from the robotic device, wherein the compressed video received from the robot device is captured by a first camera on the robotic device and the second video is captured by a second camera on the robotic device.

10. The method of claim 1, wherein determining the frame rate based on an environment of the robotic device comprises determining bandwidth availability of a channel through which the compressed video is received.

11. The method of claim 1, further comprising:

processing the extracted still frames to determine information associated with the compressed video;

determining a second frame rate at which to extract still frames based on the information associated with the compressed video;

sending a message to the robotic device requesting a second compressed video;

receiving the second compressed video from the robotic device; and

extracting still frames from the second compressed video at the second frame rate.

12. The method of claim 1, further comprising:

receiving information associated with an objective of the robotic device; and

determining the frame rate based also on the objective of the robotic device.

13. The method of claim 1, further comprising:

receiving information associated with capabilities of the robotic device; and

determining the frame rate based also on the capabilities of the robotic device.

14. The method of claim 1, further comprising:

processing the still frames so as to provide a result; and

when the result is inconclusive, sending a message to the robotic device requesting the robotic device to increase a frame rate of the compressed video.

15. The method of claim 1, further comprising:

receiving sensor data from the robotic device, the sensor data associated with one or more sensors on the robotic device; and

determining the frame rate based on the received sensor data from the robotic device.

16. A non-transitory computer readable medium having stored thereon instructions executable by a computing device to cause the computing device to perform functions comprising:

receiving a compressed video from a robotic device;

determining a frame rate at which to extract still frames from the compressed video based on an environment of the robotic device, wherein determining the frame rate based on an environment of the robotic device comprises determining a location of the robotic device; and

extracting still frames from the compressed video at the frame rate.

17. The non-transitory computer readable medium of claim 16, wherein the instructions are further executable by the computing device to cause the computing device to perform functions comprising:

receiving information associated with an objective of the robotic device;

receiving sensor data from the robotic device, the sensor data associated with one or more sensors on the robotic device; and

determining the frame rate based on the objective of the robotic device and on the received sensor data from the robotic device.

18. A device comprising:

a processor; and

memory including instructions stored therein executable by the processor to perform functions comprising;

receiving a compressed video from a robotic device;

determining a frame rate at which to extract still frames from the compressed video based on an environment of the robotic device, wherein determining the frame rate based on an environment of the robotic device comprises determining a location of the robotic device; and

extracting still frames from the compressed video at the frame rate.

19. The device of claim 18, further comprising instructions stored in the memory executable by the processor to perform functions comprising:

processing the extracted still frames to determine information associated with the compressed video;

receiving information associated with an objective of the robotic device;

receiving sensor data from the robotic device, the sensor data associated with one or more sensors on the robotic device;

determining a second frame rate based on the information associated with the compressed video, the objective of the robotic device, and the received sensor data from the robotic device; and

extracting still frames from the compressed video at the second frame rate.