Protocol and format for communicating an image from a camera to a computing environment

• US 8,625,837 B2
• Filed: 06/16/2009
• Issued: 01/07/2014
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Finocchio, Mark J., Margolis, Jeffrey
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/485,741
• Publication Number:
  US 20100304813A1
• Time in Patent Office: 1,666 Days
• Field of Search: None
• US Class Current: 382/100
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 1/032 for picture information reproduction
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/232 Content retrieval operation locally within server, e.g. reading video streams from disk arrays storage management G06F3/0604; details of querying and searching of video data from a database G06F16/739
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00

Abstract

A media feed interface may be provided that may be used to extract a media frame from a media feed. The media feed interface may access a capture device, a file, and/or a network resource. Upon accessing the capture device, file, and/or network resource, the media feed interface may populate buffers with data and then may create a media feed from the buffers. Upon request, the media feed interface may isolate a media frame within the media feed. For example, the media feed interface analyze media frames in the media feed to determine whether a media frame includes information associated with, for example, the request. If the media frame includes the requested information, the media feed interface may isolate the media frame associated with the information and may provide access to the isolated media frame.

Claims

1. A computer-implemented method for extracting captured information, the computer-implemented method comprising:

receiving a request to retrieve, from a media feed, data associated with a model of a user;

accessing the media feed; and

isolating a media frame from the media feed, wherein the media frame includes the data associated with the model of the user, wherein isolating the media frame from the media feed comprises;

determining that the media frame includes the data associated with the model of the user; and

extracting the media frame that includes the data associated with the model of the user.

2. The computer-implemented method of claim 1, wherein isolating the media frame occurs at the request of at least one of the following:

an application, an operating system, and a gesture library.

3. The computer-implemented method of claim 2 further comprising providing the media frame to at least one of the following:

the application, the operating system, and the gesture library.

4. The computer-implemented method of claim 1, wherein the request comprises a time stamp, and wherein isolating the media frame from the media feed further comprises:

determining the media frame from the media feed that corresponds to the specified time stamp in the request; and

extracting the media frame that corresponds to the specified time stamp in the request.

5. The computer-implemented method of claim 1, wherein the request comprises a filter type, and wherein isolating the media frame from the media feed further comprises:

filtering data from the media frame based on the filter type in the request; and

creating a new media feed, the new media feed including the filtered data from the media frame.

6. The computer-implemented method of claim 1, wherein receiving the media feed comprises:

accessing at least one of a capture device, a capture file, and a network resource;

receiving data from at least one of the capture device, the capture file, and the network resource;

populating one or more buffers with the received data; and

creating the media feed with the received data in the buffers, wherein the media feed includes one or more media frames for the received data in the buffers.

7. A computer-readable storage medium that is not a signal and that has stored thereon computer executable instructions for extracting captured information, the computer executable instructions comprising instructions for:

transmitting a request to a media feed interface, the request specifying data associated with a model of a user to be retrieved from a media feed, the media feed comprising at least one media frame that includes data;

accessing a media frame associated with the request, wherein the accessed media frame includes joint data associated with the model of the user; and

processing the accessed media frame.

8. The computer-readable storage medium of claim 7, wherein the request includes a time stamp associated with the accessed media frame.

9. The computer-readable storage medium of claim 8, wherein the request specifies at least one of the following type of data associated with the accessed media frame:

a depth image, an image, an RGB image, an IR image, sound data, infrared data, video data and picture data.

10. The computer-readable storage medium of claim 9, wherein the media frame accessed from the media feed interface is associated with the type of data and the time stamp in the request.

11. The computer-readable storage medium of claim 7, wherein the request includes a filter type.

12. The computer-readable storage medium of claim 11, wherein the media frame accessed from the media feed interface includes data associated with the filter type.

13. A system for extracting captured information, the system comprising:

a processor;

memory having stored therein computer executable instructions that, when executed by the processor, implement a method for extracting captured information, the method comprising;

receiving a request to retrieve, from a media feed, data associated with a model of a user;

accessing the media feed; and

isolating a media frame from the media feed, wherein the media frame includes the data associated with the model of the user, wherein isolating the media frame from the media feed comprises;

determining that the media frame includes the data associated with the model of the user; and

extracting the media frame that includes the data associated with the model of the user.

14. The system of claim 13, wherein isolating the media frame occurs at the request of at least one of the following:

an application, an operating system, and a gesture library.

15. The system of claim 14, wherein the method further comprises providing the media frame to at least one of the following:

the application, the operating system, and the gesture library.

16. The system of claim 13, wherein the request comprises a time stamp, and wherein isolating the media frame from the media feed further comprises:

determining the media frame from the media feed that corresponds to the specified time stamp in the request; and

extracting the media frame that corresponds to the specified time stamp in the request.

17. The system of claim 13, wherein the request comprises a filter type, and wherein isolating the media frame from the media feed further comprises:

filtering data from the media frame based on the filter type in the request; and

creating a new media feed, the new media feed including the filtered data from the media frame.

18. The system of claim 13, wherein receiving the media feed comprises:

accessing at least one of a capture device, a capture file, and a network resource;

receiving data from at least one of the capture device, the capture file, and the network resource;

populating one or more buffers with the received data; and

creating the media feed with the received data in the buffers, wherein the media feed includes one or more media frames for the received data in the buffers.