Protocol and format for communicating an image from a camera to a computing environment

• US 9,215,478 B2
• Filed: 11/27/2013
• Issued: 12/15/2015
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Finocchio, Mark J., Margolis, Jeffrey
• Primary Examiner(s): Seth, Manav
• Application Number:
  US14/091,503
• Publication Number:
  US 20140085193A1
• Time in Patent Office: 748 Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/424 involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 1/032 for picture information reproduction
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/232 Content retrieval operation locally within server, e.g. reading video streams from disk arrays storage management G06F3/0604; details of querying and searching of video data from a database G06F16/739
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00

Abstract

A media feed interface may be provided that may be used to extract a media frame from a media feed. The media feed interface may access a capture device, a file, and/or a network resource. Upon accessing the capture device, file, and/or network resource, the media feed interface may populate buffers with data and then may create a media feed from the buffers. Upon request, the media feed interface may isolate a media frame within the media feed. For example, the media feed interface analyze media frames in the media feed to determine whether a media frame includes information associated with, for example, the request. If the media frame includes the requested information, the media feed interface may isolate the media frame associated with the information and may provide access to the isolated media frame.

Claims

1. A system for processing a media feed, the system comprising:

a capture device to capture data; and

a processor, wherein the processor is operable to execute computer executable instructions, and wherein the computer executable instructions comprise instructions for;

accessing the capture device;

receiving the captured data from the capture device;

populating one or more buffers with the captured data; and

creating the media feed with the captured data in the buffers, wherein the media feed comprises one or more media frames that include the captured data, and wherein an interface retrieves the captured data in the buffers and provides the captured data to an application program without revealing a memory address of the buffers to the application program.

2. The system of claim 1, wherein the captured data comprises at least one of the following:

a depth image, an image, an RGB image, an IR image, sound data, infrared data, video data, picture data, or data associated with a model of a user.

3. The system of claim 1, wherein the computer executable instructions further comprise instructions for:

receiving a request to retrieve data from the media feed;

accessing the media feed; and

isolating a media frame from the media feed, wherein the media frame includes the data associated with the received request.

4. The system of claim 3, wherein the request comprises a time stamp and a specified type of data, and wherein isolating the media frame from the media feed comprises:

determining the media frame from the media feed that corresponds to the specified type of data and the time stamp in the request; and

extracting the media frame that corresponds to the specified type of data and the time stamp in the request.

5. The system of claim 3, wherein the request comprises a filter type, and wherein isolating the media frame from the media feed comprises:

filtering data from the media frame based on the filter type in the request; and

creating a new media feed, the new media feed including the filtered data from the media frame.

6. The system of claim 3, wherein the request includes a request for data associated with a model of a user, and wherein isolating the media frame from the media feed comprises:

determining whether the media frame includes the data associated with the model of the user; and

extracting the media frame that includes the data associated with the model of the user.

7. A computer-implemented method for processing a media feed comprising:

accessing a capture device;

receiving captured data from the capture device;

populating one or more buffers with the captured data; and

creating a media feed with the captured data in the buffers, wherein the media feed comprises one or more media frames that include the captured data, and wherein an interface retrieves the captured data in the buffers and provides the captured data to an application program without revealing a memory address of the buffers to the application program.

8. The computer-implemented method of claim 7, wherein the captured data comprises at least one of the following:

a depth image, an image, an RGB image, an IR image, sound data, infrared data, video data or picture data.

9. The computer-implemented method of claim 7, wherein the captured data comprises data associated with a model of a user.

10. The computer-implemented method of claim 7, further comprising:

receiving a request to retrieve data from the media feed;

accessing the media feed; and

isolating a media frame from the media feed, wherein the media frame includes the data associated with the received request.

11. The computer-implemented method of claim 10, wherein the request comprises a time stamp and a specified type of data, and wherein isolating the media frame from the media feed comprises:

determining the media frame from the media feed that corresponds to the specified type of data and the time stamp in the request; and

extracting the media frame that corresponds to the specified type of data and the time stamp in the request.

12. The computer-implemented method of claim 10, wherein the request comprises a filter type, and wherein isolating the media frame from the media feed comprises:

filtering data from the media frame based on the filter type in the request; and

creating a new media feed, the new media feed including the filtered data from the media frame.

13. The computer-implemented method of claim 10, wherein the request includes a request for data associated with a model of a user, and wherein isolating the media frame from the media feed comprises:

determining whether the media frame includes the data associated with the model of the user; and

extracting the media frame that includes the data associated with the model of the user.

14. A computer-readable storage device having stored thereon computer-executable instructions that, upon execution by one or more processors, cause the processors to perform operations comprising:

populating one or more buffers with captured data that is received from a capture device, wherein the captured data comprises data associated with a model of a user; and

creating a media feed with the captured data in the buffers, wherein the media feed comprises one or more media frames that include the captured data, and wherein an interface retrieves the captured data in the buffers and provides the captured data to an application program without revealing a memory address of the buffers to the application program.

15. The computer-readable storage device of claim 14, wherein the operations further comprise:

receiving a request to retrieve data from the media feed;

accessing the media feed; and

isolating a media frame from the media feed, wherein the media frame includes the data associated with the received request.

16. The computer-readable storage device of claim 15, wherein the request comprises a time stamp and a specified type of data.

17. The computer-readable storage device of claim 16, wherein isolating the media frame from the media feed comprises:

determining the media frame from the media feed that corresponds to the specified type of data and the time stamp in the request; and

extracting the media frame that corresponds to the specified type of data and the time stamp in the request.

18. The computer-readable storage medium of claim 15, wherein the request comprises a filter type.

19. The computer-readable storage medium of claim 18, wherein isolating the media frame from the media feed comprises:

filtering data from the media frame based on the filter type in the request; and

creating a new media feed, the new media feed including the filtered data from the media frame.

20. The computer-readable storage medium of claim 15, wherein the request includes a request for data associated with a model of a user, and wherein isolating the media frame from the media feed comprises:

determining whether the media frame includes the data associated with the model of the user; and

extracting the media frame that includes the data associated with the model of the user.