Cooperative provision of personalized user functions using shared and personal devices

• US 10,419,804 B2
• Filed: 08/18/2017
• Issued: 09/17/2019
• Est. Priority Date: 09/12/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Li, Wenlong, Young, Honesty, Wang, Randolph
• Primary Examiner(s): Tokuta, Shean
• Assistant Examiner(s): Turriate Gastulo, Juan C
• Application Number:
  US15/681,145
• Publication Number:
  US 20170374404A1
• Time in Patent Office: Days
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/4854 for modifying image parameters, e.g. image brightness, contrast
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows

Abstract

Methods, apparatuses and storage medium associated with cooperative provision of personalized user functions using shared device and personal device are disclosed herein. In various embodiments, a personal device (PD) method may include receiving, by a personal device of a user, a request to perform a user function to be cooperatively provided by the personal device and a shared device (SD) configured for use by multiple users; and cooperating with the shared device, by the personal device, to provide the requested user function personalized to the user of the personal device. In various embodiments, a SD method may include similar receiving and cooperating operations, performed by the SD. Other embodiments may be disclosed or claimed.

Claims

1. An apparatus capable of being used in association with a television, a camera, a distribution server, at least one personal device, and at least one remote server, the apparatus being capable, when the apparatus is in operation, of communicating via an Internet network with the at least one remote server, the apparatus comprising:

a box device for use in rendering, at least in part, streaming video content via the television, the streaming video content to be received by the box device from the at least one remote server via the Internet network, the box device comprising;

at least one multi-core processor;

mass storage to store program instructions that are capable of being executed, at least in part, by the at least one multi-core processor, the program instructions when executed, at least in part, by the at least one multi-core processor resulting in the box device being capable of performing operations comprising;

performing user facial recognition-related processing, based at least in part, upon image data generated, at least in part, via image capture by the camera of an area that includes a current user of the box device, the user facial recognition-related processing being for use in identifying the current user of the box device from among a plurality of possible users of the box device;

capturing, for upload via the at least one remote server to at least one social network, video clip data comprising at least a portion of the streaming video rendered via the television;

wherein;

in the operation of the apparatus, the image data is capable of being used in association with user gesture processing and user facial expression analysis to be performed, at least in part, by the box device, the user gesture processing to be used to implement, at least in part, at least one command that is to be determined based, at least in part, upon at least one gesture input of the current user, the at least one gesture input to be provided, at least in part, via the at least one personal device for use in controlling, at least in part, the box device, the at least one gesture input being capable of corresponding to at least one user request for change in picture-in-picture display, the user facial expression analysis to be used to determine, at least in part, at least one facial expression of the current user;

in the operation of the apparatus, the box device is capable of receiving, at least in part, the program instructions, via wireless communication, from the distribution server, for provision to the mass storage for execution, at least in part, by the at least one multi-core processor;

in the operation of the apparatus, the box device is capable of being used in a device registration process that is to result in generation of an association mapping of the current user with the box device, the association mapping being for use in enabling logical association of video stream data with a sender of the video stream data, the video stream data to be rendered, via the box device, by the television, as a picture-in-picture;

in the operation of the apparatus, the association mapping is capable of being updated after occurrence of at least one user status change to reflect whether the current user is no longer to be associated with the box device; and

in the operation of the apparatus, the box device is capable of being used in storage and retrieval of annotated captured video stream image content.

2. The apparatus of claim 1, wherein:

the at least one remote server comprises a cloud computing server, social networking server, and/or e-commerce server;

the at least one facial expression comprises approved, disapproved, interested, disinterested, happy, sad, angry or at peace; and

/or

the at least one gesture input comprises at least one hand gesture.

3. The apparatus of claim 2, wherein:

the at least one hand gesture comprises an enlarge gesture, a shrink gesture, a like gesture, a dislike gesture, or a swap gesture; and

/or

the box device is capable of rendering via the television at least one picture-in-picture.

4. The apparatus of claim 3, wherein:

the box device is capable of changing size of the at least one picture-in-picture based at least in part upon user command;

the box device and the television comprise mutually separate devices that are to be coupled together;

the apparatus comprises the camera;

the plurality of possible users comprises certain users associated with a registration function of the box device; and

/or

the wireless communication is via a wireless communication interface of the box device.

5. At least one computer-readable storage medium storing program instructions for execution by a box device, the box device being usable in association with a television, a camera, a distribution server, at least one personal device, and at least one remote server, the box device being usable in rendering, at least in part, when the box device is in operation, streaming video content via the television, the streaming video content to be received by the box device from the at least one remote server via an Internet network, the instructions when executed by the box device resulting in the box device being capable of performing operations comprising:

performing user facial recognition-related processing, based at least in part, upon image data generated, at least in part, via image capture by the camera of an area that includes a current user of the box device, the user facial recognition-related processing being for use in identifying the current user of the box device from among a plurality of possible users of the box device;

capturing, for upload via the at least one remote server to at least one social network, video clip data comprising at least a portion of the streaming video rendered via the television;

wherein;

in the operation of the box device, the image data is capable of being used in association with user gesture processing and user facial expression analysis to be performed, at least in part, by the box device, the user gesture processing to be used to implement at least in part at least one command that is to be determined based, at least in part, upon at least one gesture input of the current user, the at least one gesture input to be provided, at least in part, via the at least one personal device for use in controlling, at least in part, the box device, the at least one gesture input being capable of corresponding to at least one user request for change in picture-in-picture display the user facial expression analysis to be used to determine, at least in part, at least one facial expression of the current user;

in the operation of the box device, the box device is capable of receiving, at least in part, the program instructions, via wireless communication, from the distribution server, for the execution by the box device;

in the operation of the box device, the box device is capable of being used in a device registration process that is to result in generation of an association mapping of the current user with the box device, the association mapping being for use in enabling logical association of video stream data with a sender of the video stream data, the video stream data to be rendered, via the box device, by the television, as a picture-in-picture;

in the operation of the box device, the association mapping is capable of being updated after occurrence of at least one user status change to reflect whether the current user is no longer to be associated with the box device; and

in the operation of the box device, the box device is capable of being used in storage and retrieval of annotated captured video stream image content.

6. The at least one computer-readable storage medium of claim 5, wherein:

the at least one remote server comprises a cloud computing server, social networking server, and/or e-commerce server;

the at least one facial expression comprises approved, disapproved, interested, disinterested, happy, sad, angry or at peace; and

/or

the at least one gesture input comprises at least one hand gesture.

7. The at least one computer-readable storage medium of claim 6, wherein:

the at least one hand gesture comprises an enlarge gesture, a shrink gesture, a like gesture, a dislike gesture, or a swap gesture; and

/or

the box device is capable of rendering via the television at least one picture-in-picture.

8. The at least one computer-readable storage medium of claim 7, wherein:

the box device is capable of changing size of the at least one picture-in-picture based at least in part upon user command;

the box device and the television comprise mutually separate devices that are to be coupled together;

the plurality of possible users comprises certain users associated with a registration function of the box device; and

/or

the wireless communication is via a wireless communication interface of the box device.

9. A method implemented, at least in part, using a box device, the box device being usable in association with a television, a camera, distribution server, at least one personal device, and at least one remote server, the box device being usable in rendering, at least in part, when the box device is in operation, streaming video content via the television, the streaming video content to be received by the box device from the at least one remote server via an Internet network, the method comprising:

performing, at least in part by the box device, user facial recognition-related processing, based at least in part, upon image data generated, at least in part, via image capture by the camera of an area that includes a current user of the box device, the user facial recognition-related processing being for use in identifying the current user of the box device from among a plurality of possible users of the box device;

capturing, at least in part by the box device, for upload via the at least one remote server to at least one social network, video clip data comprising at least a portion of the streaming video rendered via the television;

wherein;

in the operation of the box device, the image data is capable of being used in association with user gesture processing and user facial expression analysis to be performed, at least in part, by the box device, the user gesture processing to be used to implement at least in part at least one command that is to be determined based, at least in part, upon at least one gesture input of the current user, the at least one gesture input to be provided, at least in part via the at least one personal device for use in controlling, at least in part, the box device, the at least one gesture input being capable of corresponding to at least one user request for change in picture-in-picture display the user facial expression analysis to be used to determine, at least in part, at least one facial expression of the current user;

in the operation of the box device, the box device is capable of receiving, at least in part, program instructions, via wireless communication, from the distribution server, for execution by the box device;

in the operation of the box device, the box device is capable of being used in a device registration process that is to result in generation of an association mapping of the current user with the box device, the association mapping being for use in enabling logical association of video stream data with a sender of the video stream data, the video stream data to be rendered, via the box device, by the television, as a picture-in-picture;

in the operation of the box device, the association mapping is capable of being updated after occurrence of at least one user status change to reflect whether the current user is no longer to be associated with the box device; and

in the operation of the box device, the box device is capable of being used in storage and retrieval of annotated captured video stream image content.

10. The method of claim 9, wherein:

the at least one remote server comprises a cloud computing server, social networking server, and/or e-commerce server;

the at least one facial expression comprises approved, disapproved, interested, disinterested, happy, sad, angry or at peace; and

/or

the at least one gesture input comprises at least one hand gesture.

11. The method of claim 10, wherein:

the at least one hand gesture comprises an enlarge gesture, a shrink gesture, a like gesture, a dislike gesture, or a swap gesture; and

/or

the box device is capable of rendering via the television at least one picture-in-picture.

12. The method of claim 11, wherein:

the box device is capable of changing size of the at least one picture-in-picture based at least in part upon user command;

the box device and the television comprise mutually separate devices that are to be coupled together;

the plurality of possible users comprises certain users associated with a registration function of the box device; and

/or

the wireless communication is via a wireless communication interface of the box device.

13. An apparatus capable of being used in association with a television, a camera, a distribution server, at least one personal device, and at least one remote server, the apparatus being capable, when the apparatus is in operation, of communicating via an Internet network with the at least one remote server, the apparatus comprising:

a box device for use in rendering, at least in part, streaming video content via the television, the streaming video content to be received by the box device from the at least one remote server via the Internet network, the box device comprising;

means for performing user facial recognition-related processing, based at least in part, upon image data generated, at least in part, via image capture by the camera of an area that includes a current user of the box device, the user facial recognition-related processing being for use in identifying the current user of the box device from among a plurality of possible users of the box device;

means for capturing, for upload via the at least one remote server to at least one social network, video clip data comprising at least a portion of the streaming video rendered via the television;

wherein;

in the operation of the apparatus, the image data is capable of being used in association with user gesture processing and user facial expression analysis to be performed, at least in part, by the box device, the user gesture processing to be used to implement, at least in part, at least one command that is to be determined based, at least in part, upon at least one gesture input of the current user, the at least one gesture input to be provided, at least in part, via the at least one personal device for use in controlling, at least in part, the box device, the at least one gesture input being capable of corresponding to at least one user request for change in picture-in-picture display, the user facial expression analysis to be used to determine, at least in part, at least one facial expression of the current user;

in the operation of the apparatus, the box device is capable of receiving, at least in part, program instructions, via wireless communication, from the distribution server, for provision to storage means of the box device for execution by the box device;

in the operation of the apparatus, the box device is capable of being used in a device registration process that is to result in generation of an association mapping of the current user with the box device, the association mapping being for use in enabling logical association of video stream data with a sender of the video stream data, the video stream data to be rendered, via the box device, by the television, as a picture-in-picture;

in the operation of the apparatus, the association mapping is capable of being updated after occurrence of at least one user status change to reflect whether the current user is no longer to be associated with the box device; and

in the operation of the apparatus, the box device is capable of being used in storage and retrieval of annotated captured video stream image content.

14. The apparatus of claim 13, wherein:

the at least one remote server comprises a cloud computing server, social networking server, and/or e-commerce server;

the at least one facial expression comprises approved, disapproved, interested, disinterested, happy, sad, angry or at peace; and

/or

the at least one gesture input comprises at least one hand gesture.

15. The apparatus of claim 14, wherein:

the at least one hand gesture comprises an enlarge gesture, a shrink gesture, a like gesture, a dislike gesture, or a swap gesture; and

/or

the box device is capable of rendering via the television at least one picture-in-picture.

16. The apparatus of claim 15, wherein:

the box device is capable of changing size of the at least one picture-in-picture based at least in part upon user command;

the box device and the television comprise mutually separate devices that are to be coupled together;

the apparatus comprises the camera;

the plurality of possible users comprises certain users associated with a registration function of the box device; and

/or

the wireless communication is via a wireless communication interface of the box device.

17. An apparatus capable of being used in association with a television, a camera, a distribution server, one or more personal devices, and one or more remote servers, the apparatus being capable, when the apparatus is in operation, of communicating via an Internet network with the one or more remote servers, the apparatus comprising:

a box device for use in rendering, at least in part, streaming video content via the television, the streaming video content to be received by the box device from the one or more remote servers via the Internet network, the box device comprising;

one or more multi-core processors;

mass storage to store program instructions that are capable of being executed, at least in part, by one or more multi-core processors, the program instructions when executed, at least in part, by the one or more multi-core processors resulting in the box device being capable of performing operations comprising;

performing user facial recognition-related processing based upon image data generated, at least in part, via image capture by the camera of an area that includes a current user of the box device, the user facial recognition-related processing being for use in identifying the current user of the box device from among a plurality of possible users of the box device;

capturing, for upload via the one or more remote servers to one or more social networks, video clip data comprising at least a portion of the streaming video rendered via the television;

wherein;

in the operation of the apparatus, the image data is capable of being used in user gesture processing and user facial expression analysis to be performed, at least in part, by the box device, the user gesture processing to be used to implement one or more commands that are to be determined based upon one or more gesture inputs of the current user, the one or more gesture inputs to be provided via the one or more personal device for use in controlling the box device, the one or more gesture inputs being capable of corresponding to one or more user requests for change in picture-in-picture display, the user facial expression analysis to be used to determine one or more facial expressions of the current user;

in the operation of the apparatus, the box device is capable of receiving, at least in part, the program instructions, via wireless communication, from the distribution server, for provision to the mass storage for execution, at least in part, by the one or more multicore multi-core processors;

in the operation of the apparatus, the box device is capable of being used in a device registration process that is to result in generation of an association mapping of the current user with the box device, the association mapping being for use in enabling logical association of video stream data with a sender of the video stream data, the video stream data to be rendered, via the box device, by the television, as a picture-in-picture;

in the operation of the apparatus, the association mapping is capable of being updated after occurrence of at least one user status change to reflect whether the current user is no longer to be associated with the box device; and

in the operation of the apparatus, the box device is capable of being used in storage and retrieval of annotated captured video stream image content.

18. The apparatus of claim 17, wherein:

the one or more remote servers comprise a cloud computing server, social networking server, and/or e-commerce server;

the one or more facial expressions comprise approved, disapproved, interested, disinterested, happy, sad, angry or at peace; and

/or

the one or more gesture inputs comprise one or more hand gestures.

19. The apparatus of claim 18, wherein:

the one or more hand gestures comprise an enlarge gesture, a shrink gesture, a like gesture, a dislike gesture, or a swap gesture; and

/or

the box device is capable of rendering via the television one or more pictures-in-picture.

20. The apparatus of claim 19, wherein:

the box device is capable of changing size of the one or more pictures-in-picture based upon user command;

the box device and the television comprise mutually separate devices that are to be coupled together;

the apparatus comprises the camera;

the plurality of possible users comprises certain users associated with a registration function of the box device; and

/or

the wireless communication is via a wireless communication interface of the box device.