Methods and apparatus for multimedia presentation

• US 10,705,706 B2
• Filed: 03/06/2018
• Issued: 07/07/2020
• Est. Priority Date: 03/06/2017
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Zhao, Nan, Azaria, Asaph, Paradiso, Joseph
• Primary Examiner(s): Hong, Stephen S
• Assistant Examiner(s): Hasty, Nicholas
• Application Number:
  US15/912,950
• Publication Number:
  US 20180253222A1
• Time in Patent Office: 854 Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Abstract

Closed loop control of a multimedia system may be achieved with a control space that includes at least two control axes. In each control axis, a coordinate on the axis may indicate a degree to which a multimedia scene facilitates or is perceived to facilitate a user state or a degree to which the user state is achieved. For example, the two control axes may be focus and restoration. A user may provide input that specifies a target state. The target state may be different than, or the same as, the user'"'"'s current state. The system may select a scene that has coordinates, in the control space, that are closest, by at least a threshold, to the target state, and present the scene to the user. Sensors in the system may measure a user state that results from presenting the scene. The system may revise the scene'"'"'s coordinates accordingly.

Claims

1. A system comprising:

(a) a media console;

(b) one or more sensors; and

(c) one or more computers;

wherein the system is configured

(i) to associate each scene in a set of stored scenes with coordinates in a contextual control space, which control space has at least two contextual control axes,

(ii) to accept input from a user, which input specifies coordinates of a target state, which coordinates specify a point in the contextual control space,

(iii) to select a specific scene, in the set of stored scenes, which has coordinates, in the contextual control space, that are closer, by at least a first threshold amount, to the coordinates of the target state than are the coordinates of any other scene in the set of stored scenes, including any scene then being presented by the system,

(iv) to present the specific scene to the user,

(v) to calculate, based on measurements taken by the sensors, coordinates, in the contextual control space, of a specific user state of the user that results from presenting the specific scene to the user,

(vi) to revise the coordinates of the specific scene, based on the coordinates of the specific user state that results from presenting the specific scene,

(vii) after revising the coordinates of the specific scene, to select a second scene, in the set of stored scenes, that is different than the specific scene and that has coordinates in the contextual control space which are then closer, by at least a second threshold amount, to the coordinates of the target state than are the coordinates of any other scene in the set of stored scenes, including the specific scene; and

(viii) to present the second scene to the user.

2. The system of claim 1, wherein the input specifies that the coordinates of the target state are the coordinates, in the contextual control space, of a current user state, which current user state is a user state of the user when the input is accepted.

3. The system of claim 1, wherein the system is configured to accept coordinates of the target state that are different than coordinates, in the contextual control space, of a user state of the user when the input is accepted.

4. The system of claim 1, wherein the specific scene and the second scene, when presented to the user, each comprise:

(a) a video;

(b) an audio presentation; and

(c) artificial illumination other than the video.

5. The system of claim 1, wherein:

(a) the at least two contextual control axes include a focus axis and a restoration axis;

(b) a coordinate on the focus axis is indicative of a degree to which a scene facilitates or is perceived to facilitate focus or of a degree to which focus occurs; and

(c) a coordinate on the restoration axis is indicative of a degree to which a scene facilitates or is perceived to facilitate relaxation or of a degree to which relaxation occurs.

6. A system comprising:

(a) a media console;

(b) one or more sensors; and

(c) one or more computers;

wherein the system is configured;

(i) to associate each scene in a set of stored scenes with coordinates in a contextual control space, which control space has at least two contextual control axes,

(ii) to accept input from a user, which input specifies coordinates of a target state, which coordinates specify a point in the contextual control space,

(iii) to select a specific scene, in the set of stored scenes, which has coordinates, in the control space, that are closer, by at least a threshold amount, to the coordinates of the target state than are the coordinates of any other scene in the set of stored scenes, including any scene then being presented by the system, and

(iv) to present the specific scene to the user.

7. The system of claim 6, wherein:

(a) the control space is a contextual control space; and

(b) the control axes are contextual control axes.

8. The system of claim 6, wherein the specific scene, when presented to the user, comprises:

(a) a video;

(b) an audio presentation; and

(c) artificial illumination other than the video.

9. The system of claim 6, wherein the input specifies that the coordinates of the target state are the coordinates, in the control space, of a current user state, which current user state is a user state of the user when the input is accepted.

10. The system of claim 6, wherein the system is configured to accept coordinates of the target state that are different than coordinates, in the control space, of a user state of the user when the input is accepted.

11. The system of claim 6, wherein:

(a) the at least two control axes consist of only a focus axis and a restoration axis;

(b) a coordinate on the focus axis is indicative of a degree to which a scene facilitates or is perceived to facilitate focus or of a degree to which focus occurs; and

(c) a coordinate on the restoration axis is indicative of a degree to which a scene facilitates or is perceived to facilitate relaxation or of a degree to which relaxation occurs.

12. The system of claim 6, wherein:

(a) the at least two control axes include a creativity axis; and

(b) a coordinate on the creativity axis is indicative of a degree to which a scene facilitates or is perceived to facilitate creativity or of a degree to which a user attains a creative state.

13. The system of claim 6, wherein the system is configured to modify the specific scene in such a way that coordinates, in the control space, of the specific scene move closer to the coordinates, in the control space, of the target state.

14. The system of claim 6, wherein the one or more computers are programmed to employ multiple user models for the user, one model at a time, in such a way that the coordinates of the specific scene depend on which of the user models is employed.

15. The system of claim 8, wherein the one or more computers are programmed to employ multiple user models for the user, one model at a time, in such a way that the coordinates of the specific user state depend on which of the user models is employed.

16. A method comprising:

(a) associating each scene in a set of stored scenes with coordinates in a contextual control space, which control space has at least two contextual control axes,

(b) accepting input from a user, which input specifies coordinates of a target state, which coordinates specify a point in the contextual control space,

(c) selecting a specific scene, in the set of stored scenes, which has coordinates, in the contextual control space, that are closer, by at least a first threshold amount, to the coordinates of the target state than are the coordinates of any other scene in the set of stored scenes, including any scene then being presented by the system,

(d) presenting the specific scene to the user;

(e) taking, with sensors, measurements of the user;

(f) calculating, based on the measurements, coordinates, in the contextual control space, of a specific user state of the user that results from presenting the specific scene to the user;

(g) revising the coordinates of the specific scene, based on the coordinates of the specific user state that results from presenting the specific scene;

(h) after revising the coordinates of the specific scene, selecting a second scene, in the set of stored scenes, that is different than the specific scene and that has coordinates in the contextual control space which are then closer, by at least a second threshold amount, to the coordinates of the target state than are the coordinates of any other scene in the set of stored scenes, including the specific scene; and

(i) presenting the second scene to the user.

17. The method of claim 16, wherein the input specifies that the coordinates of the target state are the coordinates, in the contextual control space, of a current user state, which current user state is a user state of the user when the input is accepted.

18. The method of claim 16, wherein the coordinates of the target state are different than coordinates, in the contextual control space, of a user state of the user when the input is accepted.

19. The method of claim 16, wherein the specific scene and the second scene, when presented to the user, each comprise:

(a) a video;

(b) an audio presentation; and

(c) artificial illumination other than the video.

20. The method of claim 16, wherein:

(a) the at least two contextual control axes include a focus axis and a restoration axis;

(b) a coordinate on the focus axis is indicative of a degree to which a scene facilitates or is perceived to facilitate focus or of a degree to which focus occurs; and

(c) a coordinate on the restoration axis is indicative of a degree to which a scene facilitates or is perceived to facilitate relaxation or of a degree to which relaxation occurs.