Attention-based rendering and fidelity

• US 9,367,117 B2
• Filed: 08/29/2013
• Issued: 06/14/2016
• Est. Priority Date: 08/29/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Timm, Paul, Cevallos, Andres Ramos, Halvorson, Ryan
• Primary Examiner(s): Haley, Joseph
• Assistant Examiner(s): FRANK, EMILY J
• Application Number:
  US14/014,199
• Publication Number:
  US 20150061989A1
• Time in Patent Office: 1,020 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G09G 2330/021 Power management, e.g. power saving
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: G09G 5/391 Resolution modifying circuits, e.g. variable screen formats
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Methods and systems for attention-based rendering on an entertainment system are provided. A tracking device captures data associated with a user, which is used to determine that a user has reacted (e.g., visually or emotionally) to a particular part of the screen. The processing power is increased in this part of the screen, which increases detail and fidelity of the graphics and/or updating speed. The processing power in the areas of the screen that the user is not paying attention to is decreased and diverted from those areas, resulting in decreased detail and fidelity of the graphics and/or decreased updating speed.

Claims

1. A method for attention-based rendering on an entertainment system, the method comprising:

receiving tracking data from at least one user by a tracking device, wherein the tracking data is captured in response to a reaction of the user to at least one area displayed on a single display of a display device, wherein the reaction from the user is associated with an event that has been recently displayed on the at least one area displayed on the single display of the display device, and wherein the reaction includes a biometric reaction;

sending the tracking data by way of the tracking device to a processor;

executing instructions stored in memory, wherein execution of the instructions by a processor;

utilizes the tracking data to determine that the user reacted to the at least one area of the single display on the display device associated with the event that has been recently displayed on the single display on the display device; and

communicates to a graphics processing unit to alter processing power used for rendering graphics with reference to the recently displayed event in the at least one area of the single display.

2. The method of claim 1, wherein the tracking data includes gesture-based tracking data.

3. The method of claim 2, wherein the processing power is increased to render greater detail and graphical fidelity in the area.

4. The method of claim 2, wherein the processing power is increased to update at least one object within the area in real-time.

5. The method of claim 1, wherein the tracking data includes retinal tracking data.

6. The method of claim 1, wherein the tracking data includes audio-based tracking data.

7. The method of claim 1, wherein the reaction includes a visual reaction.

8. The method of claim 7, wherein the visual reaction is detected by movement of the eyes of the user toward or away from the area.

9. The method of the claim 7, wherein the visual reaction is captured by a camera integrated into the entertainment system.

10. The method of claim 7, wherein the visual reaction is captured by a peripheral camera connected to the entertainment system.

11. The method of claim 1, wherein the reaction includes an emotional reaction.

12. The method of claim 11, wherein the emotional reaction includes a vocal reaction by the user.

13. The method of claim 11, wherein the vocal reaction is captured by a microphone.

14. The method of claim 1, wherein the reaction indicates that the user is focusing on the area.

15. The method of claim 1, wherein the reaction indicates that the user is not focusing on the area.

16. The method of claim 15, wherein the processing power diverted to the area is decreased based on the reaction that the user is not focusing on the area.

17. The method of claim 1, further comprising:

receiving a selection by the user indicating a preference for initiating a power-saving mode;

storing the selection in memory; and

initiating a power-saving mode when the tracking data indicates a lack of attention to the single display of the display device by the user.

18. A non-transitory computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for attention-based rendering on an entertainment system, the method comprising:

receiving tracking data from at least one user by a tracking device, wherein the tracking data is captured in response to a reaction of the user to at least one area displayed on a single display of a display device, wherein the reaction from the user is associated with an event that has been recently displayed on the at least one area displayed on the single display of the display device, and wherein the reaction includes a biometric reaction;

sending the tracking data by way of the tracking device to a processor;

utilizing the tracking data to determine that the user reacted to the at least one area of the single display on the display device associated with the event that has been recently displayed on the single display on the display device; and

communicating to a graphics processing unit to alter processing power used for rendering graphics with reference to the recently displayed event in the at least one area of the single display.

19. A system for attention-based rendering, the system comprising:

a memory;

a display device having a single display;

a tracking device for capturing tracking data associated with a user, wherein the tracking data is captured in response to a reaction of the user to at least one area displayed on the single display of a display device, wherein the reaction from the user is associated with an event that has been recently displayed on the at least one area displayed on the single display of the display device, and wherein the reaction includes a biometric reaction;

a processor for executing instructions stored in memory, wherein execution of the instructions by the processor;

utilizes the tracking data to determine that the user reacted to at least one area displayed of the single display on the display device associated with the event that has been recently displayed on the single display on the display device; and

communicates to a graphics processing unit to alter processing power used for rendering graphics with reference to the recently displayed event in the at least one area of the single display.