User Interface Display Composition with Device Sensor/State Based Graphical Effects

• US 20140092115A1
• Filed: 10/02/2012
• Published: 04/03/2014
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Mazzola, Anthony J.
• Granted Patent: US 9,430,991 B2
• Time in Patent Office: Days
• US Class Current: 345/589
• CPC Class Codes: G09G 2320/0626 for control of overall brightness
• CPC Class Codes: G09G 2320/0666 for control of colour parameters, e.g. colour temperature
• CPC Class Codes: G09G 2320/08 Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
• CPC Class Codes: G09G 2320/10 Special adaptations of display systems for operation with variable images
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/37 Details of the operation on graphic patterns G09G5/38 takes precedence
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence

Abstract

A method comprising receiving sensor data from a sensor, obtaining image data from a graphical effects shader based on the sensor data, blending the image data with a plurality of application surfaces to create a blended image, and transmitting the blended image to a display. Also disclosed is a mobile node (MN) comprising a sensor configured to generate sensor data, a display device, and a processor coupled to the sensor and the device display, wherein the processor is configured to receive the sensor data, obtain image data generated by a graphical effects shader based on the sensor data, blend the image data with an application surface associated with a plurality of applications to create a blended image, and transmit the blended image to the display.

Claims

1. A method comprising:

receiving sensor data from a sensor;

obtaining image data from a graphical effects shader based on the sensor data;

blending the image data with a plurality of application surfaces to create a blended image; and

transmitting the blended image to a display.

2. The method of claim 1 further comprising obtaining composition requirements of a mobile node (MN), composition requirements of an application that provides an application surface, or combinations thereof, and wherein blending the image data with the application surfaces is performed to meet the MN'"'"'s composition requirements, the application'"'"'s composition requirements, or combinations thereof.

3. The method of claim 1 further comprising identifying display regions impacted by the image data prior to blending the image data with the application surfaces.

4. The method of claim 1, wherein the image data and application surfaces each comprise bitmaps.

5. The method of claim 4, wherein blending the image data with the application surfaces to create the blended image comprises pixel blitting.

6. The method of claim 1, wherein the application surfaces are generated by a plurality of applications.

7. The method of claim 1, wherein blending the image data with the application surfaces to create the blended image changes pixel colors, blending, or surface pixel sampling of the application surfaces.

8. The method of claim 1, wherein the application surfaces are generated by a process that is not configured to receive sensor data.

9. The method of claim 1, wherein the sensor comprises a haptics sensor, wherein the blended image comprises distorted application surfaces, and wherein the blended image is displayed in response to vibration sensed by the haptics sensor.

10. The method of claim 1, wherein the sensor comprises a light sensor, wherein the image data comprises a green color, wherein the blended image comprises the application surfaces tinted into the green color, and wherein the blended image is displayed in response to a reduction in ambient light sensed by the light sensor.

11. The method of claim 1, wherein the sensor comprises a battery state sensor, wherein the blended image comprises the application surfaces with a colored border, and wherein the color of the border is selected in response to a change in battery state sensed by the battery state sensor.

12. The method of claim 1, wherein the image data comprises a color value modifier, and wherein the blended image comprises application surfaces with color values modified by the color value modifier.

13. The method of claim 1, wherein the blended image comprises a watermark and the applications surfaces do not comprise the watermark.

14. The method of claim 1, wherein the sensor comprises a light sensor, wherein the blended image comprises a spotlight or an animated sparkle, and wherein the blended image is displayed in response to a reduction in ambient light sensed by the light sensor.

15. The method of claim 1, wherein the sensor comprises a touch sensor, wherein the blended image comprises two substantially circular points of light separated by a space or a substantially circular primary point of light, and wherein the points of light are positioned on the application surfaces in response to user touch sensed by the touch sensor.

16. The method of claim 1, wherein the sensor comprises a touch sensor, wherein the blended image comprises the application surfaces deformed by the image data, and application surface deformities are positioned in response to user touch sensed by the touch sensor.

17. A mobile node (MN) comprising:

a sensor configured to generate sensor data;

a display device; and

a processor coupled to the sensor and the device display, wherein the processor is configured to;

receive the sensor data;

obtain image data generated by a graphical effects shader based on the sensor data;

blend the image data with an application surface associated with a plurality of applications to create a blended image; and

transmit the blended image to the display.

18. The MN of claim 17, wherein the sensor comprises an environmental sensor that indicates the environmental conditions inside of or in close proximity to the MN, and wherein obtaining image data generated by the graphical effects shader comprises requesting image data from the graphical effects shader based on the environmental conditions measured by the environmental sensor.

19. The MN of claim 18, wherein the environmental sensor comprises a light sensor, a temperature sensor, a humidity sensor, a barometric pressure sensor, or combinations thereof.

20. The MN of claim 18, wherein the sensor comprises a position sensor that indicates the position of the MN relative to external objects or geographical areas, and wherein obtaining image data generated by the graphical effects shader comprises requesting image data from the graphical effects shader based on the position of the MN as measured by the position sensor.

21. The MN of claim 20, wherein the position sensor comprises a touch sensor, a location sensor, a magnetic field sensor, an orientation sensor, a proximity sensor, or combinations thereof.

22. The MN of claim 17, wherein the sensor comprises a motion sensor that indicates motion experienced by the MN, and wherein obtaining image data generated by the graphical effects shader comprises requesting image data from the graphical effects shader based on the motion experienced by the MN as measured by the motion sensor.

23. The MN of claim 22, wherein the motion sensor comprises an accelerometer, a gravity sensor, a gyroscope, or combinations thereof.

24. The MN of claim 17, wherein the sensor comprises a battery state sensor, a haptics state sensor, audio state sensor, or combinations thereof.

25. The MN of claim 17, wherein the application surface is generated by a process that is not configured to receive sensor data.