Graphics processing unit with sensor interface

• US 10,496,579 B2
• Filed: 11/13/2017
• Issued: 12/03/2019
• Est. Priority Date: 11/13/2017
• Status: Active Grant

• Current Assignee: Bobby Gene Burrough
• Original Assignee: Bobby Gene Burrough
• Inventors: Burrough, Bobby Gene
• Primary Examiner(s): Richer, Joni
• Application Number:
  US15/811,160
• Publication Number:
  US 20190146941A1
• Time in Patent Office: 750 Days
• Field of Search: None
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 13/385 for adaptation of a particular data processing system to different peripheral devices
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04N 25/772 comprising A/D, V/T, V/F, I/T or I/F converters

Abstract

Graphics processing unit (GPU) with sensor interface. In an exemplary embodiment, an apparatus includes a GPU that includes a host interface to communicate with a host central processing unit (CPU), a sensor interface to communicate over a sensor bus, and a sensor connected to the sensor bus. The sensor communicates sensor data through the sensor interface, so that the sensor data does not flow through the host interface to the GPU. In an exemplary embodiment, a method is provided for operating a GPU having a host interface to communicate with a host CPU and a sensor interface. The method includes sending control signaling through the sensor interface to control a sensor to capture sensor data, and acquiring the sensor data sent from the sensor through the sensor interface. The sensor data does not flow through the host interface to the GPU.

Claims

1. An apparatus, comprising:

a graphics processing unit (GPU) able to process data in a highly parallel manner including;

a host interface having host interface logic to communicate over a host bus with a host central processing unit (CPU);

a graphics instruction processor (GIP) coupled to the host interface and configured to process display images in accordance with sensor data, wherein the GIP, having an instruction processor, sends control signaling over the sensor interface to control the sensor for capturing the sensor data, and wherein the control signaling does not flow through the host interface;

a display interface coupled to the GIP and configured to facilitating displaying the images; and

a sensor interface having sensor interface logic to communicate over a sensor bus;

a sensor connected to the sensor bus and configured to capture the sensor data representing surrounding information to the sensor and forward the sensor data to the GPU through the sensor interface bypassing the host CPU, wherein the sensor data includes images relating to user face, light source, and surrounding ambient light; and

a display coupled to the display interface and configured to display captured surrounding information in response to the display images.

2. The apparatus of claim 1, wherein the sensor interface comprises a mobile industry processor interface (MIPI).

3. The apparatus of claim 1, wherein the sensor interface comprises at least one interface selected from a set comprising camera serial interface (CSI), peripheral component interconnect (PCI), peripheral component interconnect express (PCIE), inter-integrated circuit (I2C), improved inter-integrated circuit (I3C), serial peripheral interface (SPI), and data bus interfaces.

4. The apparatus of claim 1, wherein the sensor comprises at least one sensor selected from a set comprising an image sensor, ambient light sensor, accelerometer sensor, gyroscope sensor, compass sensor, inertial measurement sensor, light source detector sensor, face detector sensor, global positioning system sensor, barometric pressure sensor, altimeter sensor, ultrasonic sensor, MEMS sensor, and time sensor.

5. The apparatus of claim 1, wherein the GPU further comprising a memory to store the sensor data.

6. The apparatus of claim 1, wherein the display interface includes display interface logic to communicate display data over a display bus to the display.

7. The apparatus of claim 1, the apparatus implemented on a system-on-chip (SoC) integrated circuit.

8. A method for operating a graphics processing unit (GPU) having a host interface to communicate over a host bus with a host central processing unit (CPU) and a sensor interface, the method comprising:

sending control signaling through the sensor interface to a sensor to activate a process of capturing sensor data representing surrounding information around the sensor;

acquiring the sensor data sent from the sensor to GPU through the sensor interface, and wherein the sensor data bypasses the host interface;

processing the sensor data by a graphics instruction processor (GIP) to generate display images, wherein the processing the sensor data includes processing the sensor data to generate processed data representing images relating to user face, light source, and surrounding ambient light; and

displaying images reflecting surrounding environment of the sensor in response to the display images via a display interface of the GPU.

9. The method of claim 8, wherein the sensor interface comprises a mobile industry processor interface (MIPI).

10. The method of claim 8, wherein the sensor interface comprises at least one interface selected from a set comprising camera serial interface (CSI), peripheral component interconnect (PCI), peripheral component interconnect express (PCIE), inter-integrated circuit (I2C), improved inter-integrated circuit (I3C), serial peripheral interface (SPI), and data bus interfaces.

11. The method of claim 8, wherein the sensor comprises an image sensor.

12. The method of claim 8, wherein the sensor comprises at least one of an ambient light sensor, accelerometer sensor, gyroscope sensor, compass sensor, inertial measurement sensor, light source detector sensor, face detector sensor, global positioning sensor, barometric pressure sensor, altimeter sensor, ultrasonic sensor, MEMS sensor, and time sensor.

13. The method of claim 8, further comprising storing the sensor data in a memory.

14. The method of claim 8, further comprising outputting the processed data through the display interface.

15. An apparatus, comprising:

a graphics processing unit (GPU) able to process data in a highly parallel manner including;

a host interface having host interface logic to communicate over a host bus with a host central processing unit (CPU);

a graphics instruction processor (GIP) coupled to the host interface and configured to process display images in accordance with sensor data, wherein the GIP, having an instruction processor, sends control signaling over the sensor interface to control the sensor for capturing the sensor data, and wherein the control signaling does not flow through the host interface;

a display interface coupled to the GIP and configured to facilitating displaying the images; and

a sensor interface configured for interfacing with one or more sensing devices; and

a sensor coupled to the sensor interface for interfacing between the sensor and GPU and configured to capture the sensor data representing light source and ambient light obtained from surrounding environment of the sensor, wherein the sensor transmits the sensor data to the GPU bypassing the host CPU.

16. The apparatus of claim 15, wherein the sensor interface includes at least one interface selected from a set comprising camera serial interface (CSI), peripheral component interconnect (PCI), peripheral component interconnect express (PCIE), inter-integrated circuit (I2C), improved inter-integrated circuit (I3C), serial peripheral interface (SPI), and data bus interfaces.

17. The apparatus of claim 16, wherein the sensor includes at least one sensor selected from a set comprising an ambient light sensor, accelerometer sensor, gyroscope sensor, compass sensor, inertial measurement sensor, light source detector sensor, face detector sensor, global positioning sensor, barometric pressure sensor, altimeter sensor, ultrasonic sensor, MEMS sensor, and time sensor.

18. The apparatus of claim 16, wherein the GPU is implemented on a system-on-chip (SoC) integrated circuit.