Computer-controlled video entertainment system

• US 9,486,701 B2
• Filed: 12/21/2010
• Issued: 11/08/2016
• Est. Priority Date: 12/30/2009
• Status: Active Grant

• Current Assignee: CryTek GmbH
• Original Assignee: CryTek GmbH
• Inventors: Yerli, Cevat
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): ANDRAMUNO, FRANKLIN S
• Application Number:
  US12/974,077
• Publication Number:
  US 20110162005A1
• Time in Patent Office: 2,149 Days
• Field of Search: 725/37, 725/114
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/27 characterised by a large display in a public venue, e.g. in a movie theatre, stadium or game arena
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/409 Data transfer via television network
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/2181 comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: H04N 21/2747 Remote storage of video programs received via the downstream path, e.g. from the server
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

A computer-controlled video entertainment system providing user-related input data to said system. The invention provides an rich input sensor device having direct access to an IP-based network, in particular to the Internet, for transmitting sensor signals or data derived thereof as user-related input data via the IP-based network to a remote computing device or server, preferably to a cloud of servers, which then process these input data for controlling the running entertainment program. The sensor device can be made simple, since it is mainly designed to detect user activities and to generate sensor signals which are directly transmitted as input data to the remote computer(s) or server(s). The sensor device can directly communicate with any IP-based computing device (gaming server or the like) being installed remotely and having powerful hardware and software to process the received input data.

Claims

1. A computer-controlled video entertainment system, the system comprising:

a display device comprising a network interface to directly connect to an IP-based network configured to receive output data via the IP-based network, wherein the output data transmitted via the IP-based network to the display device include rendered video data for instantly presenting rendered video to at least one user;

at least a first and a second plurality of remotely installed computing servers, each plurality of computing servers being linked to each other to form a first and second cluster, the computing servers being installed remotely from said display device, wherein one of the first and second cluster is a processing cluster configured to execute an entertainment software application for providing output data to be transmitted via the IP-based network to said display device to display the rendered video, wherein the processing cluster is further configured to host a dynamic game engine, including a rendering engine, configured to generate the rendered video data to be transmitted via the IP-based network to the display device, wherein the rendered video data is ready for display by the display device when it is transmitted by the processing cluster to the display device, and wherein the other of the first and second cluster is a content cluster configured to provide content data to the processing cluster; and

a sensor device directly connected to the IP-based network to transmit input data via the IP-based network, the sensor device being installed in the vicinity of said display device and comprising at least one optical sensory element and at least one acoustic sensory element for providing user-related input data as rich input data to be processed by said entertainment software application executed on the processing cluster, the sensor device further comprising a network interface being different from the network interface of the display device and configured to directly transmit said input data via said IP-based network to said processing cluster,

wherein the processing cluster is further configured to render graphics under control of the user-related input data, wherein the user-related input data are sensor signals of the sensor device or data derived thereof, and to transmit the rendered data directly to the display device.

2. The computer-controlled video entertainment system of claim 1, wherein the plurality of remotely installed computing servers are linked to each other to form a cluster or cloud of computing servers to form an overall cloud wherein the cluster or cloud of computing servers comprises at least one gateway server configured to be accessed by the display device, wherein the at least one gateway server is configured to build up a video streaming link to the display device.

3. The computer-controlled video entertainment system of claim 2, wherein the computing servers are linked to each other via said IP-based network.

4. The computer-controlled video entertainment system of claim 1, wherein said IP-based network is constituted by the Internet.

5. The computer-controlled video entertainment system of claim 1, wherein said network interface of said sensor device is a wired or wireless Internet module directly connecting the sensor device to the Internet.

6. The computer-controlled video entertainment system of claim 1, wherein said sensor device comprises a sensor module with at least one sensory element for detecting user activities, in particular natural user inputs such as physical gestures and/or speech, and for producing sensor signals.

7. The computer-controlled video entertainment system of claim 6, wherein said sensor device further comprises a processing unit for processing said sensor signals to generate said user-related input data representing said detected user activities.

8. The computer-controlled video entertainment system of claim 7, wherein the processing unit provides said user-related input data to said network interface for being transmitted via said IP-based network to said plurality of remotely installed computing servers.

9. The computer-controlled video entertainment system of claim 6, wherein said at least one sensor element is an optical sensor configured to detect physical gestures or appearance of the at least one user, an acoustical sensor for detecting user-produced speech, sound or noise and/or a thermal sensor for detecting physical condition by temperature.

10. The computer-controlled video entertainment system of claim 7, wherein said processing unit receives from said sensor elements the sensor signals and generates raw input data which represent said sensor signals.

11. The computer-controlled video entertainment system of claim 7, wherein said processing unit receives from said sensor elements the sensor signals and pre-processes these signals to generate said input data, in particular by applying signal and/or data recognition to generate input data which represent recognized user-related input patterns, in particular gestures and/or speech commands.

12. The computer-controlled video entertainment system of claim 11, wherein said processing unit receives sensor signals which relate to a plurality of users and processes these signals to generate collective input data, in particular by applying signal and/or data recognition to generate collective input data which represent recognized multi-user or user-group-related input patterns, in particular collective gestures and/or collective speech commands.

13. The computer-controlled video entertainment system of claim 1, wherein said sensor device further comprises a visual output or projection module to output visual data to said at least one user, in particular laser-based projection allowing user activities.

14. The computer-controlled video entertainment system of claim 1, wherein said sensor device further comprises a wired or wireless interface/networking module to receive further input data from external user input devices, in particular from gamepads, joysticks, keypads, keyboards.

15. The computer-controlled video entertainment system of claim 1, wherein the sensor device comprises at least two optical sensory elements and at least one acoustic sensory element.

16. The computer-controlled video entertainment system of claim 1, wherein said at least one interface of said sensor device is at least one of the following interfaces:

a wired or wireless network interface for an IP-based network, in particular for the Internet,

a wireless interface module, and/or

a wired interface connector for providing a wired or wireless connection between said sensor device and said plurality of remotely installed computing servers.

17. The computer-controlled video entertainment system of claim 1, wherein said sensor device is a built-in device being installed in said display device.

18. The computer-controlled video entertainment system of claim 1, wherein said display device being a consumer display device, in particular a flat screen display or a TV set.

19. The computer-controlled video entertainment system of claim 1, wherein each plurality of computing servers is installed at a different location.