SYNCHRONIZED INPUT FEEDBACK FOR MACHINE LEARNING

• US 20200134447A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Inventors: Bennett, Matthew
• Granted Patent: US 11,706,499 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/69 by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/234 Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

A method and system for providing synchronized input feedback, comprising receiving an input event, encoding the input event in an output stream wherein the encoding of the input event is synchronized to a specific event and reproducing the output stream through an output device whereby the encoded input event in the reproduced output stream is imperceptible to the user

Claims

1. A method for providing synchronized input feedback;

a) receiving an input event;

b) encoding the input event in an output stream wherein the encoding of the input event is synchronized to a specific event;

c) reproducing the output stream through an output device, wherein the encoded input event in the reproduced output stream is imperceptible to the user, when reproduced through the output device.

2. The method of claim 1 wherein the output stream includes an audio stream and the input event is encoded as a tone in the audio stream.

3. The method of claim 2 wherein the input event is encoded as an infrasonic tone.

4. The method of claim 2 wherein the input event is encoded as an ultrasonic tone.

5. The method of claim 1 wherein the output stream includes a video stream.

6. The method of claim 5 wherein the input event is encoded as a watermark over the video stream.

7. The method of claim 5 wherein the input event is encoded as metadata in the video stream.

8. The method of claim 7 wherein the metadata is supplemental enhancement information for each video frame.

9. The method of claim 1 wherein the output stream comprises a series of events and the encoding of the input event is synchronized with the series of output events.

10. The method of claim 1 wherein the encoding of the input event is synchronized to an event occurring within a videogame.

11. The method of claim 1 wherein the encoding of the input event is synchronized to an event occurring on a remote device over a network.

12. The method of claim 1 wherein the input event is a key press.

13. The method of claim 1 wherein the input event is a joystick control input.

14. The method of claim 1 wherein the input event is a steering wheel control input.

15. The method of claim 1 wherein the output stream with the encoded input is filtered to separate the encoded input event from the output stream.

16. The method of claim 15 wherein the output stream and the encoded input event is provided to a Neural Network.

17. The method of claim 16 wherein the Neural Network is trained to associate the encoded input event with the specific event.

18. The method of claim 1 wherein the input event is configured to cause the specific event.

19. Instructions embedded in a non-transitory computer readable medium that when executed to enact the method comprising:

a) receiving an input event;

b) encoding the input event in an output stream wherein the encoding of the input event is synchronized to a specific event;

c) reproducing the output stream through an output device whereby the encoded input event in the reproduced output stream is undetectable to the user.

20. A system comprising:

a processor;

memory;

non-transitory instruction in the memory that when executed cause the processor to enact the method comprising;

a) receiving an input event;

b) encoding the input event in an output stream wherein the encoding of the input event is synchronized to a specific event;

c) reproducing the output stream through an output device whereby the encoded input event in the reproduced output stream is undetectable to the user.

21. A method for improved machine learning training comprising;

a) receiving an output stream comprising an encoded input event wherein the encoded input event is undetectable to the user;

b) filtering the output stream to recover the encoded input event wherein the encoded input event is synchronized to a specific event;

c) training a neural network to associate the encoded input event with the specific event.