Systems and methods for foley-style haptic content creation

• US 10,139,907 B2
• Filed: 06/16/2014
• Issued: 11/27/2018
• Est. Priority Date: 06/16/2014
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Billington, Doug
• Primary Examiner(s): Patel, Yogeshkumar
• Application Number:
  US14/305,623
• Publication Number:
  US 20150362993A1
• Time in Patent Office: 1,625 Days
• Field of Search: 345156, 342156, 342161, 342163, 342303, 463 30, 463 38, 84737, 340 421, 3404071
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 7/0255 Display systems therefor

Abstract

One illustrative system disclosed herein includes an audio input device configured to detect an audio input and transmit an audio signal associated with the audio input. The illustrative system also includes a haptic trigger device which includes a sensor, wherein the sensor is configured to transmit a sensor signal associated with an event. Further, the illustrative system includes a processor in communication with the audio input device and the haptic trigger device, the processor configured to: receive the audio signal; receive the sensor signal substantially simultaneously to receiving the audio signal; record the audio signal to an audio track; and insert a haptic effect marker into a haptic track based in part on the sensor signal, wherein the haptic track and the audio track are associated with a video.

Claims

1. A system comprising:

an audio input device configured to detect a sound effect and transmit an audio signal associated with the sound effect;

a haptic trigger device configured to detect a manipulation of a prop and transmit a sensor signal associated with the manipulation, wherein the haptic trigger device is different from the audio input device;

a processor in communication with the audio input device and the haptic trigger device; and

a memory on which instructions executable by the processor are stored to cause the processor to;

play back a pre-recorded video file; and

while the pre-recorded video file is being played back;

receive the audio signal; and

receive the sensor signal substantially simultaneously to receiving the audio signal;

based on the audio signal, record the sound effect to an audio track associated with the pre-recorded video file; and

based on the sensor signal, insert a haptic effect marker into a haptic track associated with the pre-recorded video file, wherein the haptic effect marker comprises timestamp data.

2. The system of claim 1, wherein the haptic trigger device comprises a gyroscope, an accelerometer, a depth sensor, a camera, a phototransistor, a switch, a button, or a piezoelectric transducer.

3. The system of claim 1, wherein the haptic trigger device comprises a plurality of sensors.

4. The system of claim 1, wherein the prop is a hand-held device and the haptic trigger device is positioned on the prop.

5. The system of claim 4, wherein the prop is configured to be manipulated to generate the sound effect.

6. The system of claim 5, wherein the prop comprises a Foley artist prop.

7. The system of claim 1, wherein the haptic effect marker comprises haptic effect data associated with a haptic effect.

8. The system of claim 7, wherein the processor is further configured to:

prior to inserting the haptic effect marker into the haptic track, determine the haptic effect data based at least in part on the sensor signal.

9. The system of claim 1, further comprising a plurality of haptic trigger devices.

20. The system of claim 1, wherein the haptic effect marker is configured to cause a haptic output device to output a haptic effect configured to simulate a video event in the pre-recorded video file.

21. The system of claim 1, wherein the processor is further configured to, prior to inserting the haptic effect marker into the haptic track:

select a haptic effect from among a plurality of available haptic effects; and

determine haptic effect data to be included in the haptic effect marker based on the haptic effect.

22. The system of claim 1, wherein the processor is further configured to, prior to inserting the haptic effect marker into the haptic track:

download haptic effect data from a remote data source; and

associate the haptic effect data with the haptic effect marker.

10. A method comprising:

playing back, by a processor, a pre-recorded video file; and

while playing back the pre-recorded video file;

receiving, by the processor, an audio signal from an audio input device, the audio signal comprising a sound effect;

receiving, by the processor, a sensor signal from a haptic trigger device substantially simultaneously to receiving the audio signal, the sensor signal indicating a manipulation of a prop, wherein the haptic trigger device is different from the audio input device;

based on the audio signal, recording, by the processor, the sound effect to an audio track associated with the pre-recorded video file; and

based on the sensor signal, inserting, by the processor, a haptic effect marker into a haptic track associated with the pre-recorded video file, wherein the haptic effect marker comprises timestamp data.

11. The method of claim 10, wherein the haptic trigger device comprises a plurality of sensors.

12. The method of claim 10, wherein the prop is a hand-held device and the haptic trigger device is positioned on the prop.

13. The method of claim 12, wherein the prop comprises a Foley artist prop.

14. The method of claim 10, wherein the haptic effect marker comprises haptic effect data associated with a haptic effect.

15. The method of claim 14, further comprising:

prior to inserting the haptic effect marker into the haptic track, determining the haptic effect data based at least in part on the sensor signal.

16. A non-transient computer readable medium comprising program code, which when executed by a processor is configured to cause the processor to:

play back a pre-recorded video file; and

while playing back the pre-recorded video file;

receive an audio signal from an audio input device, the audio signal comprising a sound effect;

receive a sensor signal from a haptic trigger device substantially simultaneously to receiving the audio signal, the sensor signal indicating a manipulation of a prop, wherein the haptic trigger device is different from the audio input device;

based on the audio signal, record the sound effect to an audio track associated with the pre-recorded video file; and

based on the sensor signal, insert a haptic effect marker into a haptic track associated with the pre-recorded video file, wherein the haptic effect marker comprises timestamp data.

17. The non-transient computer readable medium of claim 16, wherein the haptic trigger device comprises a plurality of sensors.

18. The non-transient computer readable medium of claim 16, wherein the prop is a hand-held device and the haptic trigger device is positioned on the prop.

19. The non-transient computer readable medium of claim 18, wherein the prop comprises a Foley artist prop.