Real time video special effects system and method

• US 10,863,109 B2
• Filed: 06/28/2019
• Issued: 12/08/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: Henry M. Pena, Thomas F. Bryant III
• Original Assignee: Henry M. Pena, Thomas F. Bryant III
• Inventors: Pena, Henry M., Bryant, III, Thomas F.
• Primary Examiner(s): Fosselman, Joel W
• Application Number:
  US16/456,639
• Publication Number:
  US 20200137319A1
• Time in Patent Office: 529 Days
• Field of Search: 348239
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate

Abstract

The present technology is a graphical user interface for a video recording system and method of recording a video while applying special effects in real time prior to and/or while recording. The interface is associated with an electronic device including a processor in communication with a camera and a memory unit. Inputs from the interface are utilized by the processing unit to determine if the native speed rate of the raw video data is to be changed, and if so then modifies at least one frame in the raw video data to create a modified video data at a modified speed rate that is different to the native speed rate. The system then generates an output video feed that is displayed in the graphical user interface. This allows for the continuous recording of video from the camera or video feed at different speed rates without altering camera operations or settings.

Claims

1. A video recording interface system for controlling a special effects operation of live video recording data in real time, the system comprising:

a mobile electronic device in operable communication with at least one processing unit and with at least one memory, the processing unit being configured or configurable to receive video data at a first frame rate from a camera, the video data at least in part corresponding to images being captured by the camera; and

a graphical user interface associated with the mobile electronic device and displayable on the mobile electronic device, the graphical user interface comprising a video display region, and at least one first affordance configured or configurable to provide at least one input receivable and usable by the at least one processing unit;

wherein the at least one processing unit is configured to determine from the at least one input if the at least one input is associated with changing the first frame rate of the video data, and if so, to modify at least one frame in the video data to create modified video data at one or more modified frame rates that are different to the first frame rate in real time with receiving the video;

wherein the graphical user interface is configured to receive the modified video data and display the modified video data in the video display region.

2. The video recording interface system according to claim 1, wherein the camera and the graphical user interface are incorporated in the mobile electronic device.

3. The video recording interface system according to claim 2, wherein the at least one processing unit and the at least one memory are incorporated in the mobile electronic device, and wherein the processing unit is configured or configurable to create at least a portion of the modified video data.

4. The video recording interface system according to claim 3, wherein the mobile electronic device is a mobile phone or tablet.

5. The video recording interface system according to claim 2, wherein the at least one processing unit and the at least one memory are incorporated in a remote server in communication with the mobile electronic device, and wherein the at least one processing unit is configured or configurable to create at least a portion of the modified video data.

6. The video recording interface system according to claim 5, wherein the modified video data is displayed on the graphical user interface at the modified playing speed while still recording the video data at the first frame rate in real time.

7. The video recording interface system according to claim 6, wherein the graphical user interface includes a second affordance configured or configurable to provide a second input to the processing unit to initiate or stop a recording operation of the video, and a third affordance usable in determining a change in zoom factor of the video data.

8. The video recording interface system according to claim 6, further comprises a second video display region configured to display a second video feed being different to the modified video data displayed in the display region, the second video feed being any one or any combination of the following:

the video data at the first frame rate, and the modified video data at the modified frame rates.

9. The video recording interface system according to claim 6, wherein the graphical user interface further includes a record affordance configured or configurable to provide at least one record input receivable and usable by the processing unit in at least determining if a recording operation is to be started or stopped.

10. The video recording interface system according to claim 6, wherein the at least one first affordance is a touch affordance and the at least one input is provided while a touching and holding action is provided to the touch affordance, and wherein the modified video data is displayed in the video display region during the touching and holding action, and upon removal of the touching and holding action the first display region displays the video data at the first frame rate.

11. The video recording interface system according to claim 8, wherein the at least one first affordance is a touch affordance and the at least one input is provided after a first touching action is provided to the touch affordance, and wherein the modified video data displayed in the video display region is changed to the video data at the first frame rate after a second touching action is provided to the touch affordance.

12. The video recording interface system according to claim 8, wherein the at least one first affordance is a sliding affordance including multiple regions with each of the regions being associated with one modified frame rate of the one or more modified frame rates different from each other, and wherein the at least one input is based on a touching and sliding action along the sliding affordance to a first region of the multiple regions.

13. The video recording interface system according to claim 12, wherein the modified video data is displayed in the video display region at the modified frame rate associated with the first region, and then changes to a second modified frame rate associated with a second region of the multiple regions when the touching and sliding action moves to the second region, and wherein upon removal of the touching and sliding action the first display region displays the video data at the first frame rate.

14. A method for controlling a special effects operation of live video recording data in real time, the method comprising the steps of:

a) displaying a graphical user interface including at least one affordance on a display operably associated with a mobile electronic device, the mobile electronic device being in operable communication with at least one processing unit and with at least one memory;

b) receiving, by the at least one processing unit, video data at a first frame rate from a camera, the video data at least in part corresponding to images being captured by the camera in real time;

c) receiving, by the processing unit, at least one input from the at least one affordance upon activation of the at least one affordance by a user;

d) determining if the at least one input is associated with changing the first frame rate of the video data, and if so, modifying the video data to create modified video data at one or more modified frame rates that are different to the first frame rate in real time while receiving the video data; and

e) displaying in at least a first display region of the graphical user interface output video data to the display, wherein the output video data is any one or any combination of the following;

the video data at the first frame rate, and the modified video data at the modified frame rates.

15. The method according to claim 14, wherein the camera and the graphical user interface are incorporated in the mobile electronic device.

16. The method according to claim 15, wherein the at least one processing unit and the at least one memory are incorporated in the mobile electronic device, and wherein the processing unit is configured or configurable to create at least a portion of the modified video data.

17. The method according to claim 16, wherein the mobile electronic device is a mobile phone or tablet.

18. The method according to claim 15, wherein the at least one processing unit and the at least one memory are incorporated in a remote server in communication with the mobile electronic device, and wherein the at least one processing unit is configured or configurable to create at least a portion of the modified video data.

19. The method according to claim 18, wherein the modified video data is displayed on the graphical user interface at the modified playing speed while still recording the video data at the first frame rate in real time.

20. The method according to claim 19, further comprises the step of determining if the at least one input from the graphical user interface is to change the first frame rate, then determining if the modified frame rate is less than the first frame rate, and if so wherein the modifying of the video data includes adding at least one new frame to the video data to create the modified video data.

21. The method according to claim 20, wherein the adding of the new frame includes copying at least one frame to create the new frame or blending at least one frame with a next frame to create the new frame, and adding the new frame to the video data adjacent to the at least one frame.

22. The method according to claim 20, wherein the new frame is a plurality of new frames each being a copy of at least one frame from the video data.

23. The method according to claim 22, wherein each of the new frames is added to the video data adjacent to the at least one frame or adjacent to a second frame of the video data.

24. The method according to claim 19, further comprises the step of determining if the input from the graphical user interface is to change the first frame rate, then determining if the modified frame rate is greater than the first frame rate, and if so wherein the modifying of the video data includes removing at least one first frame from the video data to create the modified video data.

25. The method according to claim 24, wherein the removing the first frame includes selecting the first frame to be removed, and then removing the first frame from the video data to create the modified video data.

26. The method according to claim 25, wherein the first display region is configured or configurable to display the modified video data, and wherein the graphical user interface includes a second display region configured or configurable to display the video data or at least one still frame of the video data that was previously captured within a current session.

27. The method according to claim 25, wherein the output video data displayed in the first display region of the graphical user interface is a combination of the video data and the modified video data, the modified video data is configured or configurable to include multiple subsets each having a frame rate dependent on the input, with a transitioning between the video data and any one of the subsets or between any of the subsets being dependent on the input, and wherein the output video data is displayed in the graphical user interface in real time while receiving the video data from the camera.

28. The method according to claim 19, wherein the at least one affordance is a touch affordance and the at least one input is provided while a touching and holding action is provided to the touch affordance, and wherein the modified video data is displayed in the video display region during the touching and holding action, and upon removal of the touching and holding action the first display region displays the video data at the first frame rate.

29. The method according to claim 19, wherein the at least one affordance is a touch affordance and the at least one input is provided after a first touching action is provided to the touch affordance, and wherein the modified video data displayed in the video display region is changed to the video data at the first frame rate after a second touching action is provided to the touch affordance.

30. The method according to claim 19, wherein the at least one affordance is a sliding affordance including multiple regions with each of the regions being associated with one modified frame rate of the one or more modified frame rates different from each other, and wherein the at least one input is based on a touching and sliding action along the sliding affordance to a first region of the multiple regions.

31. The method according to claim 30, wherein the modified video data is displayed in the video display region at the modified frame rate associated with the first region, and then changes to a second modified frame rate associated with a second region of the multiple regions when the touching and sliding action moves to the second region, and wherein upon removal of the touching and sliding action the first display region displays the video data at the first frame rate.

32. A non-transitory computer readable medium with an executable program stored thereon comprising instructions for execution by at least one processing unit for controlling a special effects operation of live video recording data in real time, such that the instructions when executed by the at least one processing unit cause the at least one processing unit to:

receive, by the at least one processing unit, video data at a first frame rate from a camera, the video data at least in part corresponding to images being captured by the camera in real time;

display a graphical user interface on a mobile electronic device;

receive, by the processing unit, at least one input from an affordance of the graphical user interface upon activation of the affordance by a user, the graphical user interface being executable by the at least one processing unit and displayable on a display;

modifying, by the processing unit, the video data to create modified video data at one or more modified frame rates that are different to the first frame rate in real time while receiving video data being captured by the camera;

change, by the at least one processing unit, a playing speed of the video data being captured in real time and being displayed on the graphical user interface from the first frame rate to the modified frame rates in response to the at least one input being received by the at least one processing unit.

33. The non-transitory computer readable medium according to claim 32, wherein the camera and the graphical user interface are incorporated in the mobile electronic device.

34. The non-transitory computer readable medium according to claim 33, wherein the at least one processing unit and the non-transitory computer readable medium are incorporated in the mobile electronic device, and wherein the processing unit is configured or configurable to create at least a portion of the modified video data.

35. The non-transitory computer readable medium according to claim 34, wherein the mobile electronic device is a mobile phone or tablet.

36. The non-transitory computer readable medium according to claim 33, wherein the at least one processing unit and the at least one memory are incorporated in a remote server in communication with the mobile electronic device, and wherein the at least one processing unit is configured or configurable to create at least a portion of the modified video data.

37. The non-transitory computer readable medium according to claim 36, wherein the modified video data is displayed on the graphical user interface at the modified playing speed while still recording the video data at the first frame rate in real time.

38. The non-transitory computer readable medium according to claim 37, wherein the at least one processing unit is further caused to determine if the at least one input from the graphical user interface is to change the first frame rate, then determine if the modified frame rate is less than the first frame rate, and if so, wherein the modifying of the video data includes adding at least one new frame to the video data to create the modified video data.

39. The non-transitory computer readable medium according to claim 38, wherein adding the new frame includes copying at least one frame to create the new frame or blending at least one frame with a next frame to create the new frame, and adding the new frame to the video data adjacent to the at least one frame.

40. The non-transitory computer readable medium according to claim 38, wherein the new frame is a plurality of new frames each being a copy of at least one frame from the video data.

41. The non-transitory computer readable medium according to claim 40, wherein each of the new frames is added to the video data adjacent to the at least one frame or adjacent to a second frame of the video data.

42. The non-transitory computer readable medium according to claim 36, wherein the at least one processing unit is further caused to determine if the at least one input from the graphical user interface is to change the first frame rate, then determine if the modified frame rate is greater than the first frame rate, and if so, wherein the modifying of the video data includes removing at least one first frame from the video data to create the modified video data.

43. The non-transitory computer readable medium according to claim 42, wherein the removing the first frame includes selecting the first frame to be removed, and then removing the first frame from the video data to create the modified video data.

44. The method according to claim 36, wherein the affordance is a touch affordance and the at least one input is activated while a touching and holding action is provided to the touch affordance, and wherein the modified video data is displayed in the video display region during the touching and holding action, and upon removal of the touching and holding action the first display region displays the video data at the first frame rate.

45. The method according to claim 36, wherein the affordance is a touch affordance and the at least one input is activated after a first touching action is provided to the touch affordance, and wherein the modified video data displayed in the video display region is changed to the video data at the first frame rate after a second touching action is provided to the touch affordance.

46. The method according to claim 36, wherein the affordance is a sliding affordance including multiple regions with each of the regions being associated with one modified frame rate of the one or more modified frame rates different from each other, and wherein the at least one input is based on a touching and sliding action along the sliding affordance to a first region of the multiple regions.

47. The method according to claim 46, wherein the modified video data is displayed in the video display region at the modified frame rate associated with the first region, and then changes to a second modified frame rate associated with a second region of the multiple regions when the touching and sliding action moves to the second region, and wherein upon removal of the touching and sliding action the first display region displays the video data at the first frame rate.