SYSTEM AND METHOD FOR FLEXIBLE VIDEO CONSTRUCTION

• US 20170280065A1
• Filed: 06/12/2017
• Published: 09/28/2017
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: Telesofia Medical Ltd.
• Original Assignee: Telesofia Medical Ltd.
• Inventors: Cohen, Rami, Rotshtein, Tzvi, Ben Shitrit, Danny
• Granted Patent: US 9,912,979 B2
• Time in Patent Office: Days
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 70/00 ICT specially adapted for the handling or processing of medical references
• CPC Class Codes: G16H 70/60 relating to pathologies
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4343 Extraction or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/4825 using a list of items to be played back in a given order, e.g. playlists
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 5/91 : Television signal processing therefor
• CPC Class Codes: H04N 7/52 : Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

System and method for flexible video construction, particularly of a personalized video clip which provides instructions to a viewer with regard to health and wellness. An ordered list of video input files is chained together, to create a single output video file using a chosen container. Timestamp values are tracked, to ensure synchronization of multiple joined clips, optionally using adjustments of the audio channel or the video channel. A video construction server utilizes information from multiple sources, to construct the video clip.

Claims

1-47. -47. (canceled)

48. A method of video construction by a machine comprising at least a hardware processor, the method comprising:

(a) accepting (I) an ordered list of video input files intended for being chained together, and (II) a single output video file name;

(b) checking that all of said video input files match;

(c) creating an output file using a chosen container which matches the container of said video input files;

(d) opening each of said video input files;

(e) reading each packet from each video input file, without decoding or processing;

(f) if the packet is a video packet, then writing the packet as-is to the video stream of the output file;

(g) if the packet is an audio packet, then writing the packet as-is to the audio stream of the output file;

(h) tracking the DTS timestamp value and the PTS timestamp value of each audio packet and each video packet, to keep all streams synchronized after completion of processing of each video input file;

(i) ensuring continued synchronization of multiple joined clips by adjusting both the audio and the video channels to have the exact same duration, by performing one of;

(A) selecting the later timestamp from all the video input streams for re-synchronization, by performing either;

(I) adding a quiet audio delay, or (II) postponing the starting video frame for the next video input file in sequence;

(B) calculating an optimal time period based on the source media length, sampling rate, and codec type;

wherein the optimal time period results in a number of complete audio and video frames that start together and end together exactly within said optimal period; and

adjusting the clip so that the clip length conforms to an exact multiple of the optimal time period as above; and

removing additional frames from the clip either manually or automatically;

wherein the method is performed by said machine which comprises at least said hardware processor.

49. The method of claim 48, wherein the video clips are at least one of:

(a) informational video clips, to provide information to the viewer;

(b) instructional video clips, personalized according to at least one instructional requirement of the viewer, in which the viewer is instructed to perform at least one action.

50. The method of claim 49, comprising:

constructing the instructional or informational video clips for conveying information to a viewer;

instructing the viewer in health and wellness, regarding a proper administration of a medicament, through one or more automatically-constructed video clip.

51. The method of claim 48, comprising:

determining a common synchronization period T for (I) encoded video that is saved at F frames per second, and (II) encoded audio that is sampled at S samples per second wherein each audio frame has L samples,

wherein the common synchronization period is calculated by using the following equations,

T=V×

1/F=A×

L/S

by determining the minimum integer values of parameter A and parameter V such that the following condition holds true;

V×

S=A×

F×

L.

52. The method of claim 51, comprising:

based on said common synchronization period T, constraining the number of frames of the clip so that the audio and video are always perfectly synchronized.

53. The method of claim 52, comprising:

choosing combinations of codecs and frame-rates that result in short periods that can be applied to production media of any length.

54. The method of claim 51, comprising:

repeating steps (d) through (i) for each video input file until all said video input files are processed.

55. The method of claim 48, comprising:

constructing code to enable video clips to be constructed at a later stage, by performing;

building a repository upon a repository service, wherein a production environment is composed from a repository which is cloned from a local repository;

wherein repository cloning is triggered automatically when the contents of a specific branch is updated.

56. The method of claim 55, wherein each user works on his personal repository to create the code for a scene, wherein each scene is composed from at least:

text, parameters, shot description;

wherein each scene can depend on externally provided parameters, such that a scene can have multiple variations based on the provided input;

wherein each scene can optionally have different text content associated with each variation.

57. The method of claim 56, comprising:

as each user completes a feature upon which he works, merging said feature into a development branch in a centralized repository;

merging the content into a particular branch of the repository;

automatically synchronizing the added content to a development web server for testing;

upon completion of testing, merging the content into the production branch in the centralized repository;

issuing a trigger to synchronize the production repository to the production environment into a staging area;

when the content in the staging area is confirmed to be working properly, issuing an atomic single command to swap the new web service to the live environment, to prevent the problem partial updates while data is being synchronized to production.

58. The method of claim 48, comprising:

performing video scene production by creating a superscript that allows for different variations in the scene;

wherein the scene production comprises;

constructing a scene description, which contains;

(I) a name of the scene, and (II) parameters that the scene depends on, and (III) a text of the scene which contains multiple variations if the scene depends on external parameters, and (IV) director instructions corresponding to close-up and/or mid-shot and/or animation;

linking each scene to another scene via a link, wherein the link is either included in the scene description or is kept in a separate file.

59. The method of claim 58, wherein said linking is conditional linking;

wherein a source scene links to either a first target scene if a parameter has a first value, or to a second target scene if said parameter has a second value,

thereby creating a dynamic script that procures a different set of scenes based on the parameters it receives.

60. The method of claim 58, comprising:

adding parameters for each scene to the scene description;

wherein the parameters are provided to the script from an ordering user by populating a form and filling in said parameters, wherein the parameters are fed into the dynamic script and the dynamic script evaluates a linear sequential order of scenes that compose the end-video to be displayed for the user.

61. The method of claim 60, comprising:

compiling the description of the script into a native programming language;

before or during said compilation, inserting a section that produces calculated variables that are parameters which depend on parameters that the ordering user provides.

62. The method of claim 61, wherein the method automatically produces a first video variant if a user-gender parameter indicates that the user is a male, and the method automatically produces a second video variant if the user-gender parameter indicates that the user is a female.

63. The method of claim 48, comprising:

providing instructions about medication to be taken by a viewer of the video, by including an image of the medication and a verbal description of the medication;

wherein the image is selected from a database of photographs;

wherein the image is sourced from an official medical source or from a manufacturer of the medication;

wherein, if the image data is structured, then image data is analyzed and converted into a format for a database that can be accessed by a video construction server;

wherein, if the image data is provided in free text and is meant for human consumption, then relevant information is extracted by using text processing techniques and/or Natural Language Processing (NLP), and the information is then interpreted and loaded into the database;

wherein extracted information from all the sources is loaded into the database, and the contents from each source are merged to make the entire data set available;

utilizing the information to automatically produce rich media content, such that the video clip includes up-to-date information;

wherein differences between older and newer data sets are determined to apply customizations or improvements to the produced videos, wherein the differences are used to evaluate required changes to videos.

64. The method of claim 48, comprising:

providing instructions about medication to be taken by a viewer of the video, by including an image of the medication and a verbal description of the medication;

wherein the method comprises producing either two-dimensional or three-dimensional image of the medication, to enable the viewer to validate that the medication he is taking is the medication to which reference is made in the video;

wherein the method comprises;

(a) reading from a database a set of pill properties, wherein the pill properties are selected from the group consisting of;

shape, color, size, score, and inscription;

(b) using said properties to define and render an image of the pill, wherein the produced image of the pill matches said properties, wherein the image is created without an actual image of the pill through an approximation so that the viewer can confirm that he has the proper pill;

(c) integrating the image to produce a video describing the actual pill, by rendering the image into a movie clip along with a sound track describing the pill vocally;

wherein the movie clip is produced by taking the image and repeating it to make a movie while adding another channel providing the audio describing the pill.

65. A non-transitory storage medium having stored thereon instructions that, when executed by a hardware processor, cause the hardware processor to perform a method comprising:

(a) accepting (I) an ordered list of video input files intended for being chained together, and (II) a single output video file name;

(b) checking that all of said video input files match each other by having same encoding, same format, and same codec;

(c) creating an output file using a chosen container which matches, by its encoding and format and codec, the container of said video input files;

(d) sequentially opening each of said video input files;

(e) reading each packet from each video input file, without decoding or processing;

(f) if the packet is a video packet, then writing the packet as-is to the video stream of the output file;

(g) if the packet is an audio packet, then writing the packet as-is to the audio stream of the output file;

(h) tracking the DTS timestamp value and the PTS timestamp value of each audio packet and each video packet, to keep all streams synchronized after completion of processing of each video input file;

(i) ensuring continued synchronization of multiple joined video clips by adjusting both the audio and the video channels to have the exact same duration, by performing one of;

(A) selecting the later timestamp from all the video input streams for re-synchronization, by performing either;

(I) adding a quiet audio delay, or (II) postponing the starting video frame for the next video input file in sequence;

(B) calculating an optimal time period based on the source media length, sampling rate, and codec type;

wherein the optimal time period corresponds to a number of complete audio and video frames that start together and end together exactly within said optimal period; and

adjusting the clip so that the clip length conforms to an exact multiple of the optimal time period as above; and

removing additional frames from the clip either manually or automatically;

wherein step (i) is performed by determining a common synchronization period T for (I) encoded video that is saved at F frames per second, and (II) encoded audio that is sampled at S samples per second wherein each audio frame has L samples,

wherein the common synchronization period is calculated by using the following equations, T=V×

1/F=A×

L/S

by determining the minimum integer values of parameter A and parameter V such that the following condition holds true;

V×

S=A×

F×

L.