System and method for flexible video construction

• US 9,912,979 B2
• Filed: 06/12/2017
• Issued: 03/06/2018
• Est. Priority Date: 01/24/2013
• Status: Active Grant

• Current Assignee: Telesofia Medical Ltd.
• Original Assignee: Telesofia Medical Ltd.
• Inventors: Cohen, Rami, Rotshtein, Tzvi, Ben Shitrit, Danny
• Primary Examiner(s): Chin, Ricky
• Application Number:
  US15/619,698
• Publication Number:
  US 20170280065A1
• Time in Patent Office: 267 Days
• Field of Search: None
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G16H 20/10 relating to drugs or medications, e.g. for ensuring correct administration to patients
• CPC Class Codes: G16H 20/60 relating to nutrition control, e.g. diets
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 70/00 ICT specially adapted for the handling or processing of medical references
• CPC Class Codes: G16H 70/60 relating to pathologies
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2335 involving reformatting operations of audio signals, e.g. by converting from one coding standard to another details of audio signal transcoding G10L19/173
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/4302 Content synchronisation processes, e.g. decoder synchronisation
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4343 Extraction or processing of packetized elementary streams [PES]
• CPC Class Codes: H04N 21/4825 using a list of items to be played back in a given order, e.g. playlists
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 5/91 : Television signal processing therefor
• CPC Class Codes: H04N 7/52 : Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

System and method for flexible video construction, particularly of a personalized video clip which provides instructions to a viewer with regard to health and wellness. An ordered list of video input files is chained together, to create a single output video file using a chosen container. Timestamp values are tracked, to ensure synchronization of multiple joined clips, optionally using adjustments of the audio channel or the video channel. A video construction server utilizes information from multiple sources, to construct the video clip.

Claims

1. A method of video construction by a machine comprising at least a hardware processor, the method comprising:

(a) accepting (I) an ordered list of video input files intended for being chained together, and (II) a single output video file name;

(b) checking that all of said video input files match each other by having same encoding, same format, and same codec;

(c) creating an output file using a chosen container which matches, by its encoding and format and codec, a container of said video input files;

(d) sequentially opening each of said video input files;

(e) reading each packet from each video input file, without packet decoding, wherein each video input file comprises video packets;

(f) if the packet is a video packet, then writing the packet as-is to a video stream of the output file;

(g) if the packet is an audio packet, then writing the packet as-is to an audio stream of the output file;

(h) tracking the DTS timestamp value and the PTS timestamp value of each audio packet and each video packet, to keep all streams synchronized after completion of processing of each video input file;

(i) ensuring continued synchronization of multiple joined video input files by adjusting both the audio and the video to have the exact same duration, by performing one of;

(A) selecting the later timestamp from all the video input streams for resynchronization, by performing either;

(I) adding a quiet audio delay, or (II) postponing the starting video frame for the next video input file;

(B) calculating an optimal time period based on lengths of the video input files and their sampling rates and codec types;

wherein the optimal time period results in a number of complete audio and video frames that start together and end together exactly within said optimal period; and

adjusting each video input file so that its length conforms to an exact multiple of the optimal time period as above; and

removing additional frames from the video input files either manually or automatically;

wherein step (i) is performed by determining a common synchronization period T for (I) encoded video that is saved at F frames per second, and (II) encoded audio that is sampled at S samples per second, wherein each audio frame has L samples,

wherein the common synchronization period is calculated by using the following equations,

T=V=1/F=A×

L/S

by determining the minimum integer values of parameter A and parameter V such that the following condition holds true;

V×

S=A×

F×

L;

wherein the method is performed by said machine which comprises at least said hardware processor.

2. The method of claim 1, wherein the video input files are chained into said output file which is one of:

(a) informational video, to provide information to a viewer;

(b) instructional video, personalized according to at least one instructional requirement of the viewer, in which the viewer is instructed to perform at least one action.

3. The method of claim 2, comprising:

constructing the instructional or informational video for conveying information to the viewer;

instructing the viewer in health and wellness, regarding a proper administration of a medicament, through one or more automatically-constructed videos.

4. The method of claim 1, comprising:

based on said common synchronization period T, constraining the number of frames of the output file so that the audio and video are always perfectly synchronized.

5. The method of claim 1, comprising:

choosing combinations of codecs and frame-rates that result in short periods that can be applied to production media of any length.

6. The method of claim 1, comprising:

repeating steps (d) through (i) for each video input file until all said video input files are processed.

7. The method of claim 1, comprising:

constructing code to enable video output files to be constructed at a later stage, by performing;

building a repository upon a repository service, wherein a production environment is composed from a repository which is cloned from a local repository;

wherein repository cloning is triggered automatically when the contents of a specific branch is updated.

8. The method of claim 7, wherein each user works on his personal repository to create the code for a scene, wherein each scene is composed from at least:

text, parameters, shot description;

wherein each scene can depend on externally provided parameters, such that a scene can have multiple variations based on the provided input;

wherein each scene can optionally have different text content associated with each variation.

9. The method of claim 8,

wherein each user of a plurality of users works on his own personal repository,

wherein the method comprises:

as each user of said plurality of users completes a feature upon which he works, merging said feature into a development branch in a centralized repository;

merging the content into a particular branch of the centralized repository;

automatically synchronizing added content to a development web server for testing;

upon completion of testing, merging the content into a production branch in the centralized repository;

issuing a trigger to synchronize the production branch of the centralized repository into a staging area;

when the content in the staging area is confirmed to be working properly, issuing an atomic single command to swap a new web service to a live environment, to prevent partial updates while data is being synchronized to production.

10. The method of claim 1, comprising:

performing video scene production by creating a superscript that allows for different variations in the video scene;

wherein the video scene production comprises;

constructing a scene description, which contains;

(I) a name of the scene, and (II) parameters that the scene depends on, and (III) a text of the scene which contains multiple variations if the scene depends on external parameters, and (IV) director instructions corresponding to close-up and/or mid-shot and/or animation;

linking said scene to another scene via a link, wherein the link is either included in the scene description or is kept in a separate file.

11. The method of claim 10, wherein said linking is conditional linking;

wherein a source scene links to either a first target scene if a parameter has a first value, or to a second target scene if said parameter has a second value,

thereby creating a dynamic script that procures a different set of scenes based on parameters that it receives.

12. The method of claim 10, comprising:

adding parameters for said scene to said scene description;

wherein the parameters are provided to a dynamic script from an ordering user by populating a form and filling in said parameters, wherein the parameters are fed into the dynamic script and the dynamic script evaluates a linear sequential order of scenes that compose an end-video to be displayed for a user.

13. The method of claim 12, comprising:

compiling a description of the dynamic script into a native programming language;

before or during said compiling, inserting a section that produces calculated variables that are parameters which depend on parameters that an ordering user provides.

14. The method of claim 13, wherein the method automatically produces a first video variant if a user-gender parameter indicates that the user is a male, and the method automatically produces a second video variant if the user-gender parameter indicates that the user is a female.

15. The method of claim 1, comprising:

providing instructions about medication to be taken by a viewer, by including in a video an image of the medication and a verbal description of the medication;

wherein the image is selected from a database of photographs;

wherein the image is sourced from an official medical source or from a manufacturer of the medication;

wherein, if the image data is structured, then image data is analyzed and converted into a format for a database that can be accessed by a video construction server;

wherein, if the image data is provided in free text and is meant for human consumption, then relevant information is extracted by using text processing techniques and/or Natural Language Processing (NLP), and the information is then interpreted and loaded into the database;

wherein extracted information from all sources is loaded into the database, and contents from all sources are merged to make an entire data set available;

utilizing the information to automatically produce rich media content, such that the output file includes up-to-date information;

wherein differences between older and newer data sets are determined to apply customizations or improvements to produced videos, wherein the differences are used to evaluate required changes to videos.

16. The method of claim 1, comprising:

providing instructions about medication to be taken by a viewer, by including in a video an image of the medication and a verbal description of the medication;

wherein the method comprises producing either two-dimensional or three-dimensional image of the medication, to enable the viewer to validate that a medication he is taking is the medication to which reference is made in the video;

wherein the method comprises;

(a) reading from a database a set of pill properties, wherein the pill properties are selected from the group consisting of;

shape, color, size, score, and inscription;

(b) using said pill properties to define and render an image of a pill, wherein a produced image of the pill matches said pill properties, wherein the image is created without an actual image of the pill through an approximation so that the viewer can confirm that he has the proper pill;

(c) integrating the produced image of the pill to produce a video describing the actual pill, by rendering the produced image of the pill into a movie clip along with a soundtrack vocally describing the pill;

wherein the movie clip is produced by taking the produced image of the pill and repeating it to produce said movie clip while adding a channel providing audio describing the pill.

17. A non-transitory storage medium having stored thereon instructions that, when executed by a hardware processor, cause the hardware processor to perform a method comprising:

(a) accepting (I) an ordered list of video input files intended for being chained together, and (II) a single output video file name;

(b) checking that all of said video input files match each other by having same encoding, same format, and same codec;

(c) creating an output file using a chosen container which matches, by its encoding and format and codec, a container of said video input files;

(d) sequentially opening each of said video input files;

(e) reading each packet from each video input file, without packet decoding, wherein each video input file comprises video packets;

(f) if the packet is a video packet, then writing the packet as-is to a video stream of the output file;

(g) if the packet is an audio packet, then writing the packet as-is to an audio stream of the output file;

(h) tracking the DTS timestamp value and the PTS timestamp value of each audio packet and each video packet, to keep all streams synchronized after completion of processing of each video input file;

(i) ensuring continued synchronization of multiple joined video input files by adjusting both the audio and the video to have the exact same duration, by performing one of;

(A) selecting the later timestamp from all the video input streams for resynchronization, by performing either;

(I) adding a quiet audio delay, or (II) postponing the starting video frame for the next video input file in sequence;

(B) calculating an optimal time period based on lengths of the video input files and their sampling rates and codec types;

wherein the optimal time period corresponds to a number of complete audio and video frames that start together and end together exactly within said optimal period; and

adjusting each video input file so that its length conforms to an exact multiple of the optimal time period as above; and

removing additional frames from the video input files either manually or automatically;

wherein step (i) is performed by determining a common synchronization period T for (I) encoded video that is saved at F frames per second, and (II) encoded audio that is sampled at S samples per second, wherein each audio frame has L samples,

wherein the common synchronization period is calculated by using the following equations,

T=V×

1/F=A×

L/S

by determining the minimum integer values of parameter A and parameter V such that the following condition holds true;

V×

S=A×

F×

L.