Video event capture, storage and processing method and apparatus

• US 20030009773A1
• Filed: 06/24/2002
• Published: 01/09/2003
• Est. Priority Date: 06/25/2001
• Status: Active Grant

• Current Assignee: Nytell Software LLC (Intellectual Ventures LLC)
• Original Assignee: Aprolase Development Co LLC (Intellectual Ventures LLC)
• Inventors: Carlson, Randolph S.
• Granted Patent: US 8,934,627 B2
• Time in Patent Office: Days
• US Class Current: 725/142
• CPC Class Codes: G06F 21/62 Protecting access to data via a platform, e.g. using keys or access control rules
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00132 in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
• CPC Class Codes: H04N 1/00137 Transmission
• CPC Class Codes: H04N 1/00161 Viewing or previewing
• CPC Class Codes: H04N 1/00167 Processing or editing H04N1/00196 - H04N1/00201 take precedence
• CPC Class Codes: H04N 1/00188 Printing, e.g. prints or reprints H04N1/0019, H04N1/00196 take precedence
• CPC Class Codes: H04N 1/00291 with receiver circuitry television receiver circuitry per se H04N5/44
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/478 Supplemental services, e.g. displaying phone caller identification, shopping application
• CPC Class Codes: H04N 21/63345 by transmitting keys key distribution for secret or secure communication H04L9/08; network support of key management H04L63/06
• CPC Class Codes: H04N 21/63775 for uploading keys, e.g. for a client to communicate its public key to the server key management H04L9/08; network support of key management H04L63/06
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 2201/3242 of processing required or performed, e.g. for reproduction or before recording H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3274 : Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 5/4448 : for frame-grabbing

Abstract

Generating a still image from a sequence of video frames comprises the steps of receiving a sequence of video frames and storing a predetermined number of the sequence of video frames temporally adjacent to an user initiated capture event, the predetermined number of frames being defined as a cliplet. Video capture is performed using cliplets, which are short video segments, preferably in compressed data format. A crop-and-process instruction is added to the cliplet. All processing, handling or storage of the data is cliplet based. Editing instructions are stored with the cliplet allowing editing to be performed by a user at one resolution and then re-performed at a higher resolution at a server where it is printed. Modified reprints of an image can be obtained without resending the images and adequate storage for the cliplets can be assured. Encryption of the cliplets during transmission and storage is facilitated by cliplet-based processing.

Claims

1. An improvement to a video viewer for use in combination with a display device comprising:

a computer for receiving a sequence of video frames; and

a buffer for storing a temporal sequence of video frames corresponding to what is seen in the display device, the computer copying a predetermined number of frames temporally proximate to a capture event from the buffer to a data structure, the predetermined number of frames temporally proximate to the capture event being defined as a cliplet.

2. The improvement of claim 1 where the computer stores a cliplet in the buffer which cliplet comprises at least one tenth of a second of display time at normal frame rates.

3. The improvement of claim 1 where computer stores an instruction in the buffer associated with the cliplet relating to a selected frame within the cliplet.

4. The improvement of claim 3 where the instruction stored by the computer in the buffer associated with the cliplet further relates to a selected portion of the selected frame within the cliplet.

5. The improvement of claim 4 where the instruction stored by the computer in the buffer associated with the cliplet further relating to a selected portion of the selected frame within the cliplet includes setting information.

6. The improvement of claim 5 where the setting information relates to parameters such as crop dimensions, a crop location, an up-sample ratio, a sharpness parameter, a persistence parameter, and the number of frames to be processed to form a still image, or at least one of such parameters.

7. The improvement of claim 1 where the cliplet is communicated to the computer in a compressed data format and where the computer stores the cliplet in the buffer in the compressed data format.

8. The improvement of claim 1 where the computer processes all video frames in the form of cliplets.

9. The improvement of claim 8 where the computer performs the processes of transporting, storing, archiving and retrieving cliplets or at least one of such processes.

10. The improvement of claim 4 where the instruction stored by the computer in the buffer associated with the cliplet corresponds to a default instruction, and further includes at least one revised instruction relating to the selected frame.

11. The improvement of claim 1 where the computer is a first computer which processes the cliplet at a first resolution, and further comprising a second computer communicated to the first computer, the second computer processing the cliplet at a second higher resolution according to processing instructions determined at the first computer to generate a first still image, the cliplet being communicated between the first and second computers together with the processing instructions.

12. The improvement of claim 11 where the first computer is communicated with the second computer to transport a second set of processing instructions and a reference to the corresponding cliplet to the second computer to generate a second still image.

13. The improvement of claim 1 where the computer generates a blank cliplet in the buffer to reserve data space therein.

14. The improvement of claim 13 wherein the computer is a client computer and wherein a blank cliplet is uniquely identified to the client computer and further comprising a server computer communicated to the client computer, and wherein the server computer recognizes the cliplet as uniquely identified with the client computer when the cliplet is accessed by the server computer.

15. The improvement of claim 14 wherein the cliplet is encrypted in the buffer and communicated to the server computer in encrypted form.

16. The improvement of claim 15 where the client computer stores the blank cliplet as pseudorandom data and later when the cliplet is overwritten by the client computer onto the blank cliplet, it is combined with the pseudorandom data whereby encryption is realized without slowing capture of the cliplet.

17. The improvement of claim 15 where the client and server computers exchange an encryption key at the time the blank cliplet is stored into the buffer thereby avoiding exchange of the encryption key after the blank cliplet is overwritten and later uploaded into the server computer.

18. The improvement of claim 17 where the encryption key is exchanged between the client and server computers at a different communication session from when the server computer uploads the encrypted cliplet from the client computer.

20. The improvement of claim 19 where storing a predetermined number of the sequence of video frames comprises storing the cliplet with at least one tenth of a second of display time at normal frame rates.

21. The improvement of claim 19 storing a predetermined number of the sequence of video frames comprises storing an instruction associated with the cliplet relating to a selected frame within the cliplet.

22. The improvement of claim 21 where storing an instruction associated with the cliplet comprises storing data relating to a selected portion of the selected frame within the cliplet.

23. The improvement of claim 22 where storing data relating to a selected portion of the selected frame within the cliplet comprises storing setting information.

24. The improvement of claim 23 where storing setting information comprises storing parameters such as crop dimensions, a crop location, an up-sample ratio, a sharpness parameter, a persistence parameter, and the number of frames to be processed to form a still image, or at least one of such parameters.

25. The improvement of claim 19 where storing a predetermined number of the sequence of video frames comprises storing the cliplet in the compressed data format.

26. The improvement of claim 19 further comprising processing all video frames in the form of cliplets.

27. The improvement of claim 26 where processing all video frames in the form of cliplets comprises transporting, storing, archiving and retrieving video frames or at least one of such processes.

28. The improvement of claim 22 where storing an instruction associated with the cliplet stores a default instruction and at least one revised instruction relating to the selected frame.

29. The improvement of claim 19 further comprising processing the cliplet at a first resolution according to processing instructions determined at the first computer, communicating the cliplet and processing instructions together from a first to a second computer, processing the cliplet at a second higher resolution according to the processing instructions determined at the first computer to generate a first still image.

30. The improvement of claim 29 further comprising communicating a second set of processing instructions between the first and second computer and a reference to the corresponding cliplet to the second computer and generating a second still image.

31. The improvement of claim 19 further comprising generating a blank cliplet to reserve data space in a buffer.

32. The improvement of claim 31 further comprising uniquely identifying the blank cliplet to the client computer and recognizing in a server computer the cliplet as uniquely identified with the client computer when the cliplet is accessed by the server computer.

33. The improvement of claim 32 further comprising encrypting the cliplet and communicating it to the server computer in encrypted form.

34. The improvement of claim 33 further comprising storing pseudorandom data in the blank cliplet, later overwriting the blank cliplet by input captured data with the pseudorandom data so that encryption is realized without slowing capture of the cliplet.

35. The improvement of claim 33 further comprising exchanging an encryption key between the client and server computers at the time the blank cliplet is stored into the buffer thereby avoiding exchange of the encryption key after the blank cliplet is overwritten and later uploaded into the server computer.

36. The improvement of claim 35 further comprising exchanging the encryption key between the client and server computers during a different communication session from when the server computer uploads the encrypted cliplet from the client computer.

38. The method of claim 37 where providing at least one instruction associated with the cliplet provides an instruction which also includes spatial processing of at least one video image of the cliplet to render a still image corresponding to the captured image.

39. The method of claim 37 further comprising storing, archiving, transporting, processing, or communicating the combined data structure of the cliplet and its associated instruction as a single unit of information in a computer system.

19. An improvement in a method generating a still image from a sequence of video frames comprising:

receiving a sequence of video frames; and

storing a temporal sequence of video frames; and

copying a predetermined number of frames temporally proximate to a capture event to a data structure, the predetermined number of frames temporally proximate to the capture event being defined as a cliplet.

37. A method for representing an event as a still image comprising:

providing a temporally proximate sequence of video images, defined as a cliplet;

selecting one of the temporally proximate sequence of video images as a captured image; and

providing at least one instruction associated with the cliplet to form a combined data structure, which instruction defines temporal processing of more than one video image of the cliplet to render a still image corresponding to the captured image.