Method and system for E-commerce video editing

• US 20020094189A1
• Filed: 07/26/2001
• Published: 07/18/2002
• Est. Priority Date: 07/26/2000
• Status: Abandoned Application

• Current Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Zhang, Xiang, Navab, Nassir, Liou, Shih-Ping
• Application Number:
  US09/915,650
• Publication Number:
  US 20020094189A1
• Time in Patent Office: Days
• US Class Current: 348/207.99
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]

Abstract

A video editing system or tool for E-commerce utilizing augmented reality (AR) technology combines real and virtual worlds together to provide an interface for a user to sense and interact with virtual objects in the real world. The AR video editing system is usable in conjunction with an ordinary desktop computer and a low-cost parallel port camera. A known camera calibration algorithm is utilized together with a set of specially designed markers for camera calibration and pose estimation of the markers. OpenGL and VRML (Virtual Reality Modeling Language) for 3D virtual model rendering and superimposition. are utilized. Marker-based calibration is utilized to calibrate the camera and estimate the pose of the markers in the AR video editing system. The system comprises video input/output, image feature extraction and marker recognition, camera calibration/pose estimation, and virtual reality (VR) model rendering/augmentation. This allows a sales person to create and edit customized AR video for product presentation and advertisement. In the video, the sales person can present different aspects of the product while keeping eye-to-eye contact with customers. The system is capable of providing a user with real-time augmented reality feedback while recording a video. The augmented videos can be made available on E-Commerce Web-sites or they can be emailed to customers. Because of the real-time editing capability, the AR video can be directly broadcast on the Internet, for example, for an E-commerce advertisement. Inserted virtual objects can be hyper-linked to product specification WebPages providing more detailed product and price information.

Claims

1. A video editing system or tool for E-commerce, said system utilizing augmented reality (AR) technology for combining real and virtual worlds together to provide an interface for a user to sense and interact with virtual objects in the real world, said system comprising:

a programmable computer for performing data processing of video and calibration data;

a source of video data coupled to said computer;

a set of markers for calibration of said camera and for pose estimation of said markers, for providing calibration results;

a source of a 3-dimensional (3-D) image data model for a product;

said computer utilizing said 3-D image data and said calibration results for rendering a 3D model; and

said computer utilizing said 3D model and said video data for generating a 3-D model with superposition of said 3D model and said video data so as to provide an AR video.

2. A video editing system in accordance with claim 1, wherein said (3-D) image data model for a product comprises a VRML model.

3. A video editing system in accordance with claim 1, wherein said (3-D) image data model for a product comprises an OpenGL model.

4. A video editing system in accordance with claim 1, wherein said a source of video data is a video camera.

5. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides marker-based calibration to calibrate the camera and estimate the pose of the markers in the AR video editing system.

6. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides image feature extraction and marker recognition.

7. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides virtual reality (VR) model rendering/augmentation.

8. A video editing system in accordance with claim 1, wherein said computer performs video compression on said AR video.

9. A video editing system in accordance with claim 1, wherein said computer performs video compression on said AR video for converting said AR video to at least one of RealMedia and MPEG Movie format.

10. A video editing system in accordance with claim 1, wherein said computer adds inputted hyperlink information to said AR video after said converting said AR video, so as to produce a hyperlinked video message.

11. A video editing system in accordance with claim 10, wherein said computer data provides hyper-linking of said AR video to product specification WebPages.

13. A method for video editing as recited in claim 12, comprising the steps of:

setting hyperlink information;

compressing said AR video so as to produce a compressed AR video;

adding said hyperlink information to said compressed AR video so as to produce an ArEcVideo message with hyperlinks.

14. A method for video editing as recited in claim 13, wherein said step of setting hyperlink information comprises setting hyperlink information for hyperlinks providing product information associated with said object.

15. A method for video editing as recited in claim 12, comprising the step of:

sending said ArEcVideo message with hyperlinks on the Web.

17. A system for video editing as recited in claim 16, comprising:

means for setting hyperlink information;

means for compressing said AR video so as to produce a compressed AR video; and

means for adding said hyperlink information to said compressed AR video so as to produce an ArEcVideo message with hyperlinks.

18. A system for video editing as recited in claim 17, wherein said means for setting hyperlink information comprises means for setting hyperlink information for hyperlinks providing product information associated with said object.

19. A system for video editing as recited in claim 18, comprising:

means for sending said ArEcVideo message with hyperlinks on the Web.

20. A system for video editing as recited in claim 16, wherein said means for obtaining video image data from a source comprises a video camera.

21. A system for video editing as recited in claim 16, wherein said means for obtaining video image data from a source comprises a source of a stored video image.

23. A video editing system in accordance with claim 1, wherein said (3-D) image data model for a product comprises a VRML model.

24. A video editing system in accordance with claim 1, wherein said (3-D) image data model for a product comprises an OpenGL model.

25. A video editing system in accordance with claim 1, wherein said a source of video data is a video camera.

26. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides marker-based calibration to calibrate the camera and estimate the pose of the markers in the AR video editing system.

27. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides image feature extraction and marker recognition.

28. A video editing system in accordance with claim 1, wherein said said computer utilizing said 3D model and said video data provides virtual reality (VR) model rendering/augmentation with real time editing capability.

29. A video editing system in accordance with claim 1, wherein said computer performs video compression on said AR video.

30. A video editing system in accordance with claim 1, wherein said computer performs video compression on said AR video for converting said AR video to at least one of RealMedia and MPEG Movie format.

31. A video editing system in accordance with claim 1, wherein said computer adds inputted hyperlink information to said AR video after said converting said AR video, so as to produce a hyperlinked video message.

32. A video editing system in accordance with claim 10, wherein said computer data provides hyper-linking of said AR video to product specification WebPages.

36. A video editing system in accordance with claim 1, wherein said source of video data comprises a source for associated sound data.

38. A video editing system in accordance with claim 16, wherein said source provides sound data and wherein said means for obtaining video image data comprises means for obtaining sound data from said source.

41. A method for video editing as recited in claim 12, said method being carried out in real-time using an ordinary desktop or laptop PC type of computer.

42. A method for video editing as recited in claim 12, to be carried out in real-time that enables a user to rehearse and get visual feed-back in real time.

43. A system for video editing as recited in claim 12, for producing said AR video in real time, said video being ready to be broadcast through a network in real time.

12. A method for video editing comprising the steps of:

obtaining video image data from a source;

extracting feature information data from said video image data;

extracting marking recognition data from said video image data;

utilizing said information data and said recognition data to derive calibration data and pose estimation data for said source;

deriving 3-dimensional (3-D) model data for an object;

utilizing said calibration data, said pose estimation data, said video image data, and said 3-dimensional (3-D) model data for an object to perform volume rendering (VR) and superposition to produce an artificial reality (AR) image.

16. A system for video editing comprising:

means for obtaining video image data from a source;

means for extracting feature information data from said video image data;

means for extracting marking recognition data from said video image data;

means for utilizing said information data and said recognition data to derive calibration data and pose estimation data for said source;

means for deriving 3-dimensional (3-D) model data for an object; and

means for utilizing said calibration data, said pose estimation data, said video image data, and said 3-dimensional (3-D) model data for an object to perform volume rendering (VR) and superposition to produce an artificial reality (AR) image.

22. A video editing system or tool for E-commerce, said system utilizing augmented reality (AR) technology for combining real and virtual worlds together to provide an interface for a user to sense and interact with virtual objects in the real world, said system comprising:

a programmable computer for performing data processing of video and calibration data in real time;

a source of video data coupled to said computer;

a set of markers for calibration of said camera and for pose estimation of said markers, for providing calibration results;

a source of a 3-dimensional (3-D) image data model for a product;

said computer utilizing said 3-D image data and said calibration results for rendering a 3D model; and

said computer utilizing said 3D model and said video data for generating a 3-D model with superposition of said 3D model and said video data so as to provide an AR video in real time relative to said video data.

39. A video editing system in accordance with claim 22, wherein said video data includes associated sound data.

33. A method for video editing comprising the steps of:

obtaining video image data from a source;

extracting feature information data from said video image data;

extracting marking recognition data from said video image data;

utilizing said information data and said recognition data to derive calibration data and pose estimation data for said source;

deriving 3-dimensional (3-D) model data for an object; and

utilizing said calibration data, said pose estimation data, said video image data, and said 3-dimensional (3-D) model data for an object to perform volume rendering (VR) and superposition to produce an artificial reality (AR) image.

34. A method for video editing in accordance with claim 33 wherein said step of obtaining video image data includes a step of obtaining accompanying sound data.

37. A video editing system in accordance with claim 36, wherein said source of associated sound data comprises a microphone.

40. A video editing system in accordance with claim 33, wherein said step of obtaining video image data comprises a step of obtaining associated sound data from said source.

35. A system for video editing comprising:

means for obtaining video image data, including accompanying sound data from a source;

means for extracting feature information data from said video image data;

means for extracting marking recognition data from said video image data;

means for utilizing said information data and said recognition data to derive calibration data and pose estimation data for said source;

means for deriving 3-dimensional (3-D) model data for an object; and

means for utilizing said calibration data, said pose estimation data, said video image data, and said 3-dimensional (3-D) model data for an object to perform volume rendering (VR) and superposition to produce an artificial reality (AR) image.

44. A method for video editing comprising the steps of:

obtaining video image data and associated synchronized sound data from a source;

extracting feature information data from said video image data;

extracting marking recognition data from said video image data;

utilizing said information data and said recognition data to derive calibration data and pose estimation data for said source;

deriving 3-dimensional (3-D) model data for an object; and

utilizing said calibration data, said pose estimation data, said video image data, and said 3-dimensional (3-D) model data for an object to perform volume rendering (VR) and superposition so as to produce an artificial reality (AR) image in real time.

45. A method for video editing as recited in claim 44 including a step of providing said associated synchronized sound data to accompany said AR image in real time.