Picture presentation method and apparatus

• US 10,410,397 B2
• Filed: 12/28/2015
• Issued: 09/10/2019
• Est. Priority Date: 03/31/2015
• Status: Active Grant

• Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Original Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Inventors: Ai, Rui, Du, Tangwu, Jiang, Zhaoyan, Liu, Li, Lang, Xianpeng
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US15/301,356
• Publication Number:
  US 20170186212A1
• Time in Patent Office: 1,352 Days
• Field of Search: None
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

The present application discloses a picture presentation method and apparatus. A specific implementation of the method includes: acquiring an original picture set encompassing a same scene; selecting, from the original picture set, a reconstruction picture set for reconstructing a three-dimensional structure of the scene, and reconstructing the three-dimensional structure of the scene by using reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three-dimensional feature points; selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold; determining a presentation order of to-be-presented pictures in the to-be-presented picture set, and generating a picture presentation sequence based on the presentation order of the to-be-presented pictures; and presenting pictures in the picture presentation sequence continuously. By acquiring pictures encompassing a same scene, determining a presentation order of the pictures based on the association relationship between the pictures, and continuously presenting the pictures according to the presentation order, this implementation improves the spatial continuity of the scene in the pictures when the pictures are presented.

Claims

1. A picture presentation method, comprising:

acquiring an original picture set encompassing a same scene;

selecting, from the original picture set, a reconstruction picture set for reconstructing a three-dimensional structure of the scene, and reconstructing the three-dimensional structure of the scene by using reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three-dimensional feature points;

selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold;

determining a presentation order of to-be-presented pictures in the to-be-presented picture set, and generating a picture presentation sequence based on the presentation order of the to-be-presented pictures; and

presenting pictures in the picture presentation sequence continuously,

wherein the selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold, comprises;

selecting a reference to-be-presented picture from the reconstruction picture set, a number of three-dimensional feature points included in the reference to-be-presented picture being greater than a number of three-dimensional feature points included in any other reconstruction picture in the reconstruction picture set; and

selecting a plurality of to-be-subsequently-presented pictures satisfying a first preset condition in sequence from the reconstruction picture set by using the reference to-be-presented picture, the first preset condition comprising;

a number of different three-dimensional feature points included in the to-be-subsequently-presented picture and a to-be-subsequently-presented picture previously selected is greater than a number of different three-dimensional feature points included in a reconstruction picture that is not selected as a to-be-subsequently-presented picture and the to-be-subsequently-presented picture previously selected.

2. The method according to claim 1, wherein the selecting, from the original picture set, the reconstruction picture set for reconstructing the three-dimensional structure of the scene comprises:

extracting scale-invariant feature transform (SIFT) keypoints in original pictures in the original picture set; and

selecting, from the original picture set, the reconstruction picture set for reconstructing the three-dimensional structure of the scene based on a matching relationship between SIFT keypoints in the original pictures.

3. The method according to claim 2, wherein the reconstructing the three-dimensional structure of the scene by using the reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three- dimensional feature points, comprises:

restoring a camera parameter based on SIFT keypoints in the reconstruction pictures in the reconstruction picture set, the camera parameter comprising an intrinsic camera parameter and an extrinsic camera parameter, and the intrinsic camera parameter comprising a focal length and a principal point offset; and

reconstructing the three-dimensional structure of the scene based on the SIFT keypoints in the reconstruction pictures and the camera parameter.

4. The method according to claim 1, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

determining the presentation order of the to-be-presented pictures based on an association relationship between attribute parameters of the to-be-presented pictures, the attribute parameter of the to-be-presented picture comprising a shooting position parameter and a shooting angle parameter; and

generating the picture presentation sequence based on the presentation order of the to-be-presented pictures.

5. The method according to claim 1, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

determining the presentation order of the to-be-presented pictures based on a rendering cost between the to-be-presented pictures, the rendering cost indicating a cost for inserting a transition picture between the to-be-presented pictures; and

inserting a transition picture between every two neighboring to-be-presented pictures based on the presentation order of the to-be-presented pictures so as to generate the picture presentation sequence.

6. The method according to claim 5, wherein the method further comprises, before the determining the presentation order of the to-be-presented pictures based on the rendering cost between the to-be-presented pictures:

calculating, based on rendering-related parameters of the to-be-presented pictures, a rendering sub-cost corresponding to each of the rendering-related parameters, the rendering-related parameter comprising at least one of;

an amount-of-twist parameter, a shooting position parameter, a shooting angle parameter, a resolution parameter, and an optical flow parameter; and

determining the rendering cost between the to-be-presented pictures based on the rendering sub-cost corresponding to each of the rendering-related parameters.

7. The method according to claim 5, wherein the method further comprises, before the determining the presentation order of the to-be-presented pictures based on the rendering cost between the to-be-presented pictures:

calculating, based on rendering-related parameters of the to-be-presented pictures, a rendering sub-cost corresponding to each of the rendering-related parameters, the rendering-related parameter comprising at least one of;

an amount-of-twist parameter, a shooting position parameter, a shooting angle parameter, a resolution parameter, and an optical flow parameter; and

determining the rendering cost between the to-be-presented pictures based on the rendering sub-cost corresponding to each of the rendering-related parameters.

8. The method according to claim 1, wherein the acquiring the original picture set encompassing the same scene comprises:

receiving a scene selection instruction, the scene selection instruction comprising geographical location information corresponding to the scene; and

acquiring, based on the geographical location information, the original picture set corresponding to the geographical location information.

9. The method according to claim 8, wherein the method further comprises, after the acquiring the original picture set encompassing the same scene:

performing a preprocessing operation on the original picture set,

the preprocessing operation comprising at least any one of;

filtering out, from the original picture set, original picture having a image quality lower than a preset value; and

correcting shooting positions of the original pictures in the original picture set to a normal shooting attitude.

10. A picture presentation apparatus, comprising:

at least one processor; and

a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising;

acquiring an original picture set encompassing a same scene;

selecting, from the original picture set, a reconstruction picture set for reconstructing a three-dimensional structure of the scene, and reconstructing the three-dimensional structure of the scene by using reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three-dimensional feature points;

selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold;

determining a presentation order of to-be-presented pictures in the to-be-presented picture set, and generating a picture presentation sequence based on the presentation order of the to-be-presented pictures; and

presenting pictures in the picture presentation sequence continuously,

wherein the selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold, comprises;

selecting a reference to-be-presented picture from the reconstruction picture set, a number of three-dimensional feature points included in the reference to-be-presented picture being greater than a number of three-dimensional feature points included in any other reconstruction picture in the reconstruction picture set; and

selecting a plurality of to-be-subsequently-presented pictures satisfying a first preset condition in sequence from the reconstruction picture set by using the reference to-be- presented picture, the first preset condition comprising;

a number of different three- dimensional feature points included in the to-be-subsequently-presented picture and a to- be-subsequently-presented picture previously selected is greater than a number of different three-dimensional feature points included in a reconstruction picture that is not selected as a to-be-subsequently-presented picture and the to-be-subsequently-presented picture previously selected.

11. The apparatus according to claim 10, wherein the selecting, from the original picture set, the reconstruction picture set for reconstructing the three-dimensional structure of the scene comprises:

extracting scale- invariant feature transform (SIFT) keypoints in original pictures in the original picture set; and

selecting, from the original picture set, the reconstruction picture set for reconstructing the three- dimensional structure of the scene based on a matching relationship between SIFT keypoints in the original pictures.

12. The apparatus according to claim 11, wherein the reconstructing the three-dimensional structure of the scene by using the reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three- dimensional feature points, further comprises:

restoring a camera parameter based on SIFT keypoints in the reconstruction pictures in the reconstruction picture set, the camera parameter comprising an intrinsic camera parameter and an extrinsic camera parameter, and the intrinsic camera parameter comprising a focal length and a principal point offset; and

reconstructing the three-dimensional structure of the scene based on the SIFT keypoints in the reconstruction pictures and the camera parameter.

13. The apparatus according to claim 10, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

determining the presentation order of the to-be-presented pictures based on an association relationship between attribute parameters of the to-be-presented pictures, the attribute parameter of the to-be-presented picture comprising a shooting position parameter and a shooting angle parameter; and

generating the picture presentation sequence based on the presentation order of the to-be-presented pictures.

14. The apparatus according to claim 10, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

determining the presentation order of the to-be-presented pictures based on a rendering cost between the to-be-presented pictures, the rendering cost indicating a cost for inserting a transition picture between the to-be-presented pictures; and

inserting a transition picture between every two neighboring to-be-presented pictures based on the presentation order of the to-be- presented pictures so as to generate the picture presentation sequence.

15. The apparatus according to claim 14, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

calculating, based on rendering-related parameters of the to-be-presented pictures, a rendering sub-cost corresponding to each of the rendering- related parameters, the rendering-related parameters comprising at least one of;

an amount-of-twist parameter, a shooting position parameter, a shooting angle parameter, a resolution parameter, and an optical flow parameter; and

determining the rendering costs between the to-be- presented pictures based on the rendering sub-cost corresponding to each of the rendering-related parameters.

16. The apparatus according to claim 15, wherein the determining the presentation order of the to-be-presented pictures in the to-be-presented picture set, and generating the picture presentation sequence based on the presentation order of the to-be-presented pictures comprises:

inserting a virtual camera at at least one position between shooting positions of every two neighboring to-be-presented pictures based on the presentation order of the to-be-presented pictures, and performing an interpolation operation on the intrinsic camera parameter to obtain an intrinsic parameter of the virtual camera, the virtual camera comprising a virtual-camera imaging plane; and

mapping, based on the intrinsic parameter of the virtual camera, three-dimensional feature points included in every two neighboring to-be-presented pictures to the virtual-camera imaging plane for inserting the transition picture, and generating the picture presentation sequence.

17. A non-transitory computer storage medium, storing computer readable instructions executable by a processor, wherein when the computer readable instructions are executed by a processor, the processor executes a picture presentation method, the picture presentation method comprising:

acquiring an original picture set encompassing a same scene;

selecting, from the original picture set, a reconstruction picture set for reconstructing a three-dimensional structure of the scene, and reconstructing the three-dimensional structure of the scene by using reconstruction pictures in the reconstruction picture set, the three-dimensional structure of the scene comprising three-dimensional feature points;

selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold;

determining a presentation order of to-be-presented pictures in the to-be-presented picture set, and generating a picture presentation sequence based on the presentation order of the to-be-presented pictures; and

presenting pictures in the picture presentation sequence continuously,

wherein the selecting a to-be-presented picture set from the reconstruction picture set, a number of the three-dimensional feature points included in the to-be-presented picture set being above a preset threshold, comprises;

selecting a reference to-be-presented picture from the reconstruction picture set, a number of three-dimensional feature points included in the reference to-be-presented picture being greater than a number of three-dimensional feature points included in any other reconstruction picture in the reconstruction picture set; and

selecting a plurality of to-be-subsequently-presented pictures satisfying a first preset condition in sequence from the reconstruction picture set by using the reference to-be- presented picture, the first preset condition comprising;

a number of different three- dimensional feature points included in the to-be-subsequently-presented picture and a to- be-subsequently-presented picture previously selected is greater than a number of different three-dimensional feature points included in a reconstruction picture that is not selected as a to-be-subsequently-presented picture and the to-be-subsequently-presented picture previously selected.