User interface to assist three dimensional scanning of objects

• US 10,341,568 B2
• Filed: 03/15/2017
• Issued: 07/02/2019
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Fischler, Matthew, Pajak, Arthur, Ranganath, Mithun Kumar, Sundaresan, Sairam, Sarkis, Michel Adib, Beith, Scott
• Primary Examiner(s): Berardesca, Paul M
• Application Number:
  US15/459,924
• Publication Number:
  US 20180103209A1
• Time in Patent Office: 839 Days
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

Methods and apparatuses are disclosed for assisting a user in performing a three dimensional scan of an object. An example user device to assist with scanning may include a processor. The user device may further include a scanner coupled to the processor and configured to perform a three dimensional scan of an object. The user device may also include a display to display a graphical user interface, wherein the display is coupled to the processor. The user device may further include a memory coupled to the processor and the display, the memory including one or more instructions that when executed by the processor cause the graphical user interface to display a target marker for a three dimensional (3D) scan and display a scanner position marker to assist in moving the scanner to a preferred location and direction.

Claims

1. A user device to assist with three dimensional scanning of an object, comprising:

a processor;

a scanner coupled to the processor and configured to perform a three dimensional scan of the object;

a display to display a graphical user interface (GUI), the display coupled to the processor;

a memory coupled to the processor and the display, the memory including one or more instructions that when executed by the processor cause the GUI to;

display a target marker for the object being scanned;

display a visual boundary;

display a first scanner position marker, located on the visual boundary, the first scanner position marker indicating an orientation of the scanner and a preferred position of the scanner in relation to the target marker; and

display a sight to assist a user to move the scanner to the preferred position,

wherein the sight extends from the visual boundary along the orientation of the scanner.

2. The user device of claim 1, wherein the one or more instructions when executed by the processor further cause the GUI to:

display a scanner marker to assist a user to orient the scanner relative to the target marker.

3. The user device of claim 1, wherein the one or more instructions when executed by the processor further cause the GUI to:

superimpose at least a portion of the first scanner position marker and at least a portion of the target marker when the scanner is at the preferred position.

4. The user device of claim 3, wherein:

the visual boundary is used to define, for the object, a scan path for moving the scanner.

5. The user device of claim 4, wherein the one or more instructions when executed by the processor further cause the user device to determine the visual boundary based on at least one from the group consisting of:

type of object being scanned; and

contours of the object being scanned.

6. The user device of claim 5, wherein the memory stores a plurality of predefined visual boundaries, and wherein the one or more instructions when executed by the processor further cause the user device to:

determine the type of object being scanned; and

select the visual boundary from the stored plurality of predefined visual boundaries based on the determination.

7. The user device of claim 4, wherein the one or more instructions when executed by the processor further cause the GUI to:

upon the at least portion of the target marker and the at least portion of the first scanner position marker being superimposed, display a second scanner position marker at a new location, on the visual boundary, indicating a new preferred position of the scanner in relation to the target marker.

8. The user device of claim 7, wherein the second scanner position marker is the first scanner position marker, and the one or more instructions when executed by the processor further cause the GUI to:

move the first scanner position marker along a portion of the scan path to the new location, wherein the portion of the scan path indicates a desired path for moving the scanner from the preferred position to the new preferred position.

9. The user device of claim 7, wherein the one or more instructions when executed by the processor further cause:

the GUI to display a first portion of a plurality of scanner position markers in a determined sequence to assist moving the scanner along the scan path; and

the user device to adjust a second portion of the plurality of scanner position markers based on information from a current scan while displaying the first portion, including adjusting at least one from the group consisting of;

a quantity of the plurality of scanner position markers;

the determined sequence for displaying scanner position markers; and

a location, on the visual boundary, of one or more of the plurality of scanner position markers.

10. The user device of claim 9, wherein adjusting the second portion is further based on user information from prior scans.

11. The user device of claim 1, wherein the one or more instructions when executed by the processor further cause the GUI to display, during the three dimensional scan, a rendering of the object from collected scan information.

12. The user device of claim 11, wherein the one or more instructions when executed by the processor further cause:

the user device to determine that an actual background for the object is similar to the object being scanned; and

the GUI to display a rendered background altered from the actual background to assist a user to distinguish the object from the actual background during scanning.

13. The user device of claim 11, wherein the one or more instructions when executed by the processor further cause the GUI to zoom into a portion of the rendering of the object to assist a user to scan a portion of the object associated with the portion of the rendering.

14. A method for performing a three dimensional scan of an object by a user device including a scanner, comprising:

displaying, on a display of the user device, a graphical user interface (GUI) to assist with scanning the object, comprising;

displaying a target marker for the object being scanned;

displaying a visual boundary;

displaying a first scanner position marker, located on the visual boundary, the first scanner position marker indicating an orientation of the scanner and a preferred position of the scanner in relation to the target marker; and

displaying a sight to assist a user to move the scanner to the preferred position, the sight extending from the visual boundary along the orientation of the scanner.

15. The method of claim 14, wherein displaying the GUI further comprises:

superimposing at least a portion of the first scanner position marker and at least a portion of the target marker when the scanner is at the preferred position.

16. The method of claim 15, wherein

the visual boundary is used to define, for the object, a scan path for moving the scanner.

17. The method of claim 16, further comprising:

determining, by the user device, the visual boundary based on at least one from the group consisting of;

type of object being scanned; and

contours of the object being scanned.

18. The method of claim 17, further comprising:

storing, in a memory, a plurality of predefined visual boundaries;

determining the type of object being scanned; and

selecting the visual boundary from the stored plurality of predefined visual boundaries based on the determination.

19. The method of claim 16, wherein displaying the GUI further comprises:

upon the at least portion of the target marker and the at least portion of the first scanner position marker being superimposed, displaying a second scanner position marker at a new location, on the visual boundary, indicating a new preferred position of the scanner in relation to the target marker.

20. The method of claim 16, further comprising:

displaying, in the GUI, a first portion of a plurality of scanner position markers in a determined sequence to assist moving the scanner along the scan path; and

adjusting, by the user device, a second portion of the plurality of scanner position markers based on information from a current scan while displaying the first portion, including adjusting at least one from the group consisting of;

a quantity of the plurality of scanner position markers;

the determined sequence for displaying scanner position markers; and

a location, on the visual boundary, of one or more of the plurality of scanner position markers.

21. The method of claim 14, wherein displaying the GUI further comprises:

displaying a rendering of the object from scan information collected during scanning.

22. The method of claim 21, wherein displaying the GUI further comprises:

zooming into a portion of the rendering of the object to assist a user to scan a portion of the object associated with the portion of the rendering.

23. A non-transitory computer-readable storage medium storing one or more programs containing instructions that, when executed by one or more processors of a user device using a scanner to perform a three dimensional scan of an object, cause the user device to perform operations comprising:

displaying, on a display of the user device, a graphical user interface (GUI) to assist with scanning the object, comprising;

displaying a target marker for the object being scanned;

displaying a visual boundary;

displaying a first scanner position marker, located on the visual boundary, the first scanner position marker indicating an orientation of the scanner and a preferred position of the scanner in relation to the target marker, wherein at least a portion of the first scanner position marker and at least a portion of the target marker are superimposed when the scanner is at the preferred position;

displaying a sight to assist a user to move the scanner to the preferred position, the sight extending from the visual boundary along the orientation of the scanner;

wherein;

the visual boundary is used to define, for the object, a scan path for moving the scanner; and

upon the at least portion of the target marker and the at least portion of the first scanner position marker being superimposed, displaying a second scanner position marker at a new location, on the visual boundary, indicating a new preferred position of the scanner in relation to the target marker.

24. The non-transitory computer-readable storage medium of claim 23, wherein execution of the instructions causes the user device to perform operations further comprising:

determining a type of object being scanned; and

selecting, based on the determination, the visual boundary from a plurality of predefined visual boundaries.

25. The non-transitory computer-readable storage medium of claim 23, wherein execution of the instructions causes the user device to perform operations further comprising:

displaying, in the GUI, a first portion of a plurality of scanner position markers in a determined sequence to assist moving the scanner along the scan path; and

adjusting a second portion of the plurality of scanner position markers based on information from a current scan while displaying the first portion, wherein adjusting the second portion includes adjusting at least one from the group consisting of;

a quantity of the plurality of scanner position markers;

the determined sequence for displaying scanner position markers; and

a location, on the visual boundary, of one or more of the plurality of scanner position markers.

26. The non-transitory computer-readable storage medium of claim 23, wherein execution of the instructions causes the user device to perform operations further comprising:

displaying a rendering of the object from scan information collected during scanning.

27. The non-transitory computer-readable storage medium of claim 26, wherein execution of the instructions causes the user device to perform operations further comprising:

zooming, in the GUI, into a portion of the rendering of the object to assist a user to scan a portion of the object associated with the portion of the rendering.