Generating a three-dimensional model using a portable electronic device recording

• US 8,624,974 B2
• Filed: 09/17/2012
• Issued: 01/07/2014
• Est. Priority Date: 01/28/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Tsai, Richard, Harris, Brandon, Just, Andrew
• Primary Examiner(s): Chang, Jungwon
• Application Number:
  US13/621,740
• Publication Number:
  US 20130038609A1
• Time in Patent Office: 477 Days
• Field of Search: 345/156, 345/157, 345/419, 341/120, 382/107, 715/863, 348/333.02, 348/135, 348/148
• US Class Current: 348/135
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/422 Input-only peripherals , i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] input devices also receiving signals from specially adapted client devices H04N21/4104
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Systems and methods are provided for navigating a three-dimensional model using deterministic movement of an electronic device. An electronic device can load and provide an initial display of a three dimensional model (e.g., of an environment or of an object). As the user moves the electronic device, motion sensing components can detect the device movement and adjust the displayed portion of the three-dimensional model to reflect the movement of the device. By walking with the device in the user'"'"'s real environment, a user can virtually navigate a representation of a three-dimensional environment. In some embodiments, a user can record an object or environment using an electronic device, and tag the recorded images or video with movement information describing the movement of the device during the recording. The recorded information can then be processed with the movement information to generate a three-dimensional model of the recorded environment or object.

Claims

1. A method for displaying a three-dimensional model of an object, comprising:

displaying a first image of a first portion of a three-dimensional model of an object using an electronic device, wherein the first image comprises a virtual view of the three-dimensional model of the object from the electronic device, the electronic device having a first virtual perspective and a first virtual location with respect to the three-dimensional model of the object;

detecting, with the electronic device, a first movement of the electronic device;

determining, based on the first movement, a second virtual perspective and second virtual location of the electronic device with respect to the three-dimensional model of the object, comprising;

scaling a magnitude of the first movement, based in part on at least one of a direction of the first movement and a hysteresis or other non-linear correlation parameter; and

displaying, using the electronic device, a second image of a second portion of the three-dimensional model of the object based, at least in part, on the determined second virtual perspective and second virtual location.

2. The method of claim 1, wherein the act of detecting comprises:

receiving, from a movement detection component of the electronic device, an output that indicates the first movement.

3. The method of claim 1, wherein the act of displaying further comprises:

obtaining, from a source different from the electronic device, the second image.

4. The method of claim 1, wherein the first movement comprises a movement indicating a change in a zoom level of the three-dimensional model of the object from the first image to the second image.

5. The method of claim 1, wherein the act of determining further comprises:

determining at least one of a velocity and an acceleration of the first movement.

6. An electronic device operative to display a three-dimensional model of an object, comprising:

a display device operative to display a plurality of portions of an object;

a motion-sensing component operative to provide at least one output that reflects movement of the electronic device; and

one or more processors operative to;

display a first image of a first portion of a three-dimensional model of the object, wherein the first image comprises a virtual view of the three-dimensional model of the object from the electronic device corresponding to a first virtual perspective and a first virtual location of the electronic device with respect to the three-dimensional model of the object;

receive output provided by the motion-sensing component, the received output corresponding to a first movement of the electronic device;

determine, based at least in part on the first movement, a second virtual perspective and second virtual location of the electronic device with respect to the three-dimensional model of the object, comprising;

scaling a magnitude of the first movement, based in part on at least one of a direction of the first movement and a hysteresis or other non-linear correlation parameter; and

display a second image of a second portion of the three-dimensional model of the object corresponding, at least in part, to the second virtual perspective and the second virtual location with respect to the three-dimensional model of the object.

7. The electronic device of claim 6, wherein the one or more processors are further operative to generate the second image by:

sending the received plurality of outputs to a host device for generating the second image; and

receiving the second image from the host device.

8. The electronic device of claim 6, wherein the one or more processors are further operative to:

determine at least one of a velocity and an acceleration of the first movement.

9. A non-transitory computer readable medium for displaying a three-dimensional model, comprising computer readable code recorded thereon operative to:

generate a first image of a first portion of a three-dimensional model of an object, wherein the first image comprises a virtual view of the three-dimensional model of the object from a first viewpoint, the first viewpoint having a first virtual perspective and a first virtual location with respect to the three-dimensional model of the object;

receive information indicative of a first movement of the first viewpoint;

determine, based on the first movement information, a second viewpoint having a second virtual perspective and second virtual location with respect to the three-dimensional model of the object, comprising;

scaling a magnitude of the first movement, based in part on at least one of a direction of the first movement and a hysteresis or other non-linear correlation parameter; and

generate a second image of a second portion of the three-dimensional model of the object, wherein the second image comprises a virtual view of the three-dimensional model of the object from the second viewpoint.

10. The non-transitory computer readable medium of claim 9, further comprising computer readable code recorded thereon, operative to:

transmit the information indicative of the first movement of the first viewpoint to a host device for generating the second image; and

receive the second image from the host device.

11. The non-transitory computer readable medium of claim 9, wherein the first movement comprises a movement indicating a change in a zoom level of the three-dimensional model of the object from the first image to the second image.

12. The non-transitory computer readable medium of claim 9, wherein the computer readable code operative to determined a second viewpoint comprises:

computer readable code operative to determine at least one of a velocity and an acceleration of the first movement.