Remote control system

• US 9,204,131 B2
• Filed: 09/14/2012
• Issued: 12/01/2015
• Est. Priority Date: 09/14/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Arrasvuori, Juha, Vaittinen, Tuomas, Piippo, Petri
• Primary Examiner(s): NGUYEN, HUY THANH
• Application Number:
  US13/700,217
• Publication Number:
  US 20150181200A1
• Time in Patent Office: 1,173 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 23/632 for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters

Abstract

A method comprising: capturing, by a device equipped with a camera, at least one image about a physical object; recognizing the physical object on the basis of the at least one captured image; retrieving a 3-dimensional (3D) model of the physical object from a memory; and showing the 3D model of the physical object on a display of the device.

Claims

1. A method comprising:

receiving an image captured, by a device equipped with a camera, of a first physical object and a second physical object;

providing for recognition of the first physical object and the second physical object based, at least in part, on the at least one captured image;

providing for retrieval of a 3-dimensional (3D) model of the first physical object and retrieval of a 3D model of the second physical object from a memory;

providing for display of the 3D model of the first physical object and the 3D model of the second physical object on a display of the device, wherein the display comprises a view of a space interconnecting the first physical object and the second physical object wherein the 3D model of the first physical object is superimposed over the image of the first physical object, and the 3D model of the second physical object is superimposed over the image of the second physical object; and

providing for compression of visual information between the first physical object and the second physical object in a displayed image.

2. A method of claim 1, further comprising:

providing for recognition of the first physical object using a computer vision method.

3. A method of claim 1, further comprising:

providing for rotation of the 3D model of the first physical object presented on the display.

4. A method of claim 3, wherein the rotating of the 3D model of the first physical object presented on the display is performed on a touch sensitive display screen.

5. A method of claim 1, further comprising:

providing the 3D model of the first physical object with additional information about the physical object to be shown on the display.

6. A method of claim 1, wherein the first physical object and the second physical object are each a building or an architectural object.

7. A method of claim 6, further comprising:

establishing location information of the device and orientation information of the camera view of the device;

providing for recognition of the first physical object and the second physical object appearing in the camera view based at least in part on the location and orientation information;

receiving an input of a first point in the camera view from the user, wherein the first point indicates at least one of a foreground or a background of the camera view;

receiving an input of a second point in the camera view from the user, wherein the second point indicates another of the foreground or the background of the camera view;

receiving an input associated with space compression in the camera view from the user; and

providing for a compressed camera view with a 3D model of the first physical object superimposed on an image part of the background of the camera view such that the size of the 3D model of the first physical object is increased proportional to the input of space compression.

8. A method of claim 7, wherein said input of the first point is obtained by detecting a finger press of the user on a first part of a touch screen showing the camera view;

and wherein said input of the second point is obtained by detecting a finger press of the user on a second part of the touch screen showing the camera view, while keeping the first point touched.

9. A method of claim 8, wherein said input of the space compression is obtained by detecting a pinching gesture with the fingers pressing the first and second part of the touch screen.

10. A method according to claim 9, wherein a length of the pinching gesture defines the amount of compressing of the space between first and second points.

22. A method of claim 7, wherein the input of the first point in the camera view corresponds to the first physical object in the camera view.

23. A method of claim 22, wherein the input of the second point in the camera view corresponds to the second physical object in the camera view.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive an image captured by a camera of a first physical object and a second physical object;

recognize the first physical object and the second physical object based, at least in part, on the at least one captured image;

retrieve a 3-dimensional (3D) model of the first physical object and a 3D model of the second physical object from a memory; and

cause display of the 3D model of the first physical object and the 3D model of the second physical object on a display of the apparatus, wherein the display comprises a view of a space interconnecting the first physical object and the second physical object wherein the 3D model of the first physical object is superimposed over the image of the first physical object, and the 3D model of the second physical object is superimposed over the image of the second physical object; and

provide for compression of visual information between the first physical object and the second physical object in a displayed image.

12. An apparatus of claim 11, wherein the apparatus is further caused to:

recognize the first physical object by a computer vision method.

13. An apparatus of claim 11, wherein the apparatus is further caused to:

rotate the 3D model of the first physical object on the display.

14. An apparatus of claim 13, wherein the rotating of the 3D model of the first physical object on the display is performed on a touch sensitive display screen.

15. An apparatus of claim 11, wherein the apparatus is further caused to:

provide the 3D model of the first physical object with additional information about the first physical object to be shown on the display.

16. An apparatus of claim 11, wherein the first physical object is a building or an architectural object.

17. An apparatus of claim 16, wherein the apparatus is further caused to:

obtain location information of the device and orientation information of the camera view of the device;

recognize the first physical object and the second physical object appearing in the camera view based at least in part on the location and orientation information;

obtain an input of a first point in the camera view from the user, wherein the first point indicates at least one of a foreground or a background of the camera view;

obtain an input of a second point in the camera view from the user, wherein the second point indicates another of the foreground or the background of the camera view;

and

provide a compressed camera view with a 3D model of the first physical object superimposed on an image part of the background of the camera view such that the size of the 3D model of the first physical object is increased proportional to the input of space compression.

18. An apparatus of claim 17, wherein said input of the first point is obtained by detecting a finger press of the user on a first part of a touch screen showing the camera view;

and wherein said input of the second point is obtained by detecting a finger press of the user on a second part of the touch screen showing the camera view, while keeping the first point touched.

19. An apparatus according to claim 18, wherein said input of the space compression is obtained by detecting a pinching gesture with the fingers pressing the first and second part of the touch screen.

20. An apparatus according to claim 19, wherein a length of the pinching gesture defines the amount of compressing of the space between first and second point.

24. An apparatus of claim 17, wherein the input of the first point in the camera view corresponds to the first physical object in the camera view.

25. An apparatus of claim 24, wherein the input of the second point in the camera view corresponds to the second physical object in the camera view.

21. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program instructions for one or more programs stored therein, the computer-readable program instructions comprising program instructions configured to cause an apparatus to perform a method comprising:

receive an image captured by a camera of a first physical object and a second physical object;

recognizing the first physical object and the second physical object based, at least in part, on the at least one captured image;

retrieving a 3-dimensional (3D) model of the first physical object and a 3D model of the second physical object from a memory; and

displaying the 3D model of the first physical object and the 3D model of the second physical object on a display of the apparatus, wherein the display comprises a view of a space interconnecting the first physical object and the second physical object wherein the 3D model of the first physical object is superimposed over the image of the first physical object, and the 3D model of the second physical object is superimposed over the image of the second physical object; and

providing for compression of visual information between the first physical object and the second physical object in a displayed image.