Viewing angle image manipulation based on device rotation

• US 9,836,875 B2
• Filed: 01/10/2014
• Issued: 12/05/2017
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Flipboard Incorporated
• Original Assignee: Flipboard Incorporated
• Inventors: Ying, Charles
• Primary Examiner(s): Harrison, Chante
• Application Number:
  US14/152,462
• Publication Number:
  US 20140320535A1
• Time in Patent Office: 1,425 Days
• Field of Search: 345649, 345650, 345659, 348 36, 348 37, 348 39, 3482084, 382284, 382294
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

A user device uses sensors to determine the rotation of the user device with respect to a reference orientation. Using the user device rotation, an image manipulation module accesses images that are associated with image rotations. A set of the images are selected based on the device rotation to select images that have image orientations corresponding to the device rotation. A weight may be determined for each selected image, and the images are combined to generate a blended image using the weights. The blended image is displayed to the user, and as the rotation of the user devices changes, the process is repeated to display changing blended images based on the device rotation, thereby animating the rotation effect.

Claims

1. A method for image manipulation based on device rotation, the method comprising:

determining a device rotation of a user device, the device rotation determined based on an orientation of the user device with respect to a reference orientation;

accessing a plurality of stored images, each image of the plurality of images associated with an image orientation;

selecting a plurality of near images from the plurality of images, wherein the selected near images have image orientations relative to the reference orientation that correspond to the orientation of the user device;

generating a blended image by combining the selected plurality of near images, wherein generating the blended image comprises;

determining a first angle between the device rotation and a first image orientation of a first near image of the selected near images;

determining a second angle between the device rotation and a second image orientation of a second near image of the selected near images;

responsive to a difference between the first angle and the second angle not exceeding a threshold angle, and responsive to the first angle exceeding the second angle, excluding the first near image from the plurality of near images; and

generating a blended image by combining a plurality of remaining near images from the selected near images; and

outputting the blended image for display on the user device.

2. The method of claim 1, wherein the plurality of images are images of an object.

3. The method of claim 1, wherein the image orientation of an image in the plurality of images is based on an angle at which the image was taken relative to an object.

4. The method of claim 1, wherein the image orientation of an image in the plurality of images is based on an angle at which an object in an environment was located when the image was taken.

5. The method of claim 1, wherein the image orientation of an image in the plurality of images is based on an angle at which an object was rotated when the image was taken.

6. The method of claim 1, further comprising:

determining an updated device rotation based on a changed orientation of the user device with respect to the reference orientation; and

repeating the selecting, generating, and outputting steps using the updated device rotation.

7. The method of claim 1, further comprising determining a weight for each of the selected plurality of near images, the weight based on a difference between the device rotation and the image orientation, and wherein the generated blended image is based on the weight associated with each near image.

8. The method of claim 7, wherein the difference between the device rotation and the image orientation is measured in a distance on a spherical projection.

9. The method of claim 7, wherein generating the blended image comprises interpolating pixel values of each near image in the plurality of near images based on the associated weights.

10. The method of claim 1, further comprising:

determining that the device rotation is within a region comprising a range of device rotations; and

performing an event based on determining that the device rotation is within the region.

11. The method of claim 10, wherein the event comprises a notification, an image overlay, a sound, or launching an application.

12. The method of claim 1, wherein the device rotation is determined by a sensor on the user device.

13. The method of claim 1, wherein the device rotation is determined by a server that receives the orientation of the user device, and the server generates the blended image.

14. A computer program product for image manipulation based on device rotation, the computer program product comprising a computer-readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:

determining a device rotation of a user device, the device rotation determined based on an orientation of the user device with respect to a reference orientation;

accessing a plurality of stored images, each image of the plurality of images associated with an image orientation;

selecting a plurality of near images from the plurality of images, wherein the selected near images have image orientations relative to the reference orientation that correspond to the orientation of the user device;

generating a blended image by combining the selected plurality of near images, wherein generating the blended image comprises;

determining a first angle between the device rotation and a first image orientation of a first near image of the selected near images;

determining a second angle between the device rotation and a second image orientation of a second near image of the selected near images;

responsive to a difference between the first angle and the second angle not exceeding a threshold angle, and responsive to the first angle exceeding the second angle, excluding the first near image from the plurality of near images; and

generating a blended image by combining a plurality of remaining near images from the selected near images; and

outputting the blended image for display on the user device.

15. The computer program product of claim 14, wherein the image orientation of an image in the plurality of images is based on an angle at which the image was taken relative to an object.

16. The computer program product of claim 14, wherein the image orientation of an image in the plurality of images is based on an angle at which an object in an environment was located when the image was taken.

17. The computer program product of claim 14, wherein the image orientation of an image in the plurality of images is based on an angle at which an object was rotated when the image was taken.

18. The computer program product of claim 14, wherein the computer-readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:

determining an updated device rotation based on a changed orientation of the user device with respect to the reference orientation; and

repeating the selecting, generating, and outputting steps using the updated device rotation.

19. The computer program product of claim 14, wherein the computer-readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:

determining a weight for each of the selected plurality of near images, the weight based on a difference between the device rotation and the image orientation, and wherein the generated blended image is based on the weight associated with each near image.

20. The computer program product of claim 19, wherein the difference between the device rotation and the image orientation is measured in a distance on a spherical projection.

21. The computer program product of claim 19, wherein generating the blended image comprises interpolating pixel values of each near image in the plurality of near images based on the associated weights.

22. The computer program product of claim 14, wherein the computer-readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to perform steps comprising:

determining that the device rotation is within a region comprising a range of device rotations; and

performing an event based on determining that the device rotation is within the region.

23. The computer program product of claim 22, wherein the event comprises a notification, an image overlay, a sound, or launching an application.

24. The computer program product of claim 14, wherein the device rotation is determined by a sensor on the user device.

25. A method for image manipulation based on device rotation, the method comprising:

determining a device rotation of a user device, the device rotation determined based on an orientation of the user device with respect to a reference orientation;

accessing a plurality of stored images, each image of the plurality of images associated with an image orientation;

selecting a plurality of near images from the plurality of images, wherein the selected near images have image orientations relative to the reference orientation that correspond to the orientation of the user device;

for each selected near image;

determining an angle between the selected near image and the device orientation;

comparing the determined angle to a threshold angle associated with the device rotation;

responsive to determination that the determined angle exceeds a threshold angle, excluding the selected near image; and

combining a plurality of remaining near images from the selected near images to generate a blended image; and

outputting the blended image for display on the user device.

26. The method of claim 25, further comprising:

determining that the device rotation is within a region comprising a range of device rotations; and

performing an event based on determining that the device rotation is within the region.

27. The method of claim 26, wherein the event comprises a notification, an image overlay, a sound, or launching an application.

28. The method of claim 25, further comprising determining a weight for each of the selected plurality of near images, the weight based on a difference between the device rotation and the image orientation, and wherein the generated blended image is based on the weight associated with each near image.

29. The method of claim 28 further comprising interpolating pixel values of each selected near image in the plurality of selected near images based on the associated weights to generate the blended image.