Methods and apparatus to facilitate the capture of photographs using mobile devices

• US 9,906,712 B2
• Filed: 07/10/2015
• Issued: 02/27/2018
• Est. Priority Date: 06/18/2015
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Garcia Morate, Diego, Hurtado Garcia, Antonio, Tovar Velasco, Javier, Barrales Ruiz, Carlos
• Primary Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US14/796,813
• Publication Number:
  US 20160373647A1
• Time in Patent Office: 963 Days
• Field of Search: 382284-284, 382293-301
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/6811 based on the image signal

Abstract

Methods and apparatus to capture photographs using mobile devices are disclosed. An example method includes receiving sensor data from a sensor in a mobile device. The example method further includes presenting visual feedback to a user, via a display of the mobile device, to guide the user in capturing a photograph with a camera of the mobile device. The visual feedback based on the sensor data.

Claims

1. A method comprising:

capturing a first photograph with a mobile device when a first orientation of the mobile device is within an orientation threshold of a reference orientation, the reference orientation defined by at least one of a reference compass direction or a reference angle relative to a direction of gravity determined by a sensor of the mobile device;

updating the reference orientation for a second photograph to correspond to the first orientation of the mobile device when the first photograph was captured, the first orientation defined by at least one of a first compass direction or a first angle relative to the direction of gravity determined by the sensor;

presenting an orientation guide on a display of the mobile device to indicate a current orientation of the mobile device relative to the reference orientation;

disabling an image capturing functionality of the mobile device to capture the second photograph when a difference between the current orientation of the mobile device and the reference orientation exceeds the orientation threshold; and

enabling the image capturing functionality of the mobile device to capture the second photograph when the difference between the current orientation of the mobile device and the reference orientation does not exceed the orientation threshold.

2. The method of claim 1, wherein the orientation guide is a quadrilaterally shaped box superimposed on an image of a field of view of a camera of the mobile device.

3. The method of claim 2, wherein the quadrilaterally shaped box corresponds to a rectangle oriented in alignment with the reference orientation.

4. The method of claim 3, further including geometrically transforming the rectangle based on the current orientation of the mobile device.

5. The method of claim 1, further including:

detecting a user request to capture the second photograph with the mobile device;

presenting a visual timer that incrementally changes appearance from an initial state to a final state over a period of time during which a level of movement of the mobile device is below a movement threshold; and

capturing the second photograph with the mobile device when the visual timer reaches the final state and the difference between the current orientation of the mobile device and the reference orientation is within the orientation threshold.

6. The method of claim 1, further including overlaying an image of a field of view of a camera of the mobile device with a graphic indicative of a portion of a scene in the field of view captured in the first photograph.

7. The method of claim 1, further including:

capturing the second photograph with the mobile device; and

stitching geometric transformations of the first and second photographs together, the geometric transformations of the first and second photographs based on the first orientation of the mobile device when the first photograph was captured and a second orientation of the mobile device when the second photograph was captured.

8. The method of claim 7, further including determining the geometric transformations of the first and second photographs by:

estimating initial transformations for the first and second photographs based on the first and second orientations of the mobile device when each of the first and second photographs was captured;

calculating modified transformations for the first and second photographs based on variations in at least one of orientation values or lens parameter values for the mobile device when each of the first and second photographs was captured;

calculating an error in overlapping portions for each of the initial transformations and the modified transformations of the first and second photographs; and

selecting final transformations for the first and second photographs based on a smallest error calculated.

9. The method of claim 1, further including providing non-visual feedback to a user of the mobile device indicative of a deviation between the current orientation of the mobile device and the reference orientation.

10. The method of claim 9, wherein the non-visual feedback corresponds to at least one of sound or vibration.

11. The method of claim 9, wherein the non-visual feedback varies proportionately with an amount of the deviation between the current orientation of the mobile device and the reference orientation.

12. An apparatus comprising:

a photograph capturing controller to capture a first photograph with a mobile device when a first orientation of the mobile device is within an orientation threshold of a reference orientation, the reference orientation defined by at least one of a reference compass direction or a reference angle relative to a direction of gravity determined by a sensor of the mobile device;

an orientation monitor to be implemented on the mobile device, the orientation monitor to update the reference orientation for a second photograph to correspond to the first orientation of the mobile device when the first photograph was captured, the first orientation defined by at least one of a first compass direction or a first angle relative to the direction of gravity determined by the sensor; and

a photograph capturing interface to be implemented on the mobile device, the photograph capturing interface to present an orientation guide on a display of the mobile device to indicate a current orientation of the mobile device relative to the reference orientation

the photograph capturing controller to disable an image capturing functionality of the mobile device to capture the second photograph when a difference between the current orientation of the mobile device and the reference orientation exceeds the orientation threshold, the photograph capturing controller to enable the image capturing functionality of the mobile device to capture the second photograph when the difference between the current orientation of the mobile device and the reference orientation does not exceed the orientation threshold.

13. The apparatus of claim 12, wherein the photograph capturing interface is to present the orientation guide in a first color when the difference between the current orientation of the mobile device and the reference orientation does not exceed the orientation threshold and to present the orientation guide in a second color, different from the first color, when the difference between the current orientation of the mobile device and the reference orientation exceeds the orientation threshold.

14. The apparatus of claim 12, wherein the reference orientation corresponds to a vertically upright orientation.

15. The apparatus of claim 12, wherein the orientation guide is a quadrilaterally shaped box superimposed on an image of a field of view of a camera of the mobile device.

16. The apparatus of claim 15, wherein the quadrilaterally shaped box corresponds to a rectangle oriented in alignment with the reference orientation.

17. The apparatus of claim 16, further including a transformation calculator to geometrically transform the rectangle based on the current orientation of the mobile device.

18. The apparatus of claim 12, wherein:

the photograph capturing interface is further to present a visual timer that incrementally changes appearance from an initial state to a final state over a period of time during which a level of movement of the mobile device is below a movement threshold; and

the photograph capturing controller is further to capture the second photograph with the mobile device when the visual timer reaches the final state.

19. The apparatus of claim 12, wherein the photograph capturing interface is further to overlay an image of a field of view of a camera of the mobile device with a graphic indicative of a portion of a scene in the field of view captured in the first photograph.

20. The apparatus of claim 12, wherein the photograph capturing controller is further to capture the second photograph with the mobile device, the apparatus further including:

an image stitcher to stitch geometric transformations of the first and second photographs together, the geometric transformations of the first and second photographs based on the orientation of the mobile device when each of the first and second photographs was captured.

21. The apparatus of claim 20, further including a transformation calculator to determine the geometric transformations of the first and second photographs by:

estimating initial transformations for the first and second photographs based on the first orientation of the mobile device when the first photograph was captured and a second orientation of the mobile device when the second photograph was captured;

calculating modified transformations for the first and second photographs based on variations in at least one of orientation values or lens parameter values for the mobile device when each of the first and second photographs was captured;

calculating an error in overlapping portions for each of the initial transformations and the modified transformations of the first and second photographs; and

selecting final transformations for the first and second photographs based on a smallest error calculated.

22. The apparatus of claim 12, wherein the photograph capturing interface is to provide non-visual feedback to a user of the mobile device indicative of a deviation between the current orientation of the mobile device and the reference orientation.

23. The apparatus of claim 22, wherein the non-visual feedback corresponds to at least one of sound or vibration.

24. The apparatus of claim 22, wherein the non-visual feedback varies proportionately with an amount of the deviation between the current orientation of the mobile device and the reference orientation.

25. A tangible computer readable storage medium comprising instructions that, when executed, cause a machine to at least:

capture a first photograph with a mobile device when a first orientation of the mobile device is within an orientation threshold of a reference orientation, the reference orientation defined by at least one of a reference compass direction or a reference angle relative to a direction of gravity determined by a sensor of the mobile device;

update the reference orientation for a second photograph to correspond to the first orientation of the mobile device when the first photograph was captured, the first orientation defined by at least one of a first compass direction or a first angle relative to the direction of gravity determined by the sensor;

present an orientation guide on a display of the mobile device to indicate a current orientation of the mobile device relative to the reference orientation;

disable an image capturing functionality of the mobile device to capture the second photograph when a difference between the current orientation of the mobile device and the reference orientation exceeds the orientation threshold; and

enable the image capturing functionality of the mobile device to capture the second photograph when the difference between the current orientation of the mobile device and the reference orientation does not exceed the orientation threshold.

26. The storage medium of claim 25, wherein the instructions further cause the machine to overlay an image of a field of view of a camera of the mobile device with a graphic indicative of a portion of a scene in the field of view captured in the first photograph.

27. The storage medium of claim 25, wherein the instructions further cause the machine to:

capture the second photograph with the mobile device; and

stitch geometric transformations of the first and second photographs together, the geometric transformations of the first and second photographs based on the first orientation of the mobile device when the first photograph was captured and a second orientation of the mobile device when the second photograph was captured.

28. The storage medium of claim 27, wherein the instructions further cause the machine to determine the geometric transformations of the first and second photographs by:

estimating initial transformations for the first and second photographs based on the first and second orientations of the mobile device when each of the first and second photographs was captured;

calculating modified transformations for the first and second photographs based on variations in at least one of orientation values or lens parameter values for the mobile device when each of the first and second photographs was captured;

calculating an error in overlapping portions for each of the initial transformations and the modified transformations of the first and second photographs; and

selecting final transformations for the first and second photographs based on a smallest error calculated.