Camera-based orientation fix from portrait to landscape

• US 8,958,004 B2
• Filed: 11/04/2013
• Issued: 02/17/2015
• Est. Priority Date: 09/08/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gardiner, Koji, Chang, E-Cheng, Rukes, Jason
• Primary Examiner(s): LAM, HUNG H
• Application Number:
  US14/071,218
• Publication Number:
  US 20140055658A1
• Time in Patent Office: 470 Days
• Field of Search: 348/14.07, 348/191, 348/208.2, 348/333.01, 348/333.03, 348/333.12
• US Class Current: 348/333.12
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

A portable electronic device includes a camera and a display switchable between a portrait mode and a landscape mode in response to an inertial sensor. When it is determined that the display mode should be switched in response to movement of the device, a feature of a user of the device, such as the user'"'"'s face, is imaged with the camera. If the imaged feature changes orientation with respect to the display by less than a threshold amount, then the switch in display mode is disabled and the display remains in its current mode. Other embodiments are also described and claimed.

Claims

1. A method for use in a portable electronic device having a camera, an inertial sensor, a processor and a display screen, the method comprising:

detecting, by the inertial sensor, movement of the device;

analyzing, by the processor in response to the detecting of the movement, a first image captured by the camera to determine an orientation of a feature of a user of the device relative to an axis of the device;

comparing the determined orientation of the user feature to an orientation of the user feature determined by analyzing a second image captured by the camera before the detecting of the movement of the device; and

displaying, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode based on the comparing.

2. The method of claim 1, wherein the analyzing includes determining an orientation of a line connecting eyes of the user in the first image.

3. The method of claim 2, wherein the determined orientation of the line is an angle of the line relative to the axis of the device.

4. The method of claim 1, further comprising detecting at least one of a face of the user or eyes of the user in the first image.

5. The method of claim 1, wherein the displaying includes maintaining, after the detecting of the movement, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by less than a threshold amount.

6. The method of claim 1, wherein the displaying includes switching, after the detecting of the movement, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by more than a threshold amount.

7. The method of claim 1, further comprising receiving a selection of a facial-stabilization mode of operation of the device.

8. The method of claim 1, further comprising capturing the first image in response to the detecting of the movement of the device.

9. The method of claim 1, further comprising capturing the second image after and as a function of selecting a facial stabilizing mode of the portable electronic device.

10. The method of claim 1, wherein the capturing of the second image occurs automatically.

11. A portable electronic device, comprising:

a camera;

an inertial sensor;

a display; and

a processor, wherein the processor is configured to control the camera, inertial sensor, and display to;

detect, by the inertial sensor, movement of the device;

analyze, by the processor in response to the detecting of the movement, a first image captured by the camera to determine an orientation of a feature of a user of the device relative to an axis of the device;

compare the determined orientation of the user feature to an orientation of the user feature determined by analyzing a second image captured by the camera before the detecting of the movement of the device; and

display, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode based on the comparing.

12. The portable electronic device of claim 11, wherein the analyzing includes determining an orientation of a line connecting eyes of the user in the first image.

13. The portable electronic device of claim 12, wherein the determined orientation of the line is an angle of the line relative to the axis of the device.

14. The portable electronic device of claim 11, further comprising detecting at least one of a face of the user or eyes of the user in the first image.

15. The portable electronic device of claim 11, wherein the displaying includes maintaining, after the detecting of the movement, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by less than a threshold amount.

16. The portable electronic device of claim 11, wherein the displaying includes switching, after the detecting of the movement the device, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by more than a threshold amount.

17. A non-transitory storage medium having program instructions, which when executed by a processor perform a method for use in a portable electronic device having a camera, an inertial sensor, the processor and a display screen, the method comprising:

detecting, by the inertial sensor, movement of the device;

analyzing, by the processor in response to the detected movement, a first image captured by the camera to determine an orientation of a feature of a user of the device relative to an axis of the device;

comparing the determined orientation of the user feature to an orientation of the user feature determined by analyzing a second image captured by the camera before the detecting of the movement of the device; and

displaying, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode based on the comparing.

18. The non-transitory storage medium of claim 17, wherein the analyzing includes determining an orientation of a line connecting eyes of the user in the first image.

19. The non-transitory storage medium of claim 17, wherein the determined orientation of the line is an angle of the line relative to the axis of the device.

20. The non-transitory storage medium of claim 17, further comprising detecting at least one of a face of the user or eyes of the user in the first image.

21. The non-transitory storage medium of claim 17, wherein the displaying includes maintaining, after the detecting of the movement, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by less than a threshold amount.

22. The non-transitory storage medium of claim 17, wherein the displaying includes switching, after the detecting of the movement, a display mode existing immediately prior to the detected movement if the determined orientation changes in comparison to the orientation of the user feature determined by analyzing the second image captured by the camera before the detecting of the movement by more than a threshold amount.

23. A method for use in a portable electronic device having a camera, an inertial sensor, a processor and a display screen, the method comprising:

capturing a first image of a user using the camera;

detecting, by the inertial sensor, movement of the device after capturing the first image;

capturing a second image of the user in response to the detecting of the movement;

displaying, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode as a function of the captured first and second images of the user.

24. The method of claim 23, wherein the displaying displays the user interface in the selected display mode as a function of comparing an orientation of a user feature determined by analyzing the second image to an orientation of the user feature determined by analyzing the first image.

25. The method of claim 23, wherein the displaying maintains, after the detecting of the movement, a display mode existing immediately prior to the detected movement if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by less than a threshold amount.

26. The method of claim 23, wherein the displaying switches from, after the detecting of the movement, a display mode existing immediately prior to the detected movement to a different display mode if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by more than a threshold amount.

27. A portable electronic device, comprising:

a camera;

an inertial sensor;

a display screen; and

a processor, wherein the processor is configured to control the camera, inertial sensor, and display to;

capture a first image of a user using the camera;

detect, by the inertial sensor, movement of the device after capturing the first image;

capture a second image of the user in response to the detecting of the movement; and

display, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode as a function of the captured first and second images of the user.

28. The portable electronic device of claim 27, wherein the displaying displays the user interface in the selected display mode as a function of comparing an orientation of a user feature determined by analyzing the second image to an orientation of the user feature determined by analyzing the first image.

29. The portable electronic device of claim 27, wherein the displaying maintains, after the detecting of the movement, a display mode existing immediately prior to the detected movement if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by less than a threshold amount.

30. The portable electronic device of claim 27, wherein the displaying switches from, after the detecting of the movement, a display mode existing immediately prior to the detected movement to a different display mode if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by more than a threshold amount.

31. A non-transitory storage medium having program instructions, which when executed by a processor perform a method for use in a portable electronic device having a camera, an inertial sensor, the processor and a display screen, the method comprising:

capturing a first image of a user using the camera;

detecting, by the inertial sensor, movement of the device after capturing the first image;

capturing a second image of the user in response to the detecting of the movement;

displaying, by the display screen, a user interface in a display mode selected from at least one of a landscape mode or a portrait mode as a function of the captured first and second images of the user.

32. The non-transitory storage medium of claim 31, wherein the displaying displays the user interface in the selected display mode as a function of comparing an orientation of a user feature determined by analyzing the second image to an orientation of the user feature determined by analyzing the first image.

33. The non-transitory storage medium of claim 31, wherein the displaying maintains, after the detecting of the movement, a display mode existing immediately prior to the detected movement if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by less than a threshold amount.

34. The non-transitory storage medium of claim 31, wherein the displaying switches from, after the detecting of the movement, a display mode existing immediately prior to the detected movement to a different display mode if an orientation of a user feature determined by analyzing the second image changes in comparison to an orientation of the user feature determined by analyzing the first image by more than a threshold amount.