ORIENTING DISPLAYED ELEMENTS RELATIVE TO A USER

• US 20100066763A1
• Filed: 09/14/2009
• Published: 03/18/2010
• Est. Priority Date: 09/12/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: HILDRETH, EVAN, MACDOUGALL, FRANCIS
• Granted Patent: US 8,896,632 B2
• Time in Patent Office: Days
• US Class Current: 345/656
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

One or more elements are initially displayed on a display component of an electronic device. After the one or more elements have been displayed on the display component of the electronic device, an image of a user of the electronic device is captured, and an orientation of the electronic device relative to the user is determined based on the captured image of the user of the electronic device. Thereafter, an orientation of at least one of the displayed elements is adjusted relative to the display component of the electronic device based on the determined orientation of the electronic device relative to the user.

Claims

1. A method for orienting elements displayed on a display component of an electronic device, the method comprising:

initially displaying one or more elements on a display component of an electronic device;

after initially displaying the one or more elements on the display component of the electronic device, capturing an image of a user of the electronic device;

based on the captured image of the user of the electronic device, determining an orientation of the electronic device relative to the user; and

based on the determined orientation of the electronic device relative to the user, adjusting an orientation of at least one of the displayed elements relative to the display component of the electronic device.

2. The method of claim 1 wherein:

initially displaying one or more elements on a display component of the electronic device includes displaying the at least one displayed element with an initial orientation relative to the user; and

adjusting the orientation of the at least one displayed element relative to the display component of the electronic device includes adjusting the orientation of the at least one displayed element relative to the display component of the electronic device such that, after adjusting the orientation of the at least one displayed element relative to the display component of the electronic device, the orientation of the at least one displayed element, relative to the user, is substantially the same as the initial orientation of the at least one displayed element relative to the user.

3. The method of claim 1 wherein:

the display component of the electronic device supports displaying elements according to at least a portrait orientation relative to the display component and a landscape orientation relative to the display component;

initially displaying the one or more elements on the display component of the electronic device includes initially displaying the at least one element on the display component of the electronic device according to the landscape orientation relative to the display component;

determining an orientation of the electronic device relative to the user includes determining that the display component of the electronic device is in a portrait orientation relative to the user; and

adjusting the orientation of the at least one displayed element relative to the display component of the electronic device includes changing the orientation of the at least one displayed element from the landscape orientation relative to the display component to the portrait orientation relative to the display component in response to determining that the display component of the electronic device is in a portrait orientation relative to the user.

4. The method of claim 1 wherein:

the display component of the electronic device supports displaying elements according to at least a portrait orientation relative to the display component and a landscape orientation relative to the display component;

initially displaying the one or more elements on the display component of the electronic device includes initially displaying the at least one element on the display component of the electronic device according to the portrait orientation relative to the display component;

determining an orientation of the electronic device relative to the user includes determining that the display component of the electronic device is in a landscape orientation relative to the user; and

adjusting the orientation of the at least one displayed element relative to the display component of the electronic device includes changing the orientation of the at least one displayed element from the portrait orientation relative to the display component to the landscape orientation relative to the display component in response to determining that the display component of the electronic device is in a landscape orientation relative to the user.

5. The method of claim 1 wherein:

capturing an image of a user of the electronic device includes capturing an image of a user of the electronic device that includes at least a portion of the user'"'"'s face; and

determining an orientation of the electronic device relative to the user based on the captured image of the user of the electronic device includes;

determining an orientation of the user'"'"'s face in the captured image relative to the captured image using an AdaBoost process, and

determining an orientation of the electronic device relative to the user based on the determined orientation of the user'"'"'s face in the captured image relative to the captured image.

6. The method of claim 5 wherein:

capturing an image of a user of the electronic device that includes at least a portion of the user'"'"'s face includes capturing an image of a user of the electronic device that includes the user'"'"'s eyes;

determining an orientation of the user'"'"'s face in the captured image relative to the captured image includes determining an orientation of the user'"'"'s eyes in the captured image relative to the captured image; and

determining an orientation of the electronic device relative to the user based on the determined orientation of the user'"'"'s face in the captured image relative to the captured image includes determining an orientation of the electronic device relative to the user based on the determined orientation of the user'"'"'s eyes in the captured image relative to the captured image.

7. The method of claim 1 wherein:

capturing an image of a user of the electronic device includes capturing an image of the user of the electronic device that includes at least a portion of the user'"'"'s head; and

determining an orientation of the electronic device relative to the user based on the captured image of the user of the electronic device includes;

determining an orientation of the portion of the user'"'"'s head in the captured image relative to the captured image, and

determining an orientation of the electronic device relative to the user based on the determined orientation of the portion of the user'"'"'s head in the captured image relative to the captured image.

8. The method of claim 1 wherein:

capturing an image of a user of the electronic device includes capturing an image of the user of the electronic device that includes at least a portion of the user'"'"'s body; and

determining an orientation of the electronic device relative to the user based on the captured image of the user of the electronic device includes;

determining an orientation of the portion of the user'"'"'s body in the captured image relative to the captured image, and

determining an orientation of the electronic device relative to the user based on the determined orientation of the portion of the user'"'"'s body in the captured image relative to the captured image.

9. The method of claim 1 wherein:

capturing an image of a user of the electronic device includes capturing an image of a user of the electronic device that includes at least a portion of the user'"'"'s face; and

determining an orientation of the electronic device relative to the user based on the captured image of the user of the electronic device includes;

generating at least two different versions of the captured image, each of the different versions of the captured image being oriented differently,

performing face detection on each of the differently-oriented versions of the captured image,

based on results of performing face detection on each of the differently-oriented versions of the captured image, selecting a particular version of the captured image as the version of the captured image that represents the orientation of the electronic device relative to the user,

determining the orientation of the selected version of the captured image relative to the orientation of the originally captured image; and

determining an orientation of the electronic device relative to the user based on the determined orientation of the selected version of the captured image relative to the orientation of the originally captured image.

10. The method of claim 9 wherein:

performing face detection on each of the differently-oriented versions of the captured image includes assigning a score to each of the differently-oriented versions of the captured image, wherein the score assigned to each of the differently-oriented versions of the captured image reflects a significance of a presence of a face in the version of the captured image; and

selecting a particular version of the captured image as the version of the captured image that represents the orientation of the electronic device relative to the user includes identifying a particular version of the captured image that was assigned a score that reflects, among the differently-oriented version of the captured image, the most significant presence of a face.

11. The method of claim 1 further comprising:

after determining an orientation of the electronic device relative to the user and adjusting the orientation of the least one displayed element relative to the display component of the electronic device, capturing an image of another user of the electronic device;

based on the captured image of the other user of the electronic device, determining an orientation of the electronic device relative to the other user; and

based on the determined orientation of the electronic device relative to the other user, adjusting the orientation of the at least one displayed element relative to the display component of the electronic device.

12. The method of claim 1 wherein:

determining the orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user has changed from an earlier orientation of the electronic device relative to the user; and

adjusting the orientation of the at least one displayed element relative to the display component includes changing the orientation of the at least one displayed element relative to the display component in response to the determination that the orientation of the electronic device relative to the user has changed from the earlier orientation of the electronic device relative to the user.

13. The method of claim 1 wherein:

determining the orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user has not changed from an earlier orientation of the electronic device relative to the user; and

adjusting the orientation of the at least one displayed element includes maintaining the orientation of the at least one displayed element relative to the display component in response to the determination that the orientation of the electronic device relative to the user has not changed from the earlier orientation of the electronic device relative to the user.

14. The method of claim 1 further comprising:

determining an initial orientation of the electronic device relative to the user before initially displaying the one or more elements on the display component of the electronic device; and

determining an initial orientation for the at least one element relative to the display component based on the determined initial orientation of the electronic device relative to the user, wherein initially displaying the one or more elements on the display component of the electronic device includes displaying the at least one element on the display component of the electronic device with the initial orientation determined for the at least one element.

15. The method of claim 1 wherein adjusting the orientation of the at least one displayed element relative to the display component of the electronic device includes:

determining, based on the determined orientation of the electronic device relative to the user, that the at least one displayed element is not appropriately oriented relative to the user; and

adjusting the orientation of the least one displayed element relative to the display component of the electronic device in response to determining that the at least one displayed element is not appropriately oriented relative to the user.

16. The method of claim 1 wherein adjusting an orientation of the at least one displayed element relative to the display component of the electronic device includes automatically adjusting the orientation of the at least one displayed element relative to the display component of the electronic device based on the determined orientation of the electronic device relative to the user.

17. The method of claim 1 wherein determining an orientation of the electronic device relative to the user includes determining an orientation of the electronic device relative to the user without relying on gravity to determine the orientation of the electronic device relative to the user.

18. The method of claim 1 wherein:

initially displaying one or more elements on a display component of an electronic device includes initially displaying a first element that is to be oriented relative to a user and a second element that is to be oriented relative to gravity;

adjusting an orientation of at least one of the displayed elements relative to the display component of the electronic device includes;

determining, based on the determined orientation of the electronic device relative to the user, that the first element is not appropriately oriented relative to the user, and

adjusting the orientation of the first element relative to the display component of the electronic device in response to determining that the at least one displayed element is not appropriately oriented relative to the user; and

the method further comprises;

determining an orientation of the electronic device relative to gravity,

determining, based on the determined orientation of the electronic device relative to gravity, that the second element is not appropriately oriented relative to gravity, and

adjusting the orientation of the second element relative to the display component of the electronic device in response to determining that the at least one displayed element is not appropriately oriented relative to gravity.

19. The method of claim 18 wherein:

initially displaying a first element that is to be oriented relative to a user includes initially displaying a first element that displays text; and

initially displaying a second element that is to be oriented relative to gravity includes initially displaying a second element that displays a video.

20. A method for orienting elements displayed on a display component of an electronic device, the method comprising:

displaying one or more elements on a display component of an electronic device;

capturing one or more images of a user of the electronic device;

monitoring orientation of the electronic device relative to the user based on the one or more captured images of the user of the electronic device; and

controlling display on the display component of at least one of the displayed elements based on results of monitoring the orientation of the electronic device relative to the user.

21. The method of claim 20 wherein:

monitoring the orientation of the electronic device relative to the user includes detecting a change in the orientation of the electronic device relative to the user from an earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element on the display component includes;

in response to detecting the change in orientation of the electronic device relative to the user, determining an appropriate change to an orientation of the at least one displayed element relative to the display component based on the detected change in the orientation of the electronic device relative to the user, and

effecting the change to the orientation of the at least one displayed element relative to the display component that was determined to be appropriate based on the detected change in the orientation of the electronic device relative to the user.

22. The method of claim 21 wherein:

detecting a change in the orientation of the electronic device relative to the user from the earlier orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user rotated substantially positive ninety degrees from the earlier orientation of the electronic device relative to the user;

determining the appropriate change to the orientation of the at least one displayed element relative to the display component includes determining to rotate the orientation of the at least one displayed element substantially negative ninety degrees relative to the display component in response to determining that the orientation of the electronic device relative to the user rotated substantially positive ninety degrees from the earlier orientation of the electronic device relative to the user; and

effecting the change to the orientation of the at least one displayed element relative to the display component that was determined to be appropriate includes rotating the orientation of the at least one displayed element substantially negative ninety degrees relative to the display component.

23. The method of claim 22 wherein:

monitoring the orientation of the electronic device relative to the user further includes determining that the orientation of the electronic device relative to the user rotated another substantially positive ninety degrees after determining that the orientation of the electronic device relative to the user rotated substantially positive ninety degrees from the earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element on the display component further includes rotating the orientation of the at least one displayed element another substantially ninety degrees relative to the display component after rotating the orientation of the at least one element substantially negative ninety degrees relative to the display component of the electronic device in response to determining that the orientation of the electronic device relative to the user rotated another substantially positive ninety degrees.

24. The method of claim 21 wherein:

detecting a change in the orientation of the electronic device relative to the user from the earlier orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user rotated substantially positive one-hundred and eighty degrees from the earlier orientation of the electronic device relative to the user;

determining the appropriate change to the orientation of the at least one displayed element relative to the display component includes determining to rotate the orientation of the at least one displayed element substantially negative one-hundred and eighty degrees relative to the display component in response to determining that the orientation of the electronic device relative to the user rotated substantially positive one-hundred and eighty degrees from the earlier orientation of the electronic device relative to the user; and

effecting the change to the orientation of the at least one displayed element relative to the display component that was determined to be appropriate includes rotating the orientation of the at least one displayed element substantially negative one-hundred and eighty degrees relative to the display component.

25. The method of claim 21 wherein:

detecting a change in the orientation of the electronic device relative to the user from the earlier orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user rotated a number of degrees relative to the user that is in a range between substantially zero degrees and substantially ninety degrees in a positive direction from the earlier orientation of the electronic device relative to the user;

determining the appropriate change to the orientation of the at least one displayed element relative to the display component includes determining to rotate the orientation of the at least one displayed element substantially the number of degrees in the range between substantially zero degrees and substantially ninety degrees in a negative direction relative to the display component in response to determining that the orientation of the electronic device relative to the user rotated the number of degrees relative to the user in the range between substantially zero degrees and substantially ninety degrees; and

effecting the change to the orientation of the at least one displayed element relative to the display component includes rotating the orientation of the at least one displayed element substantially the number of degrees in the range between substantially zero degrees and substantially ninety degrees in a negative direction relative to the display component.

26. The method of claim 20 wherein:

monitoring the orientation of the electronic device relative to the user includes determining that the orientation of the electronic device relative to the user has not changed from an earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element relative to the user includes maintaining an orientation of the at least one displayed element relative to the display component based on determining that the orientation of the electronic device relative to the user has not changed from an earlier orientation of the electronic device relative to the user.

27. The method of claim 20 wherein:

monitoring the orientation of the electronic device relative to the user includes detecting a change in the orientation of the electronic device relative to the user from an earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element on the display component includes, in response to detecting the change in orientation of the electronic device relative to the user, hiding the at least one displayed element by removing the at least one displayed element from the display component.

28. The method of claim 20 wherein:

monitoring the orientation of the electronic device relative to the user includes detecting a change in the orientation of the electronic device relative to the user from an earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element on the display component includes, in response to detecting the change in orientation of the electronic device relative to the user, maintaining, on the display component, a first displayed element and a second displayed element and changing, on the display component, a position and orientation of the first displayed element relative to the second displayed element.

29. The method of claim 20 wherein:

monitoring the orientation of the electronic device relative to the user includes detecting a change in the orientation of the electronic device relative to the user from an earlier orientation of the electronic device relative to the user; and

controlling the display of the at least one displayed element on the display component includes, in response to detecting the change in orientation of the electronic device relative to the user, reformatting text displayed on the display component, including at least one of changing a font size of the text displayed on the display component and maintaining a subset of the text displayed on the display component while removing a remainder of the text displayed on the display component that is not included in the subset.

30. A method for orienting elements displayed on a display component of an electronic device, the method comprising:

capturing an image of a user of an electronic device that has a display component;

based on the captured image of the user of the electronic device, determining an initial orientation of the electronic device relative to the user of the electronic device;

before displaying an element on a display component of the electronic device, determining an initial orientation for displaying the element relative to the display component of the electronic device based on the determined initial orientation of the electronic device relative to the user of the electronic device; and

displaying the element on the display component of the electronic device with the determined initial orientation for displaying the element relative to the display component.