ORIENTING A DISPLAYED ELEMENT RELATIVE TO A USER

• US 20100066667A1
• Filed: 09/14/2009
• Published: 03/18/2010
• Est. Priority Date: 09/12/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: MacDougall, Francis, Hildreth, Evan
• Granted Patent: US 8,686,953 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas

Abstract

An element is initially displayed on an interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device. One or more images of a user of the interactive touch-screen display device are captured. The user is determined to be interacting with the element displayed on the interactive touch-screen display device. In addition, an orientation of the user relative to the interactive touch-screen display device is determined based on at least one captured image of the user of the interactive touch-screen display device. Thereafter, in response to determining that the user is interacting with the displayed element, the initial orientation of the displayed element relative to the interactive touch-screen display device is automatically adjusted based on the determined orientation of the user relative to the interactive touch-screen display device.

Claims

1. A method for automatically adjusting orientations of elements displayed on an interactive touch-screen display device, the method comprising:

initially displaying an element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device;

capturing one or more images of a user of the interactive touch-screen display device;

determining that the user is interacting with the element displayed on the interactive touch-screen display device;

determining an orientation of the user relative to the interactive touch-screen display device based on at least one captured image of the user of the interactive touch-screen display device; and

in response to determining that the user is interacting with the displayed element, automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device.

2. The method of claim 1 wherein:

capturing one or more images of the user of the interactive touch-screen display device includes capturing one or more images of the user that include at least a portion of the user'"'"'s arm after determining that the user is interacting with the displayed element; and

determining the orientation of the user relative to the interactive touch-screen display device includes;

determining an orientation of the user'"'"'s arm relative to the at least one captured image of the user that includes the portion of the user'"'"'s arm, and

determining the orientation of the user relative to the interactive touch-screen display device based on the determined orientation of the user'"'"'s arm relative to the at least one captured image of the user that includes the portion of the user'"'"'s arm.

3. The method of claim 1 wherein:

capturing one or more images of the user of the interactive touch-screen display device includes capturing one or more images of the user that include at least a portion of the user'"'"'s face after determining that the user is interacting with the displayed element; and

determining the orientation of the user relative to the interactive touch-screen display device includes;

determining an orientation of at least one facial feature of the user relative to the at least one captured image of the user that includes the portion of the user'"'"'s face, and

determining the orientation of the user relative to the interactive touch-screen display device based on the determined orientation of the at least one facial feature of the user relative to the at least one captured image of the user that includes the portion of the user'"'"'s face.

4. The method of claim 1 wherein:

capturing one or more images of the user of the interactive touch-screen display device includes capturing one or more images of the user that include at least a portion of the user'"'"'s head after determining that the user is interacting with the displayed element; and

determining the orientation of the user relative to the interactive touch-screen display device includes;

determining an orientation of the user'"'"'s head relative to the at least one captured image of the user that includes the portion of the user'"'"'s head, and

determining the orientation of the user relative to the interactive touch-screen display device based on the determined orientation of the user'"'"'s head relative to the at least one captured image of the user that includes the portion of the user'"'"'s head.

5. The method of claim 1 wherein:

capturing one or more images of the user of the interactive touch-screen display device includes capturing one or more images of the user that include at least a portion of the user'"'"'s body after determining that the user is interacting with the displayed element; and

determining the orientation of the user relative to the interactive touch-screen display device includes;

determining an orientation of the user'"'"'s body relative to the at least one captured image of the user that includes the portion of the user'"'"'s body, and

determining the orientation of the user relative to the interactive touch-screen display device based on the determined orientation of the user'"'"'s body relative to the at least one captured image of the user that includes the portion of the user'"'"'s body.

6. The method of claim 1 wherein:

determining that the user is interacting with the element displayed on the interactive touch-screen display device includes determining that the user moved the displayed element from a first location on the interactive touch-screen display device to a second location on the interactive touch-screen display device; and

automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device includes automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device as well as the second location on the interactive touch-screen display device.

7. The method of claim 1 wherein:

the displayed element has a top and a bottom; and

automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device includes automatically adjusting the initial orientation of the displayed element such that after adjusting the initial orientation of the displayed element the top of the displayed element is above the bottom of the displayed element within a field of vision of the user.

8. The method of claim 1 further comprising:

initially displaying another element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device;

capturing one or more images of another user of the interactive touch-screen display device;

determining that the other user is interacting with the other element displayed on the interactive touch-screen display device;

determining an orientation of the other user relative to the interactive touch-screen display device based on at least one captured image of the other user of the interactive touch-screen display device; and

in response to determining that the other user is interacting with the other displayed element, automatically adjusting the initial orientation of the other displayed element relative to the interactive touch-screen display device based on the determined orientation of the other user relative to the interactive touch-screen display device.

9. The method of claim 1 wherein determining that the user is interacting with the element displayed on the interactive touch-screen display device includes determining that the user is interacting with the element displayed on the interactive touch-screen display device based on at least one captured image of the user of the interactive touch-screen display device.

10. The method of claim 9 wherein determining that the user is interacting with the element displayed on the interactive touch-screen display device includes determining that the user is gesturing in the vicinity of the element displayed on the interactive touch-screen display device based on at least one captured image of the user of the interactive touch-screen display device.

11. The method of claim 1 wherein:

determining an orientation of the user relative to the interactive touch-screen display device includes determining, while the user is interacting with the element displayed on the interactive touch-screen display device, that the orientation of the user relative to the interactive touch-screen display device has changed from a first orientation of the user relative to the interactive touch-screen display device to a second orientation of the user relative to the interactive touch-screen display device; and

automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device includes automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the second orientation of the user relative to the interactive touch-screen display device.

12. The method of claim 1 further comprising:

responsive to the user'"'"'s interaction with the element displayed on the interactive touch-screen display device, displaying another element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device, wherein the initial orientation of the other element relative to the interactive touch-screen display device is based on the determined orientation of the user relative to the interactive touch-screen display device.

13. The method of claim 1 wherein determining that the user is interacting with the element displayed on the interactive touch-screen display device includes determining that the user is moving the element displayed on the interactive touch-screen display device toward the user.

14. The method of claim 1 wherein determining that the user is interacting with the element displayed on the interactive touch-screen display device includes determining that the user is moving the element displayed on the interactive touch-screen display device away from the user.

15. The method of claim 1 further comprising:

determining that the user is moving the element displayed on the interactive touch-screen display device away from the user and toward another user of the interactive touch-screen display device;

capturing one or more images of the user of the interactive touch-screen display device;

determining an orientation of the other user relative to the interactive touch-screen display device based on at least one captured image of the other user of the interactive touch-screen display device; and

in response to determining that the user is moving the element displayed on the interactive touch-screen display device away from the user and toward the other user, automatically adjusting the orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the other user relative to the interactive touch-screen display device.

16. The method of claim 1 wherein automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device includes maintaining the initial orientation of the displayed element relative to the interactive touch-screen display device unchanged based on the determined orientation of the user relative to the interactive touch-screen display device.

17. The method of claim 1 wherein automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device includes changing the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device.

18. A method for automatically adjusting orientations of elements displayed on an interactive touch-screen display device, the method comprising:

initially displaying an element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device;

capturing one or more images of a user of the interactive touch-screen display device;

determining that the user is interacting with the element displayed on the interactive touch-screen display device;

determining an orientation of the user relative to the displayed element based on at least one captured image of the user of the interactive touch-screen display device; and

in response to determining that the user is interacting with the displayed element, automatically adjusting the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the displayed element.

19. A method for automatically adjusting orientations of elements displayed on an interactive touch-screen display device, the method comprising:

capturing one or more images of a user of an interactive touch-screen display device;

determining an orientation of the user relative to the interactive touch-screen display device based on at least one captured image of the user of the interactive touch-screen display device; and

initially displaying an element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device that is based on the determined orientation of the user relative to the interactive touch-screen display device.

20. A system comprising:

an interactive touch-screen display device;

a camera configured to capture images of one or more users of the interactive touch-screen display device; and

a processing unit that is configured to;

cause a display of an element on the interactive touch-screen display device with an initial orientation relative to the interactive touch-screen display device;

determine that a user is interacting with the element displayed on the interactive touch-screen display device;

determine an orientation of the user relative to the interactive touch-screen display device based on at least one image of the user of the interactive touch-screen display device captured by the camera; and

automatically adjust the initial orientation of the displayed element relative to the interactive touch-screen display device based on the determined orientation of the user relative to the interactive touch-screen display device and in response to a determination that the user is interacting with the displayed element.