ELECTRONIC DEVICE WITH TOUCH INPUT DISPLAY SYSTEM USING HEAD-TRACKING TO REDUCE VISIBLE OFFSET FOR USER INPUT

• US 20140145966A1
• Filed: 11/27/2012
• Published: 05/29/2014
• Est. Priority Date: 11/27/2012
• Status: Abandoned Application

• Current Assignee: Research In Motion Limited (Blackberry Limited)
• Original Assignee: Research In Motion Limited (Blackberry Limited)
• Inventors: GARDENFORS, Dan Zacharias
• Application Number:
  US13/686,476
• Publication Number:
  US 20140145966A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment

Abstract

An electronic device, computer readable storage medium, and method use head tracking to reduce visible offset for user input at a touch input display. Touch coordinates indicate a location on an outer surface of a touch sensitive screen layer of the touch input display at which a contact interaction is detected. Based on image information from at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer is determined. A location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the direction of a user'"'"'s perspective field of view is determined. One or more display elements are displayed at the determined location on the one of the at least one display layer.

Claims

1. An electronic device comprising:

at least one processor/controller;

memory storing computer instructions;

a touch input display comprising;

a translucent layer;

a translucent touch sensitive screen layer communicatively coupled to the processor/controller and disposed above the translucent layer, the touch sensitive screen layer having an outer surface; and

at least one display layer disposed adjacent to the translucent layer and a given distance below the translucent layer, for rendering an image visible above the touch sensitive screen layer;

at least one camera communicatively coupled to the at least one processor/controller and positioned to have a field of view directed toward a region above the touch sensitive screen layer; and

in response to executing the computer instructions, the at least one processor performing operations comprising;

receiving a touch coordinate signal from the touch input display, the touch coordinate signal representing touch coordinates of a location on the outer surface of the touch sensitive screen layer at which a contact interaction is detected;

determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, one or more display elements at the determined location on the at least one display layer.

2. The electronic device of claim 1, wherein the at least one processor/controller, responsive to executing the computer instructions, performs operations comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on the one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the one of the at least one display layer.

3. The electronic device of claim 2, wherein the at least one processor/controller, responsive to executing the computer instructions, performs operations comprising:

associating the one or more display elements that are displayed at the determined location on the one of the at least one display layer with one or more other display elements displayed at the at least one display layer as a collection of associated display image elements;

determining, based at least on the new determined direction of a user'"'"'s perspective field of view of the location corresponding to the touch coordinates on the touch sensitive screen layer, a new respective location on the at least one display layer for respective each of the one or more other display image elements of the collection of associated display image elements, the new respective location visibly tracking the new determined location on the one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering re-display of the respective each of the one or more other display image elements on the at least one display layer at the new respective location thereby maintaining visible integrity of the collection of associated display image elements displayed on the at least one display layer.

4. The electronic device of claim 1, wherein:

the at least one display layer comprises a first display layer disposed adjacent to a second display layer, the second display layer being disposed a greater distance below the touch sensitive screen layer than the first display layer, the first display layer being at least partially translucent and permitting an image displayed on the second display layer to be visible above the first display layer and therethrough visible above the touch sensitive screen layer; and

the at least one processor/controller, responsive to executing the computer instructions, performs operations comprising;

selecting one of the first display layer and the second display layer;

determining a location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, one or more display elements at the determined location on the selected one of the first display layer and the second display layer.

5. The electronic device of claim 4, wherein the at least one processor/controller, responsive to executing the computer instructions, performs operations comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the selected one of the first display layer and the second display layer.

6. The electronic device of claim 1, wherein the one or more display elements comprise one or more pixels.

7. The electronic device of claim 1, wherein the determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view, comprises:

geometrically calculating the location on the one of the at least one display layer to visibly match the location of the touch coordinates on the touch sensitive screen layer.

8. The electronic device of claim 1, wherein the electronic device comprises at least one of:

a Personal Digital Assistant (PDA), a smart-phone, a cellular telephone, a mobile phone, a tablet computer, and a laptop PC.

9. The electronic device, wherein the touch input display comprises a touchscreen graphical display for the electronic device.

10. A computer readable storage medium comprising computer instructions which, responsive to being executed by a processor communicatively coupled with a touch input display and at least one camera, cause the processor to perform operations comprising:

receiving, with the processor, a touch coordinate signal from the touch input display, the touch input display comprising a touch sensitive screen layer disposed on a translucent layer which is disposed on at least one display layer, the touch coordinate signal representing touch coordinates of a location on the outer surface of the touch sensitive screen layer at which a contact interaction is detected;

determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, one or more display elements at the determined location on one of the at least one display layer.

11. The computer readable storage medium of claim 10, comprising computer instructions which, responsive to being executed by the processor, performs operations comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on the one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the one of the at least one display layer.

12. The computer readable storage medium of claim 11, comprising computer instructions which, responsive to being executed by the processor, performs operations comprising:

associating the one or more display elements that are displayed at the determined location on the one of the at least one display layer with one or more other display elements displayed at the at least one display layer as a collection of associated display image elements;

determining, based at least on the new determined direction of a user'"'"'s perspective field of view of the location corresponding to the touch coordinates on the touch sensitive screen layer, a new respective location on the at least one display layer for respective each of the one or more other display elements of the collection of associated display image elements, the new respective location visibly tracking the new determined location on the one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering re-display of the respective each of the one or more other display elements on the at least one display layer at the new respective location thereby maintaining visible integrity of the collection of associated display image elements displayed on the at least one display layer.

13. The computer readable storage medium of claim 10, wherein the at least one display layer comprises a first display layer disposed on a second display layer, the second display layer being disposed a greater distance below the touch sensitive screen layer than the first display layer, the first display layer being at least partially translucent and permitting an image displayed on the second display layer to be visible above the first display layer and therethrough visible above the touch sensitive screen layer, and wherein the computer readable storage medium comprising computer instructions which, responsive to being executed by the processor, performs operations comprising:

selecting one of the first display layer and the second display layer;

determining a location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, one or more display elements at the determined location on the selected one of the at least one display layer.

14. The computer readable storage medium of claim 13, comprising computer instructions which, responsive to being executed by the processor, performs operations comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the selected one of the first display layer and the second display layer.

15. The computer readable storage medium of claim 10, wherein the one or more display elements comprise one or more pixels.

16. The computer readable storage medium of claim 10, wherein the determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view, comprises:

geometrically calculating the location on the one of the at least one display layer to visibly match the location of the touch coordinates on the touch sensitive screen layer.

17. A method, with an electronic device comprising a processor communicatively coupled with a touch input display and at least one camera, the touch input display comprising a touch sensitive screen layer, and a translucent layer disposed above and adjacent to at least one display layer, the method comprising:

receiving, with the processor, a touch coordinate signal from the touch input display, the touch coordinate signal representing touch coordinates of a location on the outer surface of the touch sensitive screen layer at which a contact interaction is detected;

determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, one or more display elements at the determined location on the one of the at least one display layer.

18. The method of claim 17, further comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the one of the at least one display layer.

19. The method of claim 18, further comprising:

associating the one or more display elements that are displayed at the determined location on the one of the at least one display layer with one or more other display elements displayed at the at least one display layer as a collection of associated display image elements;

determining, based at least on the new determined direction of a user'"'"'s perspective field of view of the location corresponding to the touch coordinates on the touch sensitive screen layer, a new respective location on the at least one display layer for respective each of the one or more other display elements of the collection of associated display image elements, the new respective location visibly tracking the new determined location on the one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering re-display of the respective each of the one or more other display elements on the at least one display layer at the new respective location thereby maintaining visible integrity of the collection of associated display image elements displayed on the at least one display layer.

20. The method of claim 17, wherein the at least one display layer comprises a first display layer disposed adjacent to a second display layer, the second display layer being disposed a greater distance below the touch sensitive screen layer than the first display layer, the first display layer being at least partially translucent and permitting an image displayed on the second display layer to be visible above the first display layer and therethrough visible above the touch sensitive screen layer, and the method comprising:

selecting one of the first display layer and the second display layer;

determining a location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer, display of one or more display elements at the determined location on the selected one of the at least one display layer.

21. The method of claim 20, further comprising:

continuously determining, based on at least image information from the at least one camera and the touch coordinates, a direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer at which a contact interaction is detected;

determining, based at least on the continuously determined direction of a user'"'"'s perspective field of view being a new determined direction of a user'"'"'s perspective field of view of the location on the touch sensitive screen layer, a new location on the selected one of the first display layer and the second display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the new determined direction of a user'"'"'s perspective field of view; and

rendering, based at least on detecting the contact interaction at the location on the touch sensitive screen layer and on the new determined direction of a user'"'"'s perspective field of view, re-display of the one or more display elements at the new location on the selected one of the first display layer and the second display layer.

22. The method of claim 17, wherein the determining a location on one of the at least one display layer that visibly matches the location corresponding to the touch coordinates on the touch sensitive screen layer along the determined direction of a user'"'"'s perspective field of view, comprises:

geometrically calculating the location on the one of the at least one display layer to visibly match the location of the touch coordinates on the touch sensitive screen layer.