Method, apparatus and computer program product for joining the displays of multiple devices

• US 9,483,225 B2
• Filed: 11/12/2015
• Issued: 11/01/2016
• Est. Priority Date: 04/08/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Kernen, Jaakko, Korhonen, Hannu, Lucero, Andres
• Primary Examiner(s): LEGGETT, ANDREA C.
• Application Number:
  US14/939,494
• Publication Number:
  US 20160062724A1
• Time in Patent Office: 355 Days
• Field of Search: 715/702, 715/769, 715/761
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G09G 2356/00 Detection of the display position w.r.t. other display screens
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector

Abstract

A method is provided for enhancing a user interface with a mobile device by joining the displays of multiple devices together to function together with one another. In particular, example methods may provide for detecting a first touch or drag event, receiving a message indicating a second touch or drag event from a nearby device occurring during the first touch or drag event, creating a join event, and providing for transmission of the join event to the nearby device. The method may also include updating a registry of nearby devices in response to receiving a message regarding a touch or drag event from a nearby device. The method may further include providing for presentation of content on a display of a device and direction of content to be presented on a display of the nearby device, wherein the content to be presented on the display of the nearby device is related to the content on the display of the device.

Claims

1. A method comprising:

detecting a first drag event that occurs on a display of a device;

receiving a message indicating that a second drag event has occurred on a display of a nearby device during the first drag event;

determining a relative orientation of the display of the device and the display of the nearby device on the basis of the first drag event and the second drag event corresponding with a user performing a pinching motion; and

enabling the display of the device and the display of the nearby device to be joined in dependence on the determined relative orientation of the display of the device and the display of the nearby device.

2. The method according to claim 1, wherein the display of the device and the display of the nearby device are joined according to join event rules, created by the device, that depend on the determined relative orientation.

3. The method according to claim 1, further comprising:

expanding information presented on the display of the device, during the first drag event, to the display of the nearby device.

4. The method according to claim 1, wherein the join event rules include data regarding content to be displayed on the nearby device.

5. The method according to claim 1, wherein detecting the first drag event comprises determining a drag event direction.

6. The method according to claim 1, further comprising:

providing for presentation of content on the display of the device; and

providing for direction of content to be presented on the display of the nearby device, wherein the content to be presented on the display of the nearby device is related to the content presented on the display of the device.

7. The method according to claim 1, wherein the display of the device and the display of the nearby device are joined differently when the display of the nearby device is oriented parallel to the display of the device than when the display of the nearby device is oriented perpendicular to the display of the device.

8. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

detect a first drag event that occurs on a display of a device;

receive a message indicating that a second drag event has occurred on a display of a nearby device during the first drag event;

determine a relative orientation of the display of the device and the display of the nearby device on the basis of the first drag event and the second drag event corresponding with a user performing a pinching motion; and

enable the display of the device and the display of the nearby device to be joined in dependence on the determined relative orientation of the display of the device and the display of the nearby device.

9. The apparatus according to claim 8, wherein the first device is the apparatus or the first device comprises the apparatus.

10. The apparatus according to claim 8, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to join the display of the device and the display of the nearby device according to join event rules, created by the device, that depend on the determined relative orientation.

11. The apparatus according to claim 8, wherein the at least one memory and computer program code are configured to, with the at least one processor, cause the apparatus to:

expand information presented on the display of the device, during the first drag event, to the display of the nearby device.

12. The apparatus according to claim 8, wherein the join event rules include data regarding content to be displayed on the nearby device.

13. The apparatus according to claim 8, wherein detecting the first drag event comprises determining a drag event direction.

14. The apparatus according to claim 8, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to provide for presentation of content on the display of the device and provide for direction of content to be presented on the display of the nearby device, wherein the content to be presented on the display of the nearby device is related to the content presented on the display of the device.

15. The apparatus according to claim 8, wherein detecting the first drag event comprises detecting the direction of a touch moving from a first position to a second position on a touch-sensitive display.

16. The apparatus according to claim 8, wherein the display of the device and the display of the nearby device are joined differently when the display of the nearby device is oriented parallel to the display of the device than when the display of the nearby device is oriented perpendicular to the display of the device.

17. A non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions including instructions for causing at least the following to be performed:

detecting a first drag event that occurs on a display of a device;

receiving a message indicating that a second drag event has occurred on a display of a nearby device during the first drag event;

determining a relative orientation of the display of the device and the display of the nearby device on the basis of the first drag event and the second drag event corresponding with a user performing a pinching motion; and

enabling the display of the device and the display of the nearby device to be joined in dependence on the determined relative orientation of the display of the device and the display of the nearby device.

18. The non-transitory computer-readable storage medium according to claim 17, wherein the display of the device and the display of the nearby device are joined according to join event rules, created by the device, that depend on the determined relative orientation.

19. The non-transitory computer-readable storage medium according to claim 17, further comprising:

expanding information presented on the display of the device, during the first drag event, to the display of the nearby device.

20. The non-transitory computer-readable storage medium according to claim 17, wherein the join event rules include data regarding content to be displayed on the nearby device.