Terminal apparatus, display system, display method, and recording medium for switching between pointer mode and touch-panel mode based on handheld activation

• US 9,128,562 B2
• Filed: 05/23/2013
• Issued: 09/08/2015
• Est. Priority Date: 05/25/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Mobile Communications Incorporated (Sony Group Corp.)
• Inventors: Tokutake, Kenji
• Primary Examiner(s): Bost, Dwayne
• Assistant Examiner(s): Ritchie, Darlene M
• Application Number:
  US13/900,586
• Publication Number:
  US 20130321309A1
• Time in Patent Office: 838 Days
• Field of Search: 345173-175, 345/419, 345157-159, 345/179, 345/660, 348/333.01, 348/552, 455/566, 4555561-5562, 725/34, 340/12.22, 382/100, 206/320, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4222 Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy remote control device for a television receiver integrated into a mobile phone H04M1/72415
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

To improve ease of operation in a case where a display device and a small terminal apparatus are connected to each other, and coordinated display is performed. When a contact or approach of an object having a size greater than or equal to a threshold value with or toward the surface of a display panel is detected, a video or an instruction corresponding to the motion detected by the motion sensor is output. The display device, on the basis of the video or the instruction that is output, displays a video in which the position of a pointer or the like is changed in response to the motion detected by a motion sensor.

Claims

1. A terminal apparatus comprising:

a display panel arranged on a housing;

a display processing unit that processes an image displayed on the display panel;

an output unit that externally outputs the image processed by the display processing unit or an instruction regarding the image to an external device;

a touch-panel that detects a contact or approach of an object with or toward a surface of the display panel;

a motion sensor that detects a motion of the housing; and

a control unit having circuitry that when the touch-panel detects the contact or approach of the object having a greater size than or equal to a predetermined size, controls the output unit to output an image or an instruction corresponding to the motion detected by the motion sensor,

wherein when the control unit detects the contact or approach of the object of greater than or equal to a predetermined size, the control unit deactivates the touch-panel from detecting further contact or approach of the object in response to the output unit being activated to control an image on the external device based on a motion of the housing.

2. The terminal apparatus of claim 1, wherein

the control unit responds to an inclination movement of the housing by performing a tap operation that selects a specific location on the display panel.

3. The terminal apparatus of claim 1, wherein

the control unit responds to an up-and-down or a side-to-side movement of the housing by performing a scrolling operation on the display panel.

4. The terminal apparatus of claim 1, wherein when the control unit detects the contact or approach of the object of greater than or equal to the predetermined size, the control unit activates the motion sensor to detect the motion of the housing.

5. The terminal apparatus of claim 4, wherein

the control unit identifies a movement of the housing that is not a swinging movement as corresponding to a tap operation that selects a specific location on the display panel.

6. The terminal apparatus of claim 4, wherein the control unit identifies a swing of the housing as being a scrolling operation on the display panel when a movement speed of the housing is not smaller than or equal to a threshold value.

7. The terminal apparatus of claim 4, wherein the control unit identifies a swing of the housing as being a request to move a pointer position on the display panel when a movement speed of the housing is smaller than or equal to a threshold value.

8. The terminal apparatus of claim 7, wherein the control unit causes the pointer position to move by an amount that corresponds to a movement amount of the housing.

9. The terminal apparatus of claim 8, wherein

after the pointer position has been moved, the control unit determines that an operation other than movement of the pointer is performed in response to the housing being moved faster than the threshold value.

10. The terminal apparatus of claim 1, further comprising:

a plurality of keys, wherein

when the control unit determines that a user'"'"'s palm is placed on the touch-panel, actuation of the plurality of keys is recognized as a pointing device.

11. The terminal apparatus of claim 10, wherein

the control unit recognizes actuation of a predetermined key as a left-click operation.

12. The terminal apparatus of claim 10, wherein

the control unit recognizes actuation of a predetermined key as a right-click operation.

13. The terminal apparatus of claim 10, wherein

the control unit recognizes actuation of a predetermined key as a scroll operation.

14. The terminal apparatus of claim 1, wherein

the control unit is configured to erase the image of the display panel and insert an image of an electronic mail creation screen.

15. The terminal apparatus of claim 14, wherein

the control unit outputs the instruction to create an e-mail message in response to motions detected by the motion sensor.

16. A display method comprising:

displaying an image on a display panel arranged on a housing;

processing with a display processing unit the image displayed on the display panel;

externally outputting the image processed by the display processing unit or an instruction regarding the image to an external device;

detecting on a touch-panel a contact or approach of an object with or toward a surface of the display panel;

detecting with a motion sensor a motion of the housing; and

performing with a control unit having circuitry when the touch-panel detects the contact or approach of the object having a greater size than or equal to a predetermined size, a control of the output unit to output an image or an instruction corresponding to the motion detected by the motion sensor,

wherein when the control unit detects the contact or approach of the object of greater than or equal to a predetermined size, the control unit deactivates the touch-panel from detecting further contact or approach of the object in response to the output unit being activated to control an image on the external device based on a motion of the housing.

17. The method of claim 16, wherein

the detecting includes detecting the contact or approach of the object of greater than or equal to the predetermined size and activating the motion sensor to detect the motion of the housing.

18. A non-transitory computer readable storage device having computer readable instruction that when executed by processing circuitry cause the processing circuitry to implement a display method, the method comprising:

displaying an image on a display panel arranged on a housing;

processing with a display processing unit the image displayed on the display panel;

externally outputting the image processed by the display processing unit or an instruction regarding the image to an external device;

detecting on a touch-panel a contact or approach of an object with or toward a surface of the display panel;

detecting with a motion sensor a motion of the housing; and

performing with a control unit having circuitry when the touch-panel detects the contact or approach of the object having a greater size than or equal to a predetermined size, a control of the output unit to output an image or an instruction corresponding to the motion detected by the motion sensor,

wherein when the control unit detects the contact or approach of the object of greater than or equal to a predetermined size, the control unit deactivates the touch-panel from detecting further contact or approach of the object in response to the output unit being activated to control an image on the external device based on a motion of the housing.