Method and apparatus for area-efficient graphical user interface

• US 9,207,864 B2
• Filed: 01/19/2012
• Issued: 12/08/2015
• Est. Priority Date: 02/10/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.), Sony Corporation (Sony Group Corp.)
• Inventors: Funabashi, Yoshimitsu, Kunieda, Shunsuke
• Primary Examiner(s): To, Jennifer
• Assistant Examiner(s): Sundar, Sunil
• Application Number:
  US13/353,812
• Publication Number:
  US 20130024806A1
• Time in Patent Office: 1,419 Days
• Field of Search: 715/781, 715/863, 715/771
• US Class Current: 1/1
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/426 involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

A user operates a GUI with a thumb and an index finger while holding an information processing device with left and right hands. The GUI screen image displays a directional keys input area at the lower left on the screen image, a button input area at the lower right on the screen image, an L1/L2 button input area at the upper left on the screen image, and an R1/R2 button input area at the upper right on the screen image. The L1/L2 and the R1/R2 button input areas are in the form of circle sectors, whereby the center angles of the circle sectors are at right angles of the two upper corners of the screen, and by internally dividing the center angles further so as to divide the sectors into two sectors respectively, the L1 and L2 buttons, and the R1 and R2 buttons are distinguished.

Claims

1. An input device comprising:

a display device placed with an information processing device in a housing in an integrated fashion; and

a touch panel covering the display, the input device converting information on a point of contact with a finger or thumb of a hand that is detected by the touch panel into an operation made by a user and allows the information processing device to process the operation, wherein;

the display device displays a GUI (Graphical User Interface) screen image for user input operation as an on-screen display, wherein the GUI screen image includes at least a first GUI graphic and a second GUI graphic, representing at least a first GUI operation area and a second GUI operation area, and an output image generated as a result of information processing performed by the information processing device, such that a first GUI display mode is provided when the housing is in a vertically long orientation and a second GUI display mode is provided when the housing is in a horizontally long orientation,

the first GUI display mode provides the GUI screen image for user input operation and the output image in separate, non-overlapping areas of a display screen,

the second GUI display mode provides the GUI screen image for user input operation by at least partially overlapping the GUI screen image and the output image on the display screen, and respective GUI graphics being disposed in the GUI screen image in correspondence with a position of an index finger and a thumb of a user when the user holds the housing from both sides so that an index finger and a thumb are placed on the upper surface of the display device, opposite a middle finger that supports the housing from underneath,

at least one of the GUI operation areas included in the GUI screen image in the second GUI display mode is a combined GUI image area formed by combining at least first and second GUI graphics representing at least first and second GUI operation areas, respectively, from the GUI screen image in the first GUI display mode, wherein;

(i) the second GUI graphic includes a directional keys graphic that corresponds to the second GUI operational area, which includes directional keys, wherein a plurality of direction instruction keys are disposed on the circumference of a circle, (ii) the first GUI graphic includes a joystick graphic that corresponds to the first GUI operational area, which includes a pseudo-joystick receiving an input of an arbitrary direction and an arbitrary quantity, and (iii) the joystick graphic is disposed at the center of the directional keys graphic in the combined GUI image area,

when the user selects the first GUI graphic within the combined GUI image area, the first GUI graphic representing the first GUI operation area is displayed in place of the combined GUI image area, and

when the user selects the second GUI graphic within the combined GUI image area, the second GUI graphic representing the second GUI operation area is displayed in place of the combined GUI image area.

2. The input device according to claim 1, wherein, in the second GUI display mode, among the GUI operation areas, a GUI operation area disposed at the position of an index finger has a circle sector shape that corresponds to a range of movement of an upper part from a joint of the index finger.

3. The input device according to claim 2, wherein the GUI operation area disposed at the position of the index finger receives an individual input for each of a plurality of circular sector shaped areas formed by internally dividing the center angle of the circle sector.

4. The input device according to claim 3, wherein the plurality of circular sector shaped areas are formed so that the smaller an angle of bending produced by the index finger that operates an area is, the smaller the center angle of the area is.

5. The input device according to claim 1, wherein, in the second GUI display mode, among the GUI operation areas, a GUI operation area displayed at the position of an index finger is a GUI graphic including a plurality of buttons laid out so as to form an arc shape that corresponds to a range of movement of an upper part from a joint of the index finger.

6. The input device according to claim 5, wherein the plurality of buttons are laid out so that the smaller an angle of bending produced by the index finger that operates a button is, the larger the spacing between the button and an adjacent buttons is.

7. The input device according to claim 1, wherein, in the second GUI display mode, among the GUI operation areas, a GUI operation area disposed at the position of a thumb is

an operating buttons graphic where a plurality of buttons are disposed on a circumference of a further circle.

8. The input device according to claim 7, wherein circles corresponding to the circumferences are displayed in a partially translucent manner for the directional keys graphic and the operating buttons graphic, respectively.

9. The input device according to claim 7, wherein

the GUI operation area for specifying the direction keys graphic detects an operation made on each direction instruction key by a detection area surrounding the direction keys graphic of the direction instruction key, and

an area where detection areas for adjacent direction instruction keys overlap with each other is defined as an detection area for an operation in a direction between the two directions corresponding to the adjacent direction instruction keys.

13. The input device according to claim 1, wherein the display device further displays a configuration screen image to set which orientation of the housing the user holds, vertically long or horizontally long, when the user requests for the display, and switches positional relationship between the output image and the GUI screen according to the user'"'"'s setting to the configuration screen.

14. The input device according to claim 1, wherein a first number of GUI operation areas is included in the GUI screen image in the first GUI display mode, a second number of GUI operation areas is included in the GUI screen image in the second GUI display mode, and the first number is different from the second number.

15. The input device according to claim 1, wherein at least one of the GUI operation areas of the GUI screen image in the first GUI display mode is of a differing shape as compared to a corresponding at least one of the GUI operation areas of the GUI screen image in the second GUI display mode.

10. An information processing device comprising:

an information processing unit operative to perform information processing in accordance with an operation made by a user;

a display device operative to display an output image generated as a result of information processing performed in the information processing device, the upper surface of the display device being covered by a touch panel; and

a housing operative to store the information processing device and the display device in an integrated fashion, wherein;

the display device displays a GUI (Graphical User Interface) screen image for user input operation as an on-screen display, wherein the GUI screen image includes at least a first GUI graphic and a second GUI graphic, representing at least a first GUI operation area and a second GUI operation area, and an output image generated as a result of information processing performed by the information processing device, such that a first GUI display mode is provided when the housing is in a vertically long orientation and a second GUI display mode is provided when the housing is in a horizontally long orientation,

the first GUI display mode provides the GUI screen image for user input operation and the output image in separate, non-overlapping areas of a display screen,

the second GUI display mode provides the GUI screen image for user input operation by at least partially overlapping the GUI screen image and the output image on the display screen, and respective GUI graphics being disposed in the GUI screen image in correspondence with a position of an index finger and a thumb of a user when the user holds the housing from both sides so that an index finger and a thumb are placed on the upper surface of the display device, opposite a middle finger that supports the housing from underneath,

at least one of the GUI operation areas included in the GUI screen image in the second GUI display mode is a combined GUI image area formed by combining at least first and second GUI graphics representing at least first and second GUI operation areas, respectively, from the GUI screen image in the first GUI display mode, wherein;

(i) the second GUI graphic includes a directional keys graphic that corresponds to the second GUI operational area, which includes directional keys, wherein a plurality of direction instruction keys are disposed on the circumference of a circle, (ii) the first GUI graphic includes a joystick graphic that corresponds to the first GUI operational area, which includes a pseudo-joystick receiving an input of an arbitrary direction and an arbitrary quantity, and (iii) the joystick graphic is disposed at the center of the directional keys graphic in the combined GUI image area,

when the user selects the first GUI graphic within the combined GUI image area, the first GUI graphic representing the first GUI operation area is displayed in place of the combined GUI image area, and

when the user selects the second GUI graphic within the combined GUI image area, the second GUI graphic representing the second GUI operation area is displayed in place of the combined GUI image area, and

the information processing unit converts information on a point of contact with a finger or thumb of a hand on the GUI that is detected by the touch panel into an operation made by a user and performs information processing.

11. An input value acquiring method comprising:

providing a display device with an information processing device in a housing in an integrated fashion and a touch panel covering the display;

converting information on a point of contact with a finger or thumb of a hand that is detected by the touch panel into an operation made by a user;

allowing the information processing device to perform information processing in accordance with the operation;

displaying a GUI (Graphical User Interface) screen image for user input operation as an on-screen display, wherein the GUI screen image includes at least a first GUI graphic and a second GUI graphic, representing at least a first GUI operation area and a second GUI operation area, and an output image generated as a result of information processing performed by the information processing device, such that a first GUI display mode is provided when the housing is in a vertically long orientation and a second GUI display mode is provided when the housing is in a horizontally long orientation, wherein;

the first GUI display mode provides the GUI screen image for user input operation and the output image in separate, non-overlapping areas of a display screen,

the first GUI display mode provides the GUI screen image for user input operation and the output image in separate, non-overlapping areas of a display screen,

the second GUI display mode provides the GUI screen image for user input operation by at least partially overlapping the GUI screen image and the output image on the display screen, and respective GUI graphics being disposed in the GUI screen image in correspondence with a position of an index finger and a thumb of a user when the user holds the housing from both sides so that an index finger and a thumb are placed on the upper surface of the display device, opposite a middle finger that supports the housing from underneath,

at least one of the GUI operation areas included in the GUI screen image in the second GUI display mode is a combined GUI image area formed by combining at least first and second GUI graphics representing at least first and second GUI operation areas, respectively, from the GUI screen image in the first GUI display mode, wherein;

(i) the second GUI graphic includes a directional keys graphic that corresponds to the second GUI operational area, which includes directional keys, wherein a plurality of direction instruction keys are disposed on the circumference of a circle, (ii) the first GUI graphic includes a joystick graphic that corresponds to the first GUI operational area, which includes a pseudo-joystick receiving an input of an arbitrary direction and an arbitrary quantity, and (iii) the joystick graphic is disposed at the center of the directional keys graphic in the combined GUI image area,

when the user selects the first GUI graphic within the combined GUI image area, the first GUI graphic representing the first GUI operation area is displayed in place of the combined GUI image area, and

when the user selects the second GUI graphic within the combined GUI image area, the second GUI graphic representing the second GUI operation area is displayed in place of the combined GUI image area.

12. A non-transitory computer-readable recording medium containing a computer program, the computer program for interoperating with an information processing device having a display device, a touch panel covering the display device, and a housing in an integrated fashion, the computer program, when executed by a microprocessor causes the information processing device to carry out actions, comprising:

converting information on a point of contact with a finger or thumb of a hand that is detected by the touch panel into an operation made by a user;

performing information processing in accordance with the operation,

displaying a GUI (Graphical User Interface) screen image for user input operation as an on-screen display, wherein the GUI screen image includes at least a first GUI graphic and a second GUI graphic, representing at least a first GUI operation area and a second GUI operation area, and an output image generated as a result of information processing performed by the information processing device, such that a first GUI display mode is provided when the housing is in a vertically long orientation and a second GUI display mode is provided when the housing is in a horizontally long orientation, wherein;

the first GUI display mode provides the GUI screen image for user input operation and the output image in separate, non-overlapping areas of a display screen,

the second GUI display mode provides the GUI screen image for user input operation by at least partially overlapping the GUI screen image and the output image on the display screen, and respective GUI graphics being disposed in the GUI screen image in correspondence with a position of an index finger and a thumb of a user when the user holds the housing from both sides so that an index finger and a thumb are placed on the upper surface of the display device, opposite a middle finger that supports the housing from underneath,

at least one of the GUI operation areas included in the GUI screen image in the second GUI display mode is a combined GUI image area formed by combining at least first and second GUI graphics representing at least first and second GUI operation areas, respectively, from the GUI screen image in the first GUI display mode, wherein;

(i) the second GUI graphic includes a directional keys graphic that corresponds to the second GUI operational area, which includes directional keys, wherein a plurality of direction instruction keys are disposed on the circumference of a circle, (ii) the first GUI graphic includes a joystick graphic that corresponds to the first GUI operational area, which includes a pseudo-joystick receiving an input of an arbitrary direction and an arbitrary quantity, and (iii) the joystick graphic is disposed at the center of the directional keys graphic in the combined GUI image area,

when the user selects the first GUI graphic within the combined GUI image area, the first GUI graphic representing the first GUI operation area is displayed in place of the combined GUI image area, and

when the user selects the second GUI graphic within the combined GUI image area, the second GUI graphic representing the second GUI operation area is displayed in place of the combined GUI image area.