VEHICULAR MANIPULATION INPUT APPARATUS

• US 20110029185A1
• Filed: 03/18/2009
• Published: 02/03/2011
• Est. Priority Date: 03/19/2008
• Status: Abandoned Application

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Itoh, Masahiro, Aoki, Takashi, Niwa, Shinji, Nagata, Takashi, Akahori, Ichiro, Kawashima, Takeshi
• Application Number:
  US12/866,969
• Publication Number:
  US 20110029185A1
• Time in Patent Office: Days
• US Class Current: 701/29
• CPC Class Codes: B60K 2360/141 Activation of instrument input devices by approaching fingers or pens
• CPC Class Codes: B60K 2360/195 Blocking or enabling display functions
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/212 displaying on manual operation elements, e.g. on a knob
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60R 11/0235 of flat type, e.g. LCD
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2011/0007 Mid-console
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0312 for tracking the rotation of a spherical or circular member, e.g. optical rotary encoders used in mice or trackballs using a tracking ball or in mouse scroll wheels tracking relative movement in co-operation with a regularly or irregularly patterned surface, e.g. as in optical mice G06F3/0317; constructional details of scroll or thumb-wheels G06F3/0362; optical rotary encoders G01D5/3473
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A vehicular manipulation input apparatus for a vehicle includes a manipulation panel, and a display device which displays a position indication image for displaying an indication position on the manipulation panel. It is specified whether a finger state of a hand opposing the manipulation panel is a manipulation intention expression-use finger state. In a condition that the vehicle is in a travel state, a display of the position indication image is permitted when the finger state is specified as the manipulation intention expression-use finger state whereas a display of the position indication image is prohibited when the finger state is not specified as the manipulation intention expression-use finger state.

Claims

1. A vehicular manipulation input apparatus comprising:

a display section including a display window for executing an operation of a device in a vehicle;

a remote manipulation section provided at a position separated from the display window, the remote manipulation section having a remote manipulation area for remotely executing a manipulation input in the display window;

an input acceptance section configured to accept an input corresponding to a press manipulation position based on a press manipulation to the remote manipulation area;

a hand image capture section configured to capture an image of a hand that opposes the remote manipulation area;

a position indication image display section configured to display a position indication image, which is generated based on the captured image of the hand, at a position on the display window, the position being indicated by the hand;

a predetermined finger state specification section configured to specify whether a finger state of the hand opposing the remote manipulation area is a predetermined finger state; and

a display mode assignment section configured to execute a display mode switchover control of a display mode assigned to the position indication image display section,

the display mode switchover control

assigning the display mode with a display permission mode to permit a display of the position indication image when the predetermined finger state is specified, and

assigning the display mode with a display prohibition mode to prohibit a display of the position indication image when the predetermined finger state is not specified.

2. The vehicular manipulation input apparatus according to claim 1,

the predetermined finger state specification section including a simple manipulation-use finger state specification section configured to specify whether a finger state of the hand opposing the remote manipulation area is a predetermined simple manipulation-use finger state,

the display mode assignment section being configured to execute the display mode switchover control of the display mode assigned to the position indication image display section,

the display mode switchover control

assigning a display permission mode to permit a display of the position indication image when the simple manipulation-use finger state is specified, and

assigning a display prohibition mode to prohibit a display of the position indication image when the simple manipulation-use finger state is not specified.

3. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section configured to detect a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state.

4. The vehicular manipulation input apparatus according to claim 2, further comprising:

a manipulator specification section configured to specify whether a manipulator of the remote manipulation area is a driver or not,

the display mode assignment section being configured to

execute the display mode switchover control when the driver is specified as the manipulator of the remote manipulation area, and

assign the display permission mode to the display mode when the driver is not specified as the manipulator of the remote manipulation area.

5. The vehicular manipulation input apparatus according to claim 2, further comprising:

a manipulation input acceptance control section configured to execute a manipulation input acceptance control to

permit an acceptance of an input by the input acceptance section when the display permission mode is assigned by the display mode assignment section, and

prohibit an acceptance of an input by the input acceptance section when the display prohibition mode is assigned.

6. The vehicular manipulation input apparatus according to claim 2, further comprising:

a manipulation input acceptance control section configured to execute a manipulation input acceptance control to

permit an acceptance of an input by the input acceptance section when the simple manipulation-use finger state is specified by the simple manipulation-use finger state specification section, and

prohibit an acceptance of an input by the input acceptance section when the simple manipulation-use finger state is not specified.

7. The vehicular manipulation input apparatus according to claim 2,

the simple manipulation-use finger state specification section including a finger shape recognition section configured to recognize a shape of a finger of a hand which opposes the remote manipulation area based on an image of the hand captured by the hand image capture section,

the simple manipulation-use specification section specifying the simple manipulation-use finger state when the recognized finger shape is a predetermined finger shape for a simple manipulation use.

8. The vehicular manipulation input apparatus according to claim 7,

the simple manipulation-use finger state specification section including a finger number specification section configured to specify a number of fingers, which are in a state to be extended approximately parallel with the remote manipulation area, recognized by the finger shape recognition section,

the simple manipulation-use finger state specification section specifying the simple manipulation-use finger state when the specified number of the fingers in the state to be extended approximately parallel with the remote manipulation area is a predetermined number for the simple manipulation use.

9. The vehicular manipulation input apparatus according to claim 2,

the simple manipulation-use finger state specification section including a manipulation number specification section configured to specify a number of inputs due to press manipulations, which are made to different positions in the remote manipulation area and accepted by the input acceptance section during a predetermined period,

the simple manipulation-use finger state specification section specifying the simple manipulation-use finger state when the number of the inputs specified during the predetermined period is a predetermined number for a simple manipulation use.

10. The vehicular manipulation input apparatus according to claim 8, wherein

the predetermined number for the simple manipulation use is one.

11. The vehicular manipulation input apparatus according to claim 8, wherein

the predetermined number for the simple manipulation use is two.

12. The vehicular manipulation input apparatus according to claim 8, wherein

the predetermined number for the simple manipulation use is three.

13. The vehicular manipulation input apparatus according to claim 8, wherein

the predetermined number for the simple manipulation use is four.

14. The vehicular manipulation input apparatus according to claim 7,

the simple manipulation-use finger state specification section including a finger kind specification section configured to specify a kind of a finger, which is in a state to be extended approximately parallel with the remote manipulation area, recognized by the finger shape recognition section,

the simple manipulation-use finger state specification section specifying the simple manipulation-use finger state when the specified kind of the finger in the state to be extended approximately parallel with the remote manipulation area is a predetermined kind for the simple manipulation use.

15. The vehicular manipulation input apparatus according to claim 14, wherein

the kind of the finger for the simple manipulation use is an index finger.

16. The vehicular manipulation input apparatus according to claim 14, wherein

the kind of the finger for the simple manipulation use is a middle finger.

17. The vehicular manipulation input apparatus according to claim 14, wherein

the kind of the finger for the simple manipulation use is both an index finger and a middle finger.

18. The vehicular manipulation input apparatus according to claim 2,

the position indication image display section being configured to

generate a real finger image, which allows a specification of at least an outline of a finger of a hand, based on an image of a hand captured by the hand image capture section, and

display the generated real finger image as the position indication image.

19. The vehicular manipulation input apparatus according to claim 2,

the position indication image display section being configured to display as the position indication image one of a false finger image and a fingertip position image,

the false finger image allowing a specification of at least a direction of an extended finger and a fingertip position,

the fingertip position image allowing a specification of at least a fingertip position of a finger.

20. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as zero km/h.

21. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as a vehicle velocity range greater than 0 km/h and less than 5 km/h.

22. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as a vehicle velocity range equal to or greater than 5 km/h and less than 10 km/h.

23. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as a vehicle velocity range equal to or greater than 10 km/h and less than 30 km/h.

24. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as a vehicle velocity range equal to or greater than 30 km/h and less than 60 km/h.

25. The vehicular manipulation input apparatus according to claim 2, further comprising:

a vehicle velocity detection section (SO) which detects a vehicle velocity of the vehicle,

the display mode assignment section being configured to

execute the display mode switchover control when the vehicle is in a vehicle travel state exceeding a predetermined vehicle velocity level, and

assign the display permission mode to the display mode when the vehicle is not in the vehicle travel state,

the predetermined vehicle velocity level being defined as a vehicle velocity range equal to or greater than 60 km/h.

26. The vehicular manipulation input apparatus according to claim 2, further comprising:

a manipulation window display section which displays in a switchover manner a plurality of manipulation display windows in which an input due to a press manipulation to the remote manipulation area is accepted by the input acceptance section; and

a display mode execution applicability information storage section which stores display mode execution applicability information which allows a specification of a manipulation display window of an execution target for the display mode switchover control from among the manipulation display windows,

the display mode assignment section being configured to execute the display mode switchover control in the specified manipulation display window only when the manipulation display window displayed in the manipulation window display section is specified as an execution target for the display mode switchover control based on the display mode execution applicability information.

27. The vehicular manipulation input apparatus according to claim 26, wherein

the display mode execution applicability information defines, among the manipulation display windows,

a complicated manipulation display window as an execution target for the display mode switchover control, the complicated manipulation display window where a press manipulation to the remote manipulation area accepted as an input by the input acceptance section includes a manipulation exceeding a predetermined operation load level, and

a simple manipulation window as a fixed assignment target for the display permission mode, the simple manipulation window where a press manipulation to the remote manipulation area accepted as an input by the input acceptance section includes a manipulation not exceeding a predetermined manipulation load level.

28. The vehicular manipulation input apparatus according to claim 27, wherein

the simple manipulation window includes at least one of a menu selection window, a sound volume adjustment window, and an air quantity adjustment window.

29. The vehicular manipulation input apparatus according to claim 27, wherein

the complicated manipulation window includes at least a character input window.

30. The vehicular manipulation input apparatus according to claim 1,

the predetermined finger state specification section including a manipulation intention expression-use finger state specification section configured to specify whether a finger state of the hand opposing the remote manipulation area is a predetermined manipulation intention expression-use finger state,

the display mode assignment section being configured to execute a display mode switchover control,

the display mode switchover control

assigning the display mode, which is assigned to the position indication image display section, with the display prohibition mode to prohibit a display of the position indication image, and

switching, when the manipulation intention expression-use finger state is specified, to the display permission mode to permit the display of the position indication image only during a predetermined display permission period.

31. The vehicular manipulation input apparatus according to claim 30, wherein:

the display mode assignment section defines as the display permission period a period up to a time when a predetermined prohibition mode recovery time elapses since the display mode is switched to the display permission mode; and

the display mode assignment section returns the display mode to the display prohibition mode at the time when the prohibition mode recovery time elapses.

32. The vehicular manipulation input apparatus according to claim 30, wherein:

the display mode assignment section defines as the display permission period a period up to a time when the manipulation intention expression-use finger state becomes not-specified; and

the display mode assignment section

assigns the display permission mode when the manipulation intention expression-use finger state is specified, and

assigns the display prohibition mode when the manipulation intention expression-use finger state is not specified.

33. The vehicular manipulation input apparatus according to claim 30, further comprising:

a manipulation input acceptance control section configured to

execute a manipulation input acceptance control to permit an acceptance of an input by the input acceptance section when the display permission mode is assigned, and

prohibit an acceptance of an input by the input acceptance section when the display prohibition mode is assigned.

34. The vehicular manipulation input apparatus according to claim 30, further comprising:

a manipulation input acceptance control section configured to execute a manipulation input acceptance control to

permit an acceptance of an input by the input acceptance section when the manipulation intention expression-use state is specified, and

prohibit an acceptance of an input by the input acceptance section when the manipulation intention expression-use finger state is not specified.

35. The vehicular manipulation input apparatus according to claim 30,

the manipulation intention expression-use finger state specification section including a finger shape recognition section configured to recognize a shape of a finger of a hand which opposes the remote manipulation area based on an image of the hand captured by the hand image capture section,

the manipulation intention expression-use finger state specification section specifying the manipulation intention expression-use finger state when the recognized finger shape is a predetermined finger shape for a manipulation intention expression use.

36. The vehicular manipulation input apparatus according to claim 35,

the manipulation intention expression-use finger state specification including a finger number specification section configured to specify, when fingers extended approximately parallel with the remote manipulation area are recognized by the finger shape recognition section, a number of fingers in a state of being extended approximately parallel with the remote manipulation area,

the manipulation intention expression-use finger state specification section specifying the manipulation intention expression-use finger when the specified number of fingers in the state of being extended approximately parallel with the remote manipulation area is a predetermined number for a manipulation intention expression use.

37. The vehicular manipulation input apparatus according to claim 30, wherein

the manipulation intention expression-use finger state specification section including a manipulation number specification section configured to specify a number of inputs due to press manipulations, which are made to different positions in the remote manipulation area and accepted by the input acceptance section during a predetermined period,

the manipulation intention expression-use finger state specification section specifying the manipulation intention expression-use finger state when the number of inputs specified during the predetermined period is a predetermined number for a manipulation intention expression use.

38. The vehicular manipulation input apparatus according to claim 36, wherein

the number for the manipulation intention expression use is one.

39. The vehicular manipulation input apparatus according to claim 36, wherein

the number for the manipulation intention expression use is two.

40. The vehicular manipulation input apparatus according to claim 36, wherein

the number for the manipulation intention expression use is three.

41. The vehicular manipulation input apparatus according to claim 36, wherein

the number for the manipulation intention expression use is four.

42. The vehicular manipulation input apparatus according to claim 35,

the manipulation intention expression-use finger state specification including a finger kind specification section configured to specify, when a finger extended approximately parallel with the remote manipulation area is recognized by the finger shape recognition section, a kind of the finger in a state of being extended approximately parallel with the remote manipulation area,

the manipulation intention expression-use finger state specification section specifying the manipulation intention expression-use finger when the specified kind of the finger in the state of being extended approximately parallel with the remote manipulation area is a predetermined kind for a manipulation intention expression use.

43. The vehicular manipulation input apparatus according to claim 42, wherein

the kind of the finger for the manipulation intention expression-use is an index finger.

44. The vehicular manipulation input apparatus according to claim 42, wherein

the kind of the finger for the manipulation intention expression-use is a middle finger.

45. The vehicular manipulation input apparatus according to claim 42, wherein

the kind of the finger for the manipulation intention expression-use is an index finger and a middle finger.

46. The vehicular manipulation input apparatus according to claim 30,

the position indication image display section being configured to

generate a real finger image, which allows a specification of at least an outline of a finger of a hand, based on an image of a hand captured by the hand image capture section, and

display the generated real finger image as the position indication image.

47. The vehicular manipulation input apparatus according to claim 30,

the position indication image display section being configured to display as the position indication image one of a false finger image and a fingertip position image,

the false finger image allowing a specification of at least a direction of an extended finger and a fingertip position,

the fingertip position image allowing a specification of at least a fingertip position of a finger.

48. The vehicular manipulation input apparatus according to claim 1, wherein:

the position indication image display section displays a position indication image, when a finger opposes in front of the remote manipulation section, the position indication image being displayed at an on-window position on the display window corresponding to a position of the finger in such a manner to be combined to or superimposed on an image of a background; and

the predetermined finger state specification section specifies a finger in a hand which opposes in front of the remote manipulation section,

the apparatus further comprising;

a manipulation target finger assignment section configured to assign, from the specified finger, a manipulation target finger, which is used or estimated to be used for a manipulation to the remote manipulation section; and

a position indication image highlight display section configured to

cause the display section to display the position indication image indicating an indication position of the specified finger in such a manner to be combined to or superimposed on an image of a background on the display window, and

cause the display section to execute a highlight display of a position indication image corresponding to the assigned manipulation target finger among the displayed position indication image so as to relatively highlight more in comparison with a position indication image corresponding to a finger different from the manipulation target finger.

49. A method for displaying a position indication image in a display apparatus having a display section with a display window, and a remote manipulation section with a remote manipulation area for executing manipulation to the display window remotely,

the position indication image being displayed, when a finger opposes in front of the remote manipulation section, at an on-window position on the display window corresponding to a position of the finger in such a manner to be combined to or superimposed on an image of a background,

the method comprising:

a predetermined finger state specification step of specifying whether a finger state of the hand opposing the remote manipulation area is a predetermined finger state;

a step of assigning a display permission mode to permit a display of the position indication image when the predetermined finger state is specified; and

a step of assigning a display prohibition mode to prohibit a display of the position indication image when the predetermined finger state is not specified.

50. A method for displaying a position indication image in a display apparatus having a display section with a display window, and a remote manipulation section with a remote manipulation area for executing manipulation to the display window remotely,

the position indication image being displayed, when a finger opposes in front of the remote manipulation section, at an on-window position on the display window corresponding to a position of the finger in such a manner to be combined to or superimposed on an image of a background,

the method comprising:

a finger specification step of specifying a finger in a hand opposing in front of the remote manipulation section;

a manipulation target finger assignment step of assigning, from the specified finger, a manipulation target finger, which is used or estimated to be used for a manipulation to the remote manipulation section;

a display step of displaying the position indication image indicating an indication position of the specified finger in such a manner to be combined to or superimposed on an image of a background on the display window; and

a position indication image highlight step of executing a highlight display of a position indication image corresponding to the assigned manipulation target finger among the displayed position indication image so as to relatively highlight more in comparison with a position indication image corresponding to a finger different from the manipulation target finger.

51. The method according to claim 50, wherein:

the finger specification step is a fingertip specification step of specifying a fingertip from the hand, and a step which specifies the finger in a manner to specify the fingertip.

52. The method according to claim 50, wherein

the manipulation target finger assignment step assigns one finger to the manipulation target finger.

53. The method according to claim 50, wherein

the manipulation target finger assignment step assigns two fingers or three fingers to the manipulation target finger.

54. The method according to claim 50, further comprising:

an assignment finger information acquisition step of acquiring finger shape information concerning a shape of a finger specified by the finger specification step as assignment finger information,

the manipulation target finger assignment step assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger having the finger shape information most suitable for a predetermined selection condition about the finger shape, based on the acquired finger shape information.

55. The method according to claim 54, wherein:

the assignment finger information acquisition step is acquiring, as the finger shape information, finger length information reflecting length relation among the finger specified by the finger specification step in the direction of the remote manipulation area; and

the manipulation target finger assignment step is assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger having the finger shape information most suitable for a predetermined selection condition about the finger length information, based on the acquired finger length information.

56. The method according to claim 54, wherein:

the assignment finger information acquisition step is acquiring, as finger shape information, a finger width information reflecting width relation of the fingers specified by the finger specification step in the direction of the remote manipulation area; and

the manipulation target finger assignment step is assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger having the finger shape information most suitable for a predetermined selection condition about the finger width information, based on the acquired finger width information.

57. The method according to claim 50, further comprising:

an assignment finger information acquisition step of acquiring, as the assignment finger information, position relation information concerning a position relation in the direction of the remote manipulation area among fingers specified by the finger specification step,

the manipulation target finger assignment step assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger having the position relation information most suitable for a predetermined selection condition about the position relation information, based on the acquired position relation information.

58. The method according to claim 57, wherein:

the finger specification step is a fingertip specification step of specifying a fingertip from the hand and a step which specifies the finger in a manner to specify the fingertip;

the assignment finger information acquisition step

projects the fingertip specified by the fingertip specification step in the direction opposing the remote manipulation section,

defines one axis at a predetermined position on a projection plane to define a one-dimensional coordinate system,

specifies a coordinate value of the projected fingertip in the coordinate system, and

acquires an axis coordinates information reflecting the coordinate value as the position relation information; and

the manipulation target finger assignment step is assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger suitable for a predetermined selection condition about the axis coordinate information, based on the acquired axis coordinate information.

59. The method according to claim 57, wherein:

the finger specification step is a fingertip specification step of specifying a fingertip from the hand, and a step which specifies the finger in a manner to specify the fingertip;

the assignment finger information acquisition step

projects the fingertip specified by the fingertip specification step in the direction opposing the remote manipulation section, and

acquires, as the position relation information, distance information reflecting a distance between a reference region defined in a predetermined region on a projection plane and the projected fingertip; and

the manipulation target finger assignment step is assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger suitable for a predetermined selection condition about the distance information, based on the acquired distance information.

60. The method according to claim 50, further comprising:

an assignment finger information acquisition step of acquiring, as assignment finger information, approach distance relation information reflecting an approach distance relation to the remote manipulation area among the finger specified by the finger specification step,

the manipulation target finger assignment step assigning the manipulation target finger, among the finger specified by the finger specification step, preferentially to a finger having a fingertip closer to the remote manipulation area, based on the acquired approach distance information.

61. The method according to claim 60, further comprising:

an image capture step of radiating a light from a light source towards a hand approaching the remote manipulation area, and image-capturing, using an image capture section, a hand from a side of a rear face of the remote manipulation area in a manner to capture a reflection light from the hand,

the assignment finger information acquisition step acquiring a brightness of an image of a hand captured by the image capture section as the approach distance information,

the manipulation target finger assignment step assigning a finger of a fingertip having a highest brightness preferentially to the manipulation target finger among the finger specified by the finger specification step.

62. The method according to claim 54, further comprising:

an image capture step of radiating a light from a light source towards a hand approaching the remote manipulation area, and image-capturing, using an image capture section, a hand from a side of a rear face of the remote manipulation area in a manner to capture a reflection light from the hand,

the assignment finger information acquisition step acquiring the assignment finger information based on a capture image by the image capture step.

63. The method according to claim 50, further comprising:

an image capture step of radiating a light from a light source towards a hand approaching the remote manipulation area, and image-capturing, using an image capture section, a hand from a side of a rear face of the remote manipulation area in a manner to capture a reflection light from the hand,

the finger specification step specifying the finger from an image of the captured hand.

64. The method according to claim 50, further comprising:

an image capture step of image-capturing, using an image capture section, a hand approaching the remote manipulation area, from a rear side of the remote manipulation area,

the display step displaying a position indication image generated based on the image of the captured hand.

65. The method according to claim 50, wherein:

the remote manipulation section is a touch manipulation section

having a touch manipulation area being the remote manipulation area and

detecting presence or absence of a touch manipulation made to the touch manipulation area; and

the manipulation target finger assignment step assigns a finger which executed touch manipulation to the touch manipulation area among the finger specified by the finger specification step.

66. The method according to claim 50, wherein

the position indication image highlight display step executes the highlight display in a manner to change a highlight level step by step during a predetermined time.

67. The method according to claim 66, wherein

the position indication image highlight display step executes the highlight display, in which:

at a time of starting the highlight display, the position indication image corresponding to the manipulation target finger and the position indication image corresponding to a finger different from the manipulation target finger are displayed in highlight with an identical highlight level; and

during a period starting from the time of starting the highlight display, as the period elapses, a relative highlight level of the position indication image corresponding to the manipulation target finger is increased step by step.

68. The method according to claim 66, wherein

the position indication image highlight display step starts the relative highlight display along with the assignment of the manipulation target finger by the manipulation target finger assignment step.

69. The method according to claim 66, wherein

the position indication image highlight display step starts the relative highlight display at a time when a predetermined time elapses since the manipulation target finger is assigned by the manipulation target finger assignment step.

70. The method according to claim 50, further comprising:

a hand image capture step of capturing an image of a hand that opposes the remote manipulation area,

the finger specification step specifying a finger of a hand which opposes the remote manipulation area based on the image of the captured hand,

the position indication image highlight display step

generating a real finger image to allow a specification of at least an outline of a finger of the specified hand based on an image of the image-captured hand, and

displaying the generated real finger image as the position indication image.

71. The method according to claim 50, wherein

the position indication image highlight display step displays as the position indication image one of a fake finger image and a fingertip position image, the false finger image allowing a specification of at least a direction of an extended finger, the fingertip position image allowing a specification of at least a fingertip position of a finger.

72. The method according to claim 50, wherein

the highlight display is applied to an outline line of the position indication image.

73. The method according to claim 50, wherein

the highlight display is applied in a manner to change a display color of the position indication image.

74. The method according to claim 50, wherein

the highlight display is applied in a manner to change a contrasting density of a display color of the position indication image.

75. The method according to claim 50, wherein

the highlight display is applied in a manner to change a shape of the position indication image.

76. The method according to claim 50, wherein

the highlight display is applied in a manner to change a transmittance degree of the position indication image.

77. The method according to claim 50, wherein

the highlight display is applied in a manner to flash the position indication image.

78. The method according to claim 50, further comprising:

a manipulation target finger trace step of tracing, after the manipulation target finger is assigned by the manipulation target finger assignment step, a movement of a finger assigned to the manipulation target finger,

the manipulation target finger assignment step continuously assigning a finger identical to a finger, which was assigned to the manipulation target finger based on a trace result by the manipulation target finger trace step.

79. The method according to claim 78,

the finger specification step specifying one by one with a predetermined time interval a finger of a hand existing within a predetermined finger specification possible range in front of the remote manipulation section,

the method further comprising:

a finger trace step of

acquiring position information with respect to each finger specified one by one by the finger specification step and

storing as trace information the acquired position information in a predetermined trace information storage section to trace, with respect to the each finger, a movement of the each finger specified one by one by the finger specification step,

the manipulation target finger assignment step

specifying the trace information of the manipulation target finger from the trace information with respect to the each finger stored by the finger trace step, and

tracing a movement of a finger assigned to the manipulation target finger based on the trace information of the manipulation target finger.

80. The method according to claim 79, wherein:

the finger trace step includes a next position estimate step of computing a next position, where each finger under trace of which the trace information with respect to the each finger stored at each time of the finger specification step is estimated to be located at a next time of the finger specification step, based on the corresponding trace information;

the finger trace step specifies a pair of two fingers having a position relation which satisfies a predetermined reference condition, the position relation being between the next position of each finger under trace computed in the next position estimate step and an actual position of each finger actually specified at the next time of the finger specification step;

the finger trace step determines that a finger under trace, which is one of the two fingers of the pair, and a finger actually specified in the finger specification step are identical to each other; and

the finger trace step

determines that the finger under trace is moved to a position of a finger, which is determined to be identical and actually specified, and

continues a movement trace of the finger under trace.

81. The method according to claim 80, wherein:

the trace information is enabled to include each corresponding position information on the finger under trace specified at each of multiple times of the finger specification step; and

the next position estimate step computes the next position of the finger under trace based on each position information.

82. The method according to claim 81, wherein:

when the trace information contains position information on the finger under trace specified at a previous time of the finger specification step and position information on the finger under trace specified at a time before previous of the finger specification step,

the next position estimate step computes the next position of the finger under trace based on the position information at the previous time and the position information at the time before previous; and

when the trace information contains the position information on the finger under trace specified at the previous time of the finger specification step, but does not contain the information on the finger under trace specified at the time before previous by the finger specification step,

the next position estimate step computes the next position of the finger under trace based on the position information specified at the previous time of the finger specification step and a position of an edge portion of the finger specification possible range adjacent to a position indicated by the position information specified at the previous time.

83. The method according to claim 80, wherein:

when there is a finger under trace which does not satisfy the reference condition with the actually specified finger by the finger specification step,

the finger trace step

makes a recognition switchover from the finger under trace to a finger under trace impossible, and

stores information on the finger under trace impossible at a time of the recognition switchover as trace recovery determination information in a predetermined trace recovery determination information storage section;

when there is a finger, which is actually specified by the finger specification step and does not satisfy the reference condition with the finger under trace,

the finger trace step determines whether the actually specified finger is identical to a finger, which is recognized as under trace impossible in past based on the trace recovery determination information stored in the trace recovery determination information storage section;

when it is determined that it is identical,

the finger trace step

recognizes that the finger under trace impossible corresponding to the trace recovery determination information is a finger re-appearing at a position of the finger actually specified, and

re-starts a movement trace of the finger; and

when it is determined that it is not identical,

the finger trace step

recognizes that the finger actually specified is a finger newly appearing and

starts newly a movement trace of the finger.

84. The method according to claim 83, wherein:

the trace information includes at least newest position information on a finger specified by the finger specification step; and

the trace recovery determination information includes last position information at a last time before a corresponding finger is recognized as being under trace impossible.

85. The method according to claim 84, wherein:

when there is a finger, which is actually specified by the finger specification step and does not satisfy the reference condition with the finger under trace, and there are several fingers on which the trace recovery determination information is stored in the trace recovery determination information storage section,

the finger trace step determines that the actually specified finger is identical to a finger under trace impossible corresponding to the trace recovery determination information including the position information which indicates a position closest to the actually specified finger,

based on position information on the actually specified finger, and position information at a last time of each of the several fingers under trace impossible included in the trace recovery determination information.

86. The method according to claim 84, wherein

when there is a finger, which is actually specified by the finger specification step and does not satisfy the reference condition with the finger under trace, wherein a distance is defined as being between positions respectively indicated by the position information on the actually specified finger and the position information at the last time of the finger under trace impossible which is contained in the trace recovery determination information,

the finger trace step permits recognizing that the actually specified finger and the finger under trace impossible corresponding to the trace recovery determination information are identical to each other in cases that the distance is less than a predetermined reference distance,

whereas,

the finger trace step prohibits recognizing that the actually specified finger and the finger under trace impossible corresponding to the trace recovery determination information are identical to each other in cases that the distance is not less than the predetermined reference distance.

87. The method according to claim 84, wherein:

when there is a finger under trace, which is recognized as under trace impossible by not satisfying the reference condition with the actually specified finger by the finger specification step,

the finger trace step stores a trace impossible recognition order as trace recovery determination information in the trace recovery determination information storage section,

the trace impossible recognition order enabling a specification of an anteroposterior relation with an other finger under trace impossible in respect of a trace impossible recognition time along with information on the finger at a time of being recognized as under trace impossible;

when there is a finger, which is actually specified by the finger specification step and does not satisfy the reference condition with the finger under trace, and the trace recovery determination information on several fingers is stored in the trace recovery determination information storage section,

the finger trace step specifies the trace recovery determination information including the position information which indicates a position closest to the actually specified finger,

based on position information on the actually specified finger, and position information at a last time of each of the several fingers under trace impossible included in the trace recovery determination information;

when one piece of the trace recovery determination information is specified,

the finger trace step determines that a finger under trace impossible corresponding to the one piece of the trace recovery determination information is identical to the actually specified finger;

when two pieces of the trace recovery determination information are specified,

the finger trace step

refers to the trace impossible recognition order included in the two pieces of the trace recovery determination information, and

determines that the actually specified finger is identical to a finger recognized as under trace impossible corresponding to a newest in the trace impossible recognition time among the two pieces of the trace recovery determination information.

88. The method according to claim 84, wherein

the trace recovery determination information is the trace information on the finger stored in the trace information storage section when the finger is recognized as being trace impossible.

89. The method according to claim 79, wherein

when a finger is specified by the finger specification step and the specified finger is recognized as a finger newly appearing,

the finger trace step

assigns new identification information to the newly appearing finger and

stores the trace information, in correspondence with the identification information, in the trace information storage section.

90. The method according to claim 88, wherein:

the trace information includes at least newest position information on a finger specified by the finger specification step;

the trace recovery determination information includes last position information at a last time before a corresponding finger is recognized as being under trace impossible; and

when the finger under trace assigned with the identification information is recognized as a finger under trace impossible,

the finger trace step stores the trace recovery determination information in correspondence with the identification information in the trace recovery determination information storage section.

91. The method according to claim 90, wherein:

when a finger is specified by the finger specification step,

the finger trace step determines

whether the finger actually specified by the finger specification step is the finger under the trace, and

whether the finger actually specified by the finger specification step is the finger under trace impossible,

based on the trace information stored in the trace information storage section and, furthermore, the trace recovery determination information stored in the trace recovery determination information storage section; and

the finger trace step stores a determination result as a trace status in correspondence with the identification information in a predetermined trace status storage section.

92. The method according to claim 89, wherein

when a finger, which was specified by the finger specification step, is not specified,

the identification information on the finger and various information stored in correspondence with the identification information are erased and initialized.

93. A display apparatus comprising:

a section to achieve each step of the display method according to claim 50.

94. A manipulation input apparatus comprising:

the display apparatus according to claim 93, the display apparatus having the display section and the remote manipulation section, wherein the display window is for executing an operation of a device, wherein the remote manipulation area is provided at a position separated from the display window, the remote manipulation area for remotely executing a manipulation input in the display window;

an input acceptance section configured to accept an input corresponding to a press manipulation position based on a press manipulation to the remote manipulation area;

a hand image capture section configured to capture an image of a hand that opposes the remote manipulation area; and

a position indication image display highlight section configured to display a position indication image, which is generated based on the captured image of the hand, at a position on the display window, the position being indicated by the hand.

95. The manipulation input apparatus according to claim 94, wherein:

the device is an in-vehicle device;

the display window is arranged closer to a front side of the vehicle than a driver to be seen by the driver; and

the remote manipulation area is arranged closer to the driver than the display window.