Apparatuses, systems, and methods for representing user interactions with real-world input devices in a virtual space

• US 10,394,342 B2
• Filed: 09/27/2017
• Issued: 08/27/2019
• Est. Priority Date: 09/27/2017
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Facebook Technologies, LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Inventors: Imm, Andrew Harrison, Chew, Grantland Lee Lim
• Primary Examiner(s): Eurice, Michael J
• Application Number:
  US15/717,725
• Publication Number:
  US 20190094982A1
• Time in Patent Office: 699 Days
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 1/1692 the I/O peripheral being a secondary touch screen used as control interface, e.g. virtual buttons or sliders
• CPC Class Codes: G06F 2203/04809 Textured surface identifying touch areas, e.g. overlay structure for a virtual keyboard
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/021 Arrangements integrating additional peripherals in a keyboard, e.g. card or barcode reader, optical scanner
• CPC Class Codes: G06F 3/0219 Special purpose keyboards
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H01H 11/0018 for allowing different operating parts
• CPC Class Codes: H01H 13/70 having a plurality of operating members associated with different sets of contacts, e.g. keyboard mounting together a plurality of independent switches H02B
• CPC Class Codes: H01H 2003/0293 : with an integrated touch switch
• CPC Class Codes: H01H 2239/006 : Containing a capacitive switch or usable as such

Abstract

A key assembly for a keyboard may include a key cap that includes a touch sensor that is communicatively coupled to a control device. When the touch sensor senses a touch event, the touch sensor may send a touch signal to the control device, and when the touch sensor senses an absence of touch, the touch sensor may send a control signal to the control device. The key assembly may further include a switch that is communicatively coupled to the control device and that is movable between a pressed position and an un-pressed position. When the switch is in the pressed position, the switch may send a closed signal to the control device, and when the switch is in the un-pressed position, the switch may send an open signal to the control device. Various other apparatuses, systems, and methods are also disclosed.

Claims

1. A key assembly for a keyboard comprising:

a key cap comprising;

a connector; and

a touch sensor that is communicatively coupled to a control device, wherein;

when the touch sensor senses a touch event, the touch sensor sends a touch signal to the control device; and

when the touch sensor senses an absence of touch, the touch sensor sends a control signal to the control device; and

a switch that is communicatively coupled to the control device and that is movable between a pressed position and an un-pressed position, the switch comprising a plunger that is movable between an extended position and a compressed position, wherein;

when the switch is in the pressed position, the plunger is in the compressed position and the switch sends a closed signal to the control device; and

when the switch is in the un-pressed position, the plunger is in the extended position and the switch sends an open signal to the control device;

wherein the control device;

receives, from the switch, a switch signal comprising at least one of;

the closed signal;

or

the open signal; and

directs a head-mounted display to present, in a virtual space;

a visual representation of a fingertip; and

a visual representation of the key assembly; and

animates, when the switch signal includes the closed signal, an actuation of the visual representation of the key assembly by the visual representation of the fingertip by animating the visual representation of the fingertip moving the key assembly from the un-pressed position to the pressed position.

2. The key assembly of claim 1, wherein the touch sensor comprises a capacitive touch sensor.

3. The key assembly of claim 1, wherein the touch sensor is a resistive touch sensor.

4. The key assembly of claim 1, wherein

the key cap further comprises

an interface surface that is coupled to the touch sensor.

5. The key assembly of claim 4, wherein the interface surface defines a concave depression that is adapted to receive a fingertip of a human finger.

6. The key assembly of claim 4, wherein the touch sensor comprises a sensing portion that at least partially covers the interface surface.

7. A virtual reality interface system comprising:

a head-mounted display;

a control device that is communicatively coupled to the head-mounted display; and

a keyboard comprising at least one key assembly that comprises;

a key cap comprising;

a connector; and

a touch sensor that is communicatively coupled to the control device,

wherein;

when the touch sensor senses a touch event, the touch sensor sends a touch signal to the control device; and

when the touch sensor senses an absence of touch, the touch sensor sends a control signal to the control device; and

a switch that is communicatively coupled to the control device and that is movable between a pressed position and an un-pressed position, the switch comprising a plunger that is movable between an extended position and a compressed position, wherein;

when the switch is in the pressed position, the plunger is in the compressed position and the switch sends a closed signal to the control device; and

when the switch is in the un-pressed position, the plunger is in the extended position and the switch sends an open signal to the control device;

wherein the control device;

receives, from the switch, a switch signal comprising at least one of;

the closed signal;

or

the open signal; and

directs a head-mounted display to present, in a virtual space;

a visual representation of a fingertip; and

a visual representation of the key assembly; and

animates, when the switch signal includes the closed signal, an actuation of the visual representation of the key assembly by the visual representation of the fingertip by animating the visual representation of the fingertip moving the key assembly from the un-pressed position to the pressed position.

8. The virtual reality interface system of claim 7, wherein the touch sensor comprises a capacitive touch sensor.

9. The virtual reality interface system of claim 7, wherein

the key cap further comprises

an interface surface that is coupled to the touch sensor.

10. The virtual reality interface system of claim 9, wherein the touch sensor comprises a sensing portion that at least partially covers the interface surface.

11. The virtual reality interface system of claim 9, wherein the interface surface defines a concave depression that is adapted to receive a fingertip of a human finger.

12. The virtual reality interface system of claim 9, wherein the control device

receives, from the touch sensor, a touch sensor signal comprising at least one of:

the touch signal;

or

the control signal.

13. The virtual reality interface system of claim 12, wherein

the visual representation of the key assembly is in the pressed position when the switch signal includes the closed signal.

14. The virtual reality interface system of claim 13, wherein:

the visual representation of the fingertip is in contact with a visual representation of the interface surface of the key cap when the touch sensor signal includes the touch signal; and

the visual representation of the fingertip is disconnected from the visual representation of the key cap when the touch sensor signal includes the control signal.

15. The virtual reality interface system of claim 14, wherein:

the keyboard comprises at least one additional key assembly that is communicatively coupled to the control device;

the control device receives an additional signal from the additional key assembly; and

at least one of the visual representation of the key assembly and the visual representation of the fingertip is further based on the additional signal.

16. The virtual reality interface system of claim 14, wherein:

the keyboard further comprises a mounting surface comprising a mounting location;

the switch further comprises a mount that is coupled to the plunger and that is coupled to the mounting surface at the mounting location;

the control device receives data representative of the mounting location; and

the visual representation of the key assembly in the virtual space is further based on the data representative of the mounting location.

17. The virtual reality interface system of claim 14, wherein:

the keyboard further comprises a physical position indicator; and

the control device determines;

a position of the physical position indicator in a physical space;

a position of the keyboard in the virtual space based on a determination of the position of the physical position indicator in the physical space; and

the visual representation of the key assembly in the virtual space is further based on a determination of the position of the keyboard in the virtual space.

18. A method comprising:

sensing a touch event via a touch sensor included in a key cap that is included in a key assembly of a virtual reality interface system;

determining, while the touch sensor senses the touch event, that a switch included in the key assembly has moved from an un-pressed position to a pressed position; and

directing, while the touch sensor senses the touch event, a head-mounted display included in the virtual reality interface system to present, in a virtual space, a visual representation of the key assembly and a visual representation of a fingertip in contact with a visual representation of an interface surface of the key cap; and

animating an actuation of the visual representation of the key assembly by the visual representation of the fingertip based on sensing the touch event and determining that the switch has moved from the un-pressed position to the pressed position, the animating comprising animating the visual representation of the fingertip moving the key assembly from the un-pressed position to the pressed position.

19. The method of claim 18, further comprising:

determining, while the touch sensor senses the touch event, that the switch has moved from the pressed position to the un-pressed position; and

directing, while the switch is in the un-pressed position and the touch sensor senses the touch event, the head-mounted display to present, in the virtual space, a visual representation of the key assembly in the un-pressed position and the visual representation of the fingertip in contact with the visual representation of the interface surface of the key cap.

20. The method of claim 19, further comprising:

sensing, via the touch sensor, an absence of touch; and

directing, while the switch is in the un-pressed position and the touch sensor senses the absence of touch, the head-mounted display to present, in the virtual space, the visual representation of the key assembly in the un-pressed position and the visual representation of a fingertip disconnected from the interface surface of the key cap.