Apparatus, system, and methods for interfacing with a user and/or external apparatus by stationary state detection

• US 10,222,875 B2
• Filed: 12/09/2016
• Issued: 03/05/2019
• Est. Priority Date: 12/11/2015
• Status: Active Grant

• Current Assignee: SomniQ, Inc.
• Original Assignee: SomniQ, Inc.
• Inventors: Sakaguchi, Rikko, Ishikawa, Hidenori, Yano, Ken, Nishimoto, Takuya
• Primary Examiner(s): Boyd, Jonathan A
• Application Number:
  US15/373,989
• Publication Number:
  US 20170168595A1
• Time in Patent Office: 816 Days
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Methods and apparatuses for interfacing with a user based on the detection of a state, such as a stationary state, of the apparatus and providing a response based on the state of the apparatus are described herein. An example method of interfacing with a user includes: detecting a user in proximity of an apparatus; entering a first mode of the apparatus responsive to detection of the user in proximity; receiving motion data from one or more sensors of the apparatus responsive to movement of the apparatus; and detecting a stationary state of the apparatus based on the motion data; and entering a second mode responsive to detection of the stationary state.

Claims

1. A method of interfacing with a user, comprising:

detecting a user in proximity of an apparatus;

entering a first stillness detection mode of the apparatus from a standby mode of the apparatus responsive to detection of the user in proximity;

detecting initial movement of the apparatus;

for a period of time following the initial movement, receiving motion data from one or more sensors of the apparatus responsive to the initial movement of the apparatus;

processing the motion data to detect lack of movement of the apparatus following the initial movement;

entering a second stillness detection mode responsive to the detected lack of movement; and

providing a response with the apparatus during the second stillness detection mode.

2. The method of claim 1, wherein the one or more sensors include a motion sensor, the method further comprising activating the motion sensor responsive to entering the first mode.

3. The method of claim 1, wherein the apparatus includes a touch sensor, and wherein detecting a user in proximity of the apparatus comprises detecting a touch by the user with the touch sensor.

4. The method of claim 1, wherein the response comprises providing feedback to the user or transmitting a communication to an external apparatus.

5. The method of claim 1, wherein the providing a response comprises:

transmitting a command to an external apparatus;

receiving return communication from the external apparatus responsive to the command, wherein the return communication is based on user data retrieved by the external apparatus; and

generating feedback by the apparatus based, at least in part, on the return communication.

6. The method of claim 1, further comprising:

processing information associated with the user while the apparatus is in the second stillness detection mode,

wherein processing the information of the user during the lack of movement of the apparatus includes detecting one or more periods of lack of movement of the apparatus.

7. The method of claim 6, wherein the apparatus further comprises a microphone and a memory device, and wherein processing the information comprises:

detecting sounds from the user using the microphone;

recording audio data corresponding to the sounds on a memory device;

extracting information from the audio data by the processor;

generating an instruction to search content associated with the user and stored on an external apparatus, based on the extracted information; and

transmitting the instruction to the external apparatus.

8. The method of claim 7, further comprising:

generating an instruction to associate the extracted information with one or more content items from the content stored on the external apparatus; and

transmitting the instruction to the external apparatus.

9. The method of claim 1, further comprising recording a sequence of states of the apparatus including one or more stationary states or non-stationary states occurring in the second stillness detection mode.

10. The method of claim 9, further comprising:

determining a pattern of the sequence of states by comparing the sequence of states with a plurality of stored sequences.

11. The method of claim 10, further comprising generating a command to an external apparatus based on the pattern of the sequence of states and transmitting the command to the external apparatus.

12. The method of claim 11, further comprising identifying one of a plurality of external apparatuses based on the pattern of the sequence of states and transmitting the command to the identified external apparatus.

13. An apparatus comprising:

a proximity sensor configured to detect a user in proximity of the apparatus;

a processor configured to cause the apparatus to enter a first stillness detection mode from a standby mode responsive to detection of the user in proximity and detect initial movement of the apparatus in the first stillness detection mode; and

a motion sensor configured to provide motion data for a period of time following the initial movement responsive to the initial movement of the apparatus in the first stillness detection mode,

wherein the processor is configured to process the motion data to detect lack of movement of the apparatus following the initial movement, further configured to cause the apparatus to enter a second stillness detection mode responsive to the detection of the lack of movement, and further configured to cause the apparatus to provide a response during the second stillness detection mode.

14. The apparatus of claim 13, further comprising:

a memory device configured to record information associated with the user while the apparatus is in the second stillness detection mode; and

at least one microphone configured to detect ambient sound in the second stillness detection mode,

wherein the memory device is configured to record the ambient sound, and

wherein the processor is configured to extract the information associated with the user from the ambient sound.

15. The apparatus of claim 13, further comprising:

a memory device configured to record information associated with the user while the apparatus is in the second stillness detection mode; and

a camera configured to capture a sequence of images of the user,

wherein the memory device is configured to store the sequence of images of the user, and

wherein the processor is configured to restore the sequence of images, to interpret a gesture of the user in the sequence of images, and further configured to extract the information associated with the user from the gesture of the user.

16. The apparatus of claim 13, further comprising:

a memory device configured to record information associated with the user while the apparatus is in the second stillness detection mode,

wherein the motion sensor is further configured to capture a sequence of motion data of the user in the second stillness detection mode,

wherein the memory device is configured to store the sequence of motion data of the user, and

wherein the processor is configured to restore the sequence of motion data, to interpret a gesture of the user in the sequence of motion data, and further configured to extract the information associated with the user from the gesture of the user.

17. The apparatus of claim 13, wherein the processor is configured to provide feedback to the user or transmit a communication to an external apparatus during the second stillness detection mode.

18. The apparatus of claim 17, further comprising a wireless transmitter configured to transmit a command to an external apparatus based on the information associated with the user, the command being indicative of a function of the external apparatus.

19. The apparatus of claim 17, further comprising a light emitting diode (LED), wherein the processor is configured to blink the LED as the response, based on the information associated with the user.

20. The apparatus of claim 17, further comprising at least one speaker, wherein the processor is configured to cause the speaker to produce sound as the response, based on the information associated with the user.

21. The apparatus of claim 17, further comprising an enclosure and a vibrator, wherein the processor is configured to cause the vibrator to vibrate the enclosure as the response, based on the information associated with the user.

22. The apparatus of claim 13, further comprising an enclosure made, at least in part, from wood.