System and method for programing devices within world space volumes

• US 9,477,302 B2
• Filed: 08/10/2012
• Issued: 10/25/2016
• Est. Priority Date: 08/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kauffmann, Alejandro, Wheeler, Aaron Joseph, Chi, Liang-Yu, Dahlkamp, Hendrik, Ganapathi, Varun, Zhao, Yong, Plagemann, Christian
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): WALSH, KATHLEEN M.
• Application Number:
  US13/572,128
• Publication Number:
  US 20150153822A1
• Time in Patent Office: 1,537 Days
• Field of Search: 345/156, 345/157, 345/419, 348/14.01, 348/14.16, 348/46, 382/128
• US Class Current: 1/1
• CPC Class Codes: G01B 11/25 by projecting a pattern, e.g. one or more lines, moiré fringes on the object G01B11/255 takes precedence ; image analysis for depth or shape recovery G06T7/50
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

Aspects of the present disclosure relate to controlling the functions of various devices based on spatial relationships. In one example, a system may include a depth and visual camera and a computer (networked or local) for processing data from the camera. The computer may be connected (wired or wirelessly) to any number of devices that can be controlled by the system. A user may use a mobile device to define a volume of space relative to the camera. The volume of space may then be associated with a controlled device as well as one or more control commands. When the volume of space is subsequently occupied, the one or more control commands may be used to control the controlled device. In this regard, a user may switch a device on or off, increase volume or speed, etc. simply by occupying the volume of space.

Claims

1. A method comprising:

defining, by the one or more processors, a fixed volume of space by;

receiving user input indicating when to begin recording the volume of space;

while recording, receiving user input identifying a first location and a second location as fixed three-dimensional points defined in a real-world space by a user positioning a reference object; and

determining the volume of space by drawing a shape around the reference object and moving the shape between the first location and the second location;

receiving, by the one or more processors, input identifying a controlled device;

receiving, by the one or more processors, input command information defining a set of control commands for the controlled device;

associating, by the one or more processors, the volume of space, the controlled device, and the set of control commands together;

storing, by the one or more processors, the association in memory;

monitoring, by the one or more processors, the volume of space;

when the one or more processors detects that the volume of space is occupied by an object, identifying, by the one or more processors, the set of control commands and the controlled device associated with the volume of space; and

using, by the one or more processors, the set of control commands to initiate control operations of the controlled device based on a movement of the object in a given direction with respect to the first location and the second location, the given direction being associated with at least one of the control commands from the set for operating the controlled device.

2. The method of claim 1, wherein the reference object is a two or three-dimensional shape, receiving the user input includes receiving information indicating that the user has moved the reference object from the first location to the second location.

3. The method of claim 1, wherein the input defining the volume of space is received from a depth camera and the depth camera is used to monitor the volume of space.

4. The method of claim 3, wherein the volume of space is defined relative to a coordinate system of the depth camera.

5. A system comprising:

memory;

a processor configured to;

define a fixed volume of space by;

receiving input indicating when to begin recording the volume of space;

while recording, receiving user input identifying a first location and a second location as fixed three-dimensional points defined in a real-world space by a user positioning a reference object; and

determining the volume of space by drawing a shape around the reference object and moving the shape between the first location and the second location;

receive input identifying a controlled device;

receive input command information defining a set of control commands for the controlled device;

associate the volume of space, the controlled device, and the set of control commands together;

store the association in the memory;

monitor the volume of space;

when the processor detects that the volume of space is occupied by an object, identify the set of control commands and the controlled device associated with the volume of space; and

use the set of control commands to initiate control operations of the controlled device based on a movement of the object in a given direction with respect to the first location and the second location, the given direction being associated with at least one of the control commands from the set for operating the controlled device.

6. The system of claim 5, wherein the reference object is a two or three-dimensional shape, the processor is configured to receive the user input by receiving information indicating that the user has moved the reference object from the first location to the second location.

7. The system of claim 5, wherein the input defining the volume of space is received from a depth camera and the depth camera is used to monitor the volume of space.

8. The system of claim 7, wherein the volume of space is defined relative to a coordinate system of the depth camera.

9. A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

defining a fixed volume of space by;

receiving input indicating when to begin recording the volume of space;

while recording, receiving user input identifying a first location and a second location as fixed three-dimensional points defined in a real-world space by a user positioning a reference object; and

determining the volume of space by drawing a shape around the reference object and moving the shape between the first location and the second location;

receiving input identifying a controlled device;

receiving input command information defining a set of control commands for the controlled device;

associating the volume of space, the controlled device, and the set of control commands together;

storing the association in memory;

monitoring the volume of space;

when the processor detects that the volume of space is occupied by an object, identifying, with the processor, the set of control commands and the controlled device associated with the volume of space; and

using the set of control commands to initiate control operations of the controlled device based on a movement of the object in a given direction with respect to the first location and the second location, the given direction being associated with at least one of the control commands from the set for operating the controlled device.

10. The medium of claim 9, wherein the input defining the volume of space is received from a depth camera and the depth camera is used to monitor the volume of space.

11. The medium of claim 9, wherein the volume of space is defined relative to a coordinate system of a depth camera.