Communicating between a virtual area and a physical space

• US 9,851,863 B2
• Filed: 02/20/2017
• Issued: 12/26/2017
• Est. Priority Date: 01/15/2009
• Status: Active Grant

• Current Assignee: Sococo, Inc.
• Original Assignee: Sococo, Inc.
• Inventors: Van Wie, David
• Primary Examiner(s): Mamo, Elias
• Application Number:
  US15/437,335
• Publication Number:
  US 20170160900A1
• Time in Patent Office: 309 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06Q 50/60 Business processes related to postal services shipping G06Q10/083; franking apparatus G07B17/00
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04M 2201/42 Graphical user interfaces
• CPC Class Codes: H04M 2203/1025 of avatars
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Apparatus transmits an identifier for association with a virtual area by an administering network service, generates output data from human perceptible stimulus in a physical space, transmits the output data in connection with the virtual area, receives input data associated with the virtual area, and generates human perceptible stimulus in the physical space from the input data. A persistent association is created between the apparatus and a virtual area. A respective presence is established in the virtual area for a communicant operating a client network node connected to the virtual area. A respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus are administered in association with the virtual area. A client network node displays a graphical user interface, establishes the administered connections, and presents interaction controls associated with the object for interacting with communicants in the physical space.

Claims

1. A method, comprising by a client network node that has a client source of a respective data stream content type and a client sink of a respective data stream content type:

displaying a user interface comprising a representation of a virtual area that supports establishment of respective presences of communicants operating respective client network nodes, a representation of each of the communicants who is present in the virtual area, and a representation of an object associated with an apparatus in a physical space, wherein the apparatus has an apparatus sink that is complementary to the client source and an apparatus source that is complementary to the client sink;

establishing a respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus in association with the virtual area, wherein each connection supports transmission of the respective data stream content type between the apparatus and the client network node;

presenting interaction controls associated with the object for interacting with communicants in the physical space.

2. The method of claim 1, wherein the representation of the virtual area corresponds to a virtualized representation of the physical space.

3. The method of claim 1, further comprising receiving from a network service administering the virtual area a specification for displaying the representation of the object in spatial relation to the representation of the virtual area.

4. The method of claim 1, further comprises showing in the user interface indications of respective states of the apparatus source in connection with the representation of the object.

5. The method of claim 4, wherein the showing comprises:

displaying a first visual cue when the apparatus source is in an active state; and

displaying a second visual cue that is different from the first visual cue when the apparatus source is in an inactive state.

6. The method of claim 1, further comprising:

based on communicant input in connection with the object, establishing a connection between an audio source of the client network node and an audio sink of the apparatus.

7. The method of claim 1, further comprising:

based on communicant input in connection with the object, establishing a connection between an application sharing source of the client network node and an image projection sink of the apparatus.

8. The method of claim 1, further comprising:

based on communicant input in connection with the object, establishing a connection between a laser pointer control source of the client network node and a laser pointer control sink of the apparatus.

9. The method of claim 1, wherein the apparatus source comprises a microphone and the apparatus sink comprises a speaker, wherein the microphone is operable to generate output voice data from human voice sound projected into the physical space, the speaker is operable to project human voice sound into the physical space from input voice data associated with the virtual area, and the establishing comprises establishing connections that relay the output voice data from the apparatus to the client network node and that relay the input voice data from the client network node to the apparatus.

10. The method of claim 1, wherein the apparatus source comprises a camera operable to perform operations comprising capturing images of a scene in the physical space and generating output image data from the captured images, and the establishing comprises establishing a connection that relays the output image data from the apparatus to the client network node.

11. The method of claim 1, wherein the apparatus sink comprises a projector operable to perform operations comprising projecting images into the physical space, and the establishing comprises establishing a connection that relays input control data for controlling the projecting from the client network node to the apparatus.

12. The method of claim 1, wherein the apparatus sink comprises a laser pointer operable to perform operations comprising projecting a laser beam into the physical space, and the establishing comprises establishing a connection that relays input control data for controlling the projecting of the laser beam from the client network node to the apparatus.

13. Apparatus, comprising:

a memory storing processor-readable instructions; and

a processor coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising sending to a client network node instructions for

displaying a user interface comprising a representation of a virtual area that supports establishment of respective presences of communicants operating respective client network nodes, a representation of each of the communicants who is present in the virtual area, and a representation of an object associated with an apparatus in a physical space, wherein the apparatus has an apparatus sink of a respective data stream content type that is complementary to a client source of the client network node and an apparatus source of a respective data stream content type that is complementary to a client sink of the client network node,

establishing a respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus in association with the virtual area, wherein each connection supports transmission of the respective data stream content type between the apparatus and the client network node, and

presenting interaction controls associated with the object for interacting with communicants in the physical space.

18. The apparatus of claim 13, wherein the sending comprises sending to the client network node instructions for showing in the user interface indications of respective states of the apparatus source in connection with the representation of the object.

19. The apparatus of claim 13, wherein the sending comprises sending to the client network node instructions for establishing a connection between an application sharing source of the client network node and an image projection sink of the apparatus based on communicant input in connection with the object.

20. The apparatus of claim 13, wherein the apparatus source comprises a camera operable to perform operations comprising capturing images of a scene in the physical space and generating output image data from the captured images, and the establishing comprises establishing a connection that relays the output image data from the apparatus to the client network node.

14. At least one non-transitory computer-readable medium comprising processor-readable program code embodied therein, the processor-readable program code adapted to be executed by a client network node and comprising:

processor-executable code for displaying a user interface comprising a representation of a virtual area that supports establishment of presences of communicants operating respective client network nodes, a representation of each of the communicants who is present in the virtual area, and a representation of an object associated with an apparatus in a physical space, wherein the apparatus has an apparatus sink of a respective data stream content type that is complementary to a client source of the client network node and an apparatus source of a respective data stream content type that is complementary to a client sink of the client network node;

processor-executable code for establishing a respective connection between each active pair of complementary sources and sinks of the client network node and the apparatus in association with the virtual area, wherein each connection supports transmission of the respective data stream content type between the apparatus and the client network node;

processor-executable code for presenting interaction controls associated with the object for interacting with communicants in the physical space.

15. The least one non-transitory computer-readable medium of claim 14, further comprising processor-executable code for showing in the user interface indications of respective states of the apparatus source in connection with the representation of the object.

16. The least one non-transitory computer-readable medium of claim 14, further comprising processor-executable code for establishing a connection between an application sharing source of the client network node and an image projection sink of the apparatus based on communicant input in connection with the object.

17. The least one non-transitory computer-readable medium of claim 14, wherein the apparatus source comprises a camera operable to perform operations comprising capturing images of a scene in the physical space and generating output image data from the captured images, and the establishing comprises establishing a connection that relays the output image data from the apparatus to the client network node.