Filtering of real-time visual data transmitted to a remote recipient

• US 10,169,850 B1
• Filed: 10/05/2017
• Issued: 01/01/2019
• Est. Priority Date: 10/05/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Carey, James E., Chen, Jim C., Konik, Rafal P., Rossiter, Ryan L.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US15/726,360
• Time in Patent Office: 453 Days
• Field of Search: 348 1401- 1416, 348152, 348143
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A remote visual experience application which transmits real-time video remotely includes a filter which obscures one or more previously identified regions in space when the regions come within the camera'"'"'s field of vision. The application may be a shared visual experience application, in which the camera is mounted to a wearable appliance of a local user. The application may include a function for mapping regions of a space in which the application will be used, and defining different levels of remote access for different regions of the space. A space map may be generated before or during transmission of video to a remote user. Exemplary embodiments include a virtual vendor house call application or a game. The application may further include an audio filter for filtering an audio signal.

Claims

1. A non-transitory computer-readable media having recorded thereon a computer program having a plurality of computer-executable instructions, wherein the computer program, when executed by at least one computer system, causes the at least one computer system to perform actions comprising:

receiving input identifying a plurality of regions within a three-dimensional space, and for each region of said plurality of regions, at least one respective remote access attribute obtaining video of at least a part of the space from a movable camera, the video being obtained for a remote visual experience application;

determining regions within the three-dimensional space that are within a field of view of the movable camera, wherein said determining regions within the three-dimensional space that are within the field of vision of the movable camera comprises;

(a) receiving a respective signal from each of one or more reference markers, (b) determining a location and orientation of the movable camera using the respective signal from each of one or more reference markers, and (c) using the determined location and orientation of the movable camera to determine regions within the three dimensional space that are within the field of vision of the movable camera;

for each region of said plurality of regions which is captured in the video obtained from the movable camera, using the at least one respective remote access attribute of the respective region to modify the video by obscuring video images within a first region of said plurality of regions which is captured in the video while not obscuring images within a second region of said plurality of regions which is captured in the video; and

providing the modified video for display in real time to at least one remote user of the remote visual experience application.

2. The non-transitory computer-readable media of claim 1, wherein said camera is mounted to a wearable appliance of a local user.

3. The non-transitory computer-readable media of claim 1, wherein said remote visual experience application is a virtual vendor house call application.

4. The non-transitory computer-readable media of claim 1, wherein the computer program, when executed by the at least one computer system, further causes the at least one computer system to perform actions comprising:

generating a space map of at least a portion of said three-dimensional space, said space map defining a coordinate frame of reference with respect to at least one reference point, and identifying said plurality of regions in said coordinate frame of reference.

5. The non-transitory computer-readable media of claim 4, wherein said plurality of regions include at least one region having boundaries explicitly defined in said space map and a default region, said default region being that portion of said three-dimensional space lying outside any of the at least one region having boundaries explicitly defined in said space map.

6. The non-transitory computer-readable media of claim 4, wherein generating a space map of at least a portion of said three dimensional space comprises:

receiving user input designating a plurality of reference points;

using the plurality of reference points to select an origin and orientation in a coordinate frame of reference;

determining respective coordinates of each reference point of the plurality of reference points in the coordinate frame of reference;

determining respective coordinates associated with at least one region of the plurality of regions in the coordinate frame of reference.

7. The non-transitory computer-readable media of claim 1, wherein the computer program, when executed by at least one computer system, further causes the at least one computer system to perform actions comprising filtering an audio signal accompanying said video according to at least one pre-specified audio filtering parameter.

8. A data processing apparatus supporting execution of a remote visual experience application, comprising:

at least one physical processor;

at least one physical memory;

a remote visual experience application embodied as instructions executable on the at least one processor and storable in the at least one memory, the remote visual experience application implementing selective obscuring of portions of visual images presented to at least one remote user using a data structure identifying a plurality of regions within a three-dimensional space, and for each region of said plurality of regions, at least one respective remote access attribute;

wherein the remote visual experience application receives video of at least a part of the space from a movable camera;

wherein the remote visual experience application determines regions within the three-dimensional space that are within a field of view of the movable camera by;

(a) receiving a respective signal from each of one or more reference markers, (b) determining a location and orientation of the movable camera using the respective signal from each of one or more reference markers, and (c) using the determined location and orientation of the movable camera to determine regions within the three dimensional space that are within the field of vision of the movable camera;

wherein the remote visual experience application further, for each region of said plurality of regions which is captured in the video received from the movable camera, uses the at least one respective remote access attribute of the respective region to modify the video by obscuring video images within a first region of said plurality of regions which is captured in the video while not obscuring images within a second region of said plurality of regions which is captured in the video; and

wherein the remote visual experience application further provides the modified video for display in real time to at least one remote user of the remote visual experience application.

9. The data processing apparatus of claim 8, wherein said remote visual experience application includes a space map generating function which generates a space map of at least a portion of the three-dimensional space, said space map defining a coordinate frame of reference with respect to at least one reference point, and identifying said plurality of regions in said coordinate frame of reference.

10. The data processing apparatus of claim 9, wherein said plurality of regions include at least one region having boundaries explicitly defined in said space map and a default region, said default region being that portion of said three-dimensional space lying outside any of the at least one region having boundaries explicitly defined in said space map.

11. The data processing apparatus of claim 9, wherein the space map generating function generates a space map of at least a portion of the three dimensional space by:

receiving user input designating a plurality of reference points;

using the plurality of reference points to select an origin and orientation in a coordinate frame of reference;

determining respective coordinates of each reference point of the plurality of reference points in the coordinate frame of reference;

determining respective coordinates associated with at least one region of the plurality of regions in the coordinate frame of reference.

12. The data processing apparatus of claim 8, wherein said camera is mounted to a wearable appliance of a local user.

13. The data processing apparatus of claim 8, wherein said remote visual experience application is a virtual vendor house call application.

14. The data processing apparatus of claim 8, wherein the remote visual experience application further filters an audio signal accompanying said video according to at least one pre-specified audio filtering parameter.