Electronic Device Displays an Image of an Obstructed Target

• US 20160109940A1
• Filed: 08/11/2015
• Published: 04/21/2016
• Est. Priority Date: 10/19/2014
• Status: Active Grant

• Current Assignee: Advanced AR Systems LLC
• Original Assignee: Philip Lyren, Robert Lyren
• Inventors: Lyren, Philip, Lyren, Robert
• Granted Patent: US 9,791,919 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds

Abstract

An electronic device determines information about a target and provides the information to another electronic device that has an obstructed view of the target. The other electronic device displays an image of the target with an orientation and a location of the target.

Claims

1. A method, comprising:

determining, from information collected with first electronic glasses worn by a first user at a geographical location, a location and a first orientation of a person and a weapon that are visible with a naked eye of the first user and that are visible through the first electronic glasses;

providing, to second electronic glasses worn by a second user at the geographical location and based on the information collected with the first electronic glasses, the location of the person and the weapon;

determining, from information collected with second electronic glasses worn by the second user at the geographical location, that the person and the weapon are not visible at the location with a naked eye of the second user because objects obstruct the second user from seeing the person and the weapon at the location;

determining, based on the location of the person and the weapon, a placement location of where the person and the weapon would be visible to the naked eye of the second user in a field of view of the second user if the person and the weapon were not obstructed by the objects;

determining, based on the first orientation of the person and the weapon, a second orientation that depicts how the person and the weapon would be oriented at the location to the naked eye of the second user in the field of view of the second user if the person and the weapon were not obstructed by the objects; and

displaying, through the second electronic glasses, three dimensional (3D) images of the person and the weapon over the objects with the 3D images of the person and the weapon having the second orientation and being located at the placement location in the field of view of the second user such that the 3D images of the person and the weapon appear in the field of view of the second user where the person and the weapon would be visible to the naked eye of the second user in the field of view of the second user if the person and the weapon were not obstructed by the objects.

2. The method of claim 1 further comprising:

determining a distance from the second user to the person and the weapon;

determining, based on the distance, a size of the person and a size of the weapon that would appear in the field of view of the second user if the person and the weapon were not obstructed by the objects and visible with the naked eye of the second user in the field of view of the second user; and

displaying, through the second electronic glasses, the 3D images of the person and the weapon with the size of the person and the size of the weapon that would appear in the field of view of the second user if the person and the weapon were not obstructed by the objects and visible with the naked eye of the second user in the field of view of the second user.

3. The method of claim 1 further comprising:

determining, from the information collected with the first electronic glasses, a 3D image of an area that is proximate to the person and to the weapon; and

displaying, through the second electronic glasses, the 3D image of the area that is proximate to the person and to the weapon such that the 3D image of the area appears with the 3D images of the person and the weapon in the field of view of the second user where the area would be visible to the second user in the field of view of the second user if the area were not obstructed by the objects and visible with the naked eye of the second user in the field of view of the second user.

4. The method of claim 1 further comprising:

determining an overlap area between a field of view of the first user and a field of view of the second user;

performing a comparison of information collected by the first electronic glasses in the overlap area with information collected by the second electronic glasses in the overlap area;

determining, based on the comparison, an accuracy of information in the overlap area; and

displaying the information and the accuracy of the information to the first user through the first electronic glasses and to the second user through the second electronic glasses.

5. The method of claim 1 further comprising:

determining, from a facial image collected with the first electronic glasses, an identity of the person;

drawing the 3D images of the person to resemble the identity of the person; and

displaying, through the second electronic glasses, the 3D images of the person to resemble the identity of the person.

6. The method of claim 1 further comprising:

detecting movement of the second electronic glasses to another field of view of the second user that does not include the location of the person and the weapon if the person and the weapon were not obstructed by the objects; and

removing, in response to detecting the movement of the second electronic glasses to the another field of view, the 3D images of the person and the weapon being displayed through the second electronic glasses at a time when the field of view of the second user no longer includes the location of the person and the weapon if the person and the weapon were not obstructed by the objects.

7. The method of claim 1 further comprising:

determining, from the information collected with the first electronic glasses, movement of the person and the weapon; and

displaying, through the second electronic glasses, movement of the 3D images of the person and the weapon to coincide in real time with the movement of the person and the weapon as captured with the first electronic glasses worn by the first user.

8. The method of claim 1 further comprising:

detecting movement of the second electronic glasses to another field of view of the second user that does not include the location of the person and the weapon if the person and the weapon were not obstructed by the objects; and

displaying, with the second electronic glasses and in response to detecting the movement of the second electronic glasses to the another field of view, a visual indication that indicates a direction of the location in order to visually inform the second user where to look in order to see the 3D images of the person and the weapon at the location.

9. An electronic system, comprising:

first electronic glasses that include a processor in communication with a wireless network, a display, and a camera that captures an image of an object and a person in a field of view of a first user wearing the first electronic glasses;

second electronic glasses that include a processor in communication with the wireless network, a display, and a camera that captures an image of the object without the person since the person is located behind the object and not within a field of view of a second user wearing the second electronic glasses; and

a computer that communicates over the wireless network with the first electronic glasses and the second electronic glasses and that includes a processor and a memory with instructions that the processor of the computer executes to receive the image of the object and the person from the first electronic glasses, receive the image of the object without the person from the second electronic glasses, creates a three dimensional (3D) image of the person to resemble the person, determine a location of where the person would appear in the field of view of the second user if the object were not obstructing the person in the field of view of the second user, determine an orientation of what profile of the person that the second user would see in the field of view of the second user if the object were not obstructing the person in the field of view of the second user, and transmit the 3D image of the person to the second electronic glasses; and

wherein the display of the second electronic glasses displays the 3D image of the person at the location and with the orientation in the field of view of the second user as if the object were not obstructing the person in the field of view of the second user.

10. The electronic system of claim 9, wherein a shape, a size, and the orientation of the 3D image of the person displayed with the display of the second electronic glasses matches a shape, a size, and an orientation of the person located behind the object.

11. The electronic system of claim 9, wherein the processor of the computer further executes the instructions to move the 3D image of the person to match real time movement of the person located behind the object even though the person is not visible with a naked eye of the second user in the field of view of the second user.

12. The electronic system of claim 9, wherein the display of the second electronic glasses removes the 3D image of person when the field of view of the second user moves to a location that no longer includes the object, and the 3D image of the person reappears on the display of the second electronic glasses when the field of view of the second user moves back to a location that includes the person.

13. The electronic system of claim 9, wherein the processor of the computer further executes the instructions to create a 3D image of a portion of the object that is visible to the first user but not visible to the second user and to transmit the 3D image of the portion of the object to the second electronic glasses, and wherein the display of the second electronic glasses displays the 3D image of the portion of the object such that the second user sees the object supplemented with the 3D image of the portion of the object that is not visible to the second user.

14. The electronic system of claim 9, wherein the display of the second electronic glasses displays a distance from the second user to the person even though the person is not visible with a naked eye of the second user in the field of view of the second user.

15. The electronic system of claim 9, wherein the processor the computer further executes the instructions to analyze the image of the person from the first electronic glasses, determine that the person holds a firearm pointed in a direction, and transmit the 3D image of the person holding the firearm to the second electronic glasses;

and

wherein the display of the second electronic glasses displays the 3D image of the person holding the firearm pointed in a direction that matches the person holding the firearm pointed in the direction even though the person and the firearm are not visible with a naked eye of the second user in the field of view of the second user but are visible with a naked eye of the first user.

16. A non-transitory computer readable storage medium storing instructions that cause one or more processors to execute a method, comprising:

obtain, from first electronic glasses, an image of a person holding a weapon at a location visible in a field of view of a first user wearing the first electronic glasses;

create, from the image of the person holding the weapon at the location, 3D images of the person, the weapon, and the location such that the 3D images show the person holding the weapon at the location;

obtain, from second electronic glasses, an image of an object that blocks a field of view of a second user wearing the second electronic glasses from seeing the person holding the weapon at the location;

calculate a placement location in the field of view of the second user where to position the 3D images of the person, the weapon, and the location such that the 3D images of the person, the weapon, and the location at the placement location show the person holding the weapon at the location as the person holding the weapon at the location would appear in the field of view of the second user if the object were not blocking the second user from seeing the person holding the weapon at the location; and

display, through the second electronic glasses and over the object in the field of view of the second user, the 3D images of the person, the weapon, and the location at the placement location such that the field of view of the second user shows the person holding the weapon at the location as if the object were not blocking the second user from seeing the person holding the weapon at the location.

17. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

determine, from the image of the person holding the weapon at the location, an orientation of the weapon and a point of aim of the weapon;

adjust the 3D images of the person, the weapon, and the location to match the orientation of the weapon and the point of aim of the weapon; and

display, through the second electronic glasses and over the object in the field of view of the second user, the 3D images of the person, the weapon, and the location such that the field of view of the second user shows the person holding the weapon with the orientation and the point of aim at the location as if the object were not blocking the second user from seeing the person holding the weapon with the orientation and the point of aim.

18. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

switch views of the second electronic glasses from displaying the 3D images of the person, the weapon, and the location over the object in the field of view of the second user to displaying real time images from the first electronic glasses that show the person holding the weapon at the location in the field of view of the first user.

19. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

display, with the second electronic glasses, a visual indication that provides a compass direction of the location of the person holding the weapon with respect to a location of the second electronic glasses; and

display the 3D images of the person, the weapon, and the location when the second electronic glasses move a camera to point in the compass direction.

20. The non-transitory computer readable storage medium storing instructions of claim 16 further to cause the one or more processors to execute the method comprising:

display, on a see-through display, the 3D images of the person, the weapon, and the location in a view that has a waved circular shape with a closed configuration.