INTERACTION WITH THREE-DIMENSIONAL VIDEO

• US 20160086379A1
• Filed: 09/16/2015
• Published: 03/24/2016
• Est. Priority Date: 09/22/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Sadi, Sajid, Perdices-Gonzalez, Sergio, Budhiraja, Rahul, Lee, Brian Dongwoo, Khwaja, Ayesha Mudassir, Mistry, Pranav, Huang, Link, Kim, Cathy, Noh, Michael, Chung, Ranhee, Han, Sangwoo, Yeh, Jason, Cho, Junyeon, Nget, Soichan, Harms, Brian, Qian, Yedan, He, Ruokan
• Granted Patent: US 11,205,305 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/183 On-screen display [OSD] information, e.g. subtitles or menus
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

In one embodiment, a method includes presenting to a user, on a display of a head-worn client computing device, a three-dimensional video including images of a real-life scene that is remote from the user'"'"'s physical environment. The method also includes presenting to the user, on the display of the head-worn client computing device, a graphical object including an image of the user'"'"'s physical environment or a virtual graphical object.

Claims

1. A method comprising:

presenting to a user, on a display of a head-worn client computing device, a three-dimensional video comprising images of a real-life scene that is remote from the user'"'"'s physical environment; and

presenting to the user, on the display of the head-worn client computing device, a graphical object comprising;

an image of the user'"'"'s physical environment;

or

a virtual graphical object.

2. The method of claim 1, wherein:

the head-worn client computing device comprises an image sensor; and

presenting to the user an image of the user'"'"'s physical environment comprises presenting to the user an image captured by the image sensor.

3. The method of claim 1, wherein presenting to the user an image of the user'"'"'s physical environment comprises:

receiving, at the head-worn client computing device, an indication that an event occurred in the user'"'"'s physical environment;

determining, based on the event, to present on the display an image of at least a portion of the environment in which the event occurred; and

presenting the image on the display.

4. The method of claim 3, wherein the event comprises a sound.

5. The method of claim 4, wherein the sound comprises one or more audible words.

6. The method of claim 4, wherein the sound comprises a sonic amplitude that is greater than a threshold sonic amplitude.

7. The method of claim 3, wherein:

the head-worn client computing device comprises an image sensor; and

the event comprises an aspect of the physical environment sensed by the image sensor.

8. The method of claim 7, wherein the aspect comprises a distance between the user and an object sensed by the image sensor.

9. The method of claim 7, wherein the aspect comprises a speed of an object sensed by the image sensor.

10. The method of claim 7, wherein the aspect comprises a particular gesture performed by the user or by another person.

11. The method of claim 1, wherein the virtual graphical object comprises a notification.

12. The method of claim 11, wherein the notification comprises a message from another user or from an application.

13. The method of claim 1, wherein the virtual graphical object comprises a virtual input device.

14. The method of claim 13, wherein:

the head-worn client computing device comprises an image sensor; and

the virtual graphical object corresponds to an object sensed by the image sensor.

15. The method of claim 14, wherein the object sensed by the image sensor comprises a portion of a hand or arm of the user.

16. The method of claim 1, wherein the virtual graphical object comprises a virtual surface displaying a plurality of three-dimensional videos.

17. The method of claim 1, wherein the virtual graphical object comprises information corresponding to an object in the three-dimensional video.

18. The method of claim 17, wherein the information comprises an identification of the object.

19. The method of claim 17, wherein the information comprises an identification that the object has been selected.

20. The method of claim 1, wherein the virtual graphical object comprises content created by the user.

21. The method of claim 20, wherein the content comprises a writing.

22. The method of claim 20, wherein the content comprises a drawing.

23. The method of claim 1, wherein the virtual graphical object comprises an image corresponding to the real-life scene.

24. The method of claim 23, wherein the image corresponding to the real-life scene comprises an identification of a view of at least a portion of the real-life scene at a time different than that corresponding to the real-life scene.

25. One or more non-transitory computer-readable storage media comprising instructions that are operable when executed to:

present to a user, on a display of a head-worn client computing device, a three-dimensional video comprising images of a real-life scene that is remote from the user'"'"'s physical environment; and

present to the user, on the display of the head-worn client computing device, a graphical object comprising;

an image of the user'"'"'s physical environment;

or

a virtual graphical object.

26. The media of claim 25, wherein:

the head-worn client computing device comprises an image sensor; and

presenting to the user an image of the user'"'"'s physical environment comprises presenting to the user an image captured by the image sensor.

27. The media of claim 25, wherein the virtual graphical object comprises a notification.

28. An apparatus comprising:

one or more non-transitory computer-readable storage media embodying instructions; and

one or more processors coupled to the storage media and configured to execute the instructions to;

present to a user, on a display of a head-worn client computing device, a three-dimensional video comprising images of a real-life scene that is remote from the user'"'"'s physical environment; and

present to the user, on the display of the head-worn client computing device, a graphical object comprising;

an image of the user'"'"'s physical environment;

or

a virtual graphical object.

29. The apparatus of claim 28, wherein:

the head-worn client computing device comprises an image sensor; and

presenting to the user an image of the user'"'"'s physical environment comprises presenting to the user an image captured by the image sensor.

30. The apparatus of claim 28, wherein the virtual graphical object comprises a notification.

31. A wearable device comprising:

a display;

one or more non-transitory computer-readable storage media embodying instructions; and

one or more processors coupled to the media and configured to execute the instructions to;

present on the display a three-dimensional video comprising a sequence of images, at least two of the images comprising;

a three-dimensional image of at least a portion of a physical environment of the wearable headset device; and

a three-dimensional image of at least a portion of an environment remote from the physical environment of the wearable headset device.

32. A method comprising:

by a wearable computing device, presenting on a display of the device a three-dimensional video comprising a sequence of images, at least two of the images comprising;

a three-dimensional image of at least a portion of a physical environment of the wearable headset device; and

a three-dimensional image of at least a portion of an environment remote from the physical environment of the wearable headset device.