RECONFIGURING REALITY USING A REALITY OVERLAY DEVICE

• US 20100103075A1
• Filed: 10/24/2008
• Published: 04/29/2010
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Kalaboukis, Chris, Perry, Marc, Crane, Barry, Douris, Stephan
• Granted Patent: US 9,480,919 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/24 Constructional details thereof, e.g. game controllers with detachable joystick handles
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/90 Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/205 for detecting the geographical location of the game platform
• CPC Class Codes: A63F 2300/303 for displaying additional data, e.g. simulating a Head Up Display
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/5573 player location
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 : Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 : characterised by optical features
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 : Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 : Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 : Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 2219/2016 : Rotation, translation, scaling
• CPC Class Codes: G09G 3/001 : using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 5/003 : Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

Virtual entities are displayed alongside real world entities in a wearable reality overlay device worn by the user. Information related to an environment proximate to the wearable device is determined. For example, a position of the wearable device may be determined, a camera may capture an image of the environment, etc. Virtual entity image information representative of an entity desired to be virtually displayed is processed based on the determined information. An image of the entity is generated based on the processed image information as a non-transparent region of a lens of the wearable device, enabling the entity to appear to be present in the environment to the user. The image of the entity may conceal a real world entity that would otherwise be visible to the user through the wearable device. Other real world entities may be visible to the user through the wearable device.

Claims

1. A method, comprising:

determining information related to an environment proximate to a wearable device;

receiving image information processed based at least on the determined information, the image information being representative of an entity; and

generating an image of the entity based on the processed image information as a non-transparent region of a lens of the wearable device to enable the entity to appear to be present in the environment to a user of the wearable device.

2. The method of claim 1, wherein said generating comprises:

projecting the image of the entity on the lens.

3. The method of claim 1, wherein said generating comprises:

selectively activating pixels of the lens to form the image of the entity.

4. The method of claim 1, wherein said determining information related to an environment proximate to a wearable device comprises:

determining at least one of a location of the wearable device, an orientation of the wearable device, or a speed at which the wearable device is moving.

5. The method of claim 4, further comprising:

processing the image information representative of the entity based on the determined at least one of the location of the wearable device, the orientation of the wearable device, or the speed at which the wearable device is moving.

6. The method of claim 1, wherein said generating comprises:

optically aligning the image of the first entity on the lens with a second entity visible through the lens to at least partially conceal the second entity to the user of the wearable device.

7. The method of claim 6, wherein said determining information related to an environment proximate to a wearable device comprises:

determining at least one of a location of the second entity, an orientation of the second entity, or a speed at which the second entity is moving.

8. The method of claim 7, further comprising:

processing the image information representative of the first entity based on the determined at least one of the location of the second entity, the orientation of the second entity, or the speed at which the second entity is moving.

9. The method of claim 6, wherein said determining information related to an environment proximate to a wearable device comprises:

capturing an image of the second entity; and

processing the captured image to determine a location of the second entity.

10. The method of claim 1, further comprising:

receiving updated processed image information representative of the entity that is processed based at least on updated determined information related to the environment proximate to a wearable device; and

generating an updated image of the entity based on the updated processed image information as an updated non-transparent region of the lens.

11. The method of claim 1, further comprising:

generating a second image of the entity based on the received image information as a non-transparent region of a second lens of the wearable device.

12. The method of claim 1, wherein the image information is representative of a plurality of entities, wherein said generating comprises:

generating a plurality of images corresponding to the plurality of entities based on the processed image information as a plurality of non-transparent regions of the lens.

13. The method of claim 1, wherein the entity is a playing field, wherein said generating comprises:

generating the image of the playing field to enable the user to view the playing field using the wearable device; and

the method further comprising;

enabling the user to participate in a game configured to take place in the playing field.

14. The method of claim 13, further comprising:

receiving second image information processed based at least on the determined information, the second image information being representative of an implement of the game; and

generating an image of the implement based on the processed second image information as a second non-transparent region of the lens of the wearable device to enable the implement of the game to appear to be present in the environment to the user of the wearable device.

15. A wearable device, comprising:

a position monitor configured to determine a position of the wearable device;

a lens; and

a display generator configured to receive image information processed based at least on the determined position, the image information being representative of an entity;

wherein the display generator is configured to generate an image of the entity as a non-transparent region of the lens based on the processed image information to enable the entity to appear to be present to a user of the wearable device in an environment proximate to the wearable device.

16. The wearable device of claim 15, wherein the display generator comprises:

an image projector configured to project the image of the entity on the lens.

17. The wearable device of claim 15, wherein the display generator is a display device that includes an array of image pixels, the lens comprises the display device, the display device is configured to selectively activate pixels of the array of image pixels to form the image of the entity, and non-activated pixels of the array of image pixels are transparent.

18. The wearable device of claim 15, wherein the position monitor is further configured to determine at least one of an orientation of the wearable device or a speed of the wearable device.

19. The wearable device of claim 18, further comprising:

an image processor configured to process the image information representative of the entity based at least on the determined position of the wearable device, the determined orientation of the wearable device, or the determined speed of the wearable device to generate the processed image information.

20. The wearable device of claim 15, wherein the display generator is configured to optically align the image of the first entity on the lens with a second entity visible through the lens to at least partially conceal the second entity to the user of the wearable device based on the processed image information.

21. The wearable device of claim 20, further comprising:

an image processor configured to process the image information representative of the second entity based at least on a determined position of the second entity, a determined orientation of the second entity, or a speed of the second entity to generate the processed image information.

22. The wearable device of claim 20, further comprising:

a camera configured to capture an image of the environment;

wherein the image processor is configured to process an image of the second entity captured by the camera to determine a location of the second entity.

23. The wearable device of claim 15, further comprising:

wherein the display generator is configured to receive updated processed image information representative of the entity that is processed based at least on updated determined information related to the environment proximate to a wearable device, and to generate an updated image of the entity based on the updated processed image information as an updated non-transparent region of the lens.

24. The wearable device of claim 15, further comprising:

a second display generator configured to generate a second image of the entity based on received image information as a non-transparent region of a second lens of the wearable device.

25. The wearable device of claim 15, further comprising:

a game engine;

wherein the entity is a playing field, wherein the display generator is configured to generate the image of the playing field to enable the user to view the playing field using the wearable device; and

wherein the game engine is configured to enable the user to participate in a game configured to take place in the playing field.

26. The wearable device of claim 25, wherein the display generator is configured to receive second image information processed based at least on the determined information, the second image information being representative of an implement of the game, and to generate an image of the implement based on the processed second image information as a second non-transparent region of the lens of the wearable device to enable the implement of the game to appear to be present in the environment to the user of the wearable device.

27. The wearable device of claim 15, further comprising:

a communication interface configured to transmit information representative of the determined position information from the wearable device, and to receive the image information processed based at least on the determined position.

28. A reality overlay image processing server, comprising:

a communication interface configured to receive position information from a wearable device;

a storage that stores image information representative of an entity; and

an entity image processor configured to process the image information representative of the entity based at least on the determined position;

wherein the communication interface is configured to transmit the processed image information to the wearable device, the wearable device being configured to generate an image of the entity as a non-transparent region of a lens based on the processed image information to enable the entity to appear to be present to a user of the wearable device in an environment proximate to the wearable device.

29. The server of claim 28, further comprising;

a game engine configured to enable the user to participate in a game that includes the entity.