Portable augmented reality device and method

• US 20060038833A1
• Filed: 08/19/2004
• Published: 02/23/2006
• Est. Priority Date: 08/19/2004
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Inc. (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment Incorporated (Sony Group Corp.)
• Inventors: Marks, Richard L., Mallinson, Dominic S.
• Granted Patent: US 8,547,401 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/217 using environment-related information, i.e. information generated otherwise than by the player, e.g. ambient temperature or humidity
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/6081 generating an output signal, e.g. under timing constraints, for spatialization
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 1/00323 : with a measuring, monitoring or signaling apparatus, e.g. for transmitting measured information to a central location
• CPC Class Codes: H04W 4/33 : for indoor environments, e.g. buildings

Abstract

A portable device configured to provide an augmented reality experience is provided. The portable device has a display screen configured to display a real world scene. The device includes an image capture device associated with the display screen. The image capture device is configured to capture image data representing the real world scene. The device includes image recognition logic configured to analyze the image data representing the real world scene. Image generation logic responsive to the image recognition logic is included. The image generation logic is configured to incorporate an additional image into the real world scene. A computer readable medium and a system providing an augmented reality environment are also provided.

Claims

1. A portable device configured to provide an augmented reality experience, comprising:

a display screen configured to display a real world scene;

an image capture device associated with the display screen, the image capture device configured to capture image data representing the real world scene;

image recognition logic configured to analyze the image data representing the real world scene; and

image generation logic responsive to the image recognition logic, the image generation logic configured to incorporate an additional image into the real world scene.

2. The portable device of claim 1, wherein the image capture device is removable.

3. The portable device of claim 1, wherein the image capture device is affixed to one of a surface above the display screen or a surface behind the display screen.

4. The portable device of claim 1, wherein the image recognition logic is configured to identify a marker in the data representing the real world scene, the image recognition logic further configured to communicate identification of the marker to the image generation logic.

5. The portable device of claim 4, wherein the image generation logic constructs the additional image in response to the identification of the marker.

6. The portable device of claim 1, further comprising:

a memory card slot for inserting a memory card; and

wireless networking access capability.

7. The portable device of claim 6, further comprising:

a memory card inserted into the memory card slot, the memory card configured to be shared with other portable devices.

8. A method for augmenting display data presented to a viewer, comprising method operations of:

capturing the display data with an image capture device;

analyzing the captured display data;

identifying a marker within the captured display data;

defining additional display data in response to identifying the marker; and

presenting the captured display data and the additional display data on a display screen associated with the image capture device.

9. The method of claim 8, wherein the display data is displayed on a television.

10. The method of claim 8, wherein a portable device coupled to the image capture device is capable of executing a video game.

11. The method of claim 8, wherein the marker is hand drawn.

12. The method of claim 8, wherein the image capture device is a video capture device.

13. The method of claim 8, wherein the method operation of analyzing the captured display data includes,

comparing the captured display data with data stored in a memory of a portable device detachably linked to the image capture device.

14. A computer readable medium having program instructions for augmenting display data presented to a viewer, comprising:

program instructions for capturing the display data with an image capture device;

program instructions for analyzing the captured display data;

program instructions for identifying a marker within the captured display data;

program instructions for defining additional display data in response to identifying the marker; and

program instructions for presenting the captured display data and the additional display data on a display screen associated with the image capture device.

15. The computer readable medium of claim 14, wherein the display data is displayed on a television.

16. The computer readable medium of claim 14, wherein a portable device detachably linked to the image capture device is capable of executing a video game.

17. The computer readable medium of claim 14, wherein the marker is hand drawn.

18. The computer readable medium of claim 14, wherein the image capture device is a video capture device.

19. The computer readable medium of claim 14, wherein the program instructions for analyzing the captured display data includes,

program instructions for comparing the captured display data with data stored in a memory of a portable device coupled to the image capture device.

20. A method for providing information in a portable environment, comprising method operations of:

capturing an image of a first object in a first location;

accessing object characteristics of the first object based upon the image of the first object;

capturing an image of a second object in a second location;

accessing object characteristics of the second object based upon the image of the second object; and

presenting the object characteristics of the first object and the object characteristics of the second object.

21. The method of claim 20, wherein the method operation of accessing object characteristics of the first object based upon the image of the first object includes,

accessing a local wireless network of the first location.

22. The method of claim 20, wherein the method operation of accessing object characteristics of the second object based upon the image of the second object includes,

accessing a local wireless network of the second location;

comparing the image of the second object to image data stored within an image library associated with the local wireless; and

identifying a match between the image of the second object and image data within the image library.

23. The method of claim 20, wherein the first location and the second location are different stores.

24. The method of claim 20, wherein the method operation of presenting the object characteristics of the first object and the object characteristics of the second object includes,

comparing prices of the first object and the second object.

25. A computer readable medium having program instructions for providing information to a portable device, comprising:

program instructions for capturing an image of a first object in a first location;

program instructions for accessing object characteristics of the first object based upon the image of the first object;

program instructions for capturing an image of a second object in the second location;

program instructions for accessing object characteristics of the second object based upon the image of the second object; and

program instructions for presenting the object characteristics of the first object and the object characteristics of the second object.

26. The computer readable medium of claim 25, wherein the program instructions for accessing object characteristics of the first object based upon the image of the first object includes,

program instructions for accessing a local wireless network of the first location.

27. The computer readable medium of claim 25, wherein the program instructions for accessing object characteristics of the second object based upon the image of the second object includes,

program instructions for accessing a local wireless network of the second location;

program instructions for comparing the image of the second object to image data stored within an image library associated with the local wireless; and

program instructions for identifying a match between the image of the second object and image data within the image library.

28. The computer readable medium of claim 25, wherein the first location and the second location are different stores.

29. The computer readable medium of claim 25, wherein the program instructions for presenting the object characteristics of the first object and the object characteristics of the second object includes,

program instructions for comparing prices of the first object and the second object.

30. A system for providing an augmented reality environment, comprising:

a wireless network;

a portable device having a display screen and image capture capability, the portable device configured to access the wireless network, the portable device including,

image recognition logic configured to analyze image data captured through an image capture device, the image recognition logic further configured to compare the image data to image data accessible through the wireless network in order to determine a match; and

image/audio generation logic responsive to the image recognition logic, the image/audio generation logic configured to overlay additional image data and include an audio clip into the data captured through the image capture device.

31. The system of claim 30, wherein the additional image data and the audio clip are downloaded through the wireless network.

32. The system of claim 30, wherein the image data captured through the image capture device is a business card and the additional image data is a picture of an owner of the business card.

33. The system of claim 30, wherein the image data captured through the image capture device includes a hand drawn graphic, associated with a complex 3-dimensional graphic accessed through one of the image/audio generation logic or the wireless network.

34. The system of claim 30, wherein the additional image data and the audio clip represent a portion of a music compact disc or a video clip of a movie.

35. The system of claim 30, wherein the wireless network is located within confines of a retail store.

36. The system of claim 30, wherein the additional image data and the audio clip represent a portion of a video game.

37. The system of claim 35, wherein a catalogue of the retail store items are accessible through the wireless network.

38. The system of claim 30, wherein the additional image data is one of video data or a still image.