Information processing apparatus and method, information processing system, and providing medium

• US 9,501,142 B2
• Filed: 10/25/2012
• Issued: 11/22/2016
• Est. Priority Date: 10/19/1998
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Rekimoto, Junichi
• Primary Examiner(s): Theriault, Steven B
• Application Number:
  US13/660,332
• Publication Number:
  US 20130050261A1
• Time in Patent Office: 1,489 Days
• Field of Search: 715/848
• US Class Current: 1/1
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

The invention enables users to virtually attach information to situations in the real world, and also enables users to quickly and easily find out desired information. An IR sensor receives an IR signal transmitted from an IR beacon, and supplies the received signal to a sub-notebook PC. A CCD video camera takes in a visual ID from an object, and supplies the inputted visual ID to the sub-notebook PC. A user inputs, through a microphone, a voice to be attached to situations in the real world. The sub-notebook PC transmits position data, object data and voice data, which have been supplied to it, to a server through a communication unit. The transmitted data is received by the server via a wireless LAN. The server stores the received voice data in a database in correspondence to the position data and the object data.

Claims

1. An information processing apparatus comprising:

at least one processor; and

a memory device storing instructions which when executed by the at least one processor cause the information processing apparatus to;

receive, at the information processing apparatus, first time information indicative of a first time, first position information indicative of a physical position of a wearable augmentable reality user device at the first time, the wearable augmentable reality user device being located at a different location than the information processing apparatus, and corresponding first attachment information from the wearable augmentable reality user device at the first time;

create a virtual community space displaying the first attachment information with the first position information;

receive, at the information processing apparatus, second time information indicative of a second time after the first time and corresponding second attachment information at the second time;

update the virtual community space to display the second attachment information as new information in relation to the first attachment information representative of a time passed between the first time and the second time; and

transmit the updated virtual community space to the wearable augmentable reality user device,

wherein the virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

2. The information processing apparatus of claim 1, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

3. The information processing apparatus of claim 1, wherein the virtual community space is a bulletin board system.

4. The information processing apparatus of claim 1, wherein the first attachment information includes a uniform resource locator.

5. The information processing apparatus of claim 1, wherein the first attachment information includes a photograph and a text.

6. The information processing apparatus of claim 1, wherein the first attachment information includes an image captured by a camera in the wearable augmentable reality user device.

7. The information processing apparatus of claim 1, wherein the first attachment information includes advertisement information.

8. The information processing apparatus of claim 1, further comprising:

a storage device; and

a communication device.

9. The information processing apparatus of claim 8, wherein the storage device stores the virtual user community space in the storage device.

10. The information processing apparatus of claim 8, wherein the memory device stores instructions which when executed by the at least one processor cause the information processing apparatus to:

receive, via the communication device, a request corresponding to the first position.

11. A wearable augmentable reality user device comprising:

at least one processor; and

a memory device storing instructions which when executed by the at least one processor cause the wearable augmentable reality user device to;

receive, at the wearable augmentable reality user device, an updated virtual community space displaying first attachment information from a second wearable augmentable reality user device, the first attachment information corresponding to a first time, and second attachment information corresponding to a second time after the first time, and the first attachment information corresponding to first position information indicative of a physical position of the second wearable augmentable reality user device at the first time, the second wearable augmentable reality user device being located at a different location than the wearable augmentable reality user device; and

control a display device to display the updated virtual community space, wherein the second attachment information is displayed as new information in relation to the first attachment information representative of a time passed between the first time and the second time,

wherein the updated virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the updated virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

12. The wearable augmentable reality user device of claim 11, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

13. The wearable augmentable reality user device of claim 11, wherein the virtual community space is a bulletin board system.

14. The wearable augmentable reality user device of claim 11, wherein the memory device stores instructions which when executed by the at least one processor cause the wearable augmentable reality user device to:

start an application for displaying the updated virtual community space.

15. The wearable augmentable reality user device of claim 11, wherein the first attachment information includes a uniform resource locator.

16. The wearable augmentable reality user device of claim 11, wherein the first attachment information includes a link for on-line shopping.

17. The wearable augmentable reality user device of claim 11, wherein the first attachment information includes a photograph and a text.

18. The wearable augmentable reality user device of claim 11, wherein the first attachment information includes an image captured by a camera in a wearable augmentable reality user device.

19. The wearable augmentable reality user device of claim 11, wherein the first attachment information includes advertisement information.

20. The wearable augmentable reality user device of claim 11, further comprising:

a display device;

an input device; and

a communication device.

21. The wearable augmentable reality user device of claim 20, wherein the display device displays the virtual user community space.

22. The wearable augmentable reality user device of claim 20, wherein the memory device stores instructions which when executed by the at least one processor cause the wearable augmentable reality user device to:

receive, via the input device, a request corresponding to the first position; and

transmit, via the communication device, the request corresponding to the first position.

23. A non-transitory computer readable medium storing instructions which, when executed, cause at least one processor in an information processing apparatus to:

receive, at the information processing apparatus, first time information indicative of a first time, first position information indicative of a physical position of a wearable augmentable reality user device at the first time, the wearable augmentable reality user device being located at a different location than the information processing apparatus, and corresponding first attachment information from the wearable augmentable reality user device at the first time;

create a virtual community space displaying the first attachment information with the first position information;

receive, at the information processing apparatus, second time information indicative of a second time after the first time and corresponding second attachment information at the second time;

update the virtual community space to display the second attachment information as new information in relation to the first attachment information representative of a time passed between the first time and the second time; and

transmit the updated virtual community space to the wearable augmentable reality user device,

wherein the virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

24. The non-transitory computer readable medium of claim 23, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

25. The non-transitory computer readable medium of claim 23, wherein the virtual community space is a bulletin board system.

26. The non-transitory computer readable medium of claim 23, wherein the first attachment information includes a uniform resource locator.

27. The non-transitory computer readable medium of claim 23, wherein the first attachment information includes a photograph and a text.

28. The non-transitory computer readable medium of claim 23, wherein the first attachment information includes an image captured by a camera in the wearable augmentable reality user device.

29. The non-transitory computer readable medium of claim 23, wherein the first attachment information includes advertisement information.

30. The non-transitory computer readable medium of claim 23, wherein the information processing apparatus comprises:

a storage device; and

a communication device.

31. The non-transitory computer readable medium of claim 30, wherein the storage device stores the virtual user community space in the storage device.

32. The non-transitory computer readable medium of claim 30, wherein the information processing apparatus receives, via the communication device, a request corresponding to the first position.

33. A non-transitory computer readable medium storing instructions which, when executed, cause at least one processor in a wearable augmentable reality user device to:

receive, at the wearable augmentable reality user device, an updated virtual community space displaying first attachment information from a second wearable augmentable reality user device, the first attachment information corresponding to a first time, and second attachment information corresponding to a second time after the first time, and the first attachment information corresponding to first position information indicative of a physical position of the second wearable augmentable reality user device at the first time, the second wearable augmentable reality user device being located at a different location than the wearable augmentable reality user device; and

control a display device to display the updated virtual community space, wherein the second attachment information is displayed as new information in relation to the first attachment information representative of a time passed between the first time and the second time,

wherein the updated virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the updated virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

34. The non-transitory computer readable medium of claim 33, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

35. The non-transitory computer readable medium of claim 33, wherein the virtual community space is a bulletin board system.

36. The non-transitory computer readable medium of claim 33, wherein the wearable augmentable reality user device starts an application for displaying the updated virtual community space.

37. The non-transitory computer readable medium of claim 33, wherein the first attachment information includes a uniform resource locator.

38. The non-transitory computer readable medium of claim 33, wherein the first attachment information includes a link for on-line shopping.

39. The non-transitory computer readable medium of claim 33, wherein the first attachment information includes a photograph and a text.

40. The non-transitory computer readable medium of claim 33, wherein the first attachment information includes an image captured by a camera in the wearable augmentable reality user device.

41. The non-transitory computer readable medium of claim 33, wherein the first attachment information includes advertisement information.

42. The non-transitory computer readable medium of claim 33, wherein the wearable augmentable reality user device comprises:

a display device;

an input device; and

a communication device.

43. The non-transitory computer readable medium of claim 42, wherein the display device displays the virtual user community space.

44. The non-transitory computer readable medium of claim 42, wherein the wearable augmentable reality user device receives, via the input device, a request corresponding to the first position, and transmits, via the communication device, the request corresponding to the first position.

45. A method comprising:

receiving, at an information processing apparatus, first time information indicative of a first time, first position information indicative of a physical position of a wearable augmentable reality user device at the first time, the wearable augmentable reality user device being located at a different location than the information processing apparatus, and corresponding first attachment information from the wearable augmentable reality user device at the first time;

creating a virtual community space displaying the first attachment information with the first position information;

receiving, at the information processing apparatus, second time information indicative of a second time after the first time and corresponding second attachment information at the second time;

updating the virtual community space to display the second attachment information as new information in relation to the first attachment information representative of a time passed between the first time and the second time; and

transmitting the updated virtual community space to the wearable augmentable reality user device,

wherein the virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

46. The method of claim 45, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

47. The method of claim 45, wherein the virtual community space is a bulletin board system.

48. The method of claim 45, wherein the first attachment information includes a uniform resource locator.

49. The method of claim 45, wherein the first attachment information includes a photograph and a text.

50. The method of claim 45, wherein the first attachment information includes an image captured by a camera in the wearable augmentable reality user device.

51. The method of claim 45, wherein the first attachment information includes advertisement information.

52. The method of claim 45, wherein an information processing apparatus configured to perform the method comprises:

a storage device; and

a communication device.

53. The method of claim 52, wherein the storage device stores the virtual user community space in the storage device.

54. The method of claim 52, wherein the information processing apparatus receives, via the communication device, a request corresponding to the first position.

55. A method comprising:

receiving, at a wearable augmentable reality user device, an updated virtual community space displaying first attachment information from a second wearable augmentable reality user device, the first attachment information corresponding to a first time, and second attachment information corresponding to a second time after the first time, and the first attachment information corresponding to first position information indicative of a physical position of the second wearable augmentable reality user device at the first time, the second wearable augmentable reality user device being located at a different location than the wearable augmentable reality user device; and

controlling a display device to display the updated virtual community space, wherein the second attachment information is displayed as new information in relation to the first attachment information representative of a time passed between the first time and the second time,

wherein the updated virtual community space includes a context-aware area and a personal information tray, wherein the context-aware area corresponds to a real world associated with the wearable augmentable reality user device, and wherein an icon corresponding to the first attachment information is configured to be generated in the personal information tray, wherein a drag and drop operation of the icon is configured to be performed in the updated virtual community space, wherein the icon is configured to be dragged from a first position in the personal information tray and dropped at a second position in the context-aware area, and wherein accessing the icon provides the first attachment information.

56. The method of claim 55, wherein the virtual community space provides for an exchange of attachment information between a plurality of wearable augmentable reality user devices.

57. The method of claim 55, wherein the virtual community space is a bulletin board system.

58. The method of claim 55, further comprising:

starting an application for displaying the updated virtual community space.

59. The method of claim 55, wherein the first attachment information includes a uniform resource locator.

60. The method of claim 55, wherein the first attachment information includes a link for on-line shopping.

61. The method of claim 55, wherein the first attachment information includes a photograph and a text.

62. The method of claim 55, wherein the first attachment information includes an image captured by a camera in the wearable augmentable reality user device.

63. The method of claim 55, wherein the first attachment information includes advertisement information.

64. The method of claim 55, wherein the wearable augmentable reality user device comprises:

a display device;

an input device; and

a communication device.

65. The method of claim 64, wherein the display device displays the virtual user community space.

66. The method of claim 64, wherein the wearable augmentable reality user device receives, via the input device, a request corresponding to the first position, and transmits, via the communication device, the request corresponding to the first position.