Information processing apparatus and method, information processing system, and providing medium

• US 7,716,606 B2
• Filed: 09/19/2003
• Issued: 05/11/2010
• Est. Priority Date: 10/19/1998
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Rekimoto, Junichi
• Primary Examiner(s): Huynh; Ba
• Application Number:
  US10/667,287
• Publication Number:
  US 20040056902A1
• Time in Patent Office: 2,426 Days
• Field of Search: 715/764, 715848-851, 715/865, 715/866, 709/203, 725/114, 725/62
• US Class Current: 715/848
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

The invention enables users to virtually attach information to situations in the real world, and also enables users to quickly and easily find out desired information. An IR sensor receives an IR signal transmitted from an IR beacon, and supplies the received signal to a sub-notebook PC. A CCD video camera takes in a visual ID from an object, and supplies the inputted visual ID to the sub-notebook PC. A user inputs, through a microphone, a voice to be attached to situations in the real world. The sub-notebook PC transmits position data, object data and voice data, which have been supplied to it, to a server through a communication unit. The transmitted data is received by the server via a wireless LAN. The server stores the received voice data in a database in correspondence to the position data and the object data.

Claims

1. A method of attaching data items to a physical environment, the method comprising:

capturing surrounding contexts in a particular physical environment, the surrounding contexts including at least one of position context and object context, and capturing data items including at least one of visual data and audio data, wherein the surrounding contexts and the data items are sensed by a plurality of sensing means, wherein the position context represents a position in the particular physical environment and is based on a plurality of electric field intensities sensed by the sensing means, and where in the position context is based on a distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electric wave;

attaching data items to the surrounding contexts;

sending the attached data items and the captured surrounding contexts to an external apparatus for storing;

transmitting the captured position context to the external apparatus for retrieving a data item stored in the external apparatus; and

receiving the data item stored in the external apparatus based on the transmitted position context.

2. The method of claim 1, further comprising selecting a date and a time when attaching the data items to the surrounding contexts.

3. The method of claim 2, wherein the selected date and time includes a future or past date and time.

4. The method of claim 1, wherein the object context is for identifying at least one object in the particular physical environment.

5. The method of claim 1, wherein the capturing is continuously performed so that the data items are always captured.

6. The method of claim 1, wherein the position context identifies a room in the particular physical environment.

7. The method of claim 1, wherein the audio data includes voice data.

8. The method of claim 1, wherein the visual data includes image data.

9. The method of claim 1, further comprising sensing an IR signal transmitted from an IR beacon located in the particular physical environment, wherein the position context is based on the IR signal.

10. An apparatus for attaching data items to a physical environment, the apparatus comprising:

capturing means for capturing surrounding context in a particular physical environment, the surrounding contexts including at least one of position context and object context, and capturing data items including at least one of visual data and audio data, wherein the surrounding contexts and the data items are sensed by a plurality of sensing means, wherein the position context represents a position in the particular physical environment and is based on a plurality of electric field intensities sensed by the sensing means, and wherein the position context is based on a distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electric wave;

attaching means for attaching data items to the surrounding contexts;

sending means for sending the attached data items and the captured surrounding contexts to an external apparatus for storing;

transmitting means for transmitting the captured position context to the external apparatus for retrieving a data item stored in the external apparatus; and

receiving means for receiving the data item stored in the external apparatus based on the transmitted position context.

11. The apparatus of claim 10, further comprising selecting means for selecting a date and a time when attaching the data items to the surrounding contexts.

12. The apparatus of claim 11, wherein the selected date and time includes a future or past date and time.

13. The apparatus of claim 10, wherein the object context is for identifying at least one object in the particular physical environment.

14. The apparatus of claim 10, wherein the capturing is continuously performed so that the data items are always captured.

15. The apparatus of claim 10, wherein the position context identifies a room in the particular physical environment.

16. The apparatus of claim 10, wherein the audio data includes voice data.

17. The apparatus of claim 10, wherein the visual data includes image data.

18. The apparatus claim 10, further comprising:

IR sensing means for sensing an IR signal transmitted from an IR beacon located in the particular physical environment, wherein the position context is based on the IR signal.

19. method for attaching data items to a physical environment, the method comprising:

capturing a plurality of surrounding contexts in a particular physical environment, the surrounding contexts including location data and time data, wherein the surrounding contexts are sensed by a plurality of sensing means, wherein the location data represents a position in the particular physical environment and is based on a plurality of electric field intensities sensed by the sensing means, and wherein the location data is based on distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electric wave;

inputting keyword and data including at least one of visual data, audio data, and text data regarding the surrounding contexts, wherein the keyword and the data including at least one of the visual data, the audio data, and the text data are attached to the surrounding contexts;

sending the location data, the time data, the keyword, and the data including at least one of the visual data, the audio data, and the text data to a database for storing;

transmitting at least one of the location data, the time data, and the keyword to the database for retrieving the data including at least one of the visual data, the audio data, and the text data stored in the database;

receiving the data including at least one of the visual data, the audio data, and the text data stored in the database in a result of the retrieving; and

displaying the received data including at least one of the visual data, the audio data, and the text data.

20. The method of claim 19, further comprising sensing an IR signal transmitted from an IR beacon located in the particular physical environment, wherein the location data is based on the IR signal.

21. An apparatus for attaching data items to a physical environment, the apparatus comprising:

capturing means for capturing a plurality of surrounding contexts in a particular physical environment, the surrounding contexts including location data and time data, wherein the surrounding contexts are sensed by a plurality of sensing means, wherein the location data represents a position in the particular physical environment and is based on a plurality of electric field intensities sensed by the sensing means, and wherein the location data is based on a distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electronic wave;

inputting means for inputting keyword and data including at least one of visual data, audio data, and text data regarding the surrounding contexts, wherein the keyword and the data including at least one of the visual data, the audio data, and the text data are attached to the surrounding contexts;

sending means for sending the location data, the time data, the keyword, and the data including at least one of the visual data, the audio data, and the text data to a database for storing;

transmitting means for transmitting at least one of the location data, the time data, and the keyword to the database for retrieving a data including at least one of visual data, audio data, and text data stored in the database;

receiving means for receiving the data including at least one of the visual data, the audio data, and the text data stored in the database in a result of the retrieving; and

displaying means for displaying the received data including at least one of the visual data, the audio data, and the text data.

22. The apparatus of claim 21, further comprising:

IR sensing means for sensing an IR signal transmitted from an IR beacon located in the particular physical environment, wherein the location data is based on the IR signal.

23. A method for storing data items attached to surrounding contexts in a physical environment, the method comprising:

receiving location data, time data, keyword, and data including at least one of visual data, audio data and text data from an other apparatus, wherein the location data represents a position in the physical environment, wherein the location data is based on a distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electric wave;

storing the location data, the time data, the keyword, and the data including at least one of the visual data, the audio data, and the text data in a database in a corresponding manner, wherein the location data, the time data, and the keyword are attached as a retrieval key for retrieving the data including at least one of the visual data, the audio data, and the text data;

retrieving the database in response to a request from the other apparatus; and

sending the data, to the other apparatus, including at least one of the visual data, the audio data, and the text data in a result of the retrieving.

24. An apparatus for storing data items attached to surrounding contexts in a physical environment, the apparatus comprising:

receiving means for receiving location data, time data, keyword, and data including at least one of visual data, audio data and text data from an other apparatus, wherein the location data represents a position in the physical environment, wherein the location data is based on a distance calculated from a similarity determination among pairs of an electronic field intensity of an electric wave and a unique ID of each apparatus transmitting the electric wave;

storing means for storing the location data, the time data, the keyword, and the data including at least one of the visual data, the audio data, and the text data in a database in a corresponding manner, wherein the location data, the time data, and the keyword are attached as a retrieval key for retrieving the data including at least one of the visual data, the audio data and the text data;

retrieving means for retrieving the database in response to a request from the other apparatus; and

sending means for sending the data, to the other apparatus, including at least one of the visual data, the audio data, and the text data in a result of the retrieving.