Information display system, information display apparatus and non-transitory storage medium

• US 9,201,498 B2
• Filed: 12/07/2011
• Issued: 12/01/2015
• Est. Priority Date: 12/07/2010
• Status: Active Grant

• Current Assignee: Casio Computer Company Limited
• Original Assignee: Casio Computer Company Limited
• Inventors: Sakurai, Keiichi, Nakajima, Mitsuyasu, Yamaya, Takashi, Yoshihama, Yuki
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Wang, Yuehan
• Application Number:
  US13/313,334
• Publication Number:
  US 20120139941A1
• Time in Patent Office: 1,455 Days
• Field of Search: 345/663
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

The matching processor acquires the key information such as position information and/or the like by the key information acquirer, and notifies to an external apparatus via the communicator. The matching processor stores the acquired key information in the storer corresponding to the identification information which is acquired from the external apparatus. When the acquired identification information does not exist in a AR data management table, the matching processor acquires the AR data by notifying the identification information to the external apparatus. When there is no empty record in a key information management table, key information of which usage date and time is old is considered as a deletion target. Moreover, when there is no empty record in the AR data management table, the AR data of which usage date and time is old is considered as the deletion target.

Claims

1. An information display system for displaying information comprising:

an information display apparatus which includes an imaging unit; and

an information provision apparatus;

wherein the information display apparatus comprises;

a key information acquiring unit which acquires key information corresponding to a subject in a captured image captured by the imaging unit; and

a key information notifying unit which notifies the key information to the information provision apparatus each time the key information is acquired,

wherein the information provision apparatus comprises;

an AR data storage unit which stores augmented reality (AR) data corresponding to identification information for identifying the AR data and key information notified by the key information notifying unit, the AR data including (i) a reference image to be compared with the subject in the captured image captured by the imaging unit, and (ii) superimposing information to be superimposed on the captured image that includes the subject compared with the reference image; and

an information providing unit (i) which transmits the identification information corresponding to the key information to the information display apparatus, without transmitting the AR data corresponding to the identification information, when the key information is provided from the information display apparatus, and (ii) which transmits the AR data corresponding to the identification information to the information display apparatus when the identification information is provided from the information display apparatus,

wherein the information display apparatus further comprises;

an identification information acquiring unit which acquires the identification information provided from the information provision apparatus in accordance with a notification of the key information, without acquiring the AR data corresponding to the identification information;

a key information recording unit which adds and records new key information and new identification information in a table each time the new identification information is acquired, wherein the table contains first sets each comprising the key information and the identification information in association with each other, wherein the first sets do not include the AR data corresponding to the identification information;

an identification information notifying unit which notifies the identification information recorded in the table to the information provision apparatus;

an AR data acquiring unit which acquires the AR data provided from the information provision apparatus in accordance with a notification of the identification information;

an AR data recording unit which records second sets in the table other than the first sets, the second sets each comprising the identification information notified by the identification information notifying unit and the AR data acquired by the AR data acquiring unit in association with each other; and

an AR data displaying unit which superimposes and displays the superimposing information corresponding to the reference image on the captured image based on a result from matching the subject in the captured image with the reference image included in the AR data which is recorded by the AR data recording unit,

wherein the identification information notifying unit determines, each time the identification information is acquired, whether the AR data corresponding to the identification information is already recorded in the table, and (i) if the AR data corresponding to the identification information is determined to already be recorded, the identification information notifying unit does not notify the identification information to the information provision apparatus and the AR data acquiring unit does not reacquire the AR data which is already recorded, and (ii) if the AR data is determined to not be recorded, the identification information notifying unit notifies the identification information to the information provision apparatus, the information providing unit of the information provision apparatus transmits the AR data corresponding to the identification information to the information display apparatus in response to the notification of the identification information by the information display apparatus, the AR data acquiring unit of the information display apparatus acquires the AR data transmitted from the information provision apparatus, and the AR data recording unit records the newly acquired AR data in association with the corresponding identification information as a new second set in the table.

2. An information display apparatus comprising:

an imaging unit;

a communication unit which communicates with an external apparatus;

a key information acquiring unit which acquires key information corresponding to a subject in a captured image captured by the imaging unit;

a key information notifying unit which notifies the key information acquired by the key information acquiring unit to the external apparatus through a communication network each time the key information is acquired;

an identification information acquiring unit which acquires identification information for identifying augmented reality (AR) data which is to be transmitted by the external apparatus, the identification information acquiring unit acquiring the identification information without acquiring the AR data corresponding to the identification information, in accordance with a notification of the key information by the key information notifying unit through the communication network, the AR data including (i) a reference image to be compared with the subject in the captured image captured by the imaging unit, and (ii) superimposing information to be superimposed on the captured image that includes the subject compared with the reference image;

a key information recording unit which adds and records new key information and new identification information in a table each time the new identification information is acquired, wherein the table contains first sets each comprising the key information and the identification information in association with each other, wherein the first sets do not include the AR data corresponding to the identification information;

an identification information notifying unit which notifies the identification information recorded in the table to the external apparatus;

an AR data acquiring unit which acquires the AR data transmitted by the external apparatus in accordance with a notification of the identification information;

an AR data recording unit which records second sets in the table other than the first sets, the second sets each comprising the identification information notified by the identification information notifying unit and the AR data acquired by the AR data acquiring unit in association with each other; and

an AR data displaying unit which superimposes and displays the superimposing information corresponding to the reference image on the captured image based on a result from matching the subject in the captured image with the reference image included in the AR data which is recorded by the AR data recording unit,

wherein the identification information notifying unit determines, each time the identification information is acquired, whether the AR data corresponding to the identification information is already recorded in the table, and (i) if the AR data is determined to already be recorded, the identification information notifying unit does not notify the identification information to the external apparatus and the AR data acquiring unit does not reacquire the AR data which is already recorded, and (ii) if the AR data is determined to not be recorded, the identification information notifying unit notifies the identification information to the external apparatus, the AR data acquiring unit acquires the AR data which is transmitted by the external apparatus in response to the notification of the identification information, and the AR data recording unit records the newly acquired AR data in association with the corresponding identification information as a new second set in the table.

3. The information display device according to claim 2, wherein the table comprises:

a first table in which the first sets each comprising the key information and the identification information in association with each other are recorded by the key information recording unit, and

a second table in which the second sets each comprising the identification information and the AR data in association with each other are recorded by the AR data recording unit.

4. The information display device according to claim 3, wherein:

the first table and the second table are each defined by a number of records,

the identification information notifying unit notifies the identification information to the external apparatus when the identification information recorded in the first table is not recorded in the second table;

the key information recording unit executes a deletion based on a usage date and time of the key information already recorded, when no empty record is present in the first table at a time when the key information acquiring unit acquires the key information, and does not execute a deletion when an empty record is present; and

the AR data recording unit executes a deletion based on a usage date and time of the AR data already recorded, when no empty record is present in the second table at the time the identification information notifying unit notifies the identification information, and does not execute a deletion when an empty record is present.

5. The information display apparatus according to claim 4, wherein:

the key information recording unit records a date and time when acquiring the key information as the usage date and time of the key information, and updates the date and time of the key information when superimposing and displaying the superimposing information on the captured image using the AR data corresponding to the key information;

the AR data recording unit records a date and time when acquiring the AR data as the usage date and time of the AR data, and updates the date and time when superimposing and displaying the superimposing information on the captured image using the AR data; and

the key information recording unit and the AR data recording unit specify a record recording an oldest date and time as the usage date and time as a deletion target.

6. The information display apparatus according to claim 5, wherein:

the key information includes position information indicating a position of the information display apparatus at a time of imaging by the imaging unit; and

the key information recording unit;

(i) calculates a distance between the position indicated by the position information and a current position of the information display apparatus when the key information recorded in the record specified as the deletion target is the position information; and

(ii) excludes the record from being the deletion target when the calculated distance is smaller than a predetermined threshold.

7. The information display apparatus according to claim 6, wherein the AR data recording unit excludes the specified record from being the deletion target, when (i) the identification information recorded in the record specified as the deletion target is recorded in the first table, and (ii) the record recording the identification information in the first table has been excluded from being the deletion target by the key information recording unit.

8. An information display method, comprising:

acquiring key information corresponding to a subject in a captured image captured by an imaging unit;

notifying the key information to an external apparatus through a communication network each time the key information is acquired;

acquiring identification information for identifying augmented reality (AR) data which is to be transmitted by the external apparatus, the identification information being acquired without acquiring the AR data corresponding to the identification information, in accordance with a notification of the key information through the communication network, the AR data including (i) a reference image to be compared with the subject in the captured image captured by the imaging unit and (ii) superimposing information to be superimposed on the captured image that includes the subject compared with the reference image;

adding and recording new key information and new identification information in a table each time the new identification information is acquired, wherein the table contains first sets each comprising the key information and the identification information in association with each other, wherein the first sets do not include the AR data corresponding to the identification information;

notifying the identification information recorded in the table to the external apparatus;

acquiring the AR data transmitted by the external apparatus in accordance with a notification of the identification information;

recording second sets in the table other than the first sets, the second sets each comprising the notified identification information and the acquired AR data in association with each other;

superimposing and displaying the superimposing information corresponding to the reference image on the captured image based on a result from matching the subject in the captured image with the reference image included in the AR data which is recorded in the table,

wherein each time the identification information is acquired, it is determined whether the AR data corresponding to the identification information is already recorded in the table, and (i) if the AR data is determined to already be recorded, the identification information is not notified to the external apparatus and the AR data which is already recorded is not reacquired, and (ii) if the AR data is determined to not be recorded, the identification information is notified to the external apparatus, the AR data which is transmitted by the external apparatus in response to the notification of the identification information is acquired, and the newly acquired AR data is recorded in association with the corresponding identification information as a new second set in the table.

9. A non-transitory computer-readable storage medium which stores a program for causing a computer of an information display apparatus including an imaging unit, a communicating unit which communicates with an external apparatus, and a displaying unit, to perform functions comprising:

acquiring key information corresponding to a subject in a captured image captured by the imaging unit;

notifying the key information to the external apparatus through a communication network each time the key information is acquired;

acquiring identification information for identifying augmented reality (AR) data which is to be transmitted by the external apparatus, the identification information being acquired without acquiring the AR data corresponding to the identification information, in accordance with a notification of the key information through the communication network, the AR data including (i) a reference image to be compared with the subject in the captured image captured by the imaging unit and (ii) superimposing information to be superimposed on the captured image that includes the subject compared with the reference image;

adding and recording new key information and new identification information in a table each time the new identification information is acquired, wherein the table contains first sets each comprising the key information and the identification information in association with each other, wherein the first sets do not include the AR data corresponding to the identification information;

notifying the identification information recorded in the table to the external apparatus;

acquiring the AR data transmitted by the external apparatus in accordance with a notification of the identification information;

recording second sets in the table other than the first sets, the second sets each comprising the notified identification information and the acquired AR data in association with each other; and

superimposing and displaying the superimposing information corresponding to the reference image on the captured image based on a result from matching the subject in the captured image with the reference image included in the AR data which is recorded in the table,

wherein each time the identification information is acquired, it is determined whether the AR data corresponding to the identification information is already recorded in the table, and (i) if the AR data is determined to already be recorded, the identification information is not notified to the external apparatus and the AR data which is already recorded is not reacquired, and (ii) if the AR data is determined to not be recorded, the identification information is notified to the external apparatus, the AR data which is transmitted by the external apparatus in response to the notification of the identification information is acquired, and the newly acquired AR data is recorded in association with the corresponding identification information as a new second set in the table.