DISPLAY METHOD AND DISPLAY APPARATUS

• US 20170206417A1
• Filed: 12/16/2016
• Published: 07/20/2017
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: MIYOSHI, Kengo, AOYAMA, Hideki, OSHIMA, Mitsuaki, NAKANISHI, Koji, MAEDA, Toshiyuki, UEKI, Akihiro, MUKAI, Tsutomu
• Granted Patent: US 10,303,945 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2358/00 Arrangements for display data security
• CPC Class Codes: G09G 2360/16 Calculation or use of calculated indices related to luminance levels in display data
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 2370/18 Use of optical transmission of display information
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/377 for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence
• CPC Class Codes: H04B 10/1149 Arrangements for indoor wireless networking of information
• CPC Class Codes: H04B 10/116 Visible light communication
• CPC Class Codes: H04J 3/0635 Clock or time synchronisation in a network timer in protocols H04L69/28
• CPC Class Codes: H04L 1/0045 Arrangements at the receiver end
• CPC Class Codes: H04L 1/0061 Error detection codes
• CPC Class Codes: H04L 1/0071 Use of interleaving interleaving per se H03M13/27
• CPC Class Codes: H04L 7/0033 : Correction by delay
• CPC Class Codes: H04L 7/0091 : Transmitter details
• CPC Class Codes: H04M 1/72403 : with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72412 : using two-way short-range wireless interfaces
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 5/2628 : Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 : Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A display method is for a display apparatus to display an image, and includes: obtaining a captured display image and a decode target image by an image sensor capturing an image of a subject; obtaining a light ID by decoding the decode target image; transmitting the light ID to a server; obtaining, from the server, an AR image and recognition information which are associated with the light ID; recognizing a region according to the recognition information as a target region from the captured display image; and displaying the captured display image in which the AR image is superimposed on the target region.

Claims

1. A display method for a display apparatus to display an image, the display method comprising:

(a) obtaining a captured display image and a decode target image by an image sensor capturing an image of a subject;

(b) obtaining light identification information by decoding the decode target image;

(c) transmitting the light identification information to a server;

(d) obtaining, from the server, an augmented reality image and recognition information which are associated with the light identification information;

(e) recognizing a region according to the recognition information as a target region from the captured display image; and

(f) displaying the captured display image in which the augmented reality image is superimposed on the target region.

2. The display method according to claim 1, wherein

the subject is an object illuminated by a transmitter which transmits a signal by changing luminance,

the augmented reality image is a video which includes images, and

in (f), the video is displayed, starting with one of, among the images, an image which includes the object and a predetermined number of images which are to be displayed around a time at which the image which includes the object is to be displayed.

3. The display method according to claim 2, wherein

the predetermined number of images are ten frames.

4. The display method according to claim 2, wherein

the object is a still image, and

in (f), the video is displayed, starting with an image same as the still image.

5. The display method according to claim 1, wherein

the recognition information is reference information for locating a reference region of the captured display image, and

in (e), the reference region is located from the captured display image, based on the reference information, and the target region is recognized from the captured display image, based on a position of the reference region.

6. The display method according to claim 1, wherein

the recognition information includes reference information for locating a reference region of the captured display image, and target information indicating a relative position of the target region with respect to the reference region, and

in (e), the reference region is located from the captured display image, based on the reference information, and a region in the relative position indicated by the target information is recognized as the target region from the captured display image, based on a position of the reference region.

7. The display method according to claim 6, wherein

the reference information indicates that the position of the reference region in the captured display image matches a position of a bright line pattern region in the decode target image, the bright line pattern region including a pattern formed by bright lines which appear due to exposure lines included in the image sensor being exposed.

8. The display method according to claim 6, wherein

the reference information indicates that the reference region in the captured display image is a region in which a display is shown in the captured display image.

9. The display method according to claim 8, wherein

the reference region is an outer frame portion having a predetermined color in an image displayed on the display.

10. The display method according to claim 1, wherein

in (f), a first augmented reality image which is the augmented reality image is displayed for a predetermined display period, while preventing display of a second augmented reality image different from the first augmented reality image.

11. The display method according to claim 10, wherein

in (f), decoding a decode target image newly obtained is prohibited during the predetermined display period.

12. The display method according to claim 11, wherein

(f) further includes:

measuring an acceleration of the display apparatus using an acceleration sensor during the display period;

determining whether the measured acceleration is greater than or equal to a threshold; and

displaying the second augmented reality image instead of the first augmented reality image by no longer preventing the display of the second augmented reality image, if the measured acceleration is determined to be greater than or equal to the threshold.

13. The display method according to claim 1, wherein

(f) further includes:

determining whether a face of a user is approaching the display apparatus, based on image capturing by a face camera included in the display apparatus; and

displaying a first augmented reality image which is the augmented reality image while preventing display of a second augmented reality image different from the first augmented reality image, if the face is determined to be approaching.

14. The display method according to claim 1, wherein

(f) further includes:

determining whether a face of a user is approaching the display apparatus, based on an acceleration of the display apparatus measured by an acceleration sensor; and

displaying a first augmented reality image which is the augmented reality image while preventing display of a second augmented reality image different from the first augmented reality image, if the face is determined to be approaching.

15. The display method according to claim 1, wherein

in (a), the captured display image and the decode target image are obtained by the image sensor capturing an image which includes a plurality of displays each showing an image and being the subject,

in (e), a region in which, among the plurality of displays, a transmission display that is transmitting the light identification information is shown is recognized as the target region from the captured display image, and

in (f),

first subtitles for an image displayed on the transmission display are superimposed on the target region, as the augmented reality image, and

second subtitles obtained by enlarging the first subtitles are further superimposed on a region larger than the target region of the captured display image.

16. The display method according to claim 15, wherein

(f) further includes:

determining whether information obtained from the server includes sound information; and

preferentially outputting sound indicated by the sound information over the first subtitles and the second subtitles, if the sound information is determined to be included.

20. A display apparatus which displays an image, the display apparatus comprising:

a processor; and

a memory in which a computer program is stored, wherein

the computer program causes the processor to perform the display method according to claim 1 when the computer program is executed by the processor.

21. A non-transitory recording medium in which a computer program is stored, the computer program causing a processor to perform the display method according to claim 1 when the computer program is executed by the processor.

17. A display method, comprising:

(a) obtaining a captured image by an image sensor capturing an image of, as a subject, an object illuminated by a transmitter which transmits a signal by changing luminance;

(b) decoding the signal from the captured image; and

(c) reading a video corresponding to the decoded signal from a memory, superimposing the video on a target region corresponding to the subject in the captured image, and displaying, on a display, the captured image in which the video is superimposed on the target region, wherein

in (c), the video is displayed, starting with one of, among images included in the video, an image which includes the object and a predetermined number of images which are to be displayed around a time at which the image which includes the object is to be displayed.

18. The display method according to claim 17, wherein

the object is a still image, and

in (c), the video is displayed, starting with an image same as the still image.

19. The display method according to claim 18, wherein

the still image includes an outer frame having a predetermined color,

the display method further comprising:

recognizing the target region from the captured image, based on the predetermined color, wherein

in (c), the video is resized to a size of the recognized target region, the resized video is superimposed on the target region in the captured image, and the captured image in which the resized video is superimposed on the target region is displayed on the display.