Method of processing three-dimensional image in mobile device

• US 8,749,555 B2
• Filed: 04/07/2009
• Issued: 06/10/2014
• Est. Priority Date: 01/07/2005
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Tae Seong, Lee, Min Jeong, Cho, Hang Shin
• Primary Examiner(s): Chauhan, Ulka
• Assistant Examiner(s): Prendergast, Roberta D
• Application Number:
  US12/420,004
• Publication Number:
  US 20090195539A1
• Time in Patent Office: 1,890 Days
• Field of Search: 345/473
• US Class Current: 345/473
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/406 Transmission via wireless network, e.g. pager or GSM
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals

Abstract

A processing method of interfacing a 3D image and a camera image is provided. In the processing method, a specific image pattern defined by a user is recognized, the recognized pattern is traced within an image, and a camera image and a 3D image are interfaced based on the tracing result. A 3D object is animated and rendered using a 3D graphic engine. The rendered image of the 3D object and the camera image are integrated and displayed.

Claims

1. A method for interfacing images, the method comprising:

acquiring an image of a real object via a camera, wherein the image comprises a plurality of specific patterns and the real object is a user, wherein each specific pattern of the specific patterns is designated from a color, a region, or a previously stored image file using a user-oriented passive method for defining the specific pattern by a user, wherein a characteristic of each specific pattern of the specific patterns is designated by setting a color of the specific pattern, or setting a region of the specific pattern, or selecting the previously stored image file, and the user approves the desired pattern of the defined patterns;

recognizing a first pattern, of the plurality of specific patterns, within the acquired image, wherein the first pattern relates to a first at least one body part of the user that is moveable in order to form the first pattern;

recognizing a second pattern, of the plurality of specific patterns, within the acquired image, wherein the second pattern relates to a second at least one body part of the user that is moveable in order to form the second pattern;

displaying a virtual image comprising a first portion and a second portion which respectively correspond to the recognized first pattern and the recognized second pattern;

displaying a previously stored first graphic image in conjunction with the virtual image;

changing a location of the first portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the first at least one body part and the camera;

changing a location of the second portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the second at least one body part and the camera, wherein the changing of the location of the first portion is independent of the changing of the location of the second portion;

determining a correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the correspondence identifies that a displayed location of a portion of the displayed first graphic image intersects with a displayed location of the first portion or the second portion of the displayed virtual image;

increasing a value according to repeated correspondence between the displayed first graphic image and either the first portion or the second portion of the displayed virtual image;

performing an animation process responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the animation process causes a change in location of the displayed first graphic image such that the determined correspondence between the displayed first graphic image and the first portion or the second portion of the displayed virtual image no longer occurs, and wherein the animation process further causes a change in displayed content of a portion of the displayed first graphic image;

displaying a third graphic image that is different from each of the virtual image, and the displayed first graphic image, the displaying of the third graphic image occurring responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image; and

ceasing the displaying of the third graphic image after a period of time.

2. The method according to claim 1, wherein the displayed virtual image moves responsive to the occurrence of the correspondence.

3. The method according to claim 1, wherein the method is performed in one of a mobile phone, a PDA (personal digital assistant), a PDA phone, a smart phone, a notebook computer, or a PMP (portable multimedia player).

4. The method according to claim 1, wherein the relative movement between the first at least one body part and the camera and between the second at least one body part and the camera are detected in real time.

5. The method according to claim 1, wherein the relative movement between the first at least one body part and the camera and between the second at least one body part and the camera are detected in real time by comparing frames in the acquired image.

6. The method according to claim 1, wherein the first and second patterns are defined by the user.

7. The method according to claim 1, wherein the stored first graphic image comprises a plurality of images including a specific image representing a user controlled image.

8. The method according to claim 1, wherein the acquired image including the first and second patterns comprises an image of the user.

9. The method according to claim 1, wherein the animation process results in a different type of animation according to a degree of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image.

10. The method according to claim 1, wherein at least the first graphic image or the virtual image is displayed differently after the animation process is performed.

11. A method of controlling an apparatus by interfacing images, the method comprising:

acquiring an image of a real object via a camera, wherein the image comprises a plurality of specific patterns and the real object is a user, wherein each specific pattern of the specific patterns is designated from a color, a region, or a previously stored image file using a user-oriented passive method for defining the specific pattern by a user, wherein a characteristic of each specific pattern of the specific patterns is designated by setting a color of the specific pattern, or setting a region of the specific pattern, or selecting the previously stored image file, and the user approves the desired pattern of the defined patterns;

recognizing a first pattern, of the plurality of specific patterns, within the acquired image, wherein the first pattern relates to a first at least one body part of the user that is moveable in order to form the first pattern;

recognizing a second pattern, of the plurality of specific patterns, within the acquired image, wherein the second pattern relates to a second at least one body part of the user that is moveable in order to form the second pattern;

displaying a virtual image comprising a first portion and a second portion which respectively correspond to the recognized first pattern and the recognized second pattern;

displaying a previously stored first graphic image in conjunction with the virtual image;

changing a location of the first portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the first at least one body part and the camera, wherein the changing of the location of the first portion is independent of the changing of the location of the second portion;

changing a location of the second portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the second at least one body part and the camera;

determining a correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the correspondence identifies that a displayed location of a portion of the displayed first graphic image intersects with a displayed location of the first portion or the second portion of the displayed virtual image;

increasing a value according to repeated correspondence between the displayed first graphic image and either the first portion or the second portion of the displayed virtual image;

performing an animation process responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the animation process causes a change in location of the displayed first graphic image such that the determined correspondence between the displayed first graphic image and the first portion or the second portion of the displayed virtual image no longer occurs, and wherein the animation process further causes a change in displayed content of a portion of the displayed first graphic image;

displaying a third graphic image that is different from each of the virtual image, and the displayed first graphic image, the displaying of the third graphic image occurring responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image; and

ceasing the displaying of the third graphic image after a period of time;

controlling the apparatus in response to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the apparatus comprises one of a mobile phone, a PDA (personal digital assistant), a PDA phone, a smart phone, a notebook computer, or a PMP (portable multimedia player).

12. The method according to claim 11, wherein the displayed virtual image is movable responsive to the occurrence of the correspondence between the displayed first graphic image and either the first portion or the second portion of the displayed virtual image.

13. The method according to claim 11, wherein the relative movement between the first at least one body part and the camera and between the second at least one body part and the camera is detected in real time.

14. The method according to claim 11, wherein the relative movement between the first at least one body part and the camera and between the second at least one body part and the camera is detected by comparing frames in the acquired image.

15. The method according to claim 11, wherein the apparatus is controlled to perform a different function according to a degree of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image.

16. An apparatus comprising:

a camera configured to acquire an image of a real object, the image comprising a plurality of specific patterns and the real object is a user, wherein each specific pattern of the specific patterns is designated from a color, a region, or a previously stored image file using a user-oriented passive method for defining the specific pattern by a user, wherein a characteristic of each specific pattern of the specific patterns is designated by setting a color of the specific pattern, or setting a region of the specific pattern, or selecting the previously stored image file, and the user approves the desired pattern of the defined patterns;

a processor configured to recognize a first pattern, of the plurality of specific patterns, within the acquired image, wherein the first pattern relates to a first at least one body part of the user that is moveable in order to form the first pattern, and being further configured to recognize a second pattern, of the plurality of specific patterns, within the acquired image, wherein the second pattern relates to a second at least one body part of the user that is moveable in order to form the second pattern;

a memory configured to store a first graphic image;

a display configured to display a virtual image comprising a first portion and a second portion which respectively correspond to the recognized first pattern and the recognized second pattern;

the display further configured to display the stored first graphic image in conjunction with the virtual image,

wherein the processor is further configured to;

cause a change in location of the first portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the first at least one body part and the camera;

cause a change in location of the second portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the second at least one body part and the camera, wherein the change in the location of the first portion is independent of the change in the location of the second portion;

determine a correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the correspondence identifies that a displayed location of a portion of the displayed first graphic image intersects with a displayed location of the first portion or the second portion of the displayed virtual image;

increasing a value according to repeated correspondence between the displayed first graphic image and either the first portion or the second portion of the displayed virtual image;

perform an animation process responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the animation process causes a change in location of the displayed first graphic image such that the determined correspondence between the displayed first graphic image and the first portion or the second portion of the displayed virtual image no longer occurs, and wherein the animation process further causes a change in displayed content of a portion of the displayed first graphic image;

cause the display to display a third graphic image that is different from each of the virtual image, and the displayed first graphic image, the displaying of the third graphic image occurring responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image; and

cause the display to cease the displaying of the third graphic image after a period of time.

17. An apparatus comprising:

a camera configured to acquire an image of a real object, the image comprising a plurality of specific patterns and the real object is a user, wherein each specific pattern of the specific patterns is designated from a color, a region, or a previously stored image file using a user-oriented passive method for defining the specific pattern by a user, wherein a characteristic of each specific pattern of the specific patterns is designated by setting a color of the specific pattern, or setting a region of the specific pattern, or selecting the previously stored image file, and the user approves the desired pattern of the defined patterns;

a processor configured to recognize a first pattern, of the plurality of specific patterns, within the acquired image, wherein the first pattern relates to a first at least one body part of the user that is moveable in order to form the first pattern, and being further configured to recognize a second pattern, of the plurality of specific patterns, within the acquired image, wherein the second pattern relates to a second at least one body part of the user that is moveable in order to form the second pattern;

a memory configured to store a first graphic image; and

a display configured to display a virtual image which corresponds to the recognized specific pattern, the display further configured to display a previously stored first graphic image in conjunction with the virtual image,

wherein the processor is further configured to;

cause a change in location of the first portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the first at least one body part and the camera;

cause a change in location of the second portion of the displayed virtual image relative to the displayed first graphic image according to relative movement detected between the second at least one body part and the camera, wherein the change in the location of the first portion is independent of the change in the location of the second portion;

determine a correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the correspondence identifies that a displayed location of a portion of the displayed first graphic image intersects with a displayed location of the first portion or the second portion of the displayed virtual image;

increasing a value according to repeated correspondence between the displayed first graphic image and either the first portion or the second portion of the displayed virtual image;

perform an animation process responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image, wherein the animation process causes a change in location of the displayed first graphic image such that the determined correspondence between the displayed first graphic image and the first portion or the second portion of the displayed virtual image no longer occurs, and wherein the animation process further causes a change in displayed content of a portion of the displayed first graphic image;

cause the display to display a third graphic image that is different from each of the virtual image, and the displayed first graphic image, the displaying of the third graphic image occurring responsive to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image;

cause the display to cease the displaying of the third graphic image after a period of time; and

control the apparatus in response to an occurrence of the correspondence between the displayed first graphic image and any of the first portion or the second portion of the displayed virtual image.