Light field light source orientation method for augmented reality and virtual reality and front-end device

• US 10,002,464 B1
• Filed: 12/16/2016
• Issued: 06/19/2018
• Est. Priority Date: 11/29/2016
• Status: Active Grant

• Current Assignee: Pacific Future Limited
• Original Assignee: Pacific Future Limited
• Inventors: Thew, Ian Roy, Lee, Kien Yi
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US15/381,137
• Publication Number:
  US 20180150998A1
• Time in Patent Office: 550 Days
• Field of Search: None
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G09G 2320/0242 Compensation of deficiencies in the appearance of colours
• CPC Class Codes: G09G 2320/0646 Modulation of illumination source brightness and image signal correlated to each other
• CPC Class Codes: G09G 2320/0666 for control of colour parameters, e.g. colour temperature
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/10 Intensity circuits

Abstract

The present invention relates to a light field light source orientation method for augmented reality and virtual reality and a front-end device. The method comprises: A: identifying a target marker; B. tracking the target marker; C. analyzing the pixel color shape on the marker object; D. analyzing color difference zones on the marker object and analyzing the shape cast to calculate direction of environmental light source; E. pushing the light source directional data to an augmented reality object; and F. compensating and adjusting the imaging of an augmented reality object: the present invention can collect surrounding environmental factors, such as the light source direction, so as to project a computer-generated object into a reality environment to possess a shadow consistent with that in the reality, so that an augmented reality environment is more realistic and represents a realist shadow effect of the object.

Claims

1. A light field light source orientation method for augmented reality and virtual reality, comprising the steps of:

identifying a target mark as follows;

collecting image data through a front-end device and identifying the collected image data, the front-end device being a mobile phone or helmet glasses;

tracking the target mark as follows;

tracking the position of a certain object in the image data by the front-end device;

obtaining the surface shape and feature of the certain object, and obtaining a data value of a color space of the certain object, the data value comprising;

hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value;

analyzing a grey zone on the certain object as follows;

analyzing, by the front-end device, a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;

pushing the data to a light field of a virtual object as follows;

pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and

compensating and adjusting the imaging of the virtual object;

adjusting the virtual object and a shadow of the virtual object through the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.

2. The method of claim 1, wherein the step of analyzing a grey zone on the certain object is as follows:

converting the refraction angle, the length and the brightness into identifiable codes of the front-end device;

determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle;

determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone; and

determining the intensity of the light source according to identification codes of the brightness of the grey zone.

3. The method of claim 1, comprising steps of adjusting brightness, hue and saturation of the virtual object through the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source;

adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and

the virtual scenario is more realistic and clear.

4. A front-end device, the front-end device comprising a mobile phone or helmet glasses comprising:

a processor;

a memory coupled with the processor, wherein the processor is configured to execute programmed instructions stored in the memory for

a target mark identifying process used for collecting image data and identifying the collected image data;

a target mark tracking process used for tracking the position of a certain object in the image data, obtaining the surface shape and feature of the certain object, and obtaining a data value of a color space of the certain object, the data value comprising;

hue-color essential attributes, saturation-color purity of 0-100% value and lightness and brightness of 0-100% value;

a grey zone analyzing process used for analyzing a BLOB of the certain object, judging a refraction angle, length and brightness of the grey zone of the certain object according to the BLOB, and determining a light source direction in a place where the front-end device is located, a distance between the front-end device and a light source, and the intensity of the light source according to the refraction angle, the length and the brightness of the grey zone;

a data pushing process used for pushing the data into the light field of the virtual object after converting the data into identifiable data of the front-end device according to the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and

a compensation and adjustment process used for adjusting the virtual object and a shadow of the virtual object through the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source, so that a virtual scenario is more realistic and clear.

5. The front-end device of claim 4, wherein the grey zone analyzing process is used for converting the refraction angle, the length and the brightness into identifiable codes of the front-end device;

determining the light source direction in the place where the front-end device is located according to identification codes of the refraction angle;

determining the distance between the front-end device and the light source according to identification codes of the length of the grey zone; and

determining the intensity of the light source according to identification codes of the brightness of the grey zone.

6. The front-end device of claim 4, wherein the compensation and adjustment process is used for adjusting brightness, hue and saturation of the virtual object through the data value of the color space of the certain object, the light source direction, the distance between the front-end device and the light source and the intensity of the light source;

adjusting the refraction angle, the length and the brightness of the shadow of the virtual object according to the light source direction, the distance between the front-end device and the light source and the intensity of the light source; and

the virtual scenario is more realistic and clear.