AUGMENTED REALITY DISPLAY SYSTEM FOR OVERLAYING APPAREL AND FITNESS INFORMATION

• US 20180350148A1
• Filed: 06/06/2017
• Published: 12/06/2018
• Est. Priority Date: 06/06/2017
• Status: Active Grant

• Current Assignee: PerfectFit Systems Pvt., Ltd.
• Original Assignee: PerfectFit Systems Pvt., Ltd.
• Inventors: George, Eobin Alex
• Granted Patent: US 10,665,022 B2
• Time in Patent Office: Days
• CPC Class Codes: A63B 24/0075 Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip

Abstract

The present disclosure relates to an augmented reality display system. The augmented reality display system includes an augmented reality mirror, a display device, a scanning setup and a data processing system. The data processing system receives the first set of data of a user and collects the second set of data associated with the user. Moreover, the data processing system analyzes the first and second set of data in real time and estimates the position of each feature of the body of the user. Also, the data processing system slices each feature of the body of the user to measures a set of feature metrics. Further, the data processing system create one or more feature metric databases. Moreover, the data processing system maps the feature metric database with a pre-defined set of products. The data processing system overlay and display the information related to the fitness, health and apparel.

Claims

1. An augmented reality display system for rendering an image of a user of one or more users on a display of a mirror for overlaying one or more information on the mirror in real time, the augmented reality display system comprising:

an augmented reality mirror, the augmented reality mirror comprising a reflective coating on a first region of the augmented reality mirror and a dispersive coating on a second region of the augmented reality mirror, wherein the first region and the second region being opposite to each other;

a display device positioned at an angle near the second region of the augmented reality mirror, wherein the display device being configured for displaying an augmented reality image of the user located in proximity of the augmented reality mirror in real time;

a scanning system positioned in a vicinity of the augmented reality mirror;

wherein the scanning system being configured for obtaining body scans of the user located in the proximity of the augmented reality mirror, wherein the scanning being done in real time; and

a processor coupled to the scanning system, the processor being configured to;

receiving, at a data processing system, a first set of data comprising a plurality of depth maps of the body of the user, the first set of data characterizes a three-dimensional geometry of the body of the user from a plurality of spatial parameters, wherein the first set of data being analyzed to infer the plurality of spatial parameters of the body of the user and wherein the plurality of spatial parameters comprises a posture of the body, a position of each feature of a plurality of features of the body, an axis of reference and a position of joints;

collecting, at the data processing system, a second set of data associated with the user, wherein the second set of data comprises user preference data and a position of the user with respect to the augmented reality mirror;

analyzing, at the data processing system, the first set of data and the second set of data in real time;

estimating, at the data processing system, the position of each feature of the plurality of features of the body of the user;

slicing, at the data processing system, each feature of the plurality of features of the body of the user at a slicing angle along the axis of reference and through a geometric center of corresponding features, the slicing angle being corrected for each feature along the axis of reference and the approximate geometric center being determined for each feature of the plurality of features of the body;

measuring, at the data processing system, a set of feature metrics of each sliced feature of the plurality of features, the set of feature metrics being associated with spatial geometric measurements of the body, wherein the set of feature metrics of each sliced feature being measured by elimination of at least one of speckle noise, one or more non-feature regions, creases and folds;

creating, at the data processing system, one or more feature metric databases corresponding to a measured set of feature metrics for each sliced feature of the plurality of features, wherein each feature metric database of the one or more feature metric databases comprises a normalized size and a range of sizes for each feature of the body of the user;

mapping, at the data processing system, one or more records of the one or more feature metric databases with corresponding feature metric records of a pre-defined set of products;

overlaying, at the data processing system, at least one of one or more products over the body of the user and one or more information on the augmented reality mirror in real time, wherein the one or more products being overlaid based on one or more recommendations of products based on a positive mapping of the one or more records of the one or more feature metric databases of the body with corresponding feature metric records of the products, wherein the overlaying of the one or more products being done based on the second set of data; and

displaying, at the data processing system, at least one of the overlaid one or more products over the body of the user and the one or more fitness related information on the augmented reality mirror in real time, wherein the displaying being done through the display device in real time.

2. The augmented reality display system as recited in claim 1, wherein the processor being further configured for constructing, at the data processing system, one or more three-dimensional body models depicting the body of the user based on selection of the one or more recommendations of the products.

3. The augmented reality display system as recited in claim 1, wherein the products comprises apparels, footwear, eyewear and wearable accessories and wherein each three-dimensional body model being rendered with one or more layers of virtual fittings of one or more products of the pre-defined set of products.

4. The augmented reality display system as recited in claim 1, wherein the processor being further configured for developing, at the data processing system, a body profile, an exercise profile, a diet profile and a product selection profile of the user, wherein the body profile comprises the set of feature metrics based on date stamped and time stamped progressive scans of the body of the user, wherein the exercise profile comprises a progress data of performed exercises, data points for changes in body metrics, and a time and date of exercise routine, wherein the diet profile comprises a diet routine of the user and a set of calories consumed for each diet routine segregated on basis of an event, diet routines in a day, a week, a month and an year and wherein the product selection profile comprises a list of product selection from the one or more recommendation of products, and a preference of cloth type, material type, fabric type, fitting, color, brand, size, price, designer and reseller.

5. The augmented reality display system as recited in claim 1, wherein the processor being further configured for sharing, at the data processing system, a plurality of product features corresponding to each product of the pre-defined set of products to a third party, wherein the third party being selected from a group comprising an product designer, a product manufacturer, a product distributor and a retailer and wherein the plurality of product features comprises a set of product dimensions from body fitting, a closure type, a fabric type, a material elasticity, a material texture and a material weight of each product.

6. The augmented reality display system as recited in claim 1, wherein the set of feature metrics comprises a body height, a leg length, an arm length, a shoulder width, a chest radius, a waist radius, a buttocks radius, an arm radius, a leg radius, a torso length, a shoulder length, an angle of inclination of shoulder and a neck radius, wherein the position of each feature of the plurality of features of the body of the user is estimated based on a machine learning model trained on stochastic modeling of a plurality of data points of the set of data and wherein, the one or more non-feature regions in each sliced feature of the plurality of features of the body comprises features of background objects, worn apparels and devices.

7. The augmented reality display system as recited in claim 1, wherein the processor being further configured for monitoring, at the data processing system, a deviation in one or more exercise postures of the body of the user for each fitness exercise routine from a reference set of exercise postures in real time, wherein the monitoring being done based on the machine learning model.

8. The augmented reality display system as recited in claim 1, wherein the processor being further configured for presenting, at the data processing system, a life-sized virtual assistant to the user based on one or more gestures,one or more body postures, one or more audio inputs and one or more light or pressure activation switches by the user in real time.

9. The augmented reality display system as recited in claim 1, wherein the processor being further configured for recommending, at the data processing system, one or more fitness exercise routines and one or more diet routines to the user based on date stamped and time stamped progressive analysis of one or more changes in each feature of the plurality of features of the body, wherein the one or more fitness exercise routine being rendered on the one or more virtual fitness models of the user for a visual workout plan.

10. The augmented reality display system as recited in claim 1, wherein the processor being further configured for storing, at the data processing system, the first set of data, the second set of data, a body profile, a diet profile, a product selection profile and an exercise profile of the user, wherein the body profile, the diet profile, the product selection profile and the exercise profile being stored in real time.

11. A computer-implemented method for rendering an image of a user of one or more users on a display of a mirror for overlaying one or more information on the mirror in real time, the computer-implemented method comprising:

receiving, at a data processing system with a processor, a first set of data comprising a plurality of depth maps of the body of the user, the first set of data characterizes a three-dimensional geometry of the body of the user from a plurality of spatial parameters, wherein the first set of data being analyzed to infer the plurality of spatial parameters of the body of the user and wherein the plurality of spatial parameters comprises a posture of the body, a position of each feature of a plurality of features of the body, an axis of reference and a position of joints;

collecting, at the data processing system with the processor, a second set of data associated with the user, wherein the second set of data comprises user preference data and a position of the user with respect to an augmented reality mirror;

analyzing, at the data processing system with the processor, the first set of data and the second set of data in real time;

estimating, at the data processing system with the processor, the position of each feature of the plurality of features of the body of the user;

slicing, at the data processing system with the processor, each feature of the plurality of features of the body of the user at a slicing angle along the axis of reference and through a geometric center of corresponding features, the slicing angle being corrected for each feature along the axis of reference and the approximate geometric center being determined for each feature of the plurality of features of the body;

measuring, at the data processing system with the processor, a set of feature metrics of each sliced feature of the plurality of features, the set of feature metrics being associated with spatial geometric measurements of the body, wherein the set of feature metrics of each sliced feature being measured by elimination of at least one of speckle noise, one or more non-feature regions, creases and folds;

creating, at the data processing system with the processor, one or more feature metric databases corresponding to a measured set of feature metrics for each sliced feature of the plurality of features, wherein each feature metric database of the one or more feature metric databases comprises a normalized size and a range of sizes for each feature of the body of the user;

mapping, at the data processing system with the processor, one or more records of the one or more feature metric databases with corresponding feature metric records of a pre-defined set of products;

overlaying, at the data processing system with the processor, at least one of one or more products over the body of the user and one or more information on the augmented reality mirror in real time, wherein the one or more products being overlaid based on one or more recommendations of products based on a positive mapping of the one or more records of the one or more feature metric databases of the body with corresponding feature metric records of the products, wherein the overlaying of the one or more products being done based on the second set of data; and

displaying, at the data processing system with the processor, at least one of the overlaid one or more products over the body of the user and the one or more fitness related information on the augmented reality mirror in real time, wherein the displaying being done through the display device in real time.

12. The computer-implemented method as recited in claim 11, further comprising constructing, at the data processing system with the processor, one or more three-dimensional body models depicting the body of the user based on selection of the one or more recommendations of the products.

13. The computer-implemented method as recited in claim 11,wherein the products comprises apparels, footwear, eyewear and wearable accessories and wherein each three-dimensional body model being rendered with one or more layers of virtual fittings of one or more products of the pre-defined set of products.

14. The computer-implemented method as recited in claim 11, further comprising developing, at the data processing system with the processor, a body profile, an exercise profile, a diet profile and a product selection profile of the user, wherein the body profile comprises the set of feature metrics based on date stamped and time stamped progressive scans of the body of the user, wherein the exercise profile comprises a progress data of performed exercises, data points for changes in body metrics, and a time and date of exercise routine, wherein the diet profile comprises a diet routine of the user and a set of calories consumed for each diet routine segregated on basis of an event, diet routines in a day, a week, a month and an year and wherein the product selection profile comprises a list of product selection from the one or more recommendation of products, and a preference of cloth type, material type, fabric type, fitting, color, brand, size, price, designer and reseller.

15. The computer-implemented method as recited in claim 11, further comprising sharing, at the data processing system with the processor, a plurality of product features corresponding to each product of the pre-defined set of products to a third party, wherein the third party being selected from a group comprising an product designer, a product manufacturer, a product distributor and a retailer and wherein the plurality of product features comprises a set of product dimensions from body fitting, a closure type, a fabric type, a material elasticity, a material texture and a material weight of each product.

16. The computer-implemented method as recited in claim 11, wherein the set of feature metrics comprises a body height, a leg length, an arm length, a shoulder width, a chest radius, a waist radius, a buttocks radius, an arm radius, a leg radius, a torso length, a shoulder length, an angle of inclination of shoulder and a neck radius, wherein the position of each feature of the plurality of features of the body of the user is estimated based on a machine learning model trained on stochastic modeling of a plurality of data points of the set of data and wherein, the one or more non-feature regions in each sliced feature of the plurality of features of the body comprises features of background objects, worn apparels and devices.

17. The computer-implemented method as recited in claim 11, further comprising monitoring, at the data processing system with the processor, a deviation in one or more exercise postures of the body of the user for each fitness exercise routine from a reference set of exercise postures in real time, wherein the monitoring being done based on the machine learning model.

18. The computer-implemented method as recited in claim 11, further comprising presenting, at the data processing system with the processor, a life-sized virtual assistant to the user based on one or more gestures, one or more body postures, one or more audio inputs and one or more light or pressure activation switches by the user in real time.

19. The computer-implemented method as recited in claim 11, further comprising recommending, at the data processing system with the processor, one or more fitness exercise routines and one or more diet routines to the user based on date stamped and time stamped progressive analysis of one or more changes in each feature of the plurality of features of the body, wherein the one or more fitness exercise routine being rendered on the one or more virtual fitness models of the user for a visual workout plan.

20. The computer-implemented method as recited in claim 11, further comprising storing, at the data processing system with the processor, the first set of data, the second set of data, a body profile, a diet profile, a product selection profile and an exercise profile of the user, wherein the body profile, the diet profile, the product selection profile and the exercise profile being stored in real time.