Augmented reality display system for overlaying apparel and fitness information

• US 10,665,022 B2
• Filed: 06/06/2017
• Issued: 05/26/2020
• Est. Priority Date: 06/06/2017
• Status: Active Grant

• Current Assignee: PerfectFit Systems Pvt., Ltd.
• Original Assignee: PerfectFit Systems Pvt., Ltd.
• Inventors: George, Eobin Alex
• Primary Examiner(s): Utama, Robert J
• Application Number:
  US15/614,901
• Publication Number:
  US 20180350148A1
• Time in Patent Office: 1,085 Days
• CPC Class Codes: A63B 24/0075 Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 19/0038 Sports
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip

Abstract

The present disclosure relates to an augmented reality display system. The augmented reality display system includes an augmented reality mirror, a display device, a scanning setup and a data processing system. The data processing system receives the first set of data of a user and collects the second set of data associated with the user. Moreover, the data processing system analyzes the first and second set of data in real time and estimates the position of each feature of the body of the user. Also, the data processing system slices each feature of the body of the user to measures a set of feature metrics. Further, the data processing system create one or more feature metric databases. Moreover, the data processing system maps the feature metric database with a pre-defined set of products. The data processing system overlay and display the information related to the fitness, health and apparel.

Claims

1. An augmented reality display system for rendering an image of a user of one or more users on a display of a mirror for overlaying one or more information on the mirror in real time, the augmented reality display system comprising:

an augmented reality mirror that comprises a reflective coating on a first region of the augmented reality mirror and a dispersive coating on a second region of the augmented reality mirror, wherein the first region and the second region are opposite to each other;

a display device positioned at an angle from the second region of the augmented reality mirror, wherein the display device is configured to display an augmented reality image of the user located in proximity of the augmented reality mirror in real time;

a scanning system positioned in a vicinity of the augmented reality mirror;

wherein the scanning system comprises one or more scanners that scan body of the user with different angles and position in real time to obtain at least one of (i) a 3 Dimensional image and (ii) a 2 Dimensional image of the user; and

a data processing system that comprises a processor that is configured to;

receive a first set of data comprising a plurality of depth maps of the body of the user, wherein the plurality of depth maps are generated from a plurality of depth images of body of the user;

analyze the first set of data by inferring a plurality of spatial parameters that comprise one or more of an axis of reference for the body, a height of the body, a skeletal model of the body with a position of joints, a posture of the body, or a plurality of features of the body;

collect a second set of data associated with the user, wherein the second set of data comprises user preference data and a position of the user with respect to the augmented reality mirror;

analyze the second set of data in real time;

estimate the position of each feature of the plurality of features of the body of the user along with a geometric center of each feature of the body, wherein the geometric center of each feature of the body is estimated using an axis of reference for the body, the height of the body, the skeletal model of the body with position of the joints and the posture of the body;

determine slicing positions for each feature of the plurality of features of the body based on the estimated geometric center of each feature of the plurality of features of the body;

measure a plurality of spatial geometric parameters of each feature of the plurality of features by slicing at the determined slicing positions at a slicing angle along the axis of reference for the body and through the geometric center of each feature of the body;

generate, a set of feature metrics based on the plurality of spatial geometric parameters of each feature of the plurality of features, wherein the set of feature metrics of each sliced feature is measured by elimination of one or more speckle noise, one or more non-feature regions, creases and folds;

create one or more feature metric databases using the set of feature metrics for each sliced feature of the plurality of features, wherein each feature metric database of the one or more feature metric databases comprises a normalized size and a range of sizes for each feature of the body of the user;

map one or more records of the one or more feature metric databases to record a pre-defined set of products;

overlay at least one of one or more products over the body of the user and one or more information on the augmented reality mirror in real time, wherein the one or more products are overlaid based on one or more recommendations of products based on a positive mapping of the one or more records of the one or more feature metric databases of the body with corresponding feature metric records of the products, wherein the overlaying of the one or more products is done based on the second set of data; and

display at least one of the overlaid one or more products over the body of the user and one or more fitness related information on the augmented reality mirror in real time based on one or more fitness exercise routines and one or more diet routines of the user, wherein the displaying is done through the display device in real time.

2. The augmented reality display system as recited in claim 1, wherein the processor is further configured for generate one or more three-dimensional body models matching the body of the user in real time based on selection of the one or more recommendations of the products.

3. The augmented reality display system as recited in claim 1, wherein the pre-defined set of products comprises apparels, footwear, eyewear and wearable accessories, wherein each three-dimensional body model is rendered with one or more layers of virtual fittings of one or more products of the pre-defined set of products.

4. The augmented reality display system as recited in claim 1, wherein the processor is further configured to develop a body profile, an exercise profile, a diet profile and a product selection profile of the user, wherein the body profile comprises the set of feature metrics based on date stamped and time stamped progressive scans of the body of the user, wherein the exercise profile comprises a progress data of performed exercises, data points for changes in body metrics, and a time and date of exercise routine, wherein the diet profile comprises a diet routine of the user and a set of calories consumed for each diet routine segregated on basis of an event, diet routines in a day, a week, a month and an year, wherein the product selection profile comprises a list of product selection from the one or more recommendation of products, and a preference of cloth type, material type, fabric type, fitting, color, brand, size, price, designer and reseller.

5. The augmented reality display system as recited in claim 1, wherein the processor is further configured to share a plurality of product features corresponding to each product of the pre-defined set of products to a third party, wherein the third party is selected from a group comprising an product designer, a product manufacturer, a product distributor and a retailer wherein the plurality of product features comprises a set of product dimensions from body fitting, a closure type, a fabric type, a material elasticity, a material texture and a material weight of each product.

6. The augmented reality display system as recited in claim 1, wherein the set of feature metrics comprises a body height, a leg length, an arm length, a shoulder width, a chest radius, a waist radius, a buttocks radius, an arm radius, a leg radius, a torso length, a shoulder length, an angle of inclination of shoulder and a neck radius, wherein the position of each feature of the plurality of features of the body of the user is estimated based on a machine learning model trained on stochastic modeling of a plurality of data points of the set of data, wherein the one or more non-feature regions in each sliced feature of the plurality of features of the body comprises features of background objects, worn apparels and devices.

7. The augmented reality display system as recited in claim 1, wherein the processor is further configured to recommend the one or more fitness exercise routines and the one or more diet routines to the user based on date stamped and time stamped progressive analysis of one or more changes in each feature of the plurality of features of the body, wherein the one or more fitness exercise routines are rendered on one or more virtual fitness models of the user for a visual workout plan.

8. The augmented reality display system as recited in claim 7, wherein the processor is further configured to monitor a deviation in one or more exercise postures of the body of the user for each of the one or more fitness exercise routines from a reference set of exercise postures in real time based on the machine learning model.

9. The augmented reality display system as recited in claim 1, wherein the processor is further configured to present a life-sized virtual assistant to the user based on one or more gestures, one or more body postures, one or more audio inputs and one or more light or pressure activation switches by the user in real time.

10. The augmented reality display system as recited in claim 1, wherein the processor is further configured to store the first set of data, the second set of data, a body profile, a diet profile, a product selection profile and an exercise profile of the user, wherein the body profile, the diet profile, the product selection profile and the exercise profile are stored in real time.

11. A computer-implemented method for rendering an image of a user of one or more users on a display of a mirror for overlaying one or more information on the mirror in real time, the computer-implemented method comprising:

scanning, at a scanning system with one or more scanners, scan body of the user with different angles and position in real time to obtain at least one of (i) a 3Dimensional image and (ii) a 2Dimensional image of the user;

receiving, at a data processing system with a processor, a first set of data comprising a plurality of depth maps of body of the user, wherein the plurality of depth maps are generated from a plurality of depth images of body of the user,

analyzing the first set of data by inferring a plurality of spatial parameters that comprise one or more of an axis of reference for the body, a height of the body, a skeletal model of the body with a position of joints, a posture of the body, or a plurality of features of the body;

collecting, at the data processing system with the processor, a second set of data associated with the user, wherein the second set of data comprises user preference data and a position of the user with respect to an augmented reality mirror;

analyzing, at the data processing system with the processor, the second set of data in real time;

estimating, at the data processing system with the processor, the position of each feature of the plurality of features of the body of the user along with a geometric center of each feature of the body, wherein the geometric center of each feature of the body is estimated using an axis of reference for the body, the height of the body, the skeletal model of the body with position of the joints and the posture of the body;

determining slicing positions for each feature of the plurality of features of the body based on the estimated geometric center of each feature of the plurality of features of the body;

measuring a plurality of spatial geometric parameters of each feature of the plurality of features by slicing at the determined slicing positions at a slicing angle along the axis of reference for the body and through the geometric center of each feature of the body;

based on the plurality of spatial geometric parameters of each feature of the plurality of features, wherein the set of feature metrics of each sliced feature is measured by elimination of one or more speckle noise, one or more non-feature regions, creases and folds;

creating, at the data processing system with the processor, one or more feature metric databases using the set of feature metrics for each sliced feature of the plurality of features, wherein each feature metric database of the one or more feature metric databases comprises a normalized size and a range of sizes for each feature of the body of the user;

mapping, at the data processing system with the processor, one or more records of the one or more feature metric databases to record a pre-defined set of products;

overlaying, at the data processing system with the processor, at least one of one or more products over the body of the user and one or more information on the augmented reality mirror in real time, wherein the one or more products are overlaid based on one or more recommendations of products based on a positive mapping of the one or more records of the one or more feature metric databases of the body with corresponding feature metric records of the products, wherein the overlaying of the one or more products is done based on the second set of data; and

displaying, at the data processing system with the processor, at least one of the overlaid one or more products over the body of the user and one or more fitness related information on the augmented reality mirror in real time based on one or more fitness exercise routines and one or more diet routines of the user, wherein the displaying is done through the display device in real time.

12. The computer-implemented method as recited in claim 11, further comprising generating, at the data processing system with the processor, one or more three-dimensional body models matching the body of the user in real time with one or more static or dynamic display positions, fitted with one or more of the recommended products on the display device of the user.

13. The computer-implemented method as recited in claim 11, wherein the pre-defined set of products comprises apparels, footwear, eyewear and wearable accessories, wherein each three-dimensional body model is rendered with one or more layers of virtual fittings of one or more products of the pre-defined set of products.

14. The computer-implemented method as recited in claim 11, further comprising developing, at the data processing system with the processor, a body profile, an exercise profile, a diet profile and a product selection profile of the user, wherein the body profile comprises the set of feature metrics based on date stamped and time stamped progressive scans of the body of the user, wherein the exercise profile comprises a progress data of performed exercises, data points for changes in body metrics, and a time and date of exercise routine, wherein the diet profile comprises a diet routine of the user and a set of calories consumed for each diet routine segregated on basis of an event, diet routines in a day, a week, a month and an year, wherein the product selection profile comprises a list of product selection from the one or more recommendation of products, and a preference of cloth type, material type, fabric type, fitting, color, brand, size, price, designer and reseller.

15. The computer-implemented method as recited in claim 11, further comprising sharing, at the data processing system with the processor, a plurality of product features corresponding to each product of the pre-defined set of products to a third party, wherein the third party is selected from a group comprising an product designer, a product manufacturer, a product distributor and a retailer, wherein the plurality of product features comprises a set of product dimensions from body fitting, a closure type, a fabric type, a material elasticity, a material texture and a material weight of each product.

16. The computer-implemented method as recited in claim 11, wherein the set of feature metrics comprises a body height, a leg length, an arm length, a shoulder width, a chest radius, a waist radius, a buttocks radius, an arm radius, a leg radius, a torso length, a shoulder length, an angle of inclination of shoulder and a neck radius, wherein the position of each feature of the plurality of features of the body of the user is estimated based on a machine learning model trained on stochastic modeling of a plurality of data points of the set of data, wherein;

the one or more non-feature regions in each sliced feature of the plurality of features of the body comprises features of background objects, worn apparels and devices.

17. The computer-implemented method as recited in claim 11, further comprising recommending, at the data processing system with the processor, the one or more fitness exercise routines and the one or more diet routines to the user based on date stamped and time stamped progressive analysis of one or more changes in each feature of the plurality of features of the body, wherein the one or more fitness exercise routines are rendered on one or more virtual fitness models of the user for a visual workout plan.

18. The computer-implemented method as recited in claim 17, further comprising monitoring, at the data processing system with the processor, a deviation in one or more exercise postures of the body of the user for each of the one or more fitness exercise routines from a reference set of exercise postures in real time based on the machine learning model.

19. The computer-implemented method as recited in claim 11, further comprising presenting at the data processing system with the processor, a life-sized virtual assistant to the user based on one or more gestures, one or more body postures, one or more audio inputs and one or more light or pressure activation switches by the user in real time.

20. The computer-implemented method as recited in claim 11, further comprising storing, at the data processing system with the processor, the first set of data, the second set of data, a body profile, a diet profile, a product selection profile and an exercise profile of the user, wherein the body profile, the diet profile, the product selection profile and the exercise profile are stored in real time.