User interface and authentication for a virtual mirror

• US 8,976,160 B2
• Filed: 04/15/2014
• Issued: 03/10/2015
• Est. Priority Date: 03/01/2005
• Status: Active Grant

• Current Assignee: Eyesmatch, Ltd.
• Original Assignee: Eyesmatch, Ltd.
• Inventors: Vilcovsky, Nissi, Saban, Ofer
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/253,827
• Publication Number:
  US 20140226000A1
• Time in Patent Office: 329 Days
• Field of Search: 345/156, 345204-215, 348/333.01, 434/395
• US Class Current: 345/204
• CPC Class Codes: A47F 2007/195 Virtual display of clothes on the wearer by means of a mirror, screen or the like
• CPC Class Codes: G02B 5/08 Mirrors vehicle mirrors involving special optical features B60R1/08
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G09F 19/16 involving the use of mirrors
• CPC Class Codes: G09F 27/00 Combined visual and audible advertising or displaying, e.g. for public address
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/622 with simulation on a subsidiary picture reproducer
• CPC Class Codes: H04N 2005/2726 for simulating a person's appearance, e.g. hair style, glasses, clothes
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

A method for operating a mirror-display including: operating the system in idle mode wherein image is displayed on the monitor and the processor awaits the presence of a user in front of the camera; once a user presence is identified, switching the operation of the mirror-display into mirror mode, wherein the camera continuously sends captured images to the processor, the processor flips the captured images to reverse right and left sides and applies transformation mapping to generate modified images that appear to mimic reflection of a mirror, and displaying the modified images on the monitor. During idle mode the camera continuously sends background images to the processor, and the user presence is identified by the processor detecting changes in the background images. The processor performs face recognition process on the captured image and interrogates a database to determine whether an account corresponding to the user exists in the database.

Claims

1. A computerized system for operating a monitor, a camera, and a processor, so as to display a mirror-mimicking image on the monitor, comprising:

on a device having the processor and a memory storing a program for execution by the processor, the program including instructions for;

operating the monitor in an idle mode while sensing for presence of a user;

upon detecting the presence of a user, automatically and without the user prompting performing the steps;

initiating a mirror-mimicking mode by capturing images from the camera and operating on the images by the processor to generate mirror-mimicking images;

displaying the mirror-mimicking images on the monitor; and

initiating an authentication process to authenticate the user;

the method further comprises;

flipping the images about a vertical axis so as to reverse right and left sides of the images;

applying a transformation mapping to the images to modify the images to appears to mimic a reflection of a mirror;

resizing the images to reduce variations caused by changes in the object'"'"'s distance to the camera; and

displaying the images on the monitor after performing the flipping, transformation mapping, and resizing.

2. The computerized system of claim 1, wherein initiating an authentication process comprises prompting the user to display a code on the user'"'"'s mobile device.

3. The computerized system of claim 2, wherein prompting the user to display a code comprises displaying a frame on the monitor indicating an area for the user to present the mobile device.

4. A method for operating a mirror-display system, the mirror-display system having a camera, a monitor, and a processor, the method comprising:

operating the mirror-display system in an idle mode wherein an idle image is displayed on the monitor and the processor awaits the presence of a user in front of the camera;

once a user presence is identified in front of the camera, switching the operation of the mirror-display into mirror mode, wherein the camera continuously images the user and sends captured images to the processor, the processor;

flips the captured images about a vertical axis so as to reverse right and left sides of the captured images,

resizes the image to reduce variations caused by changes in the object'"'"'s distance to the camera,

applies transformation mapping to the captured images to generate modified images that appear to mimic reflection of a mirror, and displaying the modified images on the monitor.

5. The method of claim 4, wherein during idle mode the camera continuously sends background images to the processor, and the user presence is identified by the processor detecting changes in the background images.

6. The method of claim 4, wherein when the user presence is detected, the processor performs face recognition process on the captured image and interrogates a database to determine whether an account corresponding to the user exists in the database.

7. the method of claim 4, wherein once the user presence is detected, the processor searches the captured image to identify a code in the image.

8. The method of claim 7, wherein the code comprises an image code displayed on a smartphone.

9. The method of claim 7, wherein the code comprises an image code displayed on a card.

10. The method of claim 7, wherein once a code is identified in the image, the processor interrogates a database to identify the user and determine whether prior images of the identified users are stored in the database, and when the processor determines that prior images of the identified users are stored in the database, the method proceed by displaying thumbnail of stored images on the monitor in addition to displaying the modified images.

11. The method of claim 4, wherein once the user presence is detected, a frame is displayed within the modified image on the monitor and the processor searches the captured image to identify a code within the frame.

12. The method of claim 11, further comprising superimposing a code on the monitor, wherein the code corresponds to a hyperlink.

13. A computer implemented method for operating a system having a monitor, a camera, and a processor, so as to display a mirror-mimicking image on the monitor, comprising:

on a device having the processor and a memory storing a program for execution by the processor, the program including instructions for;

sensing for a user;

initiating a mirror-mimicking mode for displaying the mirror-mimicking image on the monitor;

initiating an authentication process;

prompting the user to control the monitor;

obtaining a digital image from the camera;

flipping the image about a vertical axis so as to reverse right and left sides of the image;

applying a transformation mapping to the image to modify the image such that it appears to mimic a reflection of a mirror;

resizing the image to reduce variations caused by changes in the object'"'"'s distance to the camera; and

displaying the image on the monitor after performing the flipping, transformation mapping, and resizing.

14. The method of claim 13, wherein the user is sensed when the user enters a specified zone in a field of view of the system.

15. The method of claim 13, the method further comprising instructions for:

prompting the user to change the user'"'"'s location until the user is located in a desired location for operation of the system.

16. the method of claim 13, wherein the system is configured to be controlled by one from the group consisting of:

biometric recognition, facial recognition, voice control, finger gestures, hand gestures, arm gestures, leg gestures, foot gestures, input via an input device, physical contact with a sensing device, virtual contact with a virtual zone displayed on the monitor, user presentment of an identification card, and input via a mobile device.

17. The method of claim 13, wherein, upon authentication of a previously authenticated and registered user, a recent recording of the user is displayed on the monitor.

18. The method of claim 13, wherein, upon authentication of a previously unauthenticated and unregistered user, the user will be presented with a new user authentication and registration sequence.

19. The method of claim 13, wherein the monitor is configured to display at least one or more from the group consisting of:

the mirror-mimicking image, a recording indicator, a QR bracket for additional identification, at least one thumbnail image, an unique account name, a split-screen mode with a live image on one side and a recorded image on another side, a field for display of a video call, and social media interaction options.

20. The method of claim 13, wherein the user is recorded for a period of time of between about 10 and 13 seconds.

21. The method of claim 13, wherein gesture control is used for basic control functions and a user application via a mobile device is used for advanced control functions.

22. The method of claim 13, wherein the system is configured to learn gesture control methods unique to the user and update a profile for the user.

23. The method of claim 13, wherein the system is configured to operate based on an assessment of the user'"'"'s familiarity with the system based on at least one or more of the group consisting of:

whether the user is a first time user, a number of images stored in the system for the user, and a last time the user tried to operate the system.

24. The method of claim 13, wherein, in response to a user'"'"'s gesture, the system is configured to recognize the virtual proximity of a body part of the user relative to a virtual representation of an object on the screen as a pointing activity.

25. The method of claim 13, wherein, in response to a user'"'"'s gesture, the system is configured to recognize one or more gestures of the user as functionally equivalent to mouse operations of pointing and clicking.