Virtual photorealistic digital actor system for remote service of customers

• US 10,152,719 B2
• Filed: 07/06/2017
• Issued: 12/11/2018
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Ratnakumar Navaratnam
• Original Assignee: Ratnakumar Navaratnam
• Inventors: Navaratnam, Ratnakumar
• Primary Examiner(s): Smith, Creighton
• Application Number:
  US15/642,365
• Publication Number:
  US 20170308905A1
• Time in Patent Office: 523 Days
• Field of Search: None
• CPC Class Codes: B25J 11/001 with emotions simulating means
• CPC Class Codes: B25J 11/0015 Face robots, animated artificial faces for imitating human expressions
• CPC Class Codes: B25J 13/00 Controls for manipulators programme controls B25J9/16; control in general G05
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/637 Administration of user profiles, e.g. generation, initialization, adaptation or distribution
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 30/016 After-sales
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2021/105 Synthesis of the lips movements from speech, e.g. for talking heads
• CPC Class Codes: G10L 21/10 Transforming into visible information
• CPC Class Codes: H04M 2203/1025 of avatars
• CPC Class Codes: H04M 3/5125 with remote located operators
• CPC Class Codes: H04N 23/611 : where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/66 : Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 7/147 : Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/157 : defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

A system for remote servicing of customers includes an interactive display unit at the customer location providing two-way audio/visual communication with a remote service/sales agent, wherein communication inputted by the agent is delivered to customers via a virtual Digital Actor on the display. The system also provides for remote customer service using physical mannequins with interactive capability having two-way audio visual communication ability with the remote agent, wherein communication inputted by the remote service or sales agent is delivered to customers using the physical mannequin. A web solution integrates the virtual Digital Actor system into a business website. A smart phone solution provides the remote service to customers via an App. In another embodiment, the Digital Actor is instead displayed as a 3D hologram. The Digital Actor is also used in an e-learning solution, in a movie studio suite, and as a presenter on TV, online, or other broadcasting applications.

Claims

1. A system for providing service to a customer by an artificial intelligence (AI) platform, the system comprising:

a text input device configured to capture text input within a preset customer perimeter and transmit the captured text input to the AI platform;

a microphone configured to capture sound input within the preset customer perimeter and transmit the captured sound input to the AI platform;

a sensor configured to detect presence of the customer in the preset customer perimeter, and to trigger said system to initiate selective capture of, at least one of, the text input and the sound input therein, and said transmission from the preset customer perimeter and the AI platform;

a means for displaying a virtual digital (VD) actor to the customer;

a speaker configured to emit sounds within the preset customer perimeter, wherein the system is configured to interact with the customer by dynamically controlling a visual appearance of the VD actor displayed and sounds emitted from the speaker based on a response comprising at least one of sounds and text, received from the AI platform, to at least one of the text input and the sound input transmitted from the preset customer perimeter to the AI platform, and wherein the AI platform converts sound input to text using a speech-to-text converter; and

a gesture database comprising at least one part sentence and a gesture associated with each of the at least one part sentence.

2. The system of claim 1 wherein at least part of the AI platform is located remotely and coupled to the said system via a network.

3. The system of claim 1 wherein the AI platform generates the response using one of a captive database comprising at least one of sounds and text stored in at least one of a local device and a cloud server, means of searching a plurality of internet websites, and a combination thereof.

4. The system of claim 3 further configured to connect through a network to a live remote service agent to obtain the response on occurrence of a predetermined event indicating failure of generating the response using the captive database and the means of searching the internet.

5. The system of claim 4 wherein the VD actor is replaced by a live stream of an audiovisual of the live remote service agent.

6. The system of claim 1 wherein the visual appearance of the VD actor is dynamically controlled using at least one gesture in said gesture database.

7. The system of claim 6 wherein the at least one gesture used is selected based on closeness in match between the response and the at least one part sentence.

8. The system of claim 1 wherein said system further comprises at least one equipment configured for performing a sale of an item, and wherein the at least one equipment is triggered into action based on capturing at least one of a predetermined text input and a predetermined sound input.

9. The system of claim 8 wherein the at least one equipment comprises at least one of a barcode scanner, a printer configured for printing at least one of a bill and a discount coupon.

10. The system of claim 1 wherein the system is configured to provide the response in a language one of a predetermined language, a language selected by the customer, a language of the sound input and a language of the text input.

11. A method for providing service to a customer by an artificial intelligence (AI) platform using a virtual digital (VD) actor displayed on a display means, the method comprising:

capturing at least one of a text input and a sound input within a preset customer perimeter and transmitting the captured at least one of the text input and the sound input to the AI platform;

sensing presence of the customer in the preset customer perimeter;

triggering on sensing the presence, selective capture of at least one of, the text input and the sound input therein, and said transmission from the preset customer perimeter to the AI platform;

interacting with the customer by dynamically controlling a visual appearance of the VD actor and sounds emitted from a speaker within the preset customer perimeter, based on a response comprising at least one of sounds and text, received from the AI platform, to at least one of the text input and the sound input, and wherein the AI platform converts sound input to text using a speech-to-text converter; and

wherein the visual appearance of the VD actor is dynamically controlled using at least one gesture from a gesture database comprising at least one part sentence and a gesture associated with each of the at least one part sentence.

12. The method of claim 11 wherein the sound input, the text input, and the response is transmitted between the AI platform and the said system via a network.

13. The method of claim 11 wherein the AI platform generates the response using one of a captive database comprising at least one of sounds and text stored in one of a local device and cloud, means of searching a plurality of internet websites, and a combination thereof.

14. The method of claim 13 further comprising connecting through a network to a live remote service agent to obtain the response on occurrence of a predetermined event indicating failure of generating the response using the captive database and the means of searching the internet.

15. The method of claim 14 wherein the VD actor is replaced by a live stream of an audiovisual of the live remote service agent.

16. The method of claim 11 wherein the at least one gesture used is selected based on closeness in match between the response and the at least one part sentence.

17. The method of claim 11 wherein said method further comprises triggering into action at least one equipment configured for performing a sale of an item on capturing at least one of a predetermined text input and a predetermined sound input.

18. The method of claim 17 wherein said action comprises at least one of scanning a barcode, and for printing at least one of a bill and a discount coupon.

19. The method of claim 11 wherein the response is received from the AI platform is in a language from one of a predetermined language, a language selected by the customer, a language of the sound input and a language of the text input.