SYSTEMS, METHODS, AND APPARATUSES FOR IMPLEMENTING IN-APP LIVE SUPPORT FUNCTIONALITY

• US 20150381931A1
• Filed: 06/30/2014
• Published: 12/31/2015
• Est. Priority Date: 06/30/2014
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Uhma, Gavin Andrew Ross, Wright, Allen Jeremy, Dennis, Adam William, Ksiezopolski, Nicholas Christopher
• Granted Patent: US 9,294,719 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: H04L 12/1895 for short real-time information, e.g. alarms, notifications, alerts, updates
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

In accordance with disclosed embodiments, there are provided systems, methods, and apparatuses for implementing in-app live support functionality. According to a particular embodiment such an apparatus may be embodied by a user client device which includes, for example: a processor and a memory to execute instructions at the user client device; an application to execute upon the user client device via the processor and the memory; a graphical interface; an audio interface; a communications interface to communicatively link the user client device with a remote service over a network; the graphical interface to receive a user event within the application triggering a user initiated service call to the remote service via the network; the communications interface to receive an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream displaying at least video of a customer service agent and establishing two way audio communication between the user client device and the remote service; and wherein the communications interface further transmits audio captured at the user client device to the remote service and transmits a representation of the application as displayed via the graphical interface of the user client device to the remote service. Other related embodiments are disclosed.

Claims

1. A user client device comprising:

a processor and a memory to execute instructions at the user client device;

an application to execute upon the user client device via the processor and the memory;

a graphical interface;

an audio interface;

a communications interface to communicatively link the user client device with a remote service over a network;

the graphical interface to receive a user event within the application triggering a user initiated service call to the remote service via the network;

the communications interface to receive an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream displaying at least video of a customer service agent and establishing two way audio communication between the user client device and the remote service; and

wherein the communications interface further transmits audio captured at the user client device to the remote service and transmits a representation of the application as displayed via the graphical interface of the user client device to the remote service; and

wherein navigation and interactions of the application are controlled by user inputs provided at the user client device without permitting the remote service to have navigable control over the application and its interactions at the user client device.

2. The user client device of claim 1, wherein the graphical interface to receive the user event within the application triggering the user initiated service call to the remote service via the network comprises:

the application to send a request to the remote service via the communications interface;

the remote service to receive the request from the user client device and responsively display the request at an operator terminal of the remote service;

the request to be accepted at the operator terminal of the remote service; and

wherein the communications interface to receive the audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call comprises the communications interface to receive the audio and video stream from the remote service contingent upon the request having been accepted at the terminal of the remote service.

3. The user client device of claim 1, wherein the user initiated service call to the remote service comprises a user initiated SOS type distress call requesting service, assistance, or support from the remote service.

4. The user client device of claim 1, wherein the graphical interface to receive the user event within the application comprises:

the graphical interface to display an SOS button as a component of a Graphical User Interface (GUI) established by the application; and

the graphical interface to receive a click, touch, swipe, or other user gesture event at the SOS button displayed programmatically triggering the user initiated service call to the remote service.

5. The user client device of claim 1, wherein the communications interface to receive an audio and video stream from the remote service via the network comprises:

the communications interface to receive the video stream and the graphical interface to display upon the user client device a live video feed of a customer service representative affiliated with the remote service.

7. The user client device of claim 1, wherein the two way audio communication between the user client device and the remote service comprises:

the audio interface of the user client device emitting voice communication of a customer service representative captured at an operator terminal of the remote service and transmitted to the user client device within the audio and video stream; and

the audio interface of the user client device further capturing voice communication of a user captured by the audio interface of the user client device for transmission to the operator terminal at the remote service within the audio and video stream.

8. The user client device of claim 1:

wherein the audio and video stream from the remote service is restricted to the two way audio communication only without video transmission or receipt within the audio and video stream when available transmission bandwidth accessible via the communications interface falls below a first threshold;

wherein the audio and video stream from the remote service is restricted to the two way audio communication with a still image or a sub-sampling of the video stream from the remote service when the available transmission bandwidth accessible via the communications interface falls below a second threshold;

or alternatively

wherein the audio and video stream from the remote service is provided with the two way audio communication and the video stream in an unrestricted form when the available transmission bandwidth accessible via the communications interface is above a third threshold.

9. The user client device of claim 1, further comprising:

a video capture device to capture and transmit via the communications interface an outgoing video stream of a user as captured by the video capture device of the user client device for transmission to an operator terminal at the remote service concurrently with the two way audio communication established by the communications interface between the user client device and the remote service.

10. The user client device of claim 1, wherein the user client device is embodied within one of a laptop computer, a mobile client device, a smartphone, or tablet client device, having integrated therewith a touch interface to display the graphical interface and to receive user input events.

11. The user client device of claim 1:

wherein the communications interface communicates with the remote service via a public Internet;

wherein the remote service is implemented by a system at a host organization that serves as a cloud based service provider to the user client device; and

wherein the cloud based service provider hosts the application which is built within the host organization by or on behalf of a customer organization which is a tenant of the host organization and wherein the cloud based service provider makes the application accessible to authorized users affiliated with the customer organization.

12. The user client device of claim 1, wherein the communications interface establishing two way audio communication between the user client device and the remote service enables a user of the user client device and a customer service operator of the remote service to converse seamlessly through the user client device in the context of the application to resolve an issue or problem experienced by the user with the application in real-time.

13. The user client device of claim 1, wherein an operator terminal of the remote service displays a representation of a Graphical User Interface (GUI) established by the application at the user client device in real time at the remote service concurrently with the established two way audio communication between the client device and the remote service.

14. The user client device of claim 13:

wherein the operator terminal of the remote service which displays the representation of the GUI established by the application at the user client device receives drawing elements input at the operator terminal on top of the displayed representation of the GUI at the operator terminal; and

wherein the received drawing elements input at the operator terminal are transmitted back to the user client device via the communications interface for overlay and display at the GUI established by the application as displayed at the user client device.

15. The user client device of claim 13:

wherein the operator terminal of the remote service further displays a likeness of the user co-located with the user client device as captured by a video capture device integrated with the user client device and transmitted to the remote service for concurrent display with the representation of the GUI established by the application at the user client device.

16. The user client device of claim 1:

wherein the application displays a Graphical User Interface (GUI) established by the application at the user client device;

wherein the communications interface to transmit the representation of the application as displayed via the graphical interface of the user client device to the remote service comprises masking, obfuscating, or blocking sub-elements of the GUI displayed at the user client device and wherein the communications interface to transmit the representation of the application as displayed via the graphical interface comprises transmitting the representation with the sub-elements of the GUI masked, obfuscated, or blocked; and

wherein the application displays the GUI at the user client device without any sub-elements of the GUI masked, obfuscated, or blocked and in which the sub-elements of the GUI are only masked, obfuscated, or blocked from view at the remote service.

17. The user client device of claim 16, wherein the application specifies what sub-elements of the GUI masked, obfuscated, or blocked as a security feature to protect sensitive customer data displayed at the user client device from being transmitted to the remote service.

18. The user client device of claim 16, wherein the sub-elements of the GUI that are masked, obfuscated, or blocked are specified by a developer of the application or configurable by a user via a customizable configuration through the application at the user client device in advance of triggering of the user initiated service call to the remote service.

20. The user client device of claim 1, wherein navigation and interactions of the application are controlled by user inputs provided at the user client device or remote inputs originated from the remote service, or both, wherein the user inputs or the remote inputs cause the application to responsively navigate through or interact with the graphical interface as displayed locally at the user client device.

21. The user client device of claim 20:

wherein a sub-set of navigable events are restricted from being instantiated via the remote inputs originated from the remote service to improve privacy and security of sensitive customer information accessible at the application of the user client device; and

wherein the sub-set of navigable events that are restricted from being instantiated via the remote inputs remain available for instantiation via the user inputs local to the user client device.

6. A user client device comprising:

a processor and a memory to execute instructions at the user client device;

an application to execute upon the user client device via the processor and the memory;

a graphical interface;

an audio interface;

a communications interface to communicatively link the user client device with a remote service over a network;

the graphical interface to receive a user event within the application triggering a user initiated service call to the remote service via the network;

the communications interface to receive an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream displaying at least video of a customer service agent and establishing two way audio communication between the user client device and the remote service;

wherein the graphical interface is to display upon the user client device a live video feed of a customer service representative affiliated with the remote service in which the live video feed of the customer service agent is superimposed within a movable overlay box on top of a Graphical User Interface (GUI) established by the application concurrently displayed by the graphical interface of the user device; and

wherein the communications interface further transmits audio captured at the user client device to the remote service and transmits a representation of the application as displayed via the graphical interface of the user client device to the remote service.

19. The user client device of claim 6, wherein navigation and interactions of the application are controlled by user inputs provided at the user client device only without permitting the remote service to have navigable control over the application and its interactions at the user client device.

22. A user client device comprising:

a processor and a memory to execute instructions at the user client device;

an application to execute upon the user client device via the processor and the memory;

a graphical interface;

an audio interface;

a communications interface to communicatively link the user client device with a remote service over a network;

the graphical interface to receive a user event within the application triggering a user initiated service call to the remote service via the network;

the communications interface to receive an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream displaying at least video of a customer service agent and establishing two way audio communication between the user client device and the remote service;

wherein the communications interface further transmits audio captured at the user client device to the remote service and transmits a representation of the application as displayed via the graphical interface of the user client device to the remote service; and

wherein the communications interface is to further transmit diagnostic information to the remote service concurrently with the representation of the application as displayed via the graphical interface, the diagnostic information including any one or more of;

diagnostic information about the user client device;

information about network conditions between the user client device and the remote service, identity and version of operating system running on the user client device, version of the application, duration of a common session between the user client device and the remote service, size of the user client device screen and resolution, a patch list at the user client device, free memory at the user client device, battery status at the user client device, and hardware identifier for the user client device.

23. The user client device of claim 22, wherein the diagnostics information, the representation of the application as displayed via the user client device, the audio and video stream displaying at least video of the customer service agent, the established two way audio communication between the user client device, and the remote service, are recorded and archived to storage at the remote service.

24. A computer-implemented method to execute within a user client device having at least a processor and a memory therein, wherein the computer-implemented method comprises:

executing an application at the user client device via the processor and the memory;

presenting a Graphical User Interface (GUI) established by the application at a graphical interface of user client device;

receiving a user event within the application triggering a user initiated service call to a remote service;

communicably interfacing the application of the user client device to the remote service via a network accessible to a communications interface of the user client device;

receiving an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream establishing two way audio communication between the user client device and the remote service;

transmitting audio captured at the user client device to the remote service;

transmitting a representation of the application as displayed via the graphical interface of the user client device to the remote service; and

wherein navigation and interactions of the application are controlled by user inputs provided at the user client device without permitting the remote service to have navigable control over the application and its interactions at the user client device.

25. Non-transitory computer readable storage media having instructions stored thereon that, when executed by a processor of a user client device, the instructions cause the user client device to perform operations comprising:

executing an application at the user client device via the processor and the memory;

presenting a Graphical User Interface (GUI) established by the application at a graphical interface of user client device;

receiving a user event within the application triggering a user initiated service call to a remote service;

communicably interfacing the application of the user client device to the remote service via a network accessible to a communications interface of the user client device;

receiving an audio and video stream from the remote service via the network responsive to the triggering of the user initiated service call, the audio and video stream establishing two way audio communication between the user client device and the remote service;

transmitting audio captured at the user client device to the remote service;

transmitting a representation of the application as displayed via the graphical interface of the user client device to the remote service; and

wherein navigation and interactions of the application are controlled by user inputs provided at the user client device without permitting the remote service to have navigable control over the application and its interactions at the user client device.