Method, apparatus, and system for communicating information of selected objects of interest displayed in a video-chat application

• US 10,136,044 B2
• Filed: 02/22/2017
• Issued: 11/20/2018
• Est. Priority Date: 02/22/2017
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Eisses, Aaron
• Primary Examiner(s): Zenati, Amal
• Application Number:
  US15/438,860
• Publication Number:
  US 20180241930A1
• Time in Patent Office: 636 Days
• Field of Search: 348 1403
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A method, apparatus and system for sharing object information which executes a video-chat session between an agent and a mobile client using an application hosted by a server. There is presented a view in common executed by a server application to both the agent and the mobile client for viewing objects captured by the mobile client during the video-chat session. There is identified by the agent a viewed object captured by the mobile client where the captured objected has been viewed by the agent in the common view during the video-chat session. The captured objected may be selected by the agent during the video-chat session and the agent may retrieve information remotely about the selected object to send to the mobile client.

Claims

1. A method for identifying object information in a video chat application hosted by a server system, comprising:

presenting a shared view for an agent device and a client device during a video chat session using the video chat application hosted by the server system;

receiving, by the server system, object images captured by the client device during the video chat session;

receiving a touch selection of an object from the captured object images through the agent device during the video chat session wherein the touch selection can be received from either the agent device or the client device during the video chat session;

identifying, by the server system, the selected object by determining an approximate center of the touch selection;

overlaying the selected object with an image color to identify the selected object within the shared view of the video chat session, wherein the overlaid image color is scaled appropriately in the display of the agent device and the display of the client device in accordance with the size of the selected object in the respective display;

generating, by the server system, object information to identify features of the selected object;

sending generated object information of the selected object to a database system associated with the server system to search for and retrieve data related to the generated object information stored by the database system;

receiving, from the database system, data related to the object information; and

sending the received data related to the object information from the server system to the client device.

2. The method of claim 1, wherein the database system comprises a knowledge database of data related to the object information.

3. The method of claim 1, further comprising:

capturing the images of the objects with a camera coupled to the client device associated with the server system.

4. The method of claim 1, further comprising:

identifying, by locations within the images by the server, the features of the objects in the session of the video chat application.

5. The method of claim 1, further comprising:

receiving instructions by the server system to perform the search of the database system for retrieving the related data.

6. The method of claim 5, further comprising:

applying artificial intelligence solutions, by the server system, to perform the search of the database system for retrieving the related data.

7. The method of claim 1, further comprising:

overlaying, by the server system, the features of the objects captured in a display of the server system to distinguish the features from a background of the display.

8. The method of claim 1, further comprising:

identifying the features of the object within an agent app hosted by the server system.

9. The method of claim 8, further comprising:

identifying, by locations within the images in the agent app by the server system, the features of the objects in the session of the video chat application.

10. A computer program product tangibly embodied in a computer-readable storage device and comprising instructions configurable to be executed by a processor to perform a method for processing requests by a server system in a video chat application, the method comprising:

presenting a shared view for an agent device and a client device during a video chat session using the video chat application hosted by the server system;

receiving, by the server system, object images captured by the client device during the video chat session;

receiving a touch selection of an object from the captured object images through the agent device during the video chat session wherein the touch selection can be received from either the agent device or the client device during the video chat session;

identifying, by the server system, the selected object by determining an approximate center of the touch selection;

overlaying the selected object with an image color to identify the selected object within the shared view of the video chat session, wherein the overlaid image color is scaled appropriately in the display of the agent device and the display of the client device in accordance with the size of the selected object in the respective display;

generating, by the server system, object information to identify features of the selected object;

sending generated object information of the selected object to a database system associated with the server system to search for and retrieve data related to the generated object information stored by the database system;

receiving, from the database system, data related to the object information; and

sending the received data related to the object information from the server system to the client device.

11. The method of claim 10, wherein the database system comprises a knowledge database of data related to the object information.

12. The method of claim 10, further comprising:

capturing the images of the objects with a camera coupled to the client device associated with the server system.

13. The method of claim 10, further comprising:

identifying, by locations within the images by the server, the features of the objects in the session of the video chat application.

14. The method of claim 10, further comprising:

receiving instructions by the server system to perform the search of the database system for retrieving the related data.

15. The method of claim 14, further comprising:

applying artificial intelligence solutions, by the server system, to perform the search of the database system for retrieving the related data.

16. The method of claim 10, further comprising:

overlaying, by the server system, the features of the objects captured in a display of the server system to distinguish the features from a background of the display.

17. The method of claim 10, the server system further comprising:

identifying the features of the object within an agent app hosted by the server system.

18. The method of claim 17, further comprising:

identifying, by locations within the images in the agent app by the server system, the features of the objects in the session of the video chat application.

19. A server system comprising:

at least one processor;

a database system; and

at least one computer-readable storage device comprising instructions configurable to be executed by the at least one processor to perform a method for processing requests to the database system using a video chat application, the method comprising;

presenting a shared view for an agent device and a client device during a video chat session using the video chat application hosted by the server system;

receiving, by the server system, object images captured by the client device during the video chat session;

receiving a touch selection of an object from the captured object images through the agent device during the video chat session wherein the touch selection can be received from either the agent device or the client device during the video chat session;

identifying, by the server system, the selected object by determining an approximate center of the touch selection;

overlaying the selected object with an image color to identify the selected object within the shared view of the video chat session, wherein the overlaid image color is scaled appropriately in the display of the agent device and the display of the client device in accordance with the size of the selected object in the respective display;

generating, by the server system, object information to identify features of the selected object;

sending generated object information of the selected object to a database system associated with the server system to search for and retrieve data related to the generated object information stored by the database system;

receiving, from the database system, data related to the object information; and

sending the received data related to the object information from the server system to the client device.

20. The system of claim 19 wherein the database system comprises a multi-tenant database system.