Searchable screen sharing sessions

• US 9,953,301 B2
• Filed: 02/14/2014
• Issued: 04/24/2018
• Est. Priority Date: 04/03/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Rohra, Anand Shyam, Morozov, Igor
• Primary Examiner(s): Gortayo, Dangelino
• Application Number:
  US14/180,472
• Publication Number:
  US 20140304216A1
• Time in Patent Office: 1,530 Days
• Field of Search: 707608, 707602, 707620, 707769, 715751, 715733, 715753, 706 45
• CPC Class Codes: G06F 16/358 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15

Abstract

Systems and methods are provided for searchable screen sharing sessions. A system records at least one image associated with a screen sharing session. The system extracts content from the at least one image. The system stores the content as searchable content associated with the screen sharing session.

Claims

1. An apparatus for searchable screen sharing sessions, the apparatus comprising:

a processor; and

one or more stored sequences of instructions which, when executed by the processor, cause the processor to;

record at least one image associated with a screen sharing session;

extract text and non-text graphical content from the at least one image;

generate metadata based on the extracted non-text graphical content, the generated metadata identifying the screen sharing session that includes the extracted non-text content as being associated with one or more enterprises;

associate the generated metadata with the screen sharing session and the one or more enterprises;

store the text content and the generated metadata as searchable content associated with the screen sharing session;

output at least one of the text content, the non-text graphical content and the screen sharing session metadata in response to a query based on at least one of the text content, the non-text graphical content and the screen sharing session metadata.

2. The apparatus of claim 1, wherein the one or more stored sequences of instructions further cause the processor to:

record audio associated with the screen sharing session;

extract audio-based content from the audio recording;

store the audio-based content as searchable audio-based content associated with the screen sharing session.

3. The apparatus of claim 2, wherein the one or more stored sequences of instructions further cause the processor to:

generate semantic information based on applying natural language processing to the text content; and

store the generated semantic information as searchable semantic information associated with the screen sharing session.

4. The apparatus of claim 3, wherein the one or more stored sequences of instructions further cause the processor to output a transcript associated with the screen sharing session based on at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information.

5. The apparatus of claim 3, wherein the one or more stored sequences of instructions further cause the processor to output at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information in response to a query based on at least one of the text content, the non-text graphical content, screen sharing session metadata, the audio-based content, and the generated semantic information.

6. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:

record at least one image associated with a screen sharing session;

extract text and non-text graphical content from the at least one image;

generate metadata based on the extracted non-text graphical content, the generated metadata identifying the screen sharing session that includes the extracted non-text content as being associated with one or more enterprises;

associate the generated metadata with the screen sharing session and the one or more enterprises;

store the text content and the generated metadata as searchable content associated with the screen sharing session; and

output at least one of the text content, the non-text graphical content and the screen sharing session metadata in response to a query based on at least one of the text content, the non-text graphical content and the screen sharing session metadata.

7. The computer program product of claim 6, wherein the program code includes further instructions to:

record audio associated with the screen sharing session;

extract audio-based content from the audio recording; and

store the audio audio-based content as searchable audio-based content associated with the screen sharing session.

8. The computer program product of claim 6, wherein the program code includes further instructions to:

generate semantic information based on applying natural language processing to the text content; and

store the generated semantic information as searchable semantic information associated with the screen sharing session.

9. The computer program product of claim 8, wherein the program code includes further instructions to output a transcript associated with the screen sharing session based on at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information.

10. The computer program product of claim 8, wherein the program code includes further instructions to output at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information in response to a query based on at least one of the text content, the non-text graphical content, screen sharing session metadata, the audio-based content, and the generated semantic information.

11. A method for searchable screen sharing sessions, the method comprising:

recording, by a processor of a computing device, at least one image associated with a screen sharing session;

extracting, by the processor of a computing device, text and non-text content from the at least one image;

generating metadata based on the extracted non-text graphical content, the generated metadata identifying the screen sharing session that includes the extracted non-text content as being associated with one or more enterprises;

associating the generated metadata with the screen sharing session and the one or more enterprises;

storing, by the processor of a computing device, the text content and the generated metadata as searchable content associated with the screen sharing session; and

outputting, by the processor, at least one of the text content, the non-text graphical content and the screen sharing session metadata in response to a query based on at least one of the text content, the non-text graphical content and the screen sharing session metadata.

12. The method of claim 11, the method further comprising:

recording, by the processor of a computing device, audio associated with the screen sharing session;

extracting, by the processor of a computing device, audio-based content from the audio recording; and

storing, by the processor of a computing device, the audio audio-based content as searchable audio-based content associated with the screen sharing session.

13. The method of claim 11, the method further comprising:

generating, by the processor of a computing device, semantic information based on applying natural language processing to the text content; and

storing, by the processor of a computing device, the generated semantic information as searchable semantic information associated with the screen sharing session.

14. The method of claim 13, the method further comprising outputting, by the processor of a computing device, a transcript associated with the screen sharing session based on at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information.

15. The method of claim 13, the method further comprising outputting, by the processor of a computing device, at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information in response to a query based on at least one of the text content, the non-text graphical content, screen sharing session metadata, the audio-based content, and the generated semantic information.

16. A method for transmitting code for searchable screen sharing sessions, the method comprising:

transmitting code to record at least one image associated with a screen sharing session;

transmitting code to extract text and non-text graphical content from the at least one image;

transmitting code to generate metadata based on the extracted non-text graphical content, the generated metadata identifying the screen sharing session that includes the extracted non-text content as being associated with one or more enterprises;

transmitting code to associate the generated metadata with the screen sharing session and the one or more enterprises;

transmitting code to store the text content and the generated metadata as searchable content associated with the screen sharing session; and

transmitting code to output at least one of the text content, the non-text graphical content and the screen sharing session metadata in response to a query based on at least one of the text content, the non-text graphical content and the screen sharing session metadata.

17. The method for transmitting code of claim 16, the method further comprising:

transmitting code to record audio associated with the screen sharing session;

transmitting code to extract audio-based content from the audio recording; and

transmitting code to store the audio audio-based content as searchable audio-based content associated with the screen sharing session.

18. The method for transmitting code of claim 16, the method further comprising:

transmitting code to generate semantic information based on applying natural language processing to the text content; and

transmitting code to store the generated semantic information as searchable semantic information associated with the screen sharing session.

19. The method for transmitting code of claim 18, the method further comprising transmitting code to output a transcript associated with the screen sharing session based on at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information.

20. The method for transmitting code of claim 18, the method further comprising transmitting code to output at least one of the text content, the non-text graphical content, audio-based content, and generated semantic information in response to a query based on at least one of the text content, the non-text graphical content, screen sharing session metadata, the audio-based content, and the generated semantic information.