Automatic narration of signal segment
First Claim
1. A computing system comprising:
- one or more processors;
one or more computer-readable media having thereon computer-executable instructions that are structured such that, when executed by the one or more processors, cause the computing system to perform a method for automatically generating a narration of what has happened in a physical space, the method comprising;
accessing a sensed feature store, the sensed feature store comprising a plurality of signal segments, each signal segment comprising signals detected by sensors from one or more physical entities located within the physical space, each signal segment associated with a time when the signals of the physical entities were detected, the sensed feature store also storing a plurality of sensed features associated with the one or more physical entities, each sensed feature also associated with a time and location within the physical space;
accessing a signal segment from the sensed feature store based on a particular one or more physical entities and a particular one or more sensed features of the particular physical entities;
determining a semantic understanding of what is depicted in the signal segment based on the particular one or more physical entities, the particular one or more sensed features of the particular physical entities, and the associated times and locations in which the signal segment evidences state of the particular one or more of the physical entities, determining the semantic understanding including determining actions that happen repeatedly in the signal segment, determining what has not changed within the signal segment, and determining actions that occur continuously within the signal segment;
determining how the particular one or more physical entities within the signal segment are acting in the signal segment using the determined semantic understanding of the signal segment and at least one feature and associated times and locations within the physical space; and
generating a narration of the actions in the signal segment using the determined semantic understanding, the particular one or more sensed features, and determined actions of the particular one or more physical entities.
1 Assignment
0 Petitions
Accused Products
Abstract
Automatic generation of a narration of what is happening in a signal segment (live or recorded). The signal segment that is to be narrated is accessed from a physical graph. In the physical graph, the signal segment evidences state of physical entities, and thus has a semantic understanding of what is depicted in the signal segment. The system then automatically determines how the physical entities are acting within the signal segment based on that semantic understanding, and builds a narration of the activities based on the determined actions. The system may determine what is interesting for narration based on a wide variety of criteria. The system could use machine learning to determine what will be interesting to narrate.
-
Citations
20 Claims
-
1. A computing system comprising:
-
one or more processors; one or more computer-readable media having thereon computer-executable instructions that are structured such that, when executed by the one or more processors, cause the computing system to perform a method for automatically generating a narration of what has happened in a physical space, the method comprising; accessing a sensed feature store, the sensed feature store comprising a plurality of signal segments, each signal segment comprising signals detected by sensors from one or more physical entities located within the physical space, each signal segment associated with a time when the signals of the physical entities were detected, the sensed feature store also storing a plurality of sensed features associated with the one or more physical entities, each sensed feature also associated with a time and location within the physical space; accessing a signal segment from the sensed feature store based on a particular one or more physical entities and a particular one or more sensed features of the particular physical entities; determining a semantic understanding of what is depicted in the signal segment based on the particular one or more physical entities, the particular one or more sensed features of the particular physical entities, and the associated times and locations in which the signal segment evidences state of the particular one or more of the physical entities, determining the semantic understanding including determining actions that happen repeatedly in the signal segment, determining what has not changed within the signal segment, and determining actions that occur continuously within the signal segment; determining how the particular one or more physical entities within the signal segment are acting in the signal segment using the determined semantic understanding of the signal segment and at least one feature and associated times and locations within the physical space; and generating a narration of the actions in the signal segment using the determined semantic understanding, the particular one or more sensed features, and determined actions of the particular one or more physical entities. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A method for automatically generating a narration of what has happened in a physical space, the method comprising:
-
accessing a sensed feature store, the sensed feature store comprising a plurality of signal segments, each signal segment comprising signals detected by sensors from one or more physical entities located within the physical space, each signal segment associated with a time when the signals of the physical entities were detected, the sensed feature store also storing a plurality of sensed features associated with the one or more physical entities, each sensed feature also associated with a time and location within the physical space; accessing a signal segment from the sensed feature store based on a particular one or more physical entities and a particular one or more sensed features of the particular physical entities; determining a semantic understanding of what is depicted in the signal segment based on the particular one or more physical entities, the particular one or more sensed features of the particular physical entities, and the associated times and locations in which the signal segment evidences state of the particular one or more of the physical entities, determining the semantic understanding including determining actions that happen repeatedly in the signal segment, determining what has not changed within the signal segment, and determining actions that occur continuously within the signal segment; determining how the particular one or more physical entities within the signal segment are acting in the signal segment using the determined semantic understanding of the signal segment and at least one feature and associated times and locations within the physical space; and generating a narration of the actions in the signal segment using the determined semantic understanding, the particular one or more sensed features, and determined actions of the particular one or more physical entities.
-
-
20. A computer program product comprising one or more computer-readable storage devices having thereon computer-executable instructions that are structured such that, when executed by the one or more processors, cause the computing system to perform a method for automatically generating a narration of what has happened in a physical space, the method comprising:
-
accessing a sensed feature store, the sensed feature store comprising a plurality of signal segments, each signal segment comprising signals detected by sensors from one or more physical entities located within the physical space, each signal segment associated with a time when the signals of the physical entities were detected, the sensed feature store also storing a plurality of sensed features associated with the one or more physical entities, each sensed feature also associated with a time and location within the physical space; accessing a signal segment from the sensed feature store based on a particular one or more physical entities and a particular one or more sensed features of the particular physical entities; determining a semantic understanding of what is depicted in the signal segment based on the particular one or more physical entities, the particular one or more sensed features of the particular physical entities, and the associated times and locations in which the signal segment evidences state of the particular one or more of the physical entities, determining the semantic understanding including determining actions that happen repeatedly in the signal segment, determining what has not changed within the signal segment, and determining actions that occur continuously within the signal segment; determining how the particular one or more physical entities within the signal segment are acting in the signal segment using the determined semantic understanding of the signal segment and at least one feature and associated times and locations within the physical space; and generating a narration of the actions in the signal segment using the determined semantic understanding, the particular one or more sensed features, and determined actions of the particular one or more physical entities.
-
Specification