SYSTEM TO AUGMENT A VISUAL DATA STREAM BASED ON A COMBINATION OF GEOGRAPHICAL AND VISUAL INFORMATION

• US 20120200743A1
• Filed: 02/08/2011
• Published: 08/09/2012
• Est. Priority Date: 02/08/2011
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Autonomy Corporation PLC (HP Inc.)
• Inventors: BLANCHFLOWER, SEAN MARK, LYNCH, MICHAEL RICHARD
• Granted Patent: US 8,488,011 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04N 21/254 Management at additional data server, e.g. shopping server, rights management server arrangements for maintenance or administration in data networks H04L41/00; network services using third party service providers H04L67/53
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4355 involving reformatting operations of additional data, e.g. HTML pages on a television screen optimising the visualization of content for information retrieval from the Internet G06F16/9577; adaptation of message content in packet-switching networks H04L51/066; media handling at the source in data packet switching networks H04L65/764
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An mobile computing device includes a video capturing module to capture a video stream, a global positioning system (GPS) module to generate geographical information associated with frames of the video stream to be captured by the video capturing device, and a video processing module to analyze the frames of the video stream and extract features of points of interest included in the frames. The video processing is configured to transmit the features of the points of interest and the geographical information to a server and to receive augment information from the server computer using wireless communication. The video processing module uses the augment information to overlay the frames of the video stream to generate an augmented video stream.

Claims

1. A client mobile computing system, comprising:

a processor, a memory, a built in battery to power the mobile computing device, built-in video camera and display screen for the mobile computing device, and built-in Wi-Fi circuitry to wirelessly communicate with a server computer connected to network;

a video capturing module coupled with the processor and configured to capture a video stream;

a global positioning system (GPS) module coupled with the video capturing module and configured to generate geographical information associated with frames of the video stream to be captured by the video capturing module;

a video processing module coupled with the video capturing module and configured to analyze the frames of the video stream and extract features of points of interest included in the frames, the video processing module further configured to cause transmission of the features of the points of interest and the geographical information to the server computer and to receive of augment information from the server computer, wherein the video processing module is configured to

     1) overlay,

     2) highlight, or

     3) combination of both the points of interests in the frames of the video stream with the augment information to generate an augmented video stream; and

a display module coupled with the video processing module and configured to display the augmented video stream.

2. The system of claim 1, wherein the features of the points of interest comprises geometrical shapes associated with the points of interest and color associated with each of the geometrical shapes, and wherein each of the geometrical shapes is associated with a pattern of X-Y coordinates.

3. The system of claim 2, further comprising a direction sensor coupled with the video camera and configured to sense direction information of the video camera, wherein the video processing module is further configured to cause transmission of the direction information of the camera to the server computer, wherein the features of the points of interest are transmitted as texts.

4. The system of claim 3, wherein the video processing module is further configured to perform compression operations prior to causing the transmission of the features of the points of interest, the geographical information, and the direction information of the video camera to the server computer.

5. The system of claim 4 wherein the compression operations use DivX compression technology.

6. The system of claim 5, wherein the video processing module is further configured to select the server computer among multiple server computers based on the location and workload of the server computer.

7. The system of claim 6, wherein the augment information comprises at least one of audio information, textual information, and video information.

8. The system of claim 6, wherein the features of the points of interest are associated with a person or an object included in the frames of the video stream.

9. The system of claim 1, further comprising an audio processing module coupled with the video processing module and configured to generate and detect chirp signals, and wherein the chirp signals are used to generate direction augment information to direct a first user toward a position of a second user based on a chirp signal associated with a mobile computing device of the second user.

10. A computer-implemented method for generating augmented video streams, the method comprising:

identifying characteristics information of points of interest included in frames of a video stream being captured;

transmitting the characteristics information and geographical information associated with the frames to a server computer connected to a network using wireless communication;

receiving augment information from the server computer, the augment information related to the points of interest included in the frames of the video stream;

overlaying the augment information onto the frames of the video stream to generate an augmented video stream; and

enabling the augmented video stream to be viewable on a display screen.

11. The method of claim 10, wherein the characteristics information is transmitted as texts.

12. The method of claim 10, further comprising performing compression using DivX compression technology prior to transmitting the characteristics information to the server computer.

13. The method of claim 10, wherein the characteristics information of the points of interest includes geometrical shapes and associated color information, and wherein each of the geometrical shapes is associated with a pattern of X-Y coordinates.

14. The method of claim 10 wherein the server computer is selected from a group of servers in the network, and wherein server selection criteria include server proximity and server workload.

15. The method of claim 10, further comprising transmitting video camera direction information to the server computer, wherein the video camera direction information is associated with a video camera used to capture the video stream, and wherein overlaying the augment information onto the frames of the video stream comprises outputting an audio portion of the augment information to a speaker.

16. The method of claim 10, further comprising:

exchanging chirp signals with another mobile computing device to determine a distance from a location of a current mobile computing device to the other mobile computing device;

receiving direction augment information from the server computer; and

presenting the direction augment information on the display screen of the current mobile computing device, wherein the direction augment information overlays a video stream being played on the display screen.

17. A computer-readable media that stores instructions, which when executed by a machine, cause the machine to perform operations comprising:

detecting that a video stream is being captured by a video camera;

identifying features of objects included a central area of frames of the video stream,

generating geometrical shapes of the features of the objects and patterns associated with

each of the geometrical shapes;

causing transmission of the patterns of the geometrical shapes, color of each geometrical shape, geographical information associated with the frames, and direction information of a video camera to a server computer connected to a network using wireless communication;

receiving augment information from the server computer, the augment information related to the objects included in the frames of the video stream; and

overlaying the augment information onto the frames of the video stream.

18. The computer-readable medium of claim 17, wherein the server computer is selected based on location and workload, and wherein a set of mirrored server computers collaborate to determine the augment information, and wherein overlaying the augment information comprises sending an audio portion of the augment information to a speaker.

19. The computer-readable medium of claim 17, wherein the features of the objects are associated with a person, and wherein the augment information received from the server computer comprises information about the person.

20. The computer-readable medium of claim 19, further comprising exchanging chirp signals with another mobile computing device to determine distance from a current mobile device to the other mobile device, and wherein based on said exchanging of the chirp signals the augment information includes direction augment information.