Virtual skywriting

• US RE43,545 E1
• Filed: 06/30/2011
• Issued: 07/24/2012
• Est. Priority Date: 09/30/2008
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cohen, Michael
• Primary Examiner(s): Patel, Ajit
• Assistant Examiner(s): Navar, Erica
• Application Number:
  US13/174,576
• Time in Patent Office: 390 Days
• Field of Search: 382/159, 382/165, 382/170, 382/181, 715230-233, 4554041-4042, 4554141-4144, 455/440, 4554561-457
• US Class Current: 455/456.3
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: H04L 67/18 in which the network application is adapted for the location of the user terminal wireless application services making use of the location of users or terminals H04W4/02 takes precedence; location based Web retrieval G06F16/9537
• CPC Class Codes: H04L 67/38 Protocols for telewriting; Protocols for networked simulations, virtual reality or games games using an electronically generated display A63F13/00; remote windowing or X-Windows G06F9/452
• CPC Class Codes: H04M 1/72544 for supporting a game or graphical animation
• CPC Class Codes: H04M 1/72572 according to a geographic location user location related services provided by wireless communication networks H04W4/02; network applications adapted for the location of the user terminal H04L29/08657
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/32101 Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
• CPC Class Codes: H04N 1/32117 in a separate transmission or protocol signal prior to or subsequent to the image data transmission, e.g. in digital identification signal [DIS], in non standard setup [NSS] or in non standard field [NSF] for mode signalling H04N1/333
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/8146 involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3273 Display
• CPC Class Codes: H04N 2201/3278 : Transmission
• CPC Class Codes: H04W 4/02 : Services making use of location information

Abstract

Users with mobile computing devices author skywriting and submit them to a virtual skywriting service. A mobile computing device having wireless network access is configured to capture a digital image of a real world scene including sky. The image is captured while the mobile computing device is in a geographical location or area. While the mobile computing device is still in the geographical location or area, it transmits via the wireless network a request for virtual skywriting. The request is received by a skywriting service that obtains or determines the geographical location/area that the computing device is in or near. The skywriting service selects a skywriting from among the previously submitted skywritings; the skywriting may be selected based on its having been entered at or associated with the geographical location. The mobile device receives the virtual skywriting from the service and displays a version of the digital image with the virtual skywriting added to the sky in the version of the digital image. Either the service or the mobile device may render the skywriting onto the image. Various effects may be applied in rendering the skywriting to give it realistic or vapor-like appearance.

Claims

1. A method implemented at least in part by a mobile computing device, the method comprising:

transmitting from the mobile computing device a request to a skywriting service for skywriting;

receiving from the skywriting service skywriting associated with a geographical location of the mobile computing device, the skywriting having been obtained by the skywriting service by identifying, among a stored plurality of skywritings, a the skywriting that corresponds to the geographical location of the mobile computing device; and

displaying, on the mobile computing device, the skywriting superimposed on an image of the sky as captured by the mobile computing device from the geographical location, wherein the skywriting is having been superimposed on the image by the mobile computing device transmitting the image captured by the imaging device to the skywriting service which first recognizing recognized a region of sky in the image using a recognition algorithm and then fitting fitted the skywriting to the recognized region of sky, and where the mobile computing device then received from the skywriting service the skywriting superimposed on the image.

2. A method according to claim 1, further comprising transmitting the image captured by the imaging device to the skywriting service, wherein the skywriting service performs the recognizing of the sky region and the superimposing, and receiving from the skywriting service the skywriting superimposed on the image.

3. A The method according to claim 1, wherein the skywriting is inputted by a user using a mobile device, the skywriting being received at the skywriting service and stored in association with a the geographic location of the mobile device.

4. A The method according to claim 1, further comprising determining an age of the skywriting and rendering the skywriting with one or more visual effects that are based on the age of the sky writing.

5. A The method according to claim 1, further comprising selecting the skywriting from among the stored plurality of skywritings based at least in part on whether the skywriting has a grouping identifier that matches a the grouping identifier specified for the request, the grouping identifier identifying a plurality of users.

6. A The method according to claim 1, wherein the skywriting, when superimposed on the region of sky, appears as vapor in the region of sky.

19. The method of claim 1, comprising transmitting from the mobile computing device a request to the skywriting service for the skywriting.

7. One or more tangible computer readable storage media storing information to enable a computing device to perform a process, the process comprising:

receiving and storing skywritings inputted from respective mobile computing devices, the skywritings comprising text or graphic data, and storing with the skywritings indicia of respective geographical areas;

receiving requests from mobile devices requesting skywriting;

determining geographical areas from which the respective requests originated;

returning to one of the requesting mobile devices one of the stored skywritings based on the correspondence of its indicia of a geographical area to the geographical area from which the mobile device'"'"'s request originated;

capturing an image at the geographical area with an image capture device of the one of the requesting mobile devices, transmitting the image to a server, the server then identifying a region of sky in the image and adding the skywriting to the image by fitting the skywriting according to the identified region of sky, and receiving the image with the added skywriting at the one of the mobile devices; and

displaying the image with the added skywriting at the one of the requesting mobile devices.

8. One or more tangible computer readable storage media according to claim 7, further comprising rendering the skywriting to appear as vapor in the region of sky.

9. One or more tangible computer readable storage media according to claim 7, wherein a stored skywriting comprises respective time information indicating an age of the skywriting.

10. One or more tangible computer readable storage media according to claim 9, wherein the combining further comprises applying an effect to the skywriting, the effect depending on the age of the skywriting.

11. One or more tangible computer readable storage media according to claim 7, further comprising scaling the skywriting and/or the image to fit the skywriting to a region of the image containing pixels corresponding to sky.

12. One or more tangible computer readable storage media according to claim 7, further comprising selecting the skywriting returned to the requesting mobile device based on one or more properties of the skywriting other than the indicia of its geographical area.

13. A mobile computing device having wireless network access and configured to perform a process, the process comprising:

capturing a digital image of a real world scene including sky with an imaging device of the mobile computing device, the image being captured while the mobile computing device is in or near a geographical location;

while the mobile computing device is still in or near the geographical location, transmitting via the wireless network a request for virtual skywriting, wherein the transmitting enables a recipient of the request to determine the geographical location that the computing device is in or near;

receiving, via the wireless network, virtual skywriting having been selected in response to the request transmitted by the mobile computing device; and

displaying a version of the digital image with the virtual skywriting added to the sky in the version of the digital image, the virtual skywriting having been added by identifying transmitting the digital image to a server that then identifies a portion of the digital image as containing sky image data and then adding adds the virtual skywriting according to the identified portion of the digital image, the mobile computing device then receiving from the server the digital image with the added virtual skywriting image.

14. A The mobile computing device according to claim 13, the process further comprising receiving the virtual skywriting in rendered form and using the rendered virtual skywriting to generate the version of the digital image.

15. A The mobile computing device according to claim 13, the process further comprising authoring a new skywriting and transmitting it the new skywriting via the wireless network to a the skywriting service that makes the new skywriting available to mobile devices in or near the computing device'"'"'s geographical location.

16. A The mobile computing device according to claim 13, the receiving the virtual skywriting further comprising receiving rendering parameters for the virtual skywriting and the mobile computing device rendering the virtual skywriting according to the rendering parameters.

17. A The mobile computing device according to claim 13, the process further comprising the mobile device transmitting with the request an indicia of the geographical location.

18. A The mobile computing device according to claim 13, wherein the virtual skywriting has also been selected based on whether the virtual skywriting is associated with a specified user list.