Methods and apparatuses for using a mobile device to provide remote assistance

• US 8,619,111 B2
• Filed: 03/22/2010
• Issued: 12/31/2013
• Est. Priority Date: 03/20/2009
• Status: Active Grant

• Current Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Original Assignee: Georgia Tech Research Corporation (University System of Georgia)
• Inventors: Roach, Peter Jr.
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US12/729,020
• Publication Number:
  US 20100238194A1
• Time in Patent Office: 1,380 Days
• Field of Search: 455/3.01, 455/92, 455/352, 455/403, 455/404.1, 455/414.1, 455/420, 348 1401- 1416, 709/204, 709/227, 345/629, 345/8
• US Class Current: 348/14.01
• CPC Class Codes: H04M 2201/38 Displays
• CPC Class Codes: H04M 2201/50 Telephonic communication in combination with video communication
• CPC Class Codes: H04M 2203/1016 Telecontrol
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 3/42 Systems providing special services or facilities to subscribers specially adapted for wireless communication networks H04W4/00
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/5125 with remote located operators
• CPC Class Codes: H04M 7/0015 First party call control architectures
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04W 36/385 of the core network

Abstract

Systems and methods for providing remote assistance to a user are disclosed. A remote assistance system may communicate with a mobile device operated by the user and may manipulate devices configured on the mobile device, such as a camera, an illumination device, a projection device, or any other types of devices. Images may be captured with a camera of the mobile phone and transmitted by the mobile phone to the remote assistance system. A remote assistant may transmit images or other visual data to the mobile phone that are then overlaid upon images presented on the mobile phone. Various image manipulation and stabilization means and methods are disclosed, as well as various ways of obtaining and providing remote assistance.

Claims

1. A method of providing remote assistance comprising:

receiving an image, captured by a mobile device, at a remote assistance device;

presenting the image on a display of the remote assistance device;

generating, at the remote assistance device, at least one layer of visual data and overlaying the at least one layer of visual data on the image captured by the mobile device; and

transmitting the at least one layer of overlaid visual data, to the mobile device.

2. The method of claim 1, wherein the overlay data comprises remote assistance data including at least one layer of visual data.

3. The method of claim 1, further comprising:

presenting the at least one layer of visual data as overlaid on the image presented on the display of the remote assistance device.

4. The method of claim 1, further comprising transmitting, to the mobile device, instructions to manipulate a component of the mobile device.

5. The method of claim 4, wherein the instructions comprise instructions to adjust at least one of a camera of the mobile device, a projection device of the mobile device, and an illumination device of the mobile device.

6. The method of claim 1, further comprising generating, at the remote assistance device, remote assistance session data and transmitting the remote assistance session data to a billing server.

7. The method of claim 1, further comprising coordinating remote assistance activities among a plurality of remote assistance providers.

8. A mobile device for use in a remote assistance system comprising:

an image capture device configured to capture images of a first area;

a projection device configured to project images onto a second area, wherein the first area and the second area substantially overlap, and wherein the mobile device is configured to output an overlapping portion of the first area and second area for display such that, as viewed by a user of the mobile device, the overlapping portion is visually overlaid on the captured images of the first area; and

a transceiver configured to

transmit image data associated with the captured images of the first area to a remote assistance system, and

receive an instruction from the remote assistance system to adjust the image capture device, wherein the instruction is based at least in part on the image data transmitted to the remote assistance system, and wherein the mobile device is configured to adjust the image capture device responsive to the transceiver receiving the instruction.

9. The mobile device of claim 8, wherein the instruction comprises one or more instructions to adjust at least one of a focus, a zoom, an image capture area, an orientation, and a direction of the image capture device.

10. The mobile device of claim 8, further comprising a display device, wherein the transceiver is further configured to receive visual data from the remote assistance system, and wherein the mobile device is configured to display the visual data on the display device.

11. The mobile device of claim 10, wherein the visual data comprises at least one of text instructions, graphics, and an indicator.

12. The mobile device of claim 10, wherein the visual data comprises a graphical indicator associated with a physical object.

13. The mobile device of claim 8, wherein the transceiver is further configured to receive a projection instruction, and wherein the mobile device is configured to manipulate the projection device responsive to the transceiver receiving the projection instruction.

14. The mobile device of claim 13, wherein the projection instruction comprises visual data, and wherein the mobile device is further configured to manipulate the projection device to project the visual data onto the second area.

15. The mobile device of claim 14, wherein manipulating the projection device to project the visual data onto the second area comprises projecting a graphical indicator associated with an area of a physical object.

16. The mobile device of claim 14, wherein manipulating the projection device to project the visual data onto the second area comprises projecting the visual data onto a physical object.

17. The mobile device of claim 8, wherein the transceiver is further configured to transmit a request to activate a remote assistance session to a server of the remote assistance system and to receive remote assistance data from the server of the remote assistance system.

18. The mobile device of claim 8, further comprising an image stabilization means configured to generate stabilized image data, wherein the transceiver is further configured to transmit the stabilized image data to the remote assistance system.

19. The mobile device of claim 8, wherein the transceiver is further configured to transmit voice communications signals to the remote assistance system while receiving data communications signals from the remote assistance system.

20. The mobile device of claim 8, further comprising an illumination device.

21. The mobile device of claim 8, wherein the first area is associated with a physical object.

22. The mobile device of claim 21, wherein the second area is associated with an area of the physical object.

23. The mobile device of claim 8, wherein projecting the images onto the second area comprises projecting visual data onto a physical object.

24. The mobile device of claim 23, wherein the visual data comprises a graphical indicator.

25. The mobile device of claim 8, wherein the images of the first area comprise real-time images of the first area and the images projected onto the second area comprise historical images of the first area.

26. The mobile device of claim 8, wherein the images of the first area comprise historical images of the first area and the images projected onto the second area comprise real-time images of the first area.