Systems and Methods for Automatically Entering Symbols into a String of Symbols Based on an Image of an Object

• US 20140267000A1
• Filed: 03/12/2013
• Published: 09/18/2014
• Est. Priority Date: 03/12/2013
• Status: Abandoned Application

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Yuen, Jenny, St. Clair, Luke
• Application Number:
  US13/797,337
• Publication Number:
  US 20140267000A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

In one embodiment, a method includes, in connection with a user entering a string of symbols into a graphical user interface (GUI) of a mobile computing device, capturing an image of an object with a sensor on the mobile computing device, automatically identifying the object from the image, automatically determining one or more symbols associated with the object as identified from the image, and automatically entering the symbols into the string of symbols.

Claims

1. A method comprising:

by a mobile computing device, in connection with a user entering a string of symbols into a graphical user interface (GUI) of the mobile computing device, capturing an image of an object with a sensor on the mobile computing device;

by the mobile computing device, automatically identifying the object from the image;

by the mobile computing device, automatically determining one or more symbols associated with the object as identified from the image; and

by the mobile computing device, automatically entering the symbols into the string of symbols.

2. The method of claim 1, wherein the sensor is a still or video camera.

3. The method of claim 1, wherein the image is captured in response to the user holding the mobile computing device with the sensor facing the object.

4. The method of claim 1, wherein:

the object is a Quick Response (QR) code; and

capturing the image of the object comprises scanning the QR code.

5. The method of claim 1, wherein capturing the image of the object comprises photographing the object.

6. The method of claim 1, wherein the object is identified at least in part by optical character recognition (OCR).

7. The method of claim 1, wherein the string of symbols comprises text.

8. The method of claim 1, wherein the symbols associated with the object comprise text.

9. The method of claim 1, wherein the symbols associated with the object comprise a pictorial representation of the object.

10. The method of claim 1, wherein automatically identifying the object from the image and automatically determining one or more symbols associated with the object as identified from the image comprises:

sending to a server computing device remote from the mobile computing device first data associated with the image; and

receiving from the server computing device second data associated with the symbols associated with the object.

11. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

in connection with a user entering a string of symbols into a graphical user interface (GUI) of a mobile computing device, capture an image of an object with a sensor on the mobile computing device;

automatically identify the object from the image;

automatically determine one or more symbols associated with the object as identified from the image; and

automatically enter the symbols into the string of symbols.

12. The media of claim 11, wherein the sensor is a still or video camera.

13. The media of claim 11, wherein the image is captured in response to the user holding the mobile computing device with the sensor facing the object.

14. The media of claim 11, wherein:

the object is a Quick Response (QR) code; and

the software is further operable when executed to capture the image of the object by scanning the QR code.

15. The media of claim 11, wherein the software is further operable when executed to capture the image of the object by photographing the object.

16. The media of claim 11, wherein the object is identified at least in part by optical character recognition (OCR).

17. The media of claim 11, wherein the string of symbols comprises text.

18. The media of claim 11, wherein the symbols associated with the object comprise text.

19. The media of claim 11, wherein the symbols associated with the object comprise a pictorial representation of the object.

20. The media of claim 11, wherein the software is further operable when executed to automatically identify the object from the image and automatically determine one or more symbols associated with the object as identified from the image by:

sending to a server computing device remote from the mobile computing device first data associated with the image; and

receiving from the server computing device second data associated with the symbols associated with the object.

21. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to;

in connection with a user entering a string of symbols into a graphical user interface (GUI) of a mobile computing device, capture an image of an object with a sensor on the mobile computing device;

automatically identify the object from the image;

automatically determine one or more symbols associated with the object as identified from the image; and

automatically enter the symbols into the string of symbols.

22. The system of claim 21, wherein the sensor is a still or video camera.

23. The system of claim 21, wherein the image is captured in response to the user holding the mobile computing device with the sensor facing the object.

24. The system of claim 21, wherein:

the object is a Quick Response (QR) code; and

the processors further operable when executing the instructions to capture the image of the object by scanning the QR code.

25. The system of claim 21, wherein the processors are further operable when executing the instructions to capture the image of the object by photographing the object.

26. The system of claim 21, wherein the object is identified at least in part by optical character recognition (OCR).

27. The system of claim 21, wherein the string of symbols comprises text.

28. The system of claim 21, wherein the symbols associated with the object comprise text.

29. The system of claim 21, wherein the symbols associated with the object comprise a pictorial representation of the object.

30. The system of claim 21, wherein the processors are further operable when executing the instructions to automatically identify the object from the image and automatically determine one or more symbols associated with the object as identified from the image by:

sending to a server computing device remote from the mobile computing device first data associated with the image; and

receiving from the server computing device second data associated with the symbols associated with the object.