Adjusting graphical characteristics for recognition interfaces

• US 9,905,033 B1
• Filed: 12/20/2013
• Issued: 02/27/2018
• Est. Priority Date: 12/20/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gray, Timothy Thomas, Cummins, Charles Eugene, Warren, Angela Kathleen, Salazar, Marc Anthony, Suzan Marashi, Sommer, Steven Michael, Glaser, Russell Edward, Elliott, Forrest, Pagan, Tito
• Primary Examiner(s): Mazumder, Tapas
• Application Number:
  US14/137,709
• Time in Patent Office: 1,530 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 13/00 Animation
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A computing device can include a recognition mode interface utilizing graphical elements, such as virtual fireflies, to indicate recognized or identified objects. The fireflies can be animated to move across a display, and the fireflies can create bounding boxes around visual representations of objects as the objects are recognized. In some cases, the object might be of a type that has specific meaning or information to be conveyed to a user. In such cases, the fireflies might be displayed with a particular size, shape, or color to convey that information. The fireflies also can be configured to form shapes or patterns in order to convey other types of information to a user, such as where audio is being recognized, light is sufficient for image capture, and the like. Other types of information can be conveyed as well via altering characteristics of the fireflies.

Claims

1. A computing device, comprising:

at least one processor;

a camera;

a display screen; and

memory including instructions that, when executed by the at least one processor, cause the computing device to;

capture video data using the camera;

present the video data via the display screen;

present, via the display screen, graphical elements over the video data, the graphical elements being at least partially dispersed across the display screen;

identify, in the video data, a representation of an object;

cause one or more of the graphical elements to appear to move towards the representation to indicate that an object recognition process is being performed;

determine a type of the object; and

cause at least a subset of the graphical elements to form a shape, the shape being configured irrespective of an edge of the representation, the shape being unconfined with respect to a boundary of the representation.

2. The computing device of claim 1, wherein the shape conveys a functionality that the computing device is configured to perform in response to a touch event occurring on the display screen at a location corresponding to the shape.

3. The computing device of claim 2, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

determine wireless functionality of the computing device is disabled,

wherein the at least the subset of the graphical elements are caused to form a Wi-Fi access icon based on determining the wireless functionality is disabled,

wherein detection of the touch event enables the wireless functionality.

4. A computer-implemented method, comprising:

capturing sensor data using at least one sensor of a computing device;

presenting the sensor data via a display of the computing device;

presenting, over the sensor data and via the display in a first time period, graphical elements;

determining information to be conveyed via the computing device, the information being based at least in part upon the sensor data; and

causing, over the sensor data in a second time period after the first time period and based on determining the information, at least a portion of the graphical elements to form a shape, the shape being configured irrespective of an edge of a representation of an object in the sensor data, the shape being unconfined with respect to a boundary of the representation.

5. The computer-implemented method of claim 4, wherein causing the at least the portion of the graphical elements to form the shape further comprises:

causing a first subset of the graphical elements to move across the display; and

causing a second subset of the graphical elements to flash in a first location and then flash in a second location to appear as if each graphical element in the second subset has moved across the display.

6. The computer-implemented method of claim 4, further comprising:

identifying the representation in the sensor data; and

causing at least a second portion of the graphical elements to indicate the representation has been identified.

7. The computer-implemented method of claim 6, wherein causing the at least the second portion to indicate the representation has been identified includes causing the at least the second portion to adjust at least one of a shape, a color, a brightness, an intensity, a rate of movement, a type of movement, or a flashing pattern.

8. The computer-implemented method of claim 4, further comprising:

causing, based on the representation being absent from the sensor data, the at least the portion to disperse about the display and appear with a set of default characteristics.

9. The computer-implemented method of claim 4, wherein the shape conveys a functionality that the computing device is configured to perform in response to a touch event occurring on the display at a location corresponding to the shape.

10. The computer-implemented method of claim 4, wherein the shape corresponds to at least one of a question mark, an arrow, an exclamation point, a circle, a light source, an ear, a symbol, or a character.

11. The computer-implemented method of claim 4, further comprising:

causing the at least the portion to move in three dimensions.

12. The computer-implemented method of claim 4, wherein the sensor data includes at least one of image data, video data, voice data, gesture data, position data, motion data, or audio data.

13. The computer-implemented method of claim 4, further comprising:

causing, in addition to causing the at least the portion to form the shape, at least one of the computing device to emit a buzzing sound or the computing device to vibrate.

14. A computing device, comprising:

a display;

at least one processor; and

memory including instruction that, when executed by the at least one processor, cause the computing device to;

capture sensor data using at least one sensor of a computing device;

present the sensor data via the display;

present, over the sensor data and via the display in a first time period, graphical elements;

determine information to be conveyed via the computing device, the information being based at least in part upon the sensor data; and

cause, over the sensor data in a second time period after the first time period and based on determining the information, at least a portion of the graphical elements to form a shape, the shape being configured irrespective of an edge of a representation of an object in the sensor data, the shape being unconfined with respect to a boundary of the representation.

15. The computing device of claim 14, wherein the graphical elements are of a type selected from a plurality of different types of graphical elements stored on the computing device.

16. The computing device of claim 14, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

adjust, of the at least the portion, at least one of a color, brightness value, intensity value, size, or shape.

17. The computing device of claim 14, wherein the instructions, when executed by the at least one processor, further cause the computing to:

cause the at least the portion to move at different depths of a three-dimensional view presented via the display.

18. The computing device of claim 14, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

receive audio data from the sensor data; and

determine an acoustic fingerprint in the audio data,

wherein the shape indicates the audio data is being analyzed.

19. The computing device of claim 18, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

cause, based on the acoustic fingerprint, the at least the portion to adjust at least one of a brightness level, a color, a size, or a height at which the at least the portion is presented.