Distracted browsing modes

• US 9,196,239 B1
• Filed: 08/30/2013
• Issued: 11/24/2015
• Est. Priority Date: 08/30/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Taylor, Brett Richard, Ames, Charley, Trahan, Matthew Lloyd, Pilarinos, Dennis
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US14/015,692
• Time in Patent Office: 816 Days
• Field of Search: 704/275
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1688 the I/O peripheral being integrated loudspeakers
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems

Abstract

Approaches to enable a computing device, such as a phone or tablet computer, to determine when a user viewing the content is being distracted or is generally viewing the content with a sufficient level of irregularity, and present an audible representation of the content during the times when the user is deemed distracted. The determination of when the user is distracted or is otherwise viewing the content with irregularity can be performed using sensor data captured by one or more sensors of the computing device. For example, the computing device may analyze the image data captured by one or more cameras, such as by tracking the movement/location of eye pupils of the user and/or tracking the head movement of the user to detect when the user is distracted.

Claims

1. A computing device, comprising:

a camera;

a display screen;

an audio output device;

at least one processor; and

memory including instructions that, when executed by the at least one processor, cause the computing device to;

display content on the display screen;

capture image data using the camera, the image data containing a representation of one or more features of a user;

analyze the image data to determine that the image data is indicative of the user being distracted in viewing the content displayed on the display screen;

convert the content displayed on the screen into an audible speech representation of the content;

begin rendering the audible speech representation of the content using the audio output device;

detect that the image data is no longer indicative of the user being distracted in viewing the content displayed on the display screen; and

discontinue presenting the audible speech representation using the audio output device.

2. The computing device of claim 1, wherein analyzing the image data further comprises:

tracking pupils of the user over a period of time to detect that the user is looking away from the display screen for at least one of;

a threshold number of times over the period of time or a threshold length of time.

3. The computing device of claim 1, wherein analyzing the image data further comprises:

tracking a head movement of the user over a period of time.

4. The computing device of claim 1, wherein the memory further comprises instructions that, when executed by the at least one processor, cause the computing device to:

activate one or more voice commands to enable the user to navigate the content.

5. The computing device of claim 1, wherein converting the content displayed on the screen into an audible speech representation further comprising:

data scraping contents of a web page displayed on a web browser to select portions of the content that are intended to be for display to the user; and

converting the selected portions of the content into the audible speech representation.

6. A computer implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

processing sensor data captured by one or more sensors of a computing device, the computing device including a display screen for displaying content;

analyzing the sensor data to determine that the sensor data is indicative of a user viewing the content with a sufficient level of irregularity; and

presenting an audible representation of the content using an audio output device in response to the sensor data being indicative of the user viewing the content with the sufficient level of irregularity.

7. The computer implemented method of claim 6, further comprising:

detecting that the sensor data is no longer indicative of the user viewing the content with the sufficient level of irregularity; and

discontinuing the presentation of the audible representation of the content using the audio output device.

8. The computer implemented method of claim 6, further comprising:

scrolling the content being displayed on the display screen according to the rendering of the audible representation of the content.

9. The computer implemented method of claim 6, wherein the sensor data further comprises at least one of:

image data captured by a camera;

location data determined by global positioning system (GPS) receiver;

light data captured by a light sensor;

orientation data captured by a gyroscope;

or

acceleration data captured by an accelerometer.

10. The computer implemented method of claim 6, wherein presenting an audible representation of the content further comprises one of:

data scraping contents of a web page displayed on a web browser to select portions of the content that are intended to be for display to the user and converting the selected portions of the content into the audible speech representation;

or

retrieving an audio file associated with the content and rendering the audio file using the audio output device.

11. The computer implemented method of claim 9, wherein the content is a webpage and wherein the audio file is provided by an owner of the webpage.

12. The computer implemented method of claim 9, wherein the audio file is generated by a backend service executed by one or more servers connected to the computing device over a network connection, and wherein the one or more servers stream the audio file to the computing device.

13. The computer implemented method of claim 6, further comprising:

detecting one or more voice commands of the user during the presentation of the audible representation of the content; and

suspending the presentation of the audible content.

14. The computer implemented method of claim 13, further comprising:

invoking one or more functions for navigating the content in response to the one or more voice commands.

15. The computer implemented method of claim 6, wherein analyzing the sensor data further comprises:

tracking pupils of the user over a period of time to detect that the user is looking away from the display screen for at least one of;

a threshold number of times over the period of time or a threshold length of time, the tracking of the pupils performed based at least in part on image data captured by a camera of the computing device.

16. The computer implemented method of claim 6, wherein analyzing the sensor data further comprises:

tracking a head movement of the user over a period of time based at least in part on image data captured by a camera of the computing device.

17. A non-transitory computer readable storage medium storing one or more sequences of instructions that, when executed by one or more processors, cause a computing system to:

process sensor data captured by one or more sensors of a computing device, the computing device including a display screen for displaying content;

analyze the sensor data to determine that the sensor data is indicative of a user viewing the content with a sufficient level of irregularity; and

present an audible representation of the content using an audio output device in response to the sensor data being indicative of the user viewing the content with the sufficient level of irregularity.

18. The non-transitory computer readable storage medium of claim 17, further comprising instructions that, when executed by the one or more processors, cause the computing system to:

detect that the sensor data is no longer indicative of the user viewing the content with the sufficient level of irregularity; and

discontinue the presentation of the audible representation of the content using the audio output device.

19. The non-transitory computer readable storage medium of claim 17, wherein the sensor data further comprises at least one of:

image data captured by a camera;

location data determined by global positioning system (GPS) receiver;

light data captured by a light sensor;

orientation data captured by a gyroscope;

or

acceleration data captured by an accelerometer.

20. The non-transitory computer readable storage medium of claim 17, wherein presenting an audible representation of the content further comprises one of:

data scraping contents of a web page displayed on a web browser to select portions of the content that are intended to be for display to the user and converting the selected portions of the content into the audible speech representation;

or

retrieving an audio file associated with the content and rendering the audio file using the audio output device.

21. The non-transitory computer readable storage medium of claim 17, further comprising instructions that, when executed by the one or more processors, cause the computing system to:

detect one or more voice commands of the user during the presentation of the audible representation of the content; and

suspend the presentation of the audible content.

22. The non-transitory computer readable storage medium of claim 21, further comprising instructions that, when executed by the one or more processors, cause the computing system to:

invoke one or more functions for navigating the content in response to the one or more voice commands.

23. The non-transitory computer readable storage medium of claim 17, wherein analyzing the sensor data further comprises:

tracking pupils of the user over a period of time to detect that the user is looking away from the display screen for at least one of;

a threshold number of times over the period of time or a threshold length of time, the tracking of the pupils performed based at least in part on image data captured by a camera of the computing device.