Visual storytelling on a mobile media-consumption device

• US 9,766,786 B2
• Filed: 07/16/2014
• Issued: 09/19/2017
• Est. Priority Date: 07/19/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Elmieh, Baback, Austin, Darren Mark, Collins, Brian M., Oftedal, Mark Jason, Pinkava, Jan J., Sweetland, Douglas Paul
• Primary Examiner(s): Kujundzic, Dino
• Application Number:
  US14/333,387
• Publication Number:
  US 20150026576A1
• Time in Patent Office: 1,161 Days
• CPC Class Codes: G06F 16/74 Browsing; Visualisation therefor end-user interfaces for requesting or interacting with video content, e.g. video on demand interfaces or electronic program guides, H04N21/472
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4333 Processing operations in response to a pause request
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/8541 involving branching, e.g. to different story endings

Abstract

Techniques and apparatuses for visual storytelling on a mobile media-consumption device are described. These techniques and apparatuses enable a user to view events central to the story while also viewing context for the story. By so doing, a user may enjoy the story as the story'"'"'s author intended without sacrificing a user'"'"'s ability to engage with the story'"'"'s context.

Claims

1. A mobile media consumption device comprising:

a display device;

one or more hardware processors; and

one or more computer-readable storage devices storing processor-executable instructions that are executable by the one or more hardware processors to implement a storytelling module configured to;

present a visual experience capable of telling a story, the story including an authored series of events and visual context, the authored series of events representing a storyline of the story, the visual context providing context for the storyline, the visual experience including story views and context views, the story views each having at least a portion of a respective event of the authored series of events, the context views each including at least a portion of the visual context for the storyline;

display a portion of the visual experience on the display device based on a display view that is initially mapped to the portion of the visual experience;

cause the display view to move across the visual experience and map to different portions of the visual experience based on a user input;

determine whether the display view corresponds to a story view or a context view based on an amount of the respective event visually included within the display view; and

perform one or more operations associated with a progression through the authored series of events based on the determination.

2. The mobile media consumption device as recited in claim 1, wherein the context views include no portion of the respective event of the authored series of events.

3. The mobile media consumption device as recited in claim 1, wherein the context views include an insufficient portion of the respective event of the authored series of events, the insufficient portion including less than the portion of the respective event of the authored series of events of the story views.

4. The mobile media consumption device as recited in claim 1, wherein at least a portion of the respective event of the authored series of events is previously determined to display one or more main characters of the story.

5. The mobile media consumption device as recited in claim 1, wherein the authored series of events are human-authored and fixed in both order and number.

6. The mobile media consumption device as recited in claim 1, wherein:

the storytelling module is further configured to cease to progress through the authored series of events based on a determination that the display view corresponds to the context view;

the progress is ceased by repeating the respective event, a set of events of which the respective event is a part, or a portion of the respective event in a loop; and

the loop enables a transition from a context view to a story view to seamlessly present the story.

7. The mobile media consumption device as recited in claim 1, wherein the visual experience is a live-action or animated motion picture, and wherein the motion picture includes annotations identifying the display view as being one of the story view or the context view.

8. The mobile media consumption device as recited in claim 1, wherein the storyline module is further configured to cease progression through the authored series of events based on the respective event of the authored series of events moving out of the portion of the visual experience mapped to the display view.

9. The mobile media consumption device as recited in claim 1, wherein the causing of the display view to move across the visual experience is based on a movement of the display device through at least one of the following:

tilts, turns, moves up, moves left, moves right, or moves down.

10. The mobile media consumption device as recited in claim 1, wherein the user input is received through at least one of:

a button of the mobile media consumption device, a capacitive sensor of the mobile media consumption device, a touch screen of the mobile media consumption device, a camera of the mobile media consumption device, or an orientation sensor of the mobile media consumption device.

11. A method comprising:

presenting a visual experience capable of telling a story via a display device associated with a mobile device, the story including an authored series of events and visual context, the authored series of events representing a storyline of the story, the visual context providing context for the storyline, the visual experience including story views each having at least a portion of a respective event of the authored series of events and context views each including at least a portion of the visual context for the storyline;

determining a first display view based on a first orientation of the display device;

displaying, via the display device, a first portion of the visual experience based on the first display view;

receiving a first selection of a first display view representing a first view into the visual experience for display via the display of the mobile device;

determining that the first display view corresponds to a context view based on an amount of a respective story view visually contained within the first view;

responsive to the determining that the first display view corresponds to the context view;

presenting the visual context via the display device; and

pausing progression through the authored series of events at a respective event;

determining a second display view based on a second orientation of the display device that is different than the first orientation;

displaying, via the display device, a second portion of the visual experience based on the second display view, the second portion different than the first portion;

determining that the second display view corresponds to a story view based on an amount of the respective event visually contained within the second display view; and

responsive to the second display view corresponding to the story view, presenting the respective event at which the progression through the authored series of events was paused.

12. The method as recited in claim 11, wherein the visual context includes a subordinate story including an authored series of subordinate events.

13. The method as recited in claim 11, wherein the first orientation and the second orientation are received through orientation sensors of the mobile device that detect a motion that modifies the display of the mobile device through at least one of the following:

tilts, turns, moves up, moves left, moves right, or moves down.

14. The method as recited in claim 11, wherein:

the visual context for the storyline is presented during presentation of the respective event at which the progression through the authored series of events was paused based on the first display view; and

the presenting of the respective event at which the storyline was paused includes repeating the respective event from a starting point of the respective event.

15. The method as recited in claim 11, wherein:

the visual experience includes a set of images including a first subset of images presenting the storyline and a second subset of images presenting the visual context for the storyline; and

a display of each said subset of images is solely dependent on user selections of display views.

16. The method as recited in claim 11, wherein the mobile device includes a device configured as glasses, a phone, a tablet computer, a laptop, or an e-book reader.

17. The method as recited in claim 11, further comprising altering audio responsive to presenting the visual context of the first display view, wherein the altering of the audio reduces a volume for sounds associated with the authored series of events of the storyline, and wherein the audio volume is based on a visual distance from the first display view to the events of the storyline.

18. The method as recited in claim 11, further comprising presenting a visual indicator indicating when the visual experience is or is not progressing through the authored series of events.

19. The method as recited in claim 11, wherein the determining that the first display view corresponds to the context view is based on the amount of the respective story view visually contained within the first display view being less than a threshold.

20. The method as recited in claim 11, wherein the determining that the second display view corresponds to the story view is based on the amount of the respective event visually contained within the second display view having a minimum event range.