Systems and methods for browsing multimedia content on small mobile devices

• US 7,546,554 B2
• Filed: 03/31/2004
• Issued: 06/09/2009
• Est. Priority Date: 03/31/2004
• Status: Active Grant

• Current Assignee: Fujifilm Business Innovation Corp. (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Xerox Company Limited (Fujifilm Holdings Corporation)
• Inventors: Kamvar, Maryam, Lertsithichai, Surapong, Casi, Sandeep, Chiu, Patrick, Fuse, Tohru, Wilcox, Lynn D.
• Primary Examiner(s): Lo; Weilun
• Assistant Examiner(s): Hassan; Rashedul
• Application Number:
  US10/814,520
• Publication Number:
  US 20050229118A1
• Time in Patent Office: 1,896 Days
• Field of Search: 715/864, 715/733, 715/760, 715/513, 715/501, 715/768, 715/719, 715/720, 715/721, 715/722, 715/723, 715/790, 715/794
• US Class Current: 715/864
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/743 a collection of video files or sequences
• CPC Class Codes: G06F 16/745 the internal structure of a single video sequence
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

The multimedia content browsing system for small mobile devices smoothly blends three key tasks: querying the multimedia contents by keywords, exploring the search results by viewing keyframes of the multimedia contents, and playing a stream of the multimedia contents, e.g., videos or video segments. Videos can be stored in a segment-based multimedia content database, which is designed to support the browsing, retrieval, and reuse of videos. A layered imaging model is introduced where each layer may have its own transparent value set individually, continuously, and interactively, and the layers can overlap on top of each other when rendered on the screen. Since a small mobile device alone may not have enough resources to handle the entire task of multimedia content browsing, a scalable architecture can be adopted to break up the task among the small mobile device, a Hard Disk Drive (HDD), and a resource-rich computing device.

Claims

1. A system to support multimedia content browsing on a mobile device, comprising:

a browsing component software application a microprocessor executing in the mobile device, wherein the mobile device comprises a display screen;

wherein the browsing component renders a keyword query text box in a first area of the display screen, multimedia contents in a second area of the display screen, and a video controller in a third area of the display screen;

wherein the multimedia contents are rendered on overlapping content layers within the second area of the display screen, the content layers comprising a first content layer that renders query results, a second content layer that renders keyframes, and a third content layer that renders videos, wherein the content layers always overlap each other in totality throughout the entire area of the second area of the display screen;

wherein the browsing component further renders a transparent widget layer on the display screen, the transparent widget layer being completely transparent within the display screen until activated by a user, wherein the transparent widget layer independently, interactively and continuously adjusts the degree of transparency of two or more of the content layers via an input device, wherein transparency values for the content layers comprise continuous gradient values between 0.0 and 1.0, a content layer having a transparency value of 0.0 being completely transparent and a content layer having a transparency value of 1.0 being completely opaque; and

wherein the browsing component receives the multimedia contents from a processing component based on a keyword query, wherein the processing component searches for and retrieves the multimedia contents relevant to the keyword query from a multimedia content database, wherein the processing component transmits the multimedia contents relevant to the keyword query to the browsing component over a communication network.

2. The system according to claim 1, wherein:

the multimedia content database comprises an external hard disk drive (HDD), a portable HDD, a wireless HDD, a Bluetooth HDD, and an internal HDD on a resource-rich computing device.

3. The system according to claim 1, wherein:

the multimedia contents comprises a video, a video segment, a keyframe, an image, a figure, a drawing, a graph, a picture, a text, and a keyword.

4. The system according to claim 1, wherein:

the processing component comprises a laptop PC, a desktop PC, a server, a workstation, and a mainframe computer.

5. The system according to claim 1, wherein:

the communication network comprises Internet, an intranet, a local area network, a wireless network, and a Bluetooth network.

6. The system according to claim 1, wherein the processing component:

composes and animates the multimedia contents using the transparency values of the multimedia contents; and

saves the composed content in the multimedia content database and transmits the composed content to the browsing component.

7. The system according to claim 1, wherein:

the mobile device comprises a PDA, a cell phone, a Tablet PC, and a Pocket PC.

8. The system according to claim 1, wherein:

the input device comprises a pen, and a stylus.

17. The system of claim 1:

wherein the first content layer comprises a list of the video titles received from the processing component; and

wherein upon reception of a selection of one of the video titles, the keyframe associated with the selected video title is rendered on the second content layer.

18. The system of claim 17:

wherein the default transparency values for both the first content layer and the second content layer are 0.8.

19. The system of claim 17:

wherein the video associated with the selected video title is played by the browser component on the third content layer; and

wherein the first content layer and the second content layer are completely transparent when the selected video is being played.

20. The system of claim 19:

wherein the video controller is activated and displayed within the third area of the display screen as the selected video is being played within the second area of the display screen, the video controller comprising a graphical user interface that allows the user to stop, pause, play and jump to other parts of the video using the input device.

21. The system of claim 17:

wherein the keyframe associated with the selected video title comprises two or more frames overlapping each other in totality, each frame having a transparency value which allows each frame to be visible.

22. The system of claim 20:

wherein the video associated with the selected video title comprises two or more video segments;

wherein keywords are associated with each video segment; and

wherein the video controller includes;

a graphical representation of each video segment of the video associated with the selected video title;

a bar within each graphical representation, the height of the bar indicating the relevance of the associated video segment to the keyword query; and

a text box which displays the keywords associated with the video segments.

23. The system of claim 22:

wherein the video associated with the selected video title includes video segments obtained from different source videos;

wherein the video controller comprises a graphical indicator when the source video is present within the multimedia content database; and

wherein the source video is displayed upon selection of the associated graphical indicator.

32. The system according to claim 1, wherein the display screen comprises a touch screen, wherein the browsing component receives user input through gestures made on the touch screen.

9. A computer-implemented method to support multimedia content browsing on a mobile device, comprising:

rendering a keyword query user interface in a first area of a display screen of the mobile device;

searching for and retrieving two or more multimedia contents from a multimedia content database based on user input received through the keyword query user interface of the mobile device;

transmitting the two or more multimedia contents over a communication network to the mobile device;

rendering the multimedia contents in a second area of the display screen, wherein the multimedia contents are rendered on overlapping content layers within the second area of the display screen, the content layers comprising a first content layer that renders query results, a second content layer that renders keyframes, and a third content layer that renders videos, wherein the content layers always overlap each other in totality throughout the entire area of the second area of the display screen;

rendering a video controller in a third area of the display screen; and

receiving user input from an input device on a widget layer rendered on the display screen, the transparent widget layer being completely transparent within the display screen until activated by a user, wherein the transparent widget layer independently, interactively and continuously adjusts the degree of transparency of two or more of the content layers via an input device, wherein transparency values for the content layers comprise continuous gradient values between 0.0 and 1.0, a content layer having a transparency value of 0.0 being completely transparent and a content layer having a transparency value of 1.0 being completely opaque.

10. The method according to claim 9, further comprising at least one of:

segmenting the two or more multimedia contents into one or more video segments;

associating one or more keywords with each of the video segments;

retrieving the two or more multimedia contents and each of the associated video segments with a keyword.

11. The method according to claim 10, further comprising:

composing the multimedia contents with one or more video segments from one or more source multimedia contents.

12. The method according to claim 9, further comprising:

composing and animating the contents of the two or more multimedia contents using the transparency values; and

storing the composed content in the multimedia content database and transmitting the composed content for rendering.

24. The method of claim 9, further comprising:

wherein the two or more multimedia contents includes videos, each video having a video title and a keyframe associated with the video;

rendering a list of the video titles associated with the relevant multimedia contents on the first content layer within the second area of the display screen;

receiving user input to select one of the video titles rendered on the first content layer; and

rendering the keyframe associated with the selected video title on the second content layer within the second area of the display screen.

25. The method of claim 24, further comprising:

playing the video associated with the selected video title, wherein the first content layer and the second content layer are completely transparent while the selected video is being played.

26. The method of claim 25, further comprising:

activating and displaying the video controller when the selected video is being played, the video controller being a graphical user interface that allows the user to stop, pause, play and jump to other parts of the video being played.

27. The method of claim 24, further comprising:

receiving user input to adjust the transparency values of the first content layer and the second content layer using the input device on the widget layer, wherein the transparency values for the first content layer can be adjusted by making a horizontal gesture over the widget layer using the input device and the transparency values for the second content layer can be adjusted by making a vertical gesture over the widget layer using the input device.

13. A machine readable medium having executable instructions stored thereon that when executed cause a system to:

render a keyword query text box in a first area of a display screen of the mobile device;

search for and retrieve two or more multimedia contents from a multimedia content database based on user input received through the keyword query text box of the mobile device;

transmit the two or more multimedia contents over a communication network to the mobile device;

render the multimedia contents in a second area of the display screen, wherein the multimedia contents are rendered on overlapping content layers within the second area of the display screen, the content layers comprising a first content layer that renders query results, a second content layer that renders keyframes, and a third content layer that renders videos, wherein the content layers always overlap each other in totality throughout the entire area of the second area of the display screen;

render a video controller in a third area of the display screen; and

receive user input from an input device on a widget layer rendered on the display screen, the transparent widget layer being completely transparent within the display screen until activated by a user, wherein the transparent widget layer independently, interactively and continuously adjusts the degree of transparency of two or more of the content layers via an input device, wherein transparency values for the content layers comprise continuous gradient values between 0.0 and 1.0, a content layer having a transparency value of 0.0 being completely transparent and a content layer having a transparency value of 1.0 being completely opaque.

14. The machine readable medium of claim 13, further comprising instructions that when executed cause the system to:

segment the two or more multimedia contents into one or more video segments;

associate one or more keywords with each of the video segments;

retrieve the two or more multimedia contents and each of the video segments with a keyword.

15. The machine readable medium of claim 14, further comprising instructions that when executed cause the system to:

compose the multimedia contents with one or more video segments from one or more source multimedia contents.

16. The machine readable medium of claim 13, further comprising instructions that when executed cause the system to:

compose and animate the contents of the two or more multimedia contents using the transparency values; and

store the composed content in the multimedia content database and transmit the composed content for rendering.

28. The machine readable medium of claim 13, further comprising instructions that when executed cause the system to:

wherein the two or more multimedia contents includes videos, each video having a video title and a keyframe associated with the video;

render a list of the video titles associated with the relevant multimedia contents on the first content layer within the second area of the display screen;

receive user input to select one of the video titles rendered on the first content layer; and

render the keyframe associated with the selected video title on the second content layer within the second area of the display screen.

29. The machine readable medium of claim 28, further comprising instructions that when executed cause the system to:

play the video associated with the selected video title, wherein the first content layer and the second content layer become completely transparent while the selected video is being played.

30. The machine readable medium of claim 29, further comprising instructions that when executed cause the system to:

activate and display the video controller when the selected video is being played, the video controller being a graphical user interface that allows the user to stop, pause, play and jump to other parts of the video being played.

31. The machine readable medium of claim 28, further comprising instructions that when executed cause the system to:

receive user input to adjust the transparency values of the first content layer and the second content layer using the input device on the widget layer, wherein the transparency values for the first content layer can be adjusted by making a horizontal gesture over the widget layer and the transparency values for the second content layer can be adjusted by making a vertical gesture over the widget layer.