Picture display controller, moving-picture information transmission/reception system, picture display controlling method, moving-picture information transmitting/receiving method, and computer program

• US 20030122949A1
• Filed: 11/04/2002
• Published: 07/03/2003
• Est. Priority Date: 11/06/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Music Entertainment Japan Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Music Entertainment Japan Incorporated (Sony Group Corp.)
• Inventors: Yoshimura, Tsukasa, Natsume, Tetsu, Kanematsu, Koichi
• Granted Patent: US 7,576,742 B2
• Time in Patent Office: Days
• US Class Current: 348/333.01
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A picture display controller is provided which can display picture data having been taken from a plurality of viewpoints in a flexible manner. For displaying moving-picture data having been taken from the plurality of viewpoints, read from a storage medium or received from a server, switching can be made at any time between a manual mode in which a picture is selected or synthesized based on designated information such as viewpoint position included in a viewpoint setting command from the controller operated by the user or from the sensor such as head tracker and displayed and a preset mode in which moving-picture data having been taken from different viewpoints are selected one after another according to a picture display program preset for a moving picture and displayed. With this system design, it is possible to make a user'"'"'s setting effective only for a picture portion in which he is interested.

Claims

1. A picture display controller for outputting, to a display means, a picture having been taken from a specific point of view, selected or synthesized from moving-picture data including pictures having been taken from a plurality of viewpoints, the apparatus comprising a control means for selecting a picture to be displayed on the display means;

the control means being designed to select and execute either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from a user'"'"'s input means or a sensor, make a shift from the preset mode in progress to the manual mode when the viewpoint setting command is entered, and select a picture having been taken from a viewpoint conforming to the entered viewpoint setting command for display on the display means.

2. The apparatus as set forth in claim 1, wherein the control means shifts to the preset mode when any new viewpoint setting command is not entered within a predetermined threshold time Ta in the course of the manual mode.

3. The apparatus as set forth in claim 1, wherein:

the input means is designed operable by the user to output at least any one of a viewpoint position, sight line direction and angle of a picture as a viewpoint setting command; and

the control means is designed to select a picture whose viewpoint corresponds to at least any one of the viewpoint position, sight line direction and view angle of the picture, designated as a viewpoint setting command.

4. The apparatus as set forth in claim 1, wherein:

the sensor is designed to output at leans any one of the viewpoint position, sight line direction and view angle of the picture designed as a viewpoint setting command based on detection information from the sensor; and

the control means is designed to select a picture having been taken from a viewpoint corresponding to the viewpoint setting command as which at least any one of the viewpoint position, sight line direction and view angle of the picture is designated.

5. The apparatus as set forth in claim 1, wherein:

the sensor includes a gyro functioning to detect a direction and is designed to output, as the viewpoint setting command, data which is based on direction information acquired by the gyro; and

the control means is designed to select a picture having been taken from a viewpoint conforming to a viewpoint setting command including data which is based on the direction information acquired by the gyro.

6. The apparatus as set forth in claim 1, wherein:

the sensor includes a sight line direction detecting means for making a picture analysis based on a picture of the user'"'"'s face and is designed to output, as the viewpoint setting command, data which is based on the direction information acquired through the picture analysis; and

the control means is designed to select a picture having been taken from a viewpoint conforming to a viewpoint setting command including data which is based on the direction information acquired through the picture analysis.

7. The apparatus as set forth in claim 1, wherein the control means is designed to synthesize a picture having been taken from a viewpoint, not included in the moving-picture data, based on the moving-picture data including pictures having been taken from the plurality of viewpoints, and output the synthetic picture to the display means.

8. The apparatus as set forth in claim 1, wherein the control means is designed to display, on the display means, displayable viewpoint position data corresponding to moving-picture data including pictures having been taken from a plurality of viewpoints and data for identification of the viewpoint position of a picture being displayed on the display means.

9. The apparatus as set forth in claim 1, wherein:

the moving-picture information including pictures having been taken from a plurality of viewpoints includes pictures having been taken from a plurality of viewpoints set correspondingly to a plurality of object positions spaced from each other; and

the control means is designed to select a picture to be displayed based on the moving-picture information including the pictures having been taken from the plurality of viewpoints set correspondingly to the plurality of object positions spaced from each other.

10. A moving-picture transmission/reception system including a server holding moving-picture data including pictures having been taken from a plurality of viewpoints and distributing the moving-picture data, and a user system to display a picture based on the moving-picture data received from the server, the user system comprising a control means for selecting a picture to be displayed on the display means;

the control means being designed to select and execute either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from a user'"'"'s input means or a sensor, make a shift from the preset mode in progress to the manual mode when the viewpoint setting command is entered, and select a picture having been taken from a viewpoint conforming to the entered viewpoint setting command for display on the display means.

11. The system as set forth in claim 10, wherein:

the server distributes picture data having been taken from all viewpoints and included in the moving-picture data;

the user system includes a storage means for storing the picture data having been taken from all viewpoints and included in the moving-picture data and the preset picture display program; and

the control means is designed to output, when in the preset mode, picture data having been taken from a specific viewpoint conforming to the present picture display program while outputting, when in the manual mode, picture data having been taken from a specific viewpoint conforming to the viewpoint setting command.

12. The system as set forth in claim 10, wherein:

the server distributes picture data having been taken from all viewpoints and included in the moving-picture data; and

the user system is designed to receive, when in the present mode, the picture data having been taken from the specific viewpoint conforming to the present picture display program set at the server and output the received data to the display means, while transmitting, when in the manual mode, picture designation information corresponding to the viewpoint setting command to the server while receiving picture data sent from the server according to the picture designation information, and output the picture data having been taken from the specific viewpoint conforming to the viewpoint setting command.

13. A picture display controlling method of outputting, to a display means, a picture having been taken from a specific point of view, selected or synthesized from moving-picture data including pictures having been taken from a plurality of viewpoints, the method comprising steps of:

setting either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from a user'"'"'s input means or a sensor;

selecting a picture to be displayed on the display means according to a mode set in the mode setting step; and

making a shift from the preset mode in progress to the manual mode when the viewpoint setting command is entered, and selecting a picture having been taken from a viewpoint conforming to the entered viewpoint setting command for display on the display means.

14. The method as set forth in claim 13, further comprising a step for shifting to the preset mode when any new viewpoint setting command is not entered within a predetermined threshold time Ta in the course of the manual mode.

15. The method as set forth in claim 13, wherein the step of selecting a picture in the manual mode is to select a picture whose viewpoint corresponds to at least any one of the viewpoint position, sight line direction and view angle of the picture, designated as a viewpoint setting command.

16. The method as set forth in claim 13, wherein:

the sensor includes a gyro functioning to detect a direction and output, as the viewpoint setting command, data which is based on direction information acquired by the gyro; and

the step of selecting a picture in the manual mode is to select a picture having been taken from a viewpoint conforming to a viewpoint setting command including data which is based on the direction information acquired by the gyro.

17. The method as set forth in claim 13, wherein:

the sensor includes a sight line direction detecting means for making a picture analysis based on a picture of the user'"'"'s face to output, as the viewpoint setting command, data which is based on the direction information acquired through the picture analysis; and

the step of selecting a picture in the manual mode is to select a picture having been taken from a viewpoint conforming to a viewpoint setting command including data which is based on the direction information acquired through the picture analysis.

18. The method as set forth in claim 13, further comprising a step of synthesizing a picture having been taken from a viewpoint, not included in the moving-picture data, based on the moving-picture data including pictures having been taken from the plurality of viewpoints, and outputting the synthetic picture to the display means.

19. The method as set forth in claim 13, further comprising a step of displaying, on the display means, displayable viewpoint position data corresponding to moving-picture data including pictures having been taken from a plurality of viewpoints and data for identification of the viewpoint position of a picture being displayed on the display means.

20. The method as set forth in claim 13, wherein:

the moving-picture data including pictures having been taken from a plurality of viewpoints includes pictures having been taken from a plurality of viewpoints set correspondingly to a plurality of object positions spaced from each other; and

there is further provided a step of selecting a picture to be displayed based on the moving-picture information including the pictures having been taken from the plurality of viewpoints set correspondingly to the plurality of object positions spaced from each other.

21. A moving-picture transmitting/receiving method for use in a moving-picture information transmission/reception system including a server holding moving-picture data including pictures having been taken from a plurality of viewpoints and distributing the moving-picture data, and a user system to display a picture based on the moving-picture data received from the server, the user system working to:

select and execute either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from a user'"'"'s input means or a sensor;

make a shift from the preset mode in progress to the manual mode when the viewpoint setting command is entered; and

select a picture having been taken from a viewpoint conforming to the entered viewpoint setting command for display on the display means.

22. The method as set forth in claim 21, wherein:

the server distributes picture data having been taken from all viewpoints and included in the moving-picture data; and

the user system stores, into a storage means, the picture data having been taken from all viewpoints and included in the moving-picture data and the preset picture display program, outputs, when in the preset mode, picture data having been taken from a specific viewpoint conforming to the present picture display program while outputting, when in the manual mode, picture data having been taken from a specific viewpoint conforming to the viewpoint setting command.

23. The method as set forth in claim 21, wherein:

the server distributes picture data having been taken from all viewpoints and included in the moving-picture data; and

the user system receives, when in the present mode, the picture data having been taken from the specific viewpoint conforming to the present picture display program set at the server while outputting the received data to the display means, and transmits, when in the manual mode, picture designation information corresponding to the viewpoint setting command to the server while receiving picture data sent from the server according to the picture designation information, and output the picture data having been taken from the specific viewpoint conforming to the viewpoint setting command.

24. A computer program to provide a picture display control for outputting, to a display means, a picture having been taken from a specific point of view, selected or synthesized from moving-picture data including pictures having been taken from a plurality of viewpoints, the program comprising steps of:

setting either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from a user'"'"'s input means or a sensor;

selecting a picture to be displayed on the display means according to a mode set in the mode setting step; and

making a shift from the preset mode in progress to the manual mode when the viewpoint setting command is entered, and selecting a picture having been taken from a viewpoint conforming to the entered viewpoint setting command for display on the display means.