Picture display controller, moving-picture information transmission/reception system, picture display controlling method, moving-picture information transmitting/receiving method, and computer program

• US 7,576,742 B2
• Filed: 11/04/2002
• Issued: 08/18/2009
• Est. Priority Date: 11/06/2001
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Music Entertainment Japan Incorporated (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Music Entertainment Japan Incorporated (Sony Group Corp.)
• Inventors: Kanematsu, Koichi, Yoshimura, Tsukasa, Natsume, Tetsu
• Primary Examiner(s): Wu; Xiao M
• Assistant Examiner(s): Hajnik; Daniel F
• Application Number:
  US10/287,149
• Publication Number:
  US 20030122949A1
• Time in Patent Office: 2,479 Days
• Field of Search: 345/427, 345/420, 715/718, 715/719, 715/720, 715/721, 715/722, 715/723, 715/725
• US Class Current: 345/427
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A picture display controller can display picture data taken from a plurality of viewpoints in a flexible manner. For displaying moving-picture data taken from the plurality of viewpoints, read from a storage medium or received from a server, switching can be made at any time between a manual mode in which a picture is selected or synthesized based on designated information such as viewpoint position included in a viewpoint setting command from the controller operated by the user or from the sensor such as head tracker and displayed and a preset mode in which moving-picture data taken from different viewpoints are selected one after another according to a picture display program preset for a moving picture and displayed. With this system design, a user'"'"'s setting can be made effective only for a picture portion in which he is interested.

Claims

1. A picture display controller for outputting, to a display unit, a picture taken from a specific point of view, selected or synthesized from moving-picture data including pictures taken from a plurality of viewpoints, the apparatus comprising:

a memory for storing the moving picture data;

an input unit; and

a selector configured to select a picture to be displayed on the display unit, select and execute either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from the input unit, shift from the preset mode to the manual mode and output to the display unit a picture taken from a viewpoint based on the viewpoint setting command when the viewpoint setting command is entered, shift from the manual mode to the preset mode and output to the display unit the picture according to the preset picture display program when a viewpoint setting command is not entered within a predetermined period of time, continue in the manual mode and output to the display unit the picture from the viewpoint based on the viewpoint setting command when the viewpoint setting command is entered again for a second time within the predetermined period of time, and continue in the manual mode and output to the display unit a picture taken from a new viewpoint when a new viewpoint setting command is entered within the predetermined period of time.

2. The controller as set forth in claim 1, wherein the input unit outputs, as part of the viewpoint setting command, at least any one of a viewpoint position, sight line direction, and angle of a picture;

and

wherein the selector selects a picture whose viewpoint corresponds to the viewpoint setting command.

3. The controller as set forth in claim 1, wherein the input unit comprises a sensor, and the sensor outputs, as part of the viewpoint setting command, at least any one of a viewpoint position, sight line direction, and view angle of the picture, which is based on detection information from the sensor;

and

wherein the selector selects a picture taken from a viewpoint corresponding to the viewpoint setting command.

4. The apparatus controller as set forth in claim 1, wherein the input unit comprises a sensor, and the sensor includes a gyro functioning to detect a direction and outputs, as part of the viewpoint setting command, data which is based on direction information acquired by the gyro;

and

wherein the selector selects a picture taken from a viewpoint conforming to the viewpoint setting command.

5. The controller as set forth in claim 1, wherein the input unit comprises a sensor, and the sensor includes a sight line direction detector configured to make a picture analysis based on a picture of the users face and output, as part of the viewpoint setting command, data which is based on direction information acquired through the picture analysis;

and

wherein the selector selects a picture taken from a viewpoint conforming to the viewpoint setting command.

6. The controller as set forth in claim 1, wherein the selector synthesizes a picture taken from a viewpoint, not included in the moving-picture data, based on the moving-picture data including pictures taken from the plurality of viewpoints, and outputs the synthetic picture to the display unit.

7. The controller as set forth in claim 1, wherein the selector displays, on the display unit, displayable viewpoint position data corresponding to moving-picture data including pictures taken from a plurality of viewpoints and data for identification of the viewpoint position of a picture being displayed on the display unit.

8. The controller as set forth in claim 1, wherein the moving-picture information includes pictures taken from a plurality of viewpoints set correspondingly to a plurality of object positions spaced from each other;

and

wherein the selector selects a picture to be displayed based on the moving-picture information including the pictures taken from the plurality of viewpoints set correspondingly to the plurality of object positions spaced from each other.

9. A moving-picture transmission/reception system comprising:

a server holding moving-picture data including pictures taken from a plurality of viewpoints and for distributing the moving-picture data; and

a user system to display a picture based on the moving-picture data received from the server, the user system comprising;

a selector configured to select a picture to be displayed on the display unit, select and execute either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from an input unit or a sensor, shift from the preset mode to the manual mode and output to the display unit a picture taken from a viewpoint based on the viewpoint setting command when the viewpoint setting command is entered, shift from the manual mode to the preset mode and output to the display unit the picture according to the preset picture display program when a viewpoint setting command is not entered within a predetermined period of time, continue in the manual mode and output to the display unit the picture from the viewpoint based on the viewpoint setting command when the viewpoint setting command is entered again for a second time within the predetermined period of time, and continue in the manual mode and output to the display unit a picture taken from a new viewpoint when a new viewpoint setting command is entered within the predetermined period of time.

10. The system as set forth in claim 9, wherein the server distributes picture data taken from the plurality of viewpoints and included in the moving-picture data;

wherein the user system includes a storage unit configured to store the picture data taken from the plurality of viewpoints, and included in the moving-picture data and the preset picture display program; and

wherein the selector outputs, when in the preset mode, picture data taken from a specific viewpoint conforming to the preset picture display program while outputting, when in the manual mode, picture data taken from a specific viewpoint conforming to the viewpoint setting command.

11. The system as set forth in claim 9, wherein the server distributes picture data taken from the plurality of viewpoints and included in the moving-picture data, and

wherein the user system receives, when in the preset mode, the picture data taken from the specific viewpoint conforming to the preset picture display program set at the server and outputs the received data to the display unit, while transmitting, when in the manual mode, picture designation information corresponding to the viewpoint setting command to the server while receiving picture data sent from the server according to the picture designation information, and outputs the picture data taken from the specific viewpoint conforming to the viewpoint setting command.

12. A picture display controlling method of outputting, to a display unit, a picture taken from a specific point of view, selected or synthesized from moving-picture data including pictures taken from a plurality of viewpoints, the method comprising:

setting either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from an input unit;

selecting a picture to be displayed on the display unit according to the selected mode;

shifting from the preset mode to the manual mode and displaying a picture taken from a viewpoint based on the viewpoint setting command when the viewpoint setting command is entered;

shifting from the manual mode to the preset mode and displaying the picture according to the preset picture display program when a viewpoint setting command is not entered within a predetermined period of time;

continuing in the manual mode and displaying the picture from the viewpoint based on the viewpoint setting command when the viewpoint setting command is entered again for a second time within the predetermined period of time; and

continuing in the manual mode and displaying a picture taken from a new viewpoint when a new viewpoint setting command is entered within the predetermined period of time.

13. The method as set forth in claim 12, further comprising selecting a picture whose viewpoint corresponds to at least any one of the viewpoint position, sight line direction, and view angle of the picture, designated as the viewpoint setting command.

14. The method as set forth in claim 12, further comprising detecting a direction and output, as the viewpoint setting command, data which is based on direction information acquired by a gyro;

and

selecting a picture taken from a viewpoint conforming to the viewpoint setting command including data which is based on the direction information acquired by the gyro.

15. The method as set forth in claim 12, further comprising making a picture analysis based on a picture of the user'"'"'s face to output, as the viewpoint setting command, data which is based on the direction information acquired through the picture analysis by a sight line direction detector;

and

selecting a picture taken from a viewpoint conforming to the viewpoint setting command including data which is based on the direction information acquired through the picture analysis.

16. The method as set forth in claim 12, further comprising synthesizing a picture taken from a viewpoint, not included in the moving-picture data, and outputting the synthetic picture to the display unit.

17. The method as set forth in claim 12, further comprising displaying, on the display unit, displayable viewpoint position data corresponding to moving-picture data and data for identification of the viewpoint position of a picture being displayed on the display unit.

18. The method as set forth in claim 13, further comprising taking pictures from a plurality of viewpoints set correspondingly to a plurality of object positions spaced from each other;

and

selecting a picture to be displayed based on the moving-picture information including the pictures taken from the plurality of viewpoints set correspondingly to the plurality of object positions spaced from each other.

19. A moving-picture transmitting/receiving method for use in a moving-picture information transmission/reception system including a server holding moving-picture data including pictures taken from a plurality of viewpoints and distributing the moving-picture data, and a user system to display a picture based on the moving-picture data received from the server, the method comprising:

selecting and executing either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from an input unit;

shifting from the preset mode to the manual mode and displaying a picture taken from a viewpoint based on the viewpoint setting command when the viewpoint setting command is entered;

shifting from the manual mode to the preset mode and displaying the picture according to the preset picture display program when a viewpoint setting command is not entered within a predetermined period of time;

continuing in the manual mode and displaying the picture from the viewpoint based on the viewpoint setting command when the viewpoint setting command is entered again for a second time within the predetermined period of time; and

continuing in the manual mode and displaying a picture taken from a new viewpoint when a new viewpoint setting command is entered within the predetermined period of time.

20. The method as set forth in claim 19, further comprising distributing picture data taken from the plurality of viewpoints and included in the moving-picture data;

and

storing taken from the plurality of viewpoints and included in the moving-picture data and the preset picture display program, outputting, when in the preset mode, picture data taken from a specific viewpoint conforming to the preset picture display program while outputting, when in the manual mode, picture data taken from a specific viewpoint conforming to the viewpoint setting command.

21. The method as set forth in claim 19, further comprising distributing picture data taken from all viewpoints and included in the moving-picture data;

and

receiving, when in the preset mode, the picture data taken from the specific viewpoint conforming to the preset picture display program set at the server while outputting the received data to the display means, and transmitting, when in the manual mode, picture designation information corresponding to the viewpoint setting command to the server while receiving picture data sent from the server according to the picture designation information, and outputting the picture data taken from the specific viewpoint conforming to the viewpoint setting command.

22. A computer-readable storage media storing a program for causing a computer to perform a method for providing a picture display control for outputting, to a display means, a picture taken from a specific point of view, selected or synthesized from moving-picture data including pictures taken from a plurality of viewpoints, the method comprising:

setting either a preset mode in which a picture is displayed according to a preset picture display program corresponding to the moving-picture data and arranged as time-series data for identifying a picture to be displayed or a manual mode in which a picture to be displayed is identified based on a viewpoint setting command supplied from an input unit;

shifting from the preset mode to the manual mode and displaying a picture taken from a viewpoint based on the viewpoint setting command when the viewpoint setting command is entered;

shifting from the manual mode to the preset mode and displaying the picture according to the preset picture display program when a viewpoint setting command is not entered within a predetermined period of time;

continuing in the manual mode and displaying the picture from the viewpoint based on the viewpoint setting command when the viewpoint setting command is entered again for a second time within the predetermined period of time; and

continuing in the manual mode and displaying a picture taken from a new viewpoint when a new viewpoint setting command is entered within the predetermined period of time.