METHOD, CLIENT DEVICE AND SERVER

• US 20110199372A1
• Filed: 12/27/2010
• Published: 08/18/2011
• Est. Priority Date: 02/15/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: PORTER, Robert Mark Stefan, Gillard, Clive Henry, Keating, Stephen Mark
• Granted Patent: US 8,965,043 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A method of detecting an object in an image at a client device is described. This method comprises the steps of: receiving from a server a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, receiving from the server position data, the position data being indicative of the position of the object within the image; receiving from the server an outline representation of the object at that position, the outline representation defining for pixel positions in a segment of the image at that position whether the pixel position represents the object; and on the basis of said outline, detecting the object in the image.

Claims

1. A method of detecting an object in an image at a client device, comprising the steps of:

receiving from a server a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, receiving from the server position data, the position data being indicative of the position of the object within the image;

receiving from the server an outline representation of the object at that position, the outline representation defining for pixel positions in a segment of the image at that position whether the pixel position represents the object; and

on the basis of said outline, detecting the object in the image.

2. A method according to claim 1, further comprising:

receiving, from the server, distance data indicative of the distance of the object from a reference point relative to the camera at that position.

3. A method according to claim 2 comprising receiving the distance data from the server periodically in time.

4. A method according to claim 2 comprising registering the client device with the server, and receiving the distance data in response to such registration.

5. A method according to claim 2 comprising receiving another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image;

receiving stitching parameters from the server;

generating at least a portion of a panoramic image of the scene by stitching the first received image to the other received image in accordance with the stitching parameters and detecting the object in the stitched image.

6. A method of generating a stereoscopic image for display;

comprising a method of detecting an object in an image at a client device as claimed in claim 2;

receiving, at the client device, display data indicative of the size of the screen upon which the stereoscopic image is to be displayed; and

generating a modified image by displacing the position of the detected object within the image by an amount, the amount of displacement being dependent upon both the distance between the reference point and the object in the scene and the display data so that when both the received two dimensional image and the modified image are displayed stereoscopically on the display, the captured object appears to be a predetermined distance from the display.

7. A method according to claim 6, further comprising receiving from the server background image information defining at least a portion of the scene behind the object relative to the position of the camera;

and in the modified image, inserting over a portion of the object a portion of the background image information which corresponds to said portion of the object.

8. A method according to claim 7, further comprising displacing said background image information in said modified image, wherein the amount of displacement is dependent upon the distance between the background and said reference point.

9. A method according to claim 6, wherein the amount of displacement is further determined in accordance with user preference.

29. A computer program containing computer readable instructions which, when loaded onto a computer configure the computer to perform a method according to claim 1 or 19.

30. A storage medium configured to store the computer program of claim 29 therein or thereon.

10. A client device connectable to a server, the client device comprising:

an interface operable to receive from the server a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, the interface being further operable to receive from the server position data, the position data being indicative of the position of the object within the image and an outline representation of the object at that position, the outline representation defining for pixel positions in a segment of the image at that position whether the pixel position represents the object; and

an object detector which on the basis of said outline, is operable to detect the object in the image.

11. A client device according to claim 10, wherein the interface is operable to receive, from the server, distance data indicative of the distance of the object from a reference point relative to the camera at that position.

12. A client device according to claim 11 wherein the interface is operable to receive the distance data from the server periodically in time.

13. A client device according to claim 11 comprising a registration device operable to register the client device with the server, and the interface is further operable to receive the distance data in response to such registration.

14. A client device according to claim 11, wherein the interface is operable to receive another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image and to receive stitching parameters from the server;

and a generator operable to generate at least a portion of a panoramic image of the scene by stitching the first received image to the other received image in accordance with the stitching parameters and detecting the object in the stitched image.

15. A client device according to claim 10 operable to further generate a stereoscopic image for display;

wherein the interface is operable to receive, at the client device, display data indicative of the size of the screen upon which the stereoscopic image is to be displayed;

and the device further comprises an image generator operable to generate a modified image by displacing the position of the detected object within the image by an amount, the amount of displacement being dependent upon both the distance between the reference point and the object in the scene and the display data so that when both the received two dimensional image and the modified image are displayed stereoscopically on the display, the captured object appears to be a predetermined distance from the display.

16. A client device according to claim 15, wherein the interface is further operable to receive from the server background image information defining at least a portion of the scene behind the object relative to the position of the camera;

and in the modified image, the image generator is further operable to insert over a portion of the object a portion of the background image information which corresponds to said portion of the object.

17. A client device according to claim 16, wherein the image generator is operable to displace said background image information in said modified image, wherein the amount of displacement is dependent upon the distance between the background and said reference point.

18. A client device according to claim 15, wherein the amount of displacement is further determined in accordance with user preference.

19. A method of providing object detection data to a client device, comprising the steps of receiving from a camera element a two dimensional image of a scene, the image comprising an object located within the scene, generating position data from the image, the position data being indicative of the position of the object within the image;

generating an outline representation of the object at that position, the outline representation defusing for pixel positions in a segment of the image at that position whether the pixel position represents the object; and

sending to the client device the two dimensional image, the position data and the outline representation.

20. A method according to claim 19, further comprising:

generating distance data indicative of the distance of the object from a reference point relative to the camera at that position.

21. A method according to claim 20 comprising sending the distance data to the client device periodically in time.

22. A method according to claim 20 comprising registering the client device at the server and sending the distance data in response to such registration.

23. A method according to claim 20, comprising sending another two dimensional image to the client device, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image;

and transmitting stitching parameters from the server.

24. A server connectable to a client device, the server comprising:

an interface operable to send to the client device a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, an object detector operable to generate position data the position data being indicative of the position of the object within the image and also to generate an outline representation of the object at that position, the outline representation defining for pixel positions in a segment of the image at that position whether the pixel position represents the object; and

an interface operable to send to the client device the position data and the outline representation.

25. A server according to claim 24, wherein the detector is operable to generate distance data indicative of the distance of the object from a reference point relative to the camera at that position, and the interface is operable to send the distance data to client device.

26. A server according to claim 25 wherein the interface is operable to send the distance data from the server periodically in time.

27. A server according to claim 25 comprising a registration device operable to register the client device, and the interface is further operable to send the distance data in response to such registration.

28. A server according to claim 25 wherein the interface is operable to send another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first sent image;

a stitching parameter generator operable to generate the stitching parameters; and

said interface is further operable to send said stitching parameters to the client device.