Method, client device and server

• US 8,965,043 B2
• Filed: 12/27/2010
• Issued: 02/24/2015
• Est. Priority Date: 02/15/2010
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Porter, Robert Mark Stefan, Keating, Stephen Mark, Gillard, Clive Henry
• Primary Examiner(s): Chu, David H
• Application Number:
  US12/978,890
• Publication Number:
  US 20110199372A1
• Time in Patent Office: 1,520 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2200/16 involving adaptation to the client's capabilities
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

A method of detecting an object in an image at a client device is described. This method comprises the steps of: receiving from a server a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, receiving from the server position data, the position data being indicative of the position of the object within the image; receiving from the server an outline representation of the object at that position, the outline representation defining for pixel positions in a segment of the image at that position whether the pixel position represents the object; and on the basis of said outline, detecting the object in the image.

Claims

1. A method of detecting an object in an image at a client device, comprising the steps of:

receiving, from a server, a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene;

receiving, from the server, position data, the position data being indicative of the position of the object within the image;

receiving, from the server, an outline representation of the object at that position, the outline representation defining, for each pixel position of a plurality of pixel positions within a bounding box segment of the image corresponding to the received position, whether a respective pixel position of the plurality of pixel positions represents the object or a background, the bounding box segment of the image being determined by the server as a portion of an entirety of the image including the object; and

on the basis of said outline representation, detecting the object in the image.

2. The method according to claim 1, further comprising:

receiving, from the server, distance data indicative of the distance of the object from a reference point relative to the camera at that position.

3. The method according to claim 2 comprising receiving the distance data from the server periodically in time.

4. The method according to claim 2 comprising registering the client device with the server, and receiving the distance data in response to such registration.

5. The method according to claim 2 comprising:

receiving another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image;

receiving stitching parameters from the server;

generating at least a portion of a panoramic image of the scene by stitching the first received image to the other received image in accordance with the stitching parameters; and

detecting the object in the stitched image.

21. A non-transitory computer readable medium having stored thereon a computer program containing computer readable instructions which, when loaded onto a computer configure the computer to perform a method according to claim 1.

6. A client device connectable to a server, the client device comprising:

an interface, implemented by circuitry and configured to receive, from the server, a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, the interface being further operable to receive, from the server, position data, the position data being indicative of the position of the object within the image and an outline representation of the object at that position, the outline representation defining, for each pixel position of a plurality of pixel positions within a bounding box segment of the image corresponding to the received position, whether a respective pixel position of the plurality of pixel positions represents the object or a background, the bounding box segment of the image being determined by the server as a portion of an entirety of the image including the object; and

an object detector, which on the basis of said outline representation, is configured to detect the object in the image.

7. The client device according to claim 6, wherein the interface is operable to receive, from the server, distance data indicative of the distance of the object from a reference point relative to the camera at that position.

8. The client device according to claim 7 wherein the interface is operable to receive the distance data from the server periodically in time.

9. The client device according to claim 7 comprising a registration device operable to register the client device with the server, and the interface is further operable to receive the distance data in response to such registration.

10. The client device according to claim 7, wherein the interface is operable to receive another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image and to receive stitching parameters from the server;

and a generator operable to generate at least a portion of a panoramic image of the scene by stitching the first received image to the other received image in accordance with the stitching parameters and detecting the object in the stitched image.

11. A method of providing object detection data to a client device, comprising the steps of:

receiving, from a camera element, a two dimensional image of a scene, the image comprising an object located within the scene;

generating position data from the image, the position data being indicative of the position of the object within the image;

generating an outline representation of the object at that position, the outline representation defining, for each pixel position of a plurality of pixel positions within a bounding box segment of the image corresponding to the received position, whether a respective pixel position of the plurality of pixel positions represents the object or a background, the bounding box segment of the image being determined by the server as a portion of an entirety of the image including the object; and

sending, to the client device, the two dimensional image, the position data and the outline representation.

12. The method according to claim 11, further comprising:

generating distance data indicative of the distance of the object from a reference point relative to the camera at that position.

13. The method according to claim 12 comprising sending the distance data to the client device periodically in time.

14. The method according to claim 12 comprising registering the client device at the server and sending the distance data in response to such registration.

15. The method according to claim 12, comprising sending another two dimensional image to the client device, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first received image;

and transmitting stitching parameters from the server.

22. A non-transitory computer readable storage medium having stored thereon a computer program containing computer readable instructions which, when loaded onto a computer configure the computer to perform a method according to claim 11.

16. A server connectable to a client device, the server comprising:

an interface operable to send to the client device a two dimensional image of a scene, the scene being captured by a camera and the image comprising an object located within the scene, an object detector operable to generate position data the position data being indicative of the position of the object within the image and also to generate an outline representation of the object at that position, the outline representation defining, for each pixel position of a plurality of pixel positions within a bounding box segment of the image corresponding to the received position, whether a respective pixel position of the plurality of pixel positions represents the object or a background, the bounding box segment of the image being determined by the server as a portion of an entirety of the image including the object; and

an interface operable to send to the client device the position data and the outline representation.

17. The server according to claim 16, wherein the detector is operable to generate distance data indicative of the distance of the object from a reference point relative to the camera at that position, and the interface is operable to send the distance data to client device.

18. The server according to claim 17 wherein the interface is operable to send the distance data from the server periodically in time.

19. The server according to claim 17 comprising a registration device operable to register the client device, and the interface is further operable to send the distance data in response to such registration.

20. The server according to claim 17 wherein the interface is operable to send another two dimensional image, the other two dimensional image having a different, but overlapping, field of view of the scene compared with the first sent image;

a stitching parameter generator operable to generate the stitching parameters; and

said interface is further operable to send said stitching parameters to the client device.