ANNOTATION METHOD AND APPARATUS

• US 20140098191A1
• Filed: 10/05/2012
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: BigTinCan Holdings Limited
• Original Assignee: Vidinoti SA (BigTinCan Holdings Limited)
• Inventors: VETTERLI, Martin, RIME, Laurent, MONNEY, Mathieu, AYER, Serge
• Granted Patent: US 9,237,263 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

The present invention relates to an annotating method comprising the steps of:

• • capturing (100) data representing a light field with a plenoptic image capture device (4);
  • matching (101) the captured data with a corresponding reference data;
  • retrieving an annotation associated with an element of said reference data (102);
  • rendering (103) a view generated from said captured data and including at least one annotation.

Claims

1. An annotating method comprising the steps of:

capturing data representing a light field with a plenoptic camera in a device;

executing program code for matching the captured data with reference data;

executing program code for retrieving an annotation associated with an element of said reference data;

executing program code for rendering a view generated from captured data and including at least one annotation.

2. The method of claim 1, wherein said reference data defines a reference light field.

3. The method of claim 2, including the step of generating said reference data from a 3D model of said scene.

4. The method of claim 2, wherein said step of matching includes matching said captured data with one piece of data among a plurality of pieces of reference data representing different light views.

5. The method of claim 1, comprising a step of detecting local features in said captured data.

6. The method of claim 5, wherein said step of detecting local features includes detecting areas where pixels at a first depth have a predefined relationship with pixels at a different depth.

7. The method of claim 5, wherein said step of detecting local features includes detecting disparities in the captured data.

8. The method of claim 5, wherein said step of detecting local features includes computing epipolar volumes or lines.

9. The method of claim 5, comprising a step of describing said local features.

10. The method of claim 9, said local features being described with descriptors in a binary form.

11. The method of claim 10, comprising a step of computing a Hamming distances between said descriptors.

12. The method of claim 5, comprising a step of matching said local features in said captured data with local features in said reference data.

13. The method of claim 5, comprising a step of computing a descriptor of said local features in said captured data, and comparing said descriptor with a descriptor of local features in said reference data.

14. The method of claim 5, comprising a step of registering said plenoptic data with said reference data, using said local features.

15. The method of claim 5, comprising a step of detecting a type of scene and determining a type of local features to detect depending on said type of scene.

16. The method of claim 12, comprising a step of selecting one or a limited number of pieces of reference data depending on the position of said device, on a selection made by the user, or on a received signal, prior to said matching.

17. The method of claim 1, said reference data including a global model of a scene.

18. The method of claim 17, comprising a step of minimizing a cost function representing the error of projection of said captured data onto said reference data.

19. The method of claim 1, said step of rendering a view including rendering a 2D view from the captured data, and superimposing annotations on the 2D view.

20. The method of claim 19, said step (109) of rendering a 2D view from the captured data including displaying annotated objects or features of an annotated image so that they are in focus, while the remaining elements of the scene are made blurry.

32. A computer program product comprising a tangible apparatus readable medium for causing said apparatus to carry out the method of claim 1.

21. An annotating method comprising the steps of:

capturing data representing a light field with a plenoptic camera;

sending data to a remote server;

retrieving annotations or annotated images corresponding to the captured data;

generating a view from the captured data and including at least one annotation.

22. The method of claim 21, wherein said step of sending data to a remote server includes sending captured data in plenoptic format to said remote server.

23. The method of claim 21, comprising a step of computing a descriptor of local features in said captured data, and wherein said step of sending data to a remote server includes sending said descriptor to said remote server.

24. The method of claim 23, wherein said step of computing a descriptor includes detecting areas where pixels at a first depth have a predefined relationship with pixels at a different depth.

25. The method of claim 23, wherein said step of computing a descriptor includes detecting disparities in the captured data.

26. The method of claim 23, wherein said step of computing a descriptor includes computing epipolar volumes or lines.

27. An annotating method comprising the steps of:

retrieving data representing a light field;

executing program code for registering the retrieved data with one reference model or with one among a plurality of pieces of reference light field data;

retrieving annotations associated with said model respectively with said one among a plurality of pieces of reference light field data.

28. The method of claim 27, comprising a step of retrieving local features in the retrieved data.

29. An apparatus for capturing and annotating data corresponding to a scene, comprising:

a plenoptic camera for capturing data representing a light field;

a processor;

a display;

programme code for causing said processor to retrieve an annotation associated with an element of data captured with said camera and for rendering on said display a view generated from the captured data and including at least one annotation when said program code is executed.

30. The apparatus of claim 29, said program code being further arranged for causing said processor to detect local features present in the captured data when said program code is executed.

31. The apparatus of claim 29, said program code being further arranged for describing each detected local feature with a binary vector.

33. An apparatus for determining annotations, comprising:

a processor;

a store;

program code for causing said processor to receive data representing a light field, to match said data with one reference data in said store, to determine an annotation associated with said reference data, and to send said annotation to a remote device when said program code is executed.