METHOD AND APPARATUS FOR ADDING ANNOTATIONS TO AN IMAGE

• US 20140181630A1
• Filed: 12/21/2012
• Published: 06/26/2014
• Est. Priority Date: 12/21/2012
• Status: Abandoned Application

• Current Assignee: Ecole polytechnique fãdãrale de lausanne epfl (Schweizerische Eidgenossenschaft)
• Original Assignee: Vidinoti SA (BigTinCan Holdings Limited)
• Inventors: MONNEY, Mathieu, Rime, Laurent, Ayer, Serge
• Application Number:
  US13/724,423
• Publication Number:
  US 20140181630A1
• Time in Patent Office: Days
• US Class Current: 715/232
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/444 Spatial browsing, e.g. 2D maps, 3D or virtual spaces
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information

Abstract

A method comprising the steps of:

• • retrieving data (100) representing a light field with a plenoptic capture device (4);
  • executing program code for matching the retrieved data with corresponding reference data (101);
  • executing program code for retrieving at least one annotation (61, 63, 64) in a plenoptic format associated with an element of said reference data (102);
  • executing program code for generating annotated data in a plenoptic format from said retrieved data and said annotation (103).

Claims

1. A method comprising the steps of:

retrieving data representing a light field with a plenoptic capture device;

executing program code for matching the retrieved data with corresponding reference data;

executing program code for retrieving at least one annotation in a plenoptic format associated with an element of said reference data;

executing program code for generating annotated data in a plenoptic format from said retrieved data and said annotation.

2. The method of claim 1, further comprising:

selecting a viewing direction;

rendering a view corresponding to said annotated data from said viewing direction,

wherein the representation of said annotation depends on said viewing direction.

3. The method of claim 1, further comprising:

rendering a first view corresponding to said annotated data from a first viewing direction;

selecting a second viewing direction;

rendering a Second view corresponding to said annotated data from said second viewing direction;

wherein the representation of said annotation is changed between said first view and said second view.

4. The method of claim 1, further comprising:

associating a first annotation with a first location and a first direction;

associating a second annotation with said first location and with a second direction;

rendering a view corresponding to said annotated data,

selecting between a first or a second viewing direction;

rendering a view including said first annotation but not said second annotation if the first viewing direction is selected, or rendering a view including said second annotation but not said first annotation if the second viewing direction is selected.

5. The method of claim 1, further comprising:

rendering a first view corresponding to a reference data in a plenoptic format and to a first viewing direction;

associating an annotation to an element in said first view;

rendering a second view corresponding to said reference data in a plenoptic format and to a second viewing direction;

associating an annotation to said element in said second view;

interpolating annotations of said element in intermediate views between said first and said second viewing direction.

6. The method of claim 5, further comprising a step of computing from said first, second and intermediate views an annotation in plenoptic format.

7. The method of claim 1, further comprising:

rendering a first view corresponding to said annotated data and to a first focus distance;

modifying the focus distance;

rendering a second view corresponding to said annotated data and to the modified focus distance;

wherein the representation of said annotation is changed between said first view and said second view.

8. The method of claim 7, further comprising:

associating a first annotation with a first location and a first depth;

associating a second annotation with said first location and a second depth;

rendering a first view corresponding to said annotated data;

selecting between a first or a second focus distance;

rendering a second view including said first annotation but not said second annotation if the first focus distance is selected, or rendering a view including said second annotation but not said first annotation if the second focus distance is selected.

9. The method of claim 1, at least one of said annotations being a sound data attached to a coordinate and associated with a particular direction.

10. The method of claim 1, at least one of said annotations being a video data.

11. The method of claim 1, at least one of the annotation acting as a filter for altering the direction of light rays at a particular location in the plenoptic space.

12. The method of claim 11, wherein one said annotation modifies the directions of light rays.

13. The method of claim 12, wherein one said annotation modifies the property of the surface or texture of an object.

14. The method of claim 1, wherein said annotations are defined by an array defining direction of light rays, or modification of direction of light rays, at different point of the plenoptic space.

15. The method of claim 2, wherein rendering comprises determining when an annotation is occluded by an element of the retrieved light field, or when an annotation occludes an element of the retrieved light field, depending on the depth of said element determined from direction of light rays corresponding to said element.

16. The method of claim 2, wherein rendering comprises retrieving one annotation in plenoptic format and applying this annotation to a plurality of consecutive retrieved light fields in a stream of retrieved light fields.

17. The method of claim 2, wherein rendering comprises merging light rays of the annotation with light rays corresponding to the retrieved data.

18. An apparatus for capturing and annotating data corresponding to a scene, comprising:

a plenoptic capturing device for capturing data representing a light field;

a processor;

a display;

programme code for causing said processor to retrieve at least one annotation in a plenoptic format associated with an element of data captured with said plenoptic capturing device and for rendering on said display a view generated from the captured data and including said at least one annotation in plenoptic format when said program code is executed.

19. The apparatus of claim 18, said program code further including a refocus module allowing a user to refocus said view and for changing the presentation of said annotations depending on the selected focus distance.

20. The apparatus of claim 18, said program code further including a viewpoint selection module allowing a user to change the viewpoint used for said rendering and for changing the presentation of said annotations depending on the selected viewpoint.

21. An apparatus for determining annotations, comprising:

a processor;

a store;

program code for causing said processor to retrieve data representing a light field, to match said retrieved data with one reference data, to determine from said store an annotation in plenoptic format associated with said reference data, and to send either said annotation in plenoptic format or an annotated image in plenoptic format to a remote device when said program code is executed.

22. The apparatus of claim 21, said program code further including a module for adding annotations in plenoptic format and associating them with a position and viewing angle in said reference data.

23. The apparatus of claim 21, further comprising a memory storing annotation as an array of light ray directions, or modification of light ray directions, in different points of the plenoptic space.

24. A method comprising the steps of:

retrieving data representing a light field;

sending the retrieved data to a remote server;

receiving from said server either an annotation in plenoptic format or an annotated image in plenoptic format.

25. A method for attaching annotations to a reference image in plenoptic format, comprising:

presenting said reference image in a plenoptic format with a viewer;

selecting an annotation;

selecting with said viewer a position for said annotation and one or a plurality of directions from which said annotation can be seen;

associating in a memory said position and said directions with said annotation and said reference image in plenoptic format.

26. The method of claim 25, comprising associating a plurality of annotations with a single position but with a plurality of different directions.

27. The method of claim 25, further comprising:

rendering a first view corresponding to a reference data in a plenoptic format and to a first viewing direction;

associating a first annotation to an element in said first view;

rendering a second view corresponding to said reference data in a plenoptic format and to a second viewing direction;

associating a second annotation different from said first annotation to said element in said second view.

28. The method of claim 25, further comprising:

rendering a first view corresponding to a reference data in a plenoptic format and to a first viewing direction;

associating an annotation to an element in said first view;

rendering a second view corresponding to said reference data in a plenoptic format and to a second viewing direction;

associating an annotation to said element in said second view;

interpolating annotations of said element in intermediate views between said first and said second viewing direction.

29. An apparatus for attaching annotations to a reference image in plenoptic format, comprising:

a processor;

programme code for causing said processor to present said reference image in a plenoptic format with a viewer (150);

to allow an user to select an annotation (151), as well as a position for said annotation (152) and one or a plurality of directions from which said annotation can be seen (153);

a memory storing said annotation, said position and said directions.