Enhanced visualization of areas of interest in image data

• US 10,121,249 B2
• Filed: 10/31/2016
• Issued: 11/06/2018
• Est. Priority Date: 04/01/2016
• Status: Active Grant

• Current Assignee: Baja Education, Inc.
• Original Assignee: Baja Education, Inc.
• Inventors: Pozos, Robert, Agraz, Jose, Leon, Alicia, Baccetti, Brent
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US15/339,604
• Publication Number:
  US 20170287135A1
• Time in Patent Office: 736 Days
• Field of Search: 382103, 382108, 382109, 382154, 382190, 382278, 382282, 382291, 348143, 348144
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/0066 using a MIDI interface
• CPC Class Codes: G10H 1/38 Chord
• CPC Class Codes: G10H 2210/391 Automatic tempo adjustment, correction or control
• CPC Class Codes: G10H 2220/005 Non-interactive screen display of musical or status data graphical user interfaces specifically adapted for electrophonic musical instruments G10H2220/091; fingering displays G10H2220/041
• CPC Class Codes: G10H 2220/126 for graphical editing of individual notes, parts or phrases represented as variable length segments on a 2D or 3D representation, e.g. graphical edition of musical collage, remix files or pianoroll representations of MIDI-like files
• CPC Class Codes: G10H 2220/441 Image sensing, i.e. capturing images or optical patterns for musical purposes or musical control purposes image analysis, inspection, positioning or tracking G06T7/00, recognising music notations G06V30/304

Abstract

A method for generating visual enhancement of areas of interest in images includes receiving data representing a plurality of images in a sequence of images; analyzing the plurality images to identify respective three dimensional (3D) locations of one or more areas of interest in the plurality of images; visually enhancing the one or more of the identified areas of interest in the plurality of the images in the sequence of images; and communicating the visually enhanced image data to a display device to be displayed.

Claims

1. A system for generating visual enhancement of one or more areas of interest in a plurality of images of a sequence of images, the system comprising:

an image area of interest unit configured to receive area of interest data identifying an area of interest and a three dimensional (3D) location in one of the plurality of images;

an image analysis unit configured to analyze the plurality of images to identify respective 3D locations of the one or more areas of interest in other ones of the plurality of images different than the one of the plurality of images,

wherein the image analysis unit is configured to identify Z coordinates of the respective 3D locations of the identified area of interest in the other ones of the plurality of images by identifying changes in the size of the one or more areas of interest moving in a direction perpendicular to the plurality of images; and

an image enhancement unit configured to visually enhance the one or more of the identified areas of interest of the plurality of the images in the sequence of images.

2. The system of claim 1, wherein the sequence of images is video and the plurality of images includes frames of the video.

3. The system of claim 1, wherein the image area of interest unit receives the area of interest data from an input device configured to enable identification of the area of interest by drawing a shape around the area of interest.

4. The system of claim 1, wherein the image enhancement unit visually enhances the one or more of the identified areas of interest by performing at least one of masking, highlighting, magnifying, tracing or motion tracking.

5. The system of claim 1, wherein the image analysis unit is configured to identify X and Y coordinates of the respective 3D locations of the one or more of the areas of interest in the other ones of the plurality of images by measuring pixel distances from left and top sides, respectively, of each image of the sequence of images.

6. The system of claim 1, wherein the sequence of images depicts a 3D graph comprising one or more traces, wherein the areas of interest comprise different portions of the one or more traces, and wherein the image enhancement unit visually enhances the one or more of the identified areas of interest by highlighting the different portions of the one or more traces.

7. A method of generating visual enhancement of one or more areas of interest in a plurality of images of a sequence of images, the method comprising:

receiving area of interest data identifying an area of interest and a three dimensional (3D) location in one of the plurality of images;

analyzing the plurality of images to identify respective 3D locations of the one or more areas of interest in other ones of the plurality of images different than the one of the plurality of images;

visually enhancing the one or more of the identified areas of interest of the plurality of the images in the sequence of images; and

identifying Z coordinates of the respective 3D locations of the identified area of interest in the other ones of the plurality of images by identifying changes in the size of the one or more areas of interest moving in a direction perpendicular to the plurality of images.

8. The method of claim 7, wherein the sequence of images is video and the plurality of images includes frames of the video.

9. The method of claim 7, wherein the area of interest is identified with an input device by drawing a shape around the area of interest.

10. The method of claim 7, wherein the one or more of the identified areas of interest is visually enhanced by performing at least one of masking, highlighting, magnifying, tracing or motion tracking.

11. The method of claim 7, further comprising identifying X and Y coordinates of the respective 3D locations of the one or more of the areas of interest in the other ones of the plurality of images by measuring pixel distances from left and top sides, respectively, of each image of the sequence of images.

12. The method of claim 7, wherein the sequence of images depicts a 3D graph comprising one or more traces, wherein the areas of interest comprise different portions of the one or more traces, and the method further comprising visually enhancing the one or more of the identified areas of interest by highlighting the different portions of the one or more traces.

13. A non-transitory computer readable medium storing a computer program to generate visual enhancement of one or more areas of interest in a plurality of images of a sequence of images, the computer program comprising executable instructions that cause a computer to:

receive area of interest data identifying an area of interest and a three dimensional (3D) location in one of the plurality of images;

analyze the plurality of images to identify respective 3D locations of the one or more areas of interest in other ones of the plurality of images different than the one of the plurality of images;

visually enhance the one or more of the identified areas of interest of the plurality of the images in the sequence of images; and

identify Z coordinates of the respective 3D locations of the identified area of interest in the other ones of the plurality of images by identifying changes in the size of the one or more areas of interest moving in a direction perpendicular to the plurality of images.

14. The non-transitory computer readable medium of claim 13, wherein the sequence of images is video and the plurality of images includes frames of the video.

15. The non-transitory computer readable medium of claim 13, further comprising executable instructions that cause the computer to identify X and Y coordinates of the respective 3D locations of the one or more of the area of interest in the other ones of the plurality of images by measuring pixel distances from left and top sides, respectively, of each image of the sequence of images.

16. The non-transitory computer readable medium of claim 13, wherein the one or more of the identified areas of interest is visually enhanced by performing at least one of masking, highlighting, magnifying, tracing or motion tracking.

17. The non-transitory computer readable medium of claim 13, wherein the sequence of images depicts a 3D graph comprising one or more traces, wherein the areas of interest comprise different portions of the one or more traces, and further comprising executable instructions that cause the computer to visually enhance the one or more of the identified areas of interest by highlighting the different portions of the one or more traces.