Mobile device identification of media objects using audio and image recognition

• US 8,165,409 B2
• Filed: 07/22/2010
• Issued: 04/24/2012
• Est. Priority Date: 06/09/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Mobile Communications AB (Sony Group Corp.)
• Inventors: Liwell, Marcus, Ritzau, Tobias Robert Jan
• Primary Examiner(s): Zarka, David
• Application Number:
  US12/841,224
• Publication Number:
  US 20100284617A1
• Time in Patent Office: 642 Days
• Field of Search: 382/224
• US Class Current: 382/224
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

A method obtains media on a device, provides identification of an object in the media via image/video recognition and audio recognition, and displays on the device identification information based on the identified media object.

Claims

1. A method performed by a mobile device, the method comprising:

obtaining media via the mobile device;

identifying an object, in the media, using image recognition and audio recognition, where identifying the object includes;

performing the audio recognition in response to obtaining the media, and

performing the image recognition in response to the audio recognition failing to identify the object within a particular level of accuracy;

comparing the identified object to a plurality of media objects;

determining that at least one of the plurality of media objects matches the identified object within the particular level of accuracy; and

displaying an ordered list that includes the identified at least one of the plurality of media objects and a level of accuracy associated with each of the identified at least one of the plurality of media objects.

2. The method of claim 1, where obtaining the media via the mobile device includes:

receiving the media from another device via a network.

3. The method of claim 1, where obtaining the media via the mobile device includes:

capturing the media with an input device of the mobile device.

4. The method of claim 1, further comprising:

marking a face in the object, where the face is used to identify the object through the image recognition.

5. The method of claim 1, further comprising:

receiving a selection from the determined at least one of the plurality of media objects; and

displaying identification information associated with the selection.

6. The method of claim 5, where the identification information comprises information identifying one of a person, a place, or a thing associated with the selection.

7. The method of claim 5, where the identification information comprises at least one of:

biographical information about the identified object;

a link to information about the identified object;

or

recommended media objects associated with the identified object.

8. The method of claim 1, where the media includes one of:

an image file;

an audio file;

a video file;

or

an animation file.

9. The method of claim 1, where the object indicates a place associated with the media.

10. A device comprising:

a processor to;

obtain media,

identify an object in the media using facial recognition and voice recognition, where the processor, when identifying the object in the media, is further to perform the voice recognition in response to obtaining the media, and perform the facial recognition in response to voice recognition failing to identify the object within a particular level of accuracy,

compare the identified object to a plurality of media objects,

display an ordered list of the plurality of media objects that match the identified object within the particular level of accuracy,

display a level of accuracy associated with each of the matching plurality of media objects,

receive a selection of one of the matching plurality of media objects, and

display identification information associated with the selection of one of the matching plurality of media objects.

11. The device of claim 10, where the device comprises a media information gatherer that provides the media to the processor, where the media information gatherer includes at least one of:

a camera;

a microphone;

a media storage device;

or

a communication device.

12. The device of claim 10, where the processor, when identifying the object through facial recognition, is further to determine a location of a face in the object.

13. The device of claim 10, where the processor, when identifying the object through facial recognition, is further to receive a user input identifying a location of a face in the object.

14. A method comprising:

playing a video on a device;

providing, by the device and while the video is playing on the device, an identification of an object in the video, where the identification of the object is performed using facial recognition and voice recognition, where providing the identification of the object in the video includes;

performing the voice recognition response to playing the video, and

performing the facial recognition in response to the voice recognition failing to identify the object within a particular level of accuracy;

comparing, by the device, the identified object to a plurality of media objects;

displaying, by the device, an ordered list of the plurality of media objects that match the identified object within the particular level of accuracy; and

displaying, on the device, a particular level of accuracy associated with each of the matching plurality of media objects.

15. The method of claim 14, where the facial recognition includes determining, in the video, a location of a face associated with the object.

16. The method of claim 14, where the device includes at least one of:

a camera;

a microphone;

a media storage device;

or

a communication device.

17. The method of claim 14, where the processor, when identifying the object through facial recognition, is further to determine a location of a face in the object.

18. The method of claim 14, where the processor, when identifying the object through facial recognition, is further to receive a user input identifying a location of a face in the object.

19. The method of claim 14, further comprising:

receiving a selection of one of the plurality of media objects; and

displaying identification information associated with the selection.

20. The method of claim 19, where the identification information comprises at least one of:

biographical information about the identified object;

a link to information about the identified object;

or

recommended media objects associated with the identified object.