Image processing device, image processing system, camera device, image processing method, and program

• US 8,970,695 B2
• Filed: 01/14/2010
• Issued: 03/03/2015
• Est. Priority Date: 02/27/2009
• Status: Expired due to Fees

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ishiko, Masatsugu, Ozawa, Hitoshi, Kobashiri, Takanobu, Fukata, Akitaka, Shimomura, Munehiro, Asano, Kyosuke
• Primary Examiner(s): YI, DAVID
• Application Number:
  US12/656,046
• Publication Number:
  US 20100220194A1
• Time in Patent Office: 1,874 Days
• Field of Search: 348/143, 348/169
• US Class Current: 348/143
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

An image processing device includes: a detecting section configured to detect a plurality of objects by type from one input image; a generating section configured to generate image data on each of the objects detected by the detecting section as images of respective different picture frames by types of the objects; and a processing section configured to subject the images of the different picture frames, the images of the different picture frames being generated by the generating section, to processing according to one of a setting and a request.

Claims

1. An image processing device comprising:

a detecting section configured for selectively detecting a plurality of objects of a plurality of types from an input image, wherein the plurality of types of objects includes;

moving objects,

faces, and

stationary objects;

a generating section configured to generate, for a given image frame of the input image, one converted image frame for each type of object detected by the detection section, where each converted image frame is based on the given frame of the input image and corresponds to one of the plurality of types of objects detected by the detecting section; and

a processing section configured to selectively subject the converted frames to processing, wherein the processing includes generating an output image frame comprising a synthesis of more than one of the converted image frames for the given image frame,

wherein the processing section is configured to be capable of selectively synthesizing a plurality of sub-images into the output image, wherein the sub-images selectable by the processing section for synthesizing into the output image include;

an image generated by combining an On-Screen Display (foreground image) with an image corresponding to the given image frame of the input image,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section and adding detection information to the image, the detection information being information indicating those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section, said image comprising a zoomed in portion of said given image frame corresponding to one those objects that were detected by the detecting section and are of the type corresponding to the converted image frame

wherein the image processing device is configured such that a user may select one or more of the plurality of detected objects to be monitored objects, and

wherein the plurality of sub-images synthesized by the processing section into the output image includes at least one converted image frame for each monitored object, each converted image frame for a monitored object being generated by combining an On-Screen Display (foreground image) with an image corresponding to the type of the respective monitored object and comprising a zoomed in portion of said given image frame corresponding to the respective monitored object.

2. The image processing device according to claim 1,

wherein said generating section is configured to selectively combine an independent On-Screen Display (foreground image) with a converted image frame and output a result thereof as a converted image frame.

3. The image processing device according to claim 1,

wherein said generating section has a function capable of, when generating a given converted image frame, without affecting other converted image frames, electronically zooming and panning/tilting the given frame of the input image such that the given converted image frame corresponds to a targeted portion of the given frame of the input image.

4. The image processing device according to claim 1,

wherein said generating section generates a converted image frame by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section and adding detection information to the image, the detection information being information indicating those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame.

5. The image processing device according to claim 1,

wherein said processing section has a function of performing streaming distribution of a plurality of generated images to different communicating devices.

6. The image processing device according to claim 5,

wherein said processing section has a function of storing an image in streaming distribution.

7. The image processing device according to claim 1,

wherein said detecting section has a function of, when a non-moving object is detected, determining an elapsed time from a point in time at which the non-moving object was initially detected as a non-moving object to a present time, and

said generating section generates a converted image frame corresponding to the non-moving object type of object by adding time data indicating the elapsed time to image data corresponding to the detected non-moving object.

8. The image processing device according to claim 1, further comprising an audio processing system,

wherein, when a moving object is detected, said processing section is configured to notify a distance from the moving object to a specific area by driving said audio processing system and changing a response sound according to a position at which the moving object is detected.

9. The image processing device according to claim 8,

wherein said processing section has a function of driving said audio processing system and outputting audio for guiding to the specific area according to the position at which the moving object is detected.

10. The image processing device according to claim 1, further comprising an audio processing system,

wherein, when a face is detected, said processing section is configured to drive said audio processing system and perform streaming distribution of audio according to information obtained by detecting said face, and to output the audio to another communicating device.

14. The image processing device of claim 4,

wherein the detection information comprises one or more shapes that are superimposed on the image so as to severally surround those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame.

15. The image processing device of claim 2,

wherein said generating section generates a converted image frame by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section, said image comprising a zoomed in portion of said given image frame corresponding to one those objects that were detected by the detecting section and are of the type corresponding to the converted image frame.

16. The image processing device of claim 1,

wherein the image processing device is configured such that a user may select one or more detecting functions corresponding to the plurality of types of objects,

wherein the detecting section detects objects of those types corresponding to the selected detecting functions.

17. The image processing device of claim 16,

wherein the output image comprises a synthesis of an image corresponding to the given image frame of the input image and converted images corresponding to those types corresponding to the selected detecting functions.

18. The image processing device of claim 1,

wherein the output image comprises a synthesis of:

an image corresponding to the given image frame of the input image,

a converted image corresponding to moving objects,

a converted image corresponding to stationary objects, and

a converted image corresponding to faces.

11. An image processing system comprising:

a network;

an image processing device capable of communication via said network; and

at least one communicating device capable of communication with said image processing device via said network;

wherein said image processing device includes

a detecting section configured for selectively detecting a plurality of objects of a plurality of types from an input image, wherein the plurality of types of objects includes;

moving objects, faces, and stationary objects;

a generating section configured to generate, for a given image frame of the input image, one converted image frame for each type of object detected by the detection section, where each converted image frame is based on the given frame of the input image and corresponds to one of the plurality of types of objects detected by the detecting section; and

a processing section configured to selectively subject the converted frames to processing, wherein the processing includes generating an output image frame comprising a synthesis of more than one of the converted image frames for the given image frame,

wherein the processing section is configured to be capable of selectively synthesizing a plurality of sub-images into the output image, wherein the sub-images selectable by the processing section for synthesizing into the output image include;

an image generated by combining an On-Screen Display (foreground image) with an image corresponding to the given image frame of the input image,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section and adding detection information to the image, the detection information being information indicating those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section, said image comprising a zoomed in portion of said given image frame corresponding to one those objects that were detected by the detecting section and are of the type corresponding to the converted image frame

wherein the image processing device is configured such that a user may select one or more of the plurality of detected objects to be monitored objects, and

wherein the plurality of sub-images synthesized by the processing section into the output image includes at least one converted image frame for each monitored object, each converted image frame for a monitored object being generated by combining an On-Screen Display (foreground image) with an image corresponding to the type of the respective monitored object and comprising a zoomed in portion of said given image frame corresponding to the respective monitored object.

12. A camera device comprising:

an image pickup section configured to obtain picked-up image data; and

an image processing device configured to subject the image data obtained by said image pickup section to image processing;

wherein said image processing includes

a detecting section configured for selectively detecting a plurality of objects of a plurality of types from an input image, wherein the plurality of types of objects includes;

moving objects, faces, and stationary objects;

a generating section configured to generate, for a given image frame of the input image, one converted image frame for each type of object detected by the detection section, where each converted image frame is based on the given frame of the input image and corresponds to one of the plurality of types of objects detected by the detecting section; and

a processing section configured to selectively subject the converted frames to processing, wherein the processing includes generating an output image frame comprising a synthesis of more than one of the converted image frames for the given image frame,

wherein the processing section is configured to be capable of selectively synthesizing a plurality of sub-images into the output image, wherein the sub-images selectable by the processing section for synthesizing into the output image include;

an image generated by combining an On-Screen Display (foreground image) with an image corresponding to the given image frame of the input image,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section and adding detection information to the image, the detection information being information indicating those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section, said image comprising a zoomed in portion of said given image frame corresponding to one those objects that were detected by the detecting section and are of the type corresponding to the converted image frame

wherein the image processing device is configured such that a user may select one or more of the plurality of detected objects to be monitored objects, and

wherein the plurality of sub-images synthesized by the processing section into the output image includes at least one converted image frame for each monitored object, each converted image frame for a monitored object being generated by combining an On-Screen Display (foreground image) with an image corresponding to the type of the respective monitored object and comprising a zoomed in portion of said given image frame corresponding to the respective monitored object.

13. An image processing device comprising:

detecting means for selectively detecting a plurality of objects of a plurality of types from one input image wherein the plurality of types of objects includes;

moving objects,

faces, and

stationary objects;

generating means for generating, for a given frame of the input image, one converted image frame for each type of object detected by the detection section, where each converted image frame is based on the given frame of the input image and corresponds to one of the plurality of types of objects detected by the detecting means; and

processing means for selectively subjecting the converted image frames to processing according to one of a setting and a request, wherein the processing includes generating an output image frame comprising a synthesis of more than one of the converted image frames for the given image frame,

wherein the processing means is configured to be capable of selectively synthesizing a plurality of sub-images into the output image, wherein the sub-images selectable by the processing section for synthesizing into the output image include;

an image generated by combining an On-Screen Display (foreground image) with an image corresponding to the given image frame of the input image,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section and adding detection information to the image, the detection information being information indicating those objects that were detected by the detecting section, appear in the image, and are of the type corresponding to the converted image frame,

a converted image frame generated by combining an On-Screen Display (foreground image) with an image corresponding to one of the types of objects detected by the detecting section, said image comprising a zoomed in portion of said given image frame corresponding to one those objects that were detected by the detecting section and are of the type corresponding to the converted image frame

wherein the image processing device is configured such that a user may select one or more of the plurality of detected objects to be monitored objects, and

wherein the plurality of sub-images synthesized by the processing means into the output image includes at least one converted image frame for each monitored object, each converted image frame for a monitored object being generated by combining an On-Screen Display (foreground image) with an image corresponding to the type of the respective monitored object and comprising a zoomed in portion of said given image frame corresponding to the respective monitored object.