Video signal generating apparatus, video signal receiving apparatus, and video signal generating and receiving system

• US 10,389,926 B2
• Filed: 08/28/2015
• Issued: 08/20/2019
• Est. Priority Date: 09/12/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Shibata, Yoshiaki
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Hasan, Syed Y
• Application Number:
  US14/839,305
• Publication Number:
  US 20150373244A1
• Time in Patent Office: Days
• Field of Search: 348 68, 348 74, 348345, 382128, 386264, 386328, 386332, 705 2, 715201, 715230, 715233, 600407, 358450, 709203, 709217, 709219, 710 62, 2502081
• CPC Class Codes: A61B 1/00059 provided with identification means for the endoscope
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/06 with illuminating arrangements
• CPC Class Codes: G06F 13/385 for adaptation of a particular data processing system to different peripheral devices
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/3027 used signal is digitally coded
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

A video signal generating apparatus is disclosed which includes: a camera configured to generate a video signal; an operating device configured to operate at least either the camera or an apparatus configured to receive the video signal; a text data generating device configured to generate text data including a meaningful term in response to the operation performed on the operating device; and a text data inserting device configured to insert the text data generated by the text data generating device into the video signal in a corresponding manner.

Claims

1. A surgical image processing apparatus comprising:

circuitry configured to;

perform image processing on surgical video image data to visually detect a target object in a frame of the surgical video image data by an application of an image pattern recognition technique,

in response to an operation performed on an endoscope coupled to the surgical image processing apparatus, the operation being related to changing settings of imaging of the endoscope, select one or more reserved words from a predetermined dictionary according to operation information about the operation performed,

generate first metadata to cause a receiving apparatus to generate a first surgical still image of the frame of the surgical video image data in which the target object is detected and first time information indicating a first time when the target object is detected by the image processing relative to a start time of the surgical video image data,

generate second metadata comprising the operation information to cause the receiving apparatus to generate a second surgical still image of the frame of the surgical video image data in which the operation is performed and second time information indicating a second time when the operation is performed relative to the start time of the surgical video image data,

superimpose the generated metadata onto an ancillary data area of the frame of the surgical video image data in the High Definition Serial Digital Interface (HD-SDI) format according to a digital video transmission standard published by the Society of Motion Picture and Television Engineers (SMPTE), and

output the surgical video image data with the superimposed metadata to the receiving apparatus, wherein

the first metadata is interpretable by the receiving apparatus for automatically generating the first surgical still image of the frame of the surgical video image data in which the target object is detected at the first time, and

the second metadata is interpretable by the receiving apparatus for automatically generating the second surgical still image of the frame of the surgical video image data in which the operation is performed at the second time.

2. The surgical image processing apparatus according to claim 1, wherein information in the first metadata is represented as a text data.

3. The surgical image processing apparatus according to claim 2, wherein the text data is selected from the predetermined dictionary.

4. The surgical image processing apparatus according to claim 1, wherein the circuitry is further configured to identify a diagnosis from the surgical video image data.

5. The surgical image processing apparatus according to claim 4, wherein the circuitry is further configured to identify a position associated with the diagnosis within the surgical video image data.

6. The surgical image processing apparatus according to claim 4, wherein the circuitry is further configured to generate diagnosis information indicating the diagnosis, and set as third time information a third time when the diagnosis is identified relative to the start time of the surgical video image data.

7. The surgical image processing apparatus according to claim 1, wherein the circuitry is further configured to identify a position in the surgical video image data having brightness above a threshold.

8. A system comprising:

a surgical imaging camera configured to capture surgical video image data from a patient;

a recorder configured to receive the captured surgical video image data from the surgical imaging camera, store the received surgical video image data, and provide the surgical video image data to a surgical image processing apparatus; and

the surgical image processing apparatus including;

circuitry configured to;

perform image processing on the surgical video image data to visually detect a target object in a frame of the surgical video image data by an application of an image pattern recognition technique,

in response to an operation performed on the surgical imaging camera coupled to the surgical image processing apparatus, the operation being related to changing settings of imaging of the surgical imaging camera, select one or more reserved words from a predetermined dictionary according to operation information about the operation performed,

generate first metadata to cause a receiving apparatus to generate a first surgical still image of the frame of the surgical video image data in which the target object is detected and first time information indicating a first time when the target object is detected by the image processing relative to a start time of the surgical video image data,

generate second metadata comprising the operation information to cause the receiving apparatus to generate a second surgical still image of the frame of the surgical video image data in which the operation is performed and second time information indicating a second time when the operation is performed relative to the start time of the surgical video image data,

superimpose the generated metadata onto an ancillary data area of the frame of the surgical video image data in the High Definition Serial Digital Interface (HD SDI) format according to a digital video transmission standard published by the Society of Motion Picture and Television Engineers (SMPTE), and

output the surgical video image data with the superimposed metadata to the receiving apparatus, wherein

the first metadata is interpretable by the receiving apparatus for automatically generating the first surgical still image of the frame of the surgical video image data in which the target object is detected at the first time, and

the second metadata is interpretable by the receiving apparatus for automatically generating the second surgical still image of the frame of the surgical video image data in which the operation is performed at the second time.

9. A surgical image processing apparatus comprising:

circuitry configured to;

perform image processing on surgical video image data to visually detect a target object in a frame of the surgical video image data by an application of an image pattern recognition technique,

in response to an operation performed on at least one of a surgical camera or a peripheral device connected to the surgical image processing apparatus, the operation being related to changing settings of imaging of the one of the surgical camera or the peripheral device, select one or more reserved words from a predetermined dictionary according to operation information about the operation performed,

generate first metadata comprising;

an operation indicator to cause the at least one of the surgical camera or the peripheral device connected to the surgical image processing apparatus or the surgical camera to generate a first surgical still image of the frame of the surgical video image data in which the target object is detected, and

first time information indicating a first time when the target object is detected relative to a start time of the surgical video image data captured by the surgical camera;

generate second metadata comprising;

the operation information to cause the at least one of the surgical camera or the peripheral device to generate a second surgical still image of the frame of the surgical video image data in which the operation is performed, and

second time information indicating a second time when the operation is performed relative to the start time of the surgical video image data;

superimpose the generated metadata onto an ancillary data area of the frame of the surgical video image data captured by the surgical camera in the High Definition Serial Digital Interface (HD-SDI) format according to a digital video transmission standard published by the Society of Motion Picture and Television Engineers (SMPTE), and

output the surgical video image data with the superimposed metadata to the surgical camera or the peripheral device, wherein

the first metadata is interpretable by the surgical camera or the peripheral device for automatically generating the first surgical still image of the frame of the surgical video image data in which the target object is detected at the first time, and

the second metadata is interpretable by the surgical camera or the peripheral device for automatically generating the second surgical still image of the frame of the surgical video image data in which the operation is performed at the second time.

10. The surgical image processing apparatus according to claim 9, wherein the peripheral device is connected to the surgical image processing apparatus, the peripheral device being a recording device.

11. The surgical image processing apparatus according to claim 9, wherein the circuitry is further configured to recognize an operation from an input audio signal, wherein the operation indicator is generated based on the recognized operation.

12. The surgical image processing apparatus according to claim 9, wherein the operation indicator is generated based on a control of a button.

13. The surgical image processing apparatus according to claim 9, wherein the operation indicator indicates capturing of the first surgical still image by the surgical camera.

14. The surgical image processing apparatus according to claim 13, wherein the first surgical still image includes an affected region of a patient.

15. The surgical image processing apparatus according to claim 14, wherein the circuitry is further configured to output the captured first surgical still image with a Digital Imaging and Communication in Medicine (DICOM) format.

16. The surgical image processing apparatus according to claim 9, wherein the operation indicator is a control signal to the surgical camera, and the control signal is at least one of a light control, a shutter speed control, a gain control, or a white balance control.

17. The surgical image processing apparatus according to claim 9, wherein the peripheral device is connected to the surgical image processing apparatus and the peripheral device is a light source.

18. A system comprising:

a surgical imaging camera configured to capture surgical video image data from a patient;

a recorder configured to receive the captured surgical video image data from the surgical imaging camera, store the received surgical video image data, and provide the surgical video image data to a surgical image processing apparatus; and

the surgical image processing apparatus including circuitry configured to;

perform image processing on surgical video image data to visually detect a target object in a frame of the surgical video image data by an application of an image pattern recognition technique;

in response to an operation performed on at least one of the surgical imaging camera or a peripheral device connected to the surgical image processing apparatus, the operation being related to changing settings of imaging of the one of the surgical imaging camera or the peripheral device, select one or more reserved words from a predetermined dictionary according to operation information about the operation performed;

generate first metadata comprising;

an operation indicator to cause the at least one of the surgical imaging camera or the peripheral device connected to the surgical image processing apparatus or the surgical imaging camera to generate a first surgical still image of the frame of the surgical video image data in which the target object is detected, and

first time information indicating a first time when the target object is detected relative to a start time of the surgical video image data;

generate second metadata comprising;

the operation information to cause the at least one of the surgical imaging camera or the peripheral device to generate a second surgical still image of the frame of the surgical video image data in which the operation is performed, and

second time information indicating a second time when the operation is performed relative to the start time of the surgical video image data;

superimpose the generated metadata onto an ancillary data area of the frame of the surgical video image data captured by the surgical imaging camera in the High Definition Serial Digital Interface (HD-SDI) format according to a digital video transmission standard published by the Society of Motion Picture and Television Engineers (SMPTE), and

output the surgical video image data with the superimposed metadata to the surgical imaging camera or the peripheral device, wherein

the first metadata is interpretable by the surgical imaging camera or the peripheral device for automatically generating the first surgical still image of the frame of the surgical video image data in which the target object is detected at the first time, and

the second metadata is interpretable by the surgical imaging camera or the peripheral device for automatically generating the second surgical still image of the frame of the surgical video image data in which the operation is performed at the second time.