Systems, devices and methods for displaying pictures in a picture

• US 9,973,722 B2
• Filed: 02/20/2014
• Issued: 05/15/2018
• Est. Priority Date: 08/27/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Deng, Fan, Shanmugavadivelu, Karthikeyan, Ma, Wan Shun Vincent, Ravirala, Narayana Karthik Sadanandam, Ma, Lei, Huang, Pengjun, Liu, Shizhong
• Primary Examiner(s): Owens, Tsion B
• Assistant Examiner(s): Slater, Alison
• Application Number:
  US14/185,683
• Publication Number:
  US 20150062434A1
• Time in Patent Office: 1,545 Days
• CPC Class Codes: G09G 2352/00 Parallel handling of streams of display data
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

Systems, devices, and methods of displaying and/or recording multiple pictures in a picture (PIP) on the same display of a digital display device are disclosed. The PIPs can show objects from the main field view of the display device, such as a front camera lens, as well as objects from a different field of view, such as a back camera lens. The PIPs can further track the objects that are being displayed.

Claims

1. A method for displaying a plurality of pictures in a picture (PIP) on an electronic device, the method comprising:

capturing a first field of view in a first direction with a first imaging sensor of the device;

capturing a second field of view in a second direction with a second imaging sensor of the device, wherein the second direction is different from the first direction;

displaying a video of the first field of view on a screen of the device;

selecting a first object in the video;

recognizing selection of the first object;

defining a region of interest (ROI) in the video that includes the first object, wherein the ROI moves with the first object as the first object moves in the video;

displaying a first PIP on the screen that is separate from the ROI in the video, wherein the first PIP is displayed in a position having a first location on the screen such that the first PIP in the first location does not interfere with the ROI in the video, wherein the first PIP continuously displays a first PIP video that includes an image of the first object;

tracking the first object and displaying the first object in the first PIP as the first object is being tracked;

displaying a second PIP on the screen, wherein the second PIP displays a second PIP video that includes an image of a second object from the second field of view; and

moving the first PIP and the second PIP together as a group in response to determining that the first PIP or the second PIP interferes with the defined ROI.

2. The method of claim 1, further comprising detecting the first object when the first object is displayed on the screen.

3. The method of claim 1, further comprising recording the video, the first PIP video and the second PIP video.

4. The method of claim 1, further comprising:

displaying a third PIP and a fourth PIP on the screen, wherein the third PIP displays a third object and the fourth PIP displays a fourth object; and

tracking the first, second, third and fourth objects as they are being displayed.

5. The method of claim 1, further comprising displaying the defined ROI on the screen.

6. The method of claim 1, wherein defining the ROI comprises defining a margin that leaves a specified amount of distance between a boundary of the first object and the ROI.

7. A system for displaying a plurality of pictures in a picture (PIP) on an electronic device, the system comprising:

at least one imaging sensor configured to capture a first field of view in a first direction and a second field of view in a second direction, wherein the second direction is different from the first direction;

a processor configured to;

recognize selection of a first object to track in a video of the first field of view;

define a region of interest (ROI) in the video that includes the first object, wherein the ROI moves with the first object as the first object moves in the video;

detect and track the first object in the video;

generate a first PIP that is separate from the ROI in the video and comprising the first object in a first PIP video as the first object is being tracked;

determine a position for the first PIP having a first location such that the first PIP in the first location does not interfere with the ROI in the video;

and

a screen configured to display the video, the first PIP comprising the first object as it is being tracked, and a second PIP comprising a second PIP video that includes a second object from the second field of view, wherein the first PIP is displayed in the position having the first location,

wherein the processor is further configured to move the first PIP and the second PIP together as a group in response to determining that the first PIP or the second PIP interferes with the defined ROI.

8. The system of claim 7, wherein the processor is further configured to record the video, the first PIP video and the second PIP video.

9. The system of claim 8, wherein the processor is further configured to record the video, the first PIP video and the second PIP video each as separate videos.

10. The system of claim 8, wherein the processor is further configured to generate a third PIP and a fourth PIP, and the screen is further configured to display the third PIP and the fourth PIP.

11. The system of claim 7, wherein the processor is further configured to track the second object and the screen is further configured to display the tracked second object in the second PIP video.

12. The system of claim 7, wherein the first PIP is resizable.

13. The system of claim 7, wherein the video is a live video.

14. The system of claim 7, wherein the video is a recorded video.

15. The system of claim 7, wherein the processor is further configured to display the defined ROI on the screen.

16. The system of claim 7, wherein the processor is further configured to define the ROI by defining a margin that leaves a specified amount of distance between a boundary of the first object and the ROI.

17. A system for displaying a plurality of pictures in a picture (PIP) on an electronic device, the system comprising:

means for capturing a first field of view in a first direction;

means for capturing a second field of view in a second direction, wherein the second direction is different from the first direction

means for displaying a video of the first field of view on a screen of the device;

means for selecting a first object in the video;

means for recognizing selection of the first object;

means for defining a region of interest (ROI) in the video that includes the first object, wherein the ROI moves with the first object as the first object moves in the video;

means for displaying a first PIP on the screen that is separate from the ROI in the video, wherein the first PIP is displayed in a position having a first location on the screen such that the first PIP in the first location does not interfere with the ROI in the video, wherein the first PIP continuously displays a first PIP video that includes an image of the first object;

means for tracking the first object and displaying the first object in the first PIP as the first object is being tracked;

means for displaying a second PIP on the screen, wherein the second PIP displays a second PIP video that includes an image of a second object from the second field of view; and

means for moving the first PIP and the second PIP together as a group in response to determining that the first PIP or the second PIP interferes with the defined ROI.

18. The system of claim 17, further comprising means for recording the video, the first PIP video and the second PIP video.

19. The system of claim 17, further comprising means for displaying the defined ROI on the screen.

20. The system of claim 17, wherein the means for defining the ROI comprises means for defining a margin that leaves a specified amount of distance between a boundary of the first object and the ROI.

21. A non-transient computer readable medium configured to store instructions that when executed by a processor perform a method for displaying a plurality of pictures in a picture (PIP) orientation on an electronic device, the method comprising:

capturing a first field of view in a first direction with a first imaging sensor of the device;

capturing a second field of view in a second direction with a second imaging sensor of the device, wherein the second direction is different from the first direction;

displaying a video of the first field of view on a screen of the device;

selecting a first object in the video;

recognizing selection of the first object;

defining a region of interest (ROI) in the video that includes the first object, wherein the ROI moves with the first object as the first object moves in the video;

displaying a first PIP on the screen that is separate from the ROI in the video, wherein the first PIP is displayed in a position having a first location on the screen such that the first PIP in the first location does not interfere with the ROI in the video, wherein the first PIP continuously displays a first PIP video that includes an image of the first object;

tracking the first object and displaying the first object in the first PIP as the first object is being tracked;

displaying a second PIP on the screen, wherein the second PIP displays a second PIP video that includes an image of a second object from the second field of view;

moving the first PIP and the second PIP together as a group in response to determining that the first PIP or the second PIP interferes with the defined ROI.

22. The computer readable medium of claim 21, the method further comprising recording the video, the first PIP video and the second PIP video.

23. The computer readable medium of claim 21, wherein the method further comprises displaying the defined ROI on the screen.

24. The computer readable medium of claim 21, wherein defining the ROI comprises defining a margin that leaves a specified amount of distance between a boundary of the first object and the ROI.