Modulation of background substitution based on camera attitude and motion

• US 9,253,416 B2
• Filed: 06/19/2008
• Issued: 02/02/2016
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Motorola Solutions, Inc.
• Inventors: Dunn, Gregory J., Blake, Terance B.
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US12/142,297
• Publication Number:
  US 20090315915A1
• Time in Patent Office: 2,784 Days
• Field of Search: 345/629, 345/158
• US Class Current: 1/1
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04M 2250/62 user interface aspects of conference calls
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A device and method of background substitution are disclosed. One or more cameras in a mobile device obtain a depth image. A processor in or external to the device segments the foreground from the background of the image. The original background is removed and a stored background image or video is substituted in place of the original background. The substituted background is altered dependent on the attitude and motion of the device, which is sensed by one or more sensors in the device. A portion of the stored background selected as the substitute background varies in correspondence with the device movement.

Claims

1. A mobile device comprising:

an imager configured to provide a first depth image including a first foreground and a first original background and a subsequent second depth image including a second foreground and a second original background;

a storage configured to store a substitute background;

a sensor configured to sense movement of the mobile device; and

a processor configured to;

separate the first foreground from the first original background of the first depth image,

replace the first original background with the substitute background to form a first composite image

modulate the substitute background as a function of a movement of the mobile device sensed by the sensor to form a modulated substitute background, and

replace the second original background with the modulated substitute background to form a second composite image.

2. The mobile device of claim 1, further comprising a user-actuated input to adjust a virtual barrier defining a separation point between the respective first and second foregrounds and the respective first and second original backgrounds.

3. The mobile device of claim 1, wherein the sensor comprises an accelerometer.

4. The mobile device of claim 1, further comprising:

a user-activated input; and

a memory configured to store a plurality of substitute backgrounds that are selectable by the user-activated input.

5. The mobile device of claim 1, further comprising a transceiver and a user-activated input configured to select the substitute background from a plurality of substitute backgrounds stored in a remote site and to cause the processor to retrieve the substitute background via the transceiver.

6. The mobile device of claim 5, further comprising a memory configured to store the selected substitute background prior to replacing the first original background.

7. The mobile device of claim 1, wherein the substitute background is selected from a first portion of a larger background, and wherein modulating the substitute background comprises shifting a current selection of the larger background from the first portion to a second portion of the larger background different from the first portion.

8. The mobile device of claim 1, wherein the imager is a stereo camera system.

9. The mobile device of claim 1, wherein the modulating of the substitute background is executed at a rate of motion and an attitude to match a rate of movement and attitude of movement of the mobile station sensed by the sensor.

10. The mobile device of claim 1, wherein the substitute background contains a separate motion that is independent of the sensed movement, and wherein the modulation of the substitute background is executed as a function of both the separate motion and the sensed movement.

11. The mobile device of claim 1, further comprising a display configured to display the first and second composite images to the user.

12. The mobile device of claim 1, wherein the processor is configured to automatically adjust a position of a virtual barrier defining a separation point between the respective first and second foregrounds and the respective first and second original backgrounds as a function of the movement of the mobile device sensed by the sensor.

13. A method of providing composite images from a mobile device, the method comprising:

obtaining a first depth image using an imager in the mobile device, the depth image including a first foreground and a first original background;

detecting movement of the mobile device using a sensor in the mobile device;

obtaining a second depth image using the imager in the mobile device, the second depth image including a second foreground and a second original background;

separating the first foreground from the first original background of the first depth image;

replacing the first original background with a pre-stored substitute background to form a first composite image;

modulating the substitute background as a function of the movement of the mobile device detected by the sensor to form a modulated substitute background;

replacing the second original background with the modulated substitute background to form a second composite image.

14. The method of claim 13, further comprising automatically adjusting a virtual barrier defining a separation point between the respective first and second foregrounds and the respective first and second original backgrounds as a function of the movement of the mobile device sensed by the sensor.

15. The method of claim 13, further comprising:

storing a plurality of substitute backgrounds in a memory external to the mobile device;

retrieving a selected one of the substitute backgrounds from the memory; and

locally storing the selected one of the substitute backgrounds prior to replacing the first original background in a memory in the mobile device.

16. The method of claim 13, wherein the substitute background is selected from a first portion of a larger background, and wherein modulating the substitute background comprises shifting a current selection of the larger background from the first portion to a second portion of the larger background different from the first portion.

17. The method of claim 13, wherein the substitute background contains a separate motion that is independent of the detected movement, and wherein modulating the substitute background comprises modulating the substitute image as a function of both the separate motion and the sensed movement.

18. The method of claim 13, further comprising displaying the first and second composite images on a display of the mobile device.

19. The method of claim 13, wherein the separating, replacing, and modulating is performed in the mobile device.

20. The method of claim 13, wherein at least one of the separating, replacing, and modulating is performed external to the mobile device.