MODULATION OF BACKGROUND SUBSTITUTION BASED ON CAMERA ATTITUDE AND MOTION

• US 20090315915A1
• Filed: 06/19/2008
• Published: 12/24/2009
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: BLAKE, TERANCE B., DUNN, GREGORY J.
• Granted Patent: US 9,253,416 B2
• Time in Patent Office: Days
• US Class Current: 345/632
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04M 2250/62 user interface aspects of conference calls
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

A device and method of background substitution are disclosed. One or more cameras in a mobile device obtain a depth image. A processor in or external to the device segments the foreground from the background of the image. The original background is removed and a stored background image or video is substituted in place of the original background. The substituted background is altered dependent on the attitude and motion of the device, which is sensed by one or more sensors in the device. A portion of the stored background selected as the substitute background varies in correspondence with the device movement.

Claims

1. A mobile device comprising:

an imager configured to provide a depth image including a foreground and an original background;

a sensor configured to sense movement of the mobile device; and

a processor configured to separate the foreground from the original background, to replace the original background with a substitute background to form a substitute image, and to change the substitute background based on the sensed movement of the mobile device.

2. The mobile device of claim 1, further comprising a user-actuated input to adjust a virtual barrier defining the foreground from the original background.

3. The mobile device of claim 1, wherein the sensor comprises an accelerometer.

4. The mobile device of claim 1, further comprising:

a user-activated input; and

a memory configured to store a plurality of substitute backgrounds that are selectable by the user-activated input.

5. The mobile device of claim 1, further comprising a user-activated input configured to select the substitute background from a plurality of substitute backgrounds stored in a remote site.

6. The mobile device of claim 5, further comprising a memory configured to buffer the selected substitute background prior to replacing the original background.

7. The mobile device of claim 1, wherein the substitute background is selected from a portion of a larger background, the portion being selected dependent on the sensed movement of the mobile device.

8. The mobile device of claim 1, wherein the imager is a stereo camera system.

9. The mobile device of claim 1, wherein a rate of motion and an attitude of the substitute background matches a rate of movement and attitude, respectively, sensed by the sensor.

10. The mobile device of claim 1, wherein the substitute background contains motion that is independent of the sensed movement.

11. The mobile device of claim 1, further comprising a display configured to display the substitute image to the user.

12. The mobile device of claim 1, wherein the processor is configured to automatically adjust a position of a virtual barrier defining the foreground from the original background dependent on the sensed movement.

13. A method of providing an image from a mobile device, the method comprising:

obtaining a depth image using an imager in the mobile device, the depth image including a foreground and an original background;

detecting movement of the mobile device using a sensor in the mobile device;

separating the foreground from the original background;

replacing the original background with a stored substitute background to form a substitute image; and

changing the substitute background with respect to the foreground to correspond with the detected movement of the mobile device.

14. The method of claim 13, further comprising automatically adjusting a virtual barrier defining the foreground from the original background depending on the detected movement.

15. The method of claim 13, further comprising:

storing a plurality of substitute backgrounds in a memory external to the mobile device; and

buffering the selected substitute background prior to replacing the original background in a memory in the mobile device.

16. The method of claim 13, wherein the substitute background is a portion of a larger background, the method further comprising moving from where in the larger background the substitute background is selected based on the detected movement of the mobile device.

17. The method of claim 13, wherein the substitute background contains motion that is independent of the detected movement.

18. The method of claim 13, further comprising displaying the substitute image on a display of the mobile device.

19. The method of claim 13, wherein the separating, replacing, and moving is performed in the mobile device.

20. The method of claim 13, wherein at least some of the separating, replacing, and moving is performed external to the mobile device.