FOCUSED AREAS IN AN IMAGE

• US 20100134642A1
• Filed: 06/26/2007
• Published: 06/03/2010
• Est. Priority Date: 10/02/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Thorn, Ola
• Granted Patent: US 8,767,082 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 7/287 including a sight line detecting device
• CPC Class Codes: G03B 13/36 Autofocus systems
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: H04M 1/0264 for a camera module assembly
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions

Abstract

The invention discloses a method and a portable device for focusing an image in the device (10), which device (10) comprises; a first image recording arrangement (24a) for recording images of an user (50) of the device (10); a second image recording arrangement (24b) comprising an autofocus arrangement for recording images of the environment surrounding the device (10); and a display arrangement (22) for reproducing the recorded images of the environment surrounding the device (10), which method comprises the step of: obtaining a plurality of gazing directions of the user (50) from images recorded by the first image recording arrangement (24a); selecting focusing areas depending on the obtained gazing directions; processing said at least one image of the environment so as to create a final image that is focused within areas defined by the focusing areas.

Claims

1. A method for focusing an image in a portable device (10), which portable device (10) comprises:

a first camera arrangement (24a) for recording images of a face of an user (50) of the portable device (10);

a second camera arrangement (24b) comprising an autofocus arrangement for recording images of an environment surrounding the portable device (10); and

a display arrangement (22) for reproducing the recorded images of the environment surrounding the portable device (10),

which method comprises;

recording at least one image of the environment by the second camera arrangement (24b);

obtaining a plurality of gazing directions for the user (50) from images recorded by the first camera arrangement (24a);

selecting a plurality of focusing areas in the image reproduced by the display arrangement (22), depending on the obtained gazing directions; and

processing said at least one image of the environment so as to create a final image that is focused within areas defined by the plurality of focusing areas.

2. A method according to claim 1 further comprising:

recording a plurality of images of the environment, and

processing said plurality of images by combining the images that are better focused within an area defined by the plurality of focusing areas compared to the focus within a corresponding area in the other images, so as to create a final image that is focused within areas defined by the plurality of focusing areas.

3. A method according to claim 2 where the second camera arrangement (24b) comprises an autofocus arrangement, and where each image of the environment is recorded when the image within a focusing area has been focused by the autofocus arrangement.

4. A method according to claim 1 where the recording at least one image includes:

recording a single image of the environment, and

processing said single image by digital image processing so as to focus the single image within areas defined by the plurality of focusing areas, so as to create a final image that is focused within areas defined by the plurality of focusing areas.

5. A method according to claim 1 further comprising:

recording an image of the environment that is focused within a predetermined depth of field, and

processing said image by at least blurring the image within the depth of field so that the image remains focused within areas defined by the plurality of focusing areas.

6. A method according to claim 1 where the obtaining a plurality of gazing directions includes:

selecting a focusing area when a gaze direction has remained stable for a predetermined period.

7. A method according to claim 1 where the obtaining a plurality of gazing directions includes:

obtaining at least one of gazing directions towards the environment surrounding the device (10) or gazing directions towards the display arrangement (22).

8. A method according to claim 1 where the obtaining a plurality of gazing directions includes:

obtaining a gazing direction for at least one of the face of the user (50) or at least one eye of the user (50).

9. A method according to claim 8 where selecting a plurality of focusing areas includes:

selecting a focusing area depending on both the gazing directions for the face and said at least one eye of the user (50).

10. A method according to claim 1 where selecting a plurality of focusing areas includes:

selecting a focusing area in a grid (700) of focusing areas or moving a movable focusing frame (400), depending on the obtained gazing direction so as to identify a focusing area.

11. A method according to claim 10 further comprising:

changing a shape of the grid (700) or the focusing frame (400) on commands from the user (50).

12. A method according to any of claim 10 where the focusing grid (700) or focusing frame (400) is visible to the user (50).

13. A method according to claim 1 where the portable device (10) is a portable communication device.

14. A method according to claim 1 where the first camera arrangement (24a) is to record images of the user (50) in a first operational mode for obtaining a gazing direction of the user (50), and in a second operational mode for providing images of the user (50) during a video-call.

15. A method according to claim 13 where the portable communication device is a cell phone (10).

16. A portable device (10) comprising;

a first camera arrangement (24a) for recording images of a face of an user (50) of the portable device (10);

a second camera arrangement (24b) comprising an autofocus arrangement for recording at least one image of an environment surrounding the portable device (10);

a display arrangement (22) for reproducing the recorded images of the environment surrounding the portable device (10); and

an autofocus-control (40) for focusing an image in the portable device (10), the autofocus-control (40) is to;

obtain a plurality of gazing directions for the user (50) from images recorded by the first camera arrangement (24a);

select a plurality of focusing areas in the images reproduced by the display arrangement (22), depending on the obtained gazing directions; and

process said at least one image of the environment so as to create a final image that is focused within the areas defined by the plurality of focusing areas.

17. A portable device (10) according to claim 16 where the autofocus-control (40) is to:

record a plurality of images of the environment, and

process said plurality of images by combining the images that are better focused within an area defined by the plurality of focusing areas combining to the focus within a corresponding area in the other images, so as to create a final image that is focused within areas defined by the focusing areas.

18. A portable device (10) according to claim 16 where the autofocus-control (40) is further to order the second camera arrangement (24b) to record an image of the environment when the image within a focusing area has been focused by the autofocus arrangement.

19. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

order the second camera arrangement (24b) to record single image of the environment, and

process said single image by digital image processing so as to focus the single image within areas defined by the plurality of focusing areas, so as to create a final image that is focused within areas defined by the plurality of focusing areas.

20. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

record an image of the environment that is focused within a depth of field, and

process said image by at least blurring the image within the depth of field so that the image remains focused within areas defined by the plurality of focusing areas.

21. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

select a focusing area when a gaze direction has remained stable for a predetermined period.

22. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

obtain at least one of gazing directions towards the environment surrounding the portable device (10) or gazing directions towards the display arrangement (22).

23. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

obtain a gazing direction for at least one of the face of the user (50) or at least one eye of the user (50).

24. A portable device (10) according to claim 23 where the autofocus-control (40) is further to:

select a focusing area depending on the obtained gazing direction for both the face and said least one eye of the user (50).

25. A portable device (10) according to claim 16 where the autofocus-control (40) is further to:

select a focusing area in a grid (700) of focusing areas or to move a movable focusing frame (400), depending on the obtained gazing direction so as to identify a focusing area.

26. A portable device (10) according to claim 25 where the autofocus-control (40) is further to:

change a shape of the grid (700) or the focusing frame (400) on commands from the user (50).

28. A portable device (10) according to claim 16 where the portable device (10) is a portable communication device.

29. A portable device (20) according to claim 16 where the first camera arrangement (24a) is to:

record images of the user (50) in a first operational mode for obtaining a gazing direction of the user (50), and

record images of the user (50) in a second operational mode for providing images of the user (50) during a video-call.

30. A portable device (10) according to claim 28 where the portable communication device is a cell phone (10).

27. A portable device (10) according to 25 where the focusing grid (700) or focusing frame (400) is visible to the user (50).

31. A computer program product stored on a computer usable medium, comprising readable program instruction for causing a portable device (10) to execute a method, when said program instructions are loaded in the portable device (10), comprising:

a first camera arrangement (24a) for recording images of a face of an user (50) of the portable device (10);

a second camera arrangement (24b) comprising an autofocus arrangement for recording images of an environment surrounding the portable device (10);

a display arrangement (22) for reproducing the recorded images of the environment surrounding the portable device (10); and

an autofocus-control (40) for focusing an image in the portable device (10), the method comprising;

recording at least one image of the environment by the second camera arrangement (24b);

obtaining a plurality of gazing directions of the user (50) from images recorded by the first camera arrangement (24a);

selecting a plurality of focusing areas in the image reproduced by the display arrangement (22) depending on the obtained gazing directions; and

processing said at least one image of the environment so as to create a final image that is focused within areas defined by the plurality of focusing areas.

32. A computer program element having a program recorded thereon, where the program is to make a portable device (10) to execute a method, when said program is loaded in the portable device (10), the portable device comprising:

a first camera arrangement (24a) for recording images of a face of an user (50) of the portable device (10);

a second camera arrangement (24b) comprising an autofocus arrangement for recording images of an environment surrounding the portable device (10);

a display arrangement (22) for reproducing the recorded images of the environment surrounding the portable device (10); and

an autofocus-control (40) for focusing an image in the portable device (10), the method comprising;

recording at least one image of the environment by the second camera arrangement (24b);

obtaining a plurality of gazing directions for the user (50) from images Recorded by the first camera arrangement (24a), at least one gazing direction of the plurality of gazing directions being obtained based on at least one of the face of the user or at least one eye of the user;

selecting a plurality of focusing area'"'"'s in the image reproduced by the display (22) depending on the obtained gazing directions; and

processing said at least one image of the environment so as to create a final image that is focused within areas defined by the plurality of focusing areas.