MODULAR MOBILE CONNECTED PICO PROJECTORS FOR A LOCAL MULTI-USER COLLABORATION

• US 20120249416A1
• Filed: 03/29/2012
• Published: 10/04/2012
• Est. Priority Date: 03/29/2011
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: MACIOCCI, Giuliano, MABBUTT, Paul, EVERITT, Andrew J., BERRY, David T.
• Application Number:
  US13/434,258
• Publication Number:
  US 20120249416A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 9/3173 wherein the projection device is specially adapted for enhanced portability

Abstract

The various embodiments include systems and methods for rendering images in a virtual or augmented reality system that may include capturing scene images of a scene in a vicinity of a first and a second projector, capturing spatial data with a sensor array in the vicinity of the first and second projectors, analyzing captured scene images to recognize body parts, and projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts. Additional rendering operations may include tracking movements of the recognized body parts, applying a detection algorithm to the tracked movements to detect a predetermined gesture, applying a command corresponding to the detected predetermined gesture, and updating the projected images in response to the applied command.

Claims

1. A method of projecting images from first and second projectors coupled to a computing device, comprising:

capturing scene images of a scene in a vicinity of the first and second projectors;

capturing spatial data with a sensor array in the vicinity of the first and second projectors;

analyzing captured scene images to recognize body parts; and

projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts.

2. The method of claim 1, wherein the first and the second projectors are pico projectors.

3. The method of claim 1, wherein projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting images on a surface in front of users in the vicinity of the projectors.

4. The method of claim 1, further comprising:

orienting the projected images on a vertical or horizontal surface appropriate for users in the vicinity of the projectors.

5. The method of claim 1, further comprising:

tracking movements of the recognized body parts;

applying a detection algorithm to the tracked movements to detect a predetermined gesture;

applying a command corresponding to the detected predetermined gesture; and

modifying the projected images in response to the applied command.

6. The method of claim 5, wherein analyzing captured images to recognize body parts comprises:

applying an anatomical analysis to the images to determine whether objects within the captured images match a portion of an anatomical model.

7. The method of claim 5, wherein:

the projected images are of a virtual input device; and

applying a command corresponding to the detected predetermined gesture comprises applying a command consistent with those of a user input device.

8. The method of claim 5, wherein applying a command corresponding to the detected predetermined gesture comprises applying a command corresponding to moving the virtual object, and modifying the projected images in response to the applied command comprises displaying the virtual object as moving from a first location to a second location in response to detecting the predetermined gesture.

9. The method of claim 1, further comprising:

connecting the first projector to the second projector via connectors on exteriors of the first and second projectors in a back to back configuration, wherein projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least two images in approximately opposite directions.

10. The method of claim 1, further comprising:

connecting the first projector to a third projector via connectors on exteriors of the projectors; and

connecting the third projector to the second projector via connectors on exteriors of the projectors, wherein projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least three projected images such that they are separated by approximately ninety degrees.

11. The method of claim 1, further comprising:

connecting the first projector to a third projector and a fourth projector via connectors on exteriors of the projectors; and

connecting the third and fourth projectors to the second projector via connectors on exteriors of the projectors, wherein projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting four images such that they are separated by approximately ninety degrees.

12. The method of claim 1, further comprising:

defining a reference point in the scene;

determining an orientation of each of the first and the second projectors based on the defined reference point; and

orienting the projected images based on the defined reference point and the orientation of the first and the second projectors.

13. The method of claim 12, wherein:

defining a reference point in the scene comprises connecting the first projector to the second projector by a connection point; and

defining the connection point as the reference point.

14. A projector system for projecting images, comprising:

a first projector;

a second projector;

a computing device coupled to the first and second projectors, the computing device comprising;

a memory; and

a processor coupled to the memory, wherein the processor is configured with processor-executable instructions to perform operations comprising;

capturing scene images of a scene in a vicinity of the first and second projectors;

capturing spatial data in the vicinity of the first and second projectors;

analyzing captured scene images to recognize body parts; and

causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts.

15. The projector system of claim 14, wherein the first and the second projectors are pico projectors.

16. The projector system of claim 14, wherein the processor is configured with processor-executable instructions such that causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts comprises causing the projectors to project images on a surface in front of users in the vicinity of the projectors.

17. The projector system of claim 14, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

causing each of the first and second projectors to orient the projected images on a vertical or horizontal surface appropriate for users in the vicinity of the projectors.

18. The projector system of claim 14, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

tracking movements of the recognized body parts;

applying a detection algorithm to the tracked movements to detect a predetermined gesture;

applying a command corresponding to the detected predetermined gesture; and

causing the projectors to modify the projected images in response to the applied command.

19. The projector system of claim 18, wherein the processor is configured with processor-executable instructions such that analyzing captured images to recognize body parts comprises:

applying an anatomical analysis to the images to determine whether objects within the captured images match a portion of an anatomical model.

20. The projector system of claim 18, wherein the processor is configured with processor-executable instructions such that:

causing each of the first and second projectors to project images comprises causing each of the first and second projectors to project images of a virtual input device; and

applying a command corresponding to the detected predetermined gesture comprises applying a command consistent with those of a user input device.

21. The projector system of claim 18, wherein the processor is configured with processor-executable instructions such that:

applying a command corresponding to the detected predetermined gesture comprises applying a command corresponding to moving the virtual object; and

modifying the projected images in response to the applied command comprises displaying the virtual object as moving from a first location to a second location in response to detecting the predetermined gesture.

22. The projector system of claim 14, wherein the first projector is connected to the second projector via connectors on exteriors of the first and second projectors in a back to back configuration, and the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least two images in approximately opposite directions.

23. The projector system of claim 14, further comprising:

a third projector connected to each of the first and second projectors via connectors on exteriors of the projectors and coupled to the computing device, wherein the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least three projected images such that they are separated by approximately ninety degrees.

24. The projector system of claim 14, further comprising:

a third projector coupled to the computing device; and

a forth projector coupled to the computing device, wherein each of the first and second projectors are connected to each of the third and a fourth projectors via connectors on exteriors of the projectors, and wherein the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting four images separated by approximately ninety degrees.

25. The projector system of claim 14, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

defining a reference point in the scene;

determining an orientation of each of the first and the second projectors based on the defined reference point; and

causing the projectors to orient the projected images based on the defined reference point and the orientation of the first and the second projectors.

26. The projector system of claim 25, wherein the processor is configured with processor-executable instructions such that defining a reference point in the scene comprises connecting the first projector to the second projector by a connection point and defining the connection point as the reference point.

27. A computing device comprising:

means for capturing scene images of a scene in a vicinity of a first projector and a second projector;

means for capturing spatial data in the vicinity of the first and second projectors;

means for analyzing captured scene images to recognize body parts; and

means for causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts.

28. The computing device of claim 27, wherein means for causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts comprises means for causing pico projectors to project images in a shape and orientation determined based on the recognized body parts.

29. The computing device of claim 27, wherein means for causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts comprises means for causing the projectors to project images on a surface in front of users in the vicinity of the projectors.

30. The computing device of claim 27, further comprising:

means for causing each of the first and second projectors to orient the projected images on a vertical or horizontal surface appropriate for users in the vicinity of the projectors.

31. The computing device of claim 27, further comprising:

means for tracking movements of the recognized body parts;

means for applying a detection algorithm to the tracked movements to detect a predetermined gesture;

means for applying a command corresponding to the detected predetermined gesture; and

means for causing the projectors to modify the projected images in response to the applied command.

32. The computing device of claim 31, wherein means for analyzing captured images to recognize body parts comprises means for applying an anatomical analysis to the images to determine whether objects within the captured images match a portion of an anatomical model.

33. The computing device of claim 31, wherein:

means for causing each of the first and second projectors to project images comprises means for causing each of the first and second projectors to project images of a virtual input device; and

means for applying a command corresponding to the detected predetermined gesture comprises means for applying a command consistent with those of a user input device.

34. The computing device of claim 31, wherein:

means for applying a command corresponding to the detected predetermined gesture comprises means for applying a command corresponding to moving the virtual object; and

means for modifying the projected images in response to the applied command comprises means for displaying the virtual object as moving from a first location to a second location in response to detecting the predetermined gesture.

35. The computing device of claim 27, wherein means for projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises means for projecting at least two images in approximately opposite directions.

36. The computing device of claim 27, wherein means for projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises means for projecting at least three projected images from three different projectors such that they are separated by approximately ninety degrees.

37. The computing device of claim 27, wherein means for projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises means projecting four images from four different projectors such that they are separated by approximately ninety degrees.

38. The computing device of claim 27, further comprising:

means for defining a reference point in the scene;

means for determining an orientation of each of the first and the second projectors based on the defined reference point; and

means for causing the projectors to orient the projected images based on the defined reference point and the orientation of the first and the second projectors.

39. The computing device of claim 38, wherein means for defining a reference point in the scene comprises means for connecting the first projector to the second projector by a connection point and means for defining the connection point as the reference point.

40. A computing device, comprising:

a memory;

a processor coupled to the memory, wherein the processor is configured with processor-executable instructions to perform operations comprising;

capturing scene images of a scene in a vicinity of a first projector and a second projector;

capturing spatial data in the vicinity of the first and second projectors;

analyzing captured scene images to recognize body parts; and

causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts.

41. The computing device of claim 40, wherein the processor is configured with processor-executable instructions such that causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts comprises causing pico projectors to project images.

42. The computing device of claim 40, wherein the processor is configured with processor-executable instructions such that causing each of the first and second projectors to project images in a shape and orientation determined based on the recognized body parts comprises causing the projectors to project images on a surface in front of users in the vicinity of the projectors.

43. The computing device of claim 40, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

causing each of the first and second projectors to orient the projected images on a vertical or horizontal surface appropriate for users in the vicinity of the projectors.

44. The computing device of claim 40, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

tracking movements of the recognized body parts;

applying a detection algorithm to the tracked movements to detect a predetermined gesture;

applying a command corresponding to the detected predetermined gesture; and

causing the projectors to modify the projected images in response to the applied command.

45. The computing device of claim 44, wherein the processor is configured with processor-executable instructions such that analyzing captured images to recognize body parts comprises:

applying an anatomical analysis to the images to determine whether objects within the captured images match a portion of an anatomical model.

46. The computing device of claim 44, wherein the processor is configured with processor-executable instructions such that:

causing each of the first and second projectors to project images comprises causing each of the first and second projectors to project images of a virtual input device; and

applying a command corresponding to the detected predetermined gesture comprises applying a command consistent with those of a user input device.

47. The computing device of claim 44, wherein the processor is configured with processor-executable instructions such that:

applying a command corresponding to the detected predetermined gesture comprises applying a command corresponding to moving the virtual object; and

modifying the projected images in response to the applied command comprises displaying the virtual object as moving from a first location to a second location in response to detecting the predetermined gesture.

48. The computing device of claim 40, wherein the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least two images in approximately opposite directions

49. The computing device of claim 40, wherein the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least three projected images from three different projectors such that they are separated by approximately ninety degrees.

50. The computing device of claim 40, wherein the processor is configured with processor-executable instructions such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting four images from four different projectors such that they are separated by approximately ninety degrees.

51. The computing device of claim 40, wherein the processor is configured with processor-executable instructions to perform operations further comprising:

defining a reference point in the scene;

determining an orientation of each of the first and the second projectors based on the defined reference point; and

causing the projectors to orient the projected images based on the defined reference point and the orientation of the first and the second projectors.

52. The computing device of claim 51, wherein the processor is configured with processor-executable instructions such that defining a reference point in the scene comprises connecting the first projector to the second projector by a connection point and defining the connection point as the reference point.

53. A non-transitory computer readable storage medium having stored thereon processor-executable software instructions configured to cause a processor to perform operations for projecting images from first and second projectors, the operations comprising

capturing scene images of a scene in a vicinity of a first projector and a second projector;

capturing spatial data in the vicinity of the first and second projectors;

analyzing captured scene images to recognize body parts; and

projecting images from each of the first and the second projectors in a shape and orientation determined based on the recognized body parts.

54. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that projecting images from each of the first and the second projectors in a shape and orientation determined based on the recognized body parts comprises projecting images from pico projectors.

55. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that projecting images from each of the first and the second projectors in a shape and orientation determined based on the recognized body parts comprises causing the projectors to project images on a surface in front of users in the vicinity of the projectors.

56. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations comprising:

orienting the projected images on a vertical or horizontal surface appropriate for users in the vicinity of the projectors.

57. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations comprising:

tracking movements of the recognized body parts;

applying a detection algorithm to the tracked movements to detect a predetermined gesture;

applying a command corresponding to the detected predetermined gesture; and

causing the projectors to modify the projected images in response to the applied command.

58. The non-transitory computer readable storage medium of claim 57, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that analyzing captured images to recognize body parts comprises:

applying an anatomical analysis to the images to determine whether objects within the captured images match a portion of an anatomical model.

59. The non-transitory computer readable storage medium of claim 57, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that:

projecting images from each of the first and the second projectors comprises projecting images from each of the first and the second projectors of a virtual input device; and

applying a command corresponding to the detected predetermined gesture comprises applying a command consistent with those of a user input device.

60. The non-transitory computer readable storage medium of claim 57, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that:

applying a command corresponding to the detected predetermined gesture comprises applying a command corresponding to moving the virtual object; and

modifying the projected images in response to the applied command comprises displaying the virtual object as moving from a first location to a second location in response to detecting the predetermined gesture.

61. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least two images in approximately opposite directions

62. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting at least three projected images from three different projectors such that they are separated by approximately ninety degrees.

63. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that projecting images from each of the first and the second projectors with a shape and orientation determined based on the recognized body parts comprises projecting four images from four different projectors such that they are separated by approximately ninety degrees.

64. The non-transitory computer readable storage medium of claim 53, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations comprising:

defining a reference point in the scene;

determining an orientation of each of the first and the second projectors based on the defined reference point; and

orienting the projected images based on the defined reference point and the orientation of the first and the second projectors.

65. The non-transitory computer readable storage medium of claim 64, wherein the stored processor-executable software instructions are configured to cause a processor to perform operations such that defining a reference point in the scene comprises connecting the first projector to the second projector by a connection point and defining the connection point as the reference point.