Connecting video objects and physical objects for handheld projectors

• US 9,132,346 B2
• Filed: 04/04/2012
• Issued: 09/15/2015
• Est. Priority Date: 04/04/2012
• Status: Active Grant

• Current Assignee: Kenneth J. Huebner
• Original Assignee: Kenneth J. Huebner
• Inventors: Huebner, Kenneth J.
• Primary Examiner(s): Dowling, William C
• Application Number:
  US13/439,026
• Publication Number:
  US 20130265502A1
• Time in Patent Office: 1,259 Days
• Field of Search: 353/39, 353/42, 353/121, 353/122, 353/69, 353/70, 353/79, 353/28, 353/30, 353/46
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/825 Fostering virtual characters
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/203 Image generating hardware
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/8058 Virtual breeding, e.g. tamagotchi
• CPC Class Codes: G03B 21/145 Housing details, e.g. position adjustments thereof
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 : Using real world measurements to influence rendering
• CPC Class Codes: G09G 2340/04 : Changes in size, position or resolution of an image
• CPC Class Codes: H04N 9/3173 : wherein the projection device is specially adapted for enhanced portability
• CPC Class Codes: H04N 9/3179 : Video signal processing therefor
• CPC Class Codes: H04N 9/3185 : Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 : including sensor feedback

Abstract

A handheld projecting device that illuminates a visible image comprised of one or more video objects that appear to interact, connect, and/or form paths with physical objects and/or other video objects within a user'"'"'s environment. In at least one embodiment, a projecting device animates and projects one or more video objects that appear to responsively adapt to the position and orientation of at least one physical object for unique graphic and tactile effects. In some embodiments, a projecting device may connect together physical objects and video objects to form a user-defined object path, such as a racetrack or adventure trail. Video objects may be rendered and illuminated as three-dimensional objects to simulate real world objects.

Claims

1. An interactive image projecting system, comprising:

at least one physical object; and

a handheld projecting device including;

an outer housing;

a control unit within the outer housing;

an image projector operatively coupled to the control unit and operable to project a visible image comprised of one or more video objects generated by the control unit;

an image sensor operatively coupled to the control unit and operable to observe a spatial view including the at least one physical object located on a remote surface; and

an object tracker operable to analyze the observed spatial view including, the at least one physical object and determine a position and orientation of the at least one physical object from the observed spatial view,

wherein the control unit modifies the visible image comprised of one or more video objects based upon the position and orientation of the at least one physical object relative to the projected visible image, that the one or more video objects adapt to the position and orientation of the at least one physical object.

2. The device of claim 1 further comprising an object connector operable to connect the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially in the vicinity of the at least one physical object.

3. The device of claim 1 further comprising an object connector operable to connect the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially affixed to the at least one physical object.

4. The device of claim 1, wherein the object tracker further identifies the at least one physical object by analyzing at least one object tag associated with the physical object, wherein the object tag is comprised of an optical machine-readable pattern that represents data.

5. The device of claim 1 further comprising an object path maker operable to create an object path among a plurality of physical objects.

6. The device of claim 5, wherein the object path is an auto-generated object path that is substantially defined by the handheld projecting device.

7. The device of claim 5, wherein the object path is a user-defined object path that is substantially defined by a user.

8. A computer-implemented method for presenting video frames projected by a handheld projecting device, comprising the steps of:

obtaining, a spatial view located in front of the handheld projecting device;

determining a position and orientation of at least one physical object in the spatial view by the handheld projecting device; and

rendering one or more video frames comprised of one or more video objects that are projected by the handheld projecting device, wherein the one or more video objects appear to responsively adapt to the position and orientation of the at least one physical object in the spatial view.

9. The method of claim 8, wherein the handheld projecting device connects the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially in the vicinity of the at least one physical object.

10. The method of claim 8, wherein the handheld projecting device connects the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially affixed to the at least one physical object.

11. The method of claim 8, wherein the handheld projecting device further creates an object path among a plurality of physical objects.

12. The method of claim 11, wherein at least a first video object of the one or more video objects is a path video object that graphically defines the object path.

13. The method of claim 11, wherein at least a first video object of the one or more video objects is moved along, the object path by moving the handheld projecting device.

14. The method of claim 11, wherein the object path forms a closed path or a racetrack.

15. The method of claim 11, wherein the object path is an auto-generated object path that is substantially defined by the handheld projecting device.

16. The method of claim 11, wherein the object path is a user-defined object path that is substantially defined by a user.

17. A computer-readable storage medium comprised of computer-executable instructions that, when processed by a control unit, perform an operation for presenting video frames projected by a handheld projecting device, the operation comprising:

obtaining a spatial view located in front of the handheld projecting device;

determining a position and orientation of at least one physical object in the spatial view by the handheld projecting device; and

rendering one or more video frames comprised of one or more video objects that are projected by the handheld projecting device, wherein the one or more video objects appear to responsively adapt to the position and orientation of the at least one physical object in the spatial view.

18. The computer-readable storage medium of claim 17, wherein the handheld projecting device connects the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially in the vicinity of the at least one physical object.

19. The computer-readable storage medium of claim 17, wherein the handheld projecting device connects the one or more video objects to the at least one physical object such that the one or more video objects appear to remain substantially affixed to the at least one physical object.

20. The computer-readable storage medium of claim 17, wherein the handheld projecting device further creates an object path among a plurality of physical objects.

21. The computer-readable storage medium of claim 20, wherein at least a first video object of the one or more video objects is a path video object that graphically defines the object path.

22. The computer-readable storage medium of claim 20, wherein at least a first video object of the one or more video objects moved along the object path by moving the handheld projecting device.

23. The computer-readable storage medium of claim 20, wherein the object path forms a closed path or a racetrack.

24. The computer-readable storage medium of claim 20, wherein the object path is an auto-generated object path that is substantially defined by the handheld projecting device.

25. The computer-readable storage medium of claim 20, wherein the object path is a user-defined object path that is substantially defined by a user.