System and method of interaction for mobile devices

• US 20140248950A1
• Filed: 02/27/2014
• Published: 09/04/2014
• Est. Priority Date: 03/01/2013
• Status: Abandoned Application

• Current Assignee: Martin Tosas Bautista
• Original Assignee: Martin Tosas Bautista
• Inventors: Tosas Bautista, Martin
• Application Number:
  US14/191,549
• Publication Number:
  US 20140248950A1
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A system and method of interaction with one or more applications running on a mobile device. The system maps the visual output of one or more applications running on the mobile device onto one or more virtual surfaces located within a user defined coordinate system, the mobile device being within this coordinate system, the coordinate system being attach to an arbitrary scene. The system estimates the pose of the mobile device within the coordinate system and according to this pose displays an interactive view of the virtual surfaces. The displayed view enables interaction with the one or more applications running on the mobile device.

Claims

1. A system enabling a user to interact with one or more applications running in a mobile device, the system comprising:

A means of estimating the pose of the mobile device, the pose being defined on a coordinate system, the origin and orientation of the coordinate system being attached to a part of a scene;

A means of mapping the visual output of one or more applications running in the mobile device onto one or more virtual surfaces located within the coordinate system;

A means of rendering on a display associated with the mobile device a view of the visual output mapped onto one or more virtual surfaces according to the relative poses of the mobile device and the one or more virtual surfaces;

A means of accepting a user input related with the view rendered on the display associated with the mobile device and translating this input into a corresponding input to one or more applications running on the mobile device, thereby enabling the user of the system to interact with one or more applications running on the mobile device.

2. A system according to claim 1, wherein the estimation of the pose of the mobile device is achieved by tracking and mapping of the scene, thereby enabling to extend the usable part of the coordinate system beyond the part of the scene to which it was originally attached to.

3. A system according to claim 2, further comprising a means of reattaching the origin and orientation of the coordinate system to a different part of the scene.

4. A system according to claim 2, further comprising a means of recording the attachment between the coordinate system and the scene such that it may be recovered at a later time.

5. A system according to claim 1, wherein the system of interaction with one or more applications running on the mobile device can be interrupted and reinstated without interfering with the normal operation of the applications running on the mobile device.

6. A system according to claim 3, wherein the applications running on the mobile device include a web browser.

7. A system according to claim 2, wherein the applications running on the mobile device include an application whose visual output includes a container region that can be populated programatically.

8. A system according to claim 7, further comprising a means of recording the contents of a container region defined on one of the virtual surfaces such that they can be recovered at a later time.

9. A system according to claim 2, wherein contents within a bounded region can be exported to an area outside the bounded region and within the virtual surface.

10. A system according to claim 9, wherein the contents exported to the area outside the bounded region are updated when the content within the bounded region is updated.

11. A system according to claim 2, wherein the coordinate system can be shared with other mobile devices, thereby enabling estimating the pose of each mobile device within the same coordinate system.

12. A system according to claim 11, wherein the other mobile devices include HMDs.

13. A system according to claim 2, wherein the coordinate system can be shared with other mobile devices, thereby enabling estimating the pose of each mobile device within the same coordinate system.

14. A system according to claim 13, wherein the other mobile devices include HMDs.

15. A system enabling a mobile device to create video games based on scenes, the system comprising:

A means of estimating the pose of the mobile device, the pose being defined on a coordinate system, the origin and orientation of the coordinate system being attached to a part of a scene;

A means of creating a map of the scene;

A means of identifying features on the map of the scene and interpreting these features as game objects;

A means of rendering on a display associated with the mobile device a view of the scene overlaying the game objects and according to the estimated pose of the mobile device.

16. A system according claim 15, wherein the identified features are interpreted as platforms.

17. A system according to claim 16, wherein the features are identified after the map of the scene has been created and according to one or more game rules.

18. A system according to claim 16, wherein the features are identified while the map of the scene is being created and according to one or more game rules and a consistency constraint with previously identified features.

19. A system according to claim 15, wherein the created map of the scene and the identified features on that map can be recorded such that they may be recovered at a later time making unnecessary to create a map of the scene and identify features on that map of the scene.

20. A system according to claim 15, wherein the map of the scene and the identified features are unrelated to the scene where the pose of the mobile device is being estimated.