Object control device, object control method, computer-readable recording medium, and integrated circuit

• US 8,881,059 B2
• Filed: 02/09/2012
• Issued: 11/04/2014
• Est. Priority Date: 03/15/2011
• Status: Expired due to Fees

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Adachi, Takao
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): Lu, Hua
• Application Number:
  US13/640,176
• Publication Number:
  US 20130031511A1
• Time in Patent Office: 999 Days
• Field of Search: None
• US Class Current: 715/825
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A virtual object display determination unit identifies from real object display determination information a priority corresponding to a movement of a user indicated by user movement information notified by a state communication unit and, at the same time, identifies from real object attribute information a priority corresponding to a state change indicated by state change information notified by the state communication unit. By comparing the two identified priorities, the virtual object display determination unit determines whether or not to change a display mode of a virtual object. A UI generation unit generates a UI to be presented to the user based on a determination result of the virtual object display determination unit, and causes the UI to be displayed by a UI display unit.

Claims

1. An object control processing device that controls display of a virtual object in a virtual reality space,

the processing device comprising:

a user movement detection circuit that detects a movement of a user;

a user movement notification circuit that notifies user movement information indicating a user movement detected by the user movement detection circuit;

a real object state detection circuit that detects a change in a state of a real object existing around the user in the virtual reality space;

a real object state notification circuit that notifies state change information indicating a change in the state of the real object detected by the real object state detection circuit;

a state communication circuit that notifies user movement information notified by the user movement notification circuit and state change information notified by the real object state notification circuit;

a virtual object information management circuit that manages virtual object management information including a position of the virtual object in the virtual reality space;

a virtual object display determination circuit that extracts virtual objects positioned within a certain region in front of the user as display virtual objects from the virtual objects managed by the virtual object information management circuit, extracts from the extracted virtual objects a virtual object existing at a position where a field of the user is blocked with respect to the real object whose state has changed, and determines whether or not to change a display mode of the extracted virtual object so as to expose the real object whose state has changed based on the user movement information and the state change information notified by the state communication circuit;

a UI generation circuit that generates a UI (user interface) to be presented to the user based on a determination result by the virtual object display determination circuit; and

a UI display circuit that displays the UI generated by the UI generation circuit, wherein

the UI generation circuit generates, when the virtual object display determination circuit determines to change the display mode of the extracted virtual object, the UI so as to change only the display mode of the determined virtual object among the display virtual objects and to maintain display modes of remaining virtual objects, and

the UI display circuit superimposes the image around the user and the generated UI, and generates an image of the virtual reality space.

2. The object control device according to claim 1, further comprising:

a real object information management circuit which manages real object display determination information in which a user movement and a priority are associated with each other and real object attribute information in which a state change of the real object and a priority are associated with each other, wherein

the virtual object display determination circuit identifies from the real object display determination information a priority corresponding to a user movement indicated by the user movement information notified by the state communication circuit and identifies from the real object attribute information a priority corresponding to a state change indicated by the state change information notified by the state communication circuit, and by comparing the two identified priorities, determines whether or not to change a display mode of the virtual object.

3. The object control device according to claim 2, wherein

with respect to the priorities included in the real object display determination information and the real object attribute information, the greater the value, the higher the priority, and

when the priority identified from the real object display determination information is equal to or higher than the priority identified from the real object attribute information, the virtual object display determination circuit makes a determination to change the display mode of the virtual object.

4. The object control device according to claim 2, wherein when the user movement information notified by the state communication circuit is not included in the real object display determination information, the virtual object display determination circuit approximates the user movement indicated by the user movement information to user movement information included in the real object display determination information and determines whether or not to change the display mode of the virtual object.

5. The object control device according to claim 1, wherein the virtual object display determination circuit sets a region enclosed by a real object whose state has changed and a current position of the user as a field-of-view region, and changes the display mode of the virtual object at least partially included in the field-of-view region.

6. The object control device according to claim 1, wherein the virtual object display determination circuit sets a region which, in a top view, has an arc that is set on a side of a reached-out hand of a user and which has a fan-shape with a certain radius from a real object whose state has changed as an expanded field-of-view region, and changes a display mode of a virtual object included in at least one of the expanded field-of-view region and the field-of-view region.

7. The object control device according to claim 6, wherein the expanded field-of-view region is set such that, in a top view, one side of the fan shape is set from a center of a real object whose state has changed toward the user, and a central angle of the fan shape decreases as an angle formed between a straight line connecting the user and the center of the real object and an orientation of a reached-out hand of the user increases.

8. The object control device according to claim 1,

wherein

the virtual object display determination circuit identifies a position of the virtual object using the virtual object management information.

9. The object control device according to claim 1, further comprising

a user information management circuit that manages user position information including a current position of a user, wherein

the virtual object display determination circuit identifies a current position of the user using the user position information.

10. The object control device according to claim 1, wherein when the display mode of the virtual object is changed and then a predetermined gesture to return the display mode to an original mode is made by the user, the virtual object display determination circuit restores the display mode to the original mode.

11. The object control device according to claim 1, wherein the display mode change involves hiding the virtual object.

12. The object control device according to claim 1, wherein the display mode change involves relocating a display position of the virtual object.

13. An object control method used by an object control device that controls display of a virtual object in a virtual reality space,

the object control method comprising:

a user movement detection step of detecting a movement of a user by a user movement detection unit;

a user movement notification step of notifying by a user movement notification unit user movement information indicating a user movement detected by the user movement detection unit;

a real object state detection step of detecting by a real object state detection unit a change in a state of a real object existing around the user in the virtual reality space;

a real object state notification step of notifying by a real object state notification unit state change information indicating a change in the state of the real object detected by the real object state detection unit;

a state communication step of notifying by a state communication unit user movement information notified by the user movement notification unit and state change information notified by the real object state notification unit;

a virtual object display determination step of extracting virtual objects positioned within a certain region in front of the user as display virtual objects from the virtual objects managed by the virtual object information management unit, extracts from the extracted virtual objects a virtual object existing at a position where a field of the user is blocked with respect to the real object whose state has changed, and determining by a virtual object display determination unit whether or not to change a display mode of the virtual object so as to expose the extracted real object whose state has changed based on the user movement information and the state change information notified by the state communication unit;

a UI generation step of generating by a UI generation unit a UI (user interface) to be presented to the user based on a determination result by the virtual object display determination unit; and

a UI display step of displaying by a UI display unit the UI generated by the UI generation unit, wherein

the UI generation step generates, when the virtual object display determination unit determines to change the display mode of the extracted virtual object, the UI so as to change only the display mode of the determined virtual object among the display virtual objects and to maintain display modes of remaining virtual objects, and

the UI display step superimposes the image around the user and the generated UI, and generates an image of the virtual reality space.

14. A non-transitory computer-readable recording medium which stores an object control program which causes a computer to operate as an object control device that controls display of a virtual object in a virtual reality space, the object control program causing the computer to function as:

a user movement notification unit that notifies user movement information indicating a movement of a user detected by a user movement detection unit;

a real object state notification unit that notifies state change information indicating a change in a state of a real object existing in a periphery of the user in the virtual reality space which is detected by a real object state detection unit;

a state communication unit that notifies user movement information notified by the user movement notification unit and state change information notified by the real object state notification unit;

a virtual object information management unit that manages virtual object management information including a position of the virtual object in the virtual reality space;

a virtual object display determination unit that extracts virtual objects positioned within a certain region in front of the user as display virtual objects from the virtual objects managed by the virtual object information management unit, extracts from the extracted virtual objects a virtual object existing at a position where a field of the user is blocked with respect to the real object whose state has changed, and determines whether or not to change a display mode of the extracted virtual object so as to expose the real object whose state has changed based on the user movement information and the state change information notified by the state communication unit; and

a UI generation unit which generates a UI (user interface) to be presented to the user based on a determination result by the virtual object display determination unit and which causes the UI to be displayed by a UI display unit, wherein

the UI generation unit generates, when the virtual object display determination unit determines to change the display mode of the extracted virtual object, the UI so as to change only the display mode of the determined virtual object among the display virtual objects and to maintain display modes of remaining virtual objects.

15. An integrated circuit for an object control device that controls display of a virtual object in a virtual reality space,

the integrated circuit comprising:

a processor;

a user movement notification unit that notifies user movement information indicating a movement of a user detected by a user movement detection unit;

a real object state notification unit that notifies state change information indicating a change in a state of a real object existing in the virtual reality space which is detected by a real object state detection unit and positioned in a periphery of the user in the virtual reality space;

a state communication unit that notifies user movement information notified by the user movement notification unit and state change information;

a virtual object information management unit that manages virtual object management information including a position of the virtual object in the virtual reality space;

a virtual object display determination unit that extracts virtual objects positioned within a certain region in front of the user as display virtual objects from the virtual objects managed by the virtual object information management unit, extracts from the extracted virtual objects a virtual object existing at a position where a field of the user is blocked with respect to the real object whose state has changed, and determines whether or not to change a display mode of the virtual object so as to expose the real object whose state has changed based on the user movement information and the state change information notified by the state communication unit; and

a UI generation unit which generates a UI (user interface) to be presented to the user based on a determination result by the virtual object display determination unit and which causes the UI to be displayed by a UI display unit, wherein

the UI generation unit generates, when the virtual object display determination unit determines to change the display mode of the extracted virtual object, the UI so as to change only the display mode of the determined virtual object among the display virtual objects and to maintain display modes of remaining virtual objects.