Audio-visual navigation and communication dynamic memory architectures

• US 10,146,399 B2
• Filed: 07/01/2016
• Issued: 12/04/2018
• Est. Priority Date: 09/26/2007
• Status: Active Grant

• Current Assignee: AQ Media, Inc.
• Original Assignee: AQ Media, Inc.
• Inventors: Roos, Jan Peter
• Primary Examiner(s): Nabi, Reza
• Assistant Examiner(s): Rodriguez, Daniel
• Application Number:
  US15/200,877
• Publication Number:
  US 20160313892A1
• Time in Patent Office: 886 Days
• Field of Search: 715851
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/162 Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising
• CPC Class Codes: G06Q 30/08 Auctions
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

According to one embodiment, a plurality of spatial publishing objects (SPOs) is provided in a multidimensional space in a user interface. Each of the plurality of spatial publishing objects is associated with digital media data from at least one digital media source. The user interface has a field for the digital media data. A user is provided via the user interface with a user presence that is optionally capable of being represented in the user interface relative to the plurality of spatial publishing objects. The digital media data associated with the at least one spatial publishing object are combined to generate a media output corresponding to the combined digital media data.

Claims

1. A computer-implemented method comprising the steps of:

providing, using a user interface module, a three-dimensional user interface on a display of a client computing device that is in communication with a remote server computing device, the three-dimensional user interface comprising a first spatial publishing object associated with a user presence and a user point-of-view of a user, the user point-of-view separable from the user presence, and the first spatial publishing object comprising an attribute that is searchable, viewable, and listenable, the first spatial publishing object representing a user account created from an internet form-based account sign-up and authentication selection, the first spatial publishing object comprising an interior communication space defined by three dimensions and an exterior surface capable of displaying live or on-demand media;

navigating through other spatial publishing objects represented in a three dimensional space via the first spatial publishing object that is associated with the user presence and the user point-of-view of the user, wherein the other spatial publishing objects have a plurality of properties and states including auditory and visual fields associated with at least one live or on-demand media source;

selecting and saving one or more of the other spatial publishing objects to the user account to create a grouping; and

creating a three dimensional connected communication grouping between the first spatial publishing object and the one or more of the other spatial publishing objects that have been saved to the user account as the grouping.

2. The computer-implemented method of claim 1, wherein user inputs manipulate a property of a second spatial publishing object of the other spatial publishing objects, the user inputs comprising one or more of a three-dimensional input, a voice input, an image input, a proximity input, a touch input, a positional selection, and a remote control input.

3. The computer-implemented method of claim 1, wherein an appearance of the first spatial publishing object or the other spatial publishing objects including a scale, a position, and a rotation can be modified by the user and can take such shapes as furniture, architecture, video screens, billboards, buttons, menus, geometric forms, or anthropomorphic forms.

4. The computer-implemented method of claim 1, wherein location coordinates can be associated with the first spatial publishing object or the other spatial publishing objects to determine then spatial position.

5. The computer-implemented method of claim 1, wherein the user opens a two-way audiovisual communication channel connecting the first spatial publishing object to a second spatial publishing object thereby adding the second spatial publishing object to the three-dimensional connected communication grouping.

6. The computer-implemented method of claim 5, wherein the first spatial publishing object is connected to the second spatial publishing object by exceeding a preset time threshold for maintaining proximity with the second spatial publishing object.

7. The computer-implemented method of claim 1, wherein the first spatial publishing object further comprises an incrementing counter to represent the number of connections established between the first spatial publishing object and the other spatial publishing objects.

8. The computer-implemented method of claim 7, further comprising comparing the incrementing counter with adjustable preset connect count thresholds to instantiate a real-time change of display properties of the first spatial publishing object or the other spatial publishing objects or to generate display-alerting events when any of the adjustable preset connect count thresholds are exceeded.

9. The computer-implemented method of claim 8, wherein a display property of the first spatial publishing object or the other spatial publishing objects comprises a scale, and wherein the scale increases to an adjustable preset scale when an adjustable preset connect count threshold is exceeded.

10. The computer-implemented method of claim 8, wherein the display-altering event provides a real-time inclusion of an advertisement spatial publishing object, published by an advertiser who pre-bids payments on the adjustable preset connect count thresholds related to specific publishing categories and viewer demographics, within the three-dimensional user interface.

11. The computer-implemented method of claim 10, wherein the advertisement spatial publishing object is inserted in a space surrounding a spatial publishing object whose connect count has exceeded an adjustable preset connect count threshold.

12. The computer-implemented method of claim 11, wherein the advertisement spatial publishing object has a spatial layout within an advertisement gauntlet and wherein users navigate through the advertisement spatial publishing object on their way in to connect to or view the spatial publishing object whose connect count has exceeded an adjustable preset connect count threshold.

13. The computer-implemented method of claim 11, wherein the advertisement spatial publishing object is placed within a communication space of the first spatial publishing object through insertion within audio or video, or as signs and banners on a surface of a three-dimensional interior of the communication space or on an exterior of the spatial publishing object.

14. The computer-implemented method of claim 1, further comprising making a selection to enter a communication space associated with the user presence or one of the other spatial publishing objects.

15. The computer-implemented method of claim 1, wherein users, after entering a communication space, can fill in pre-built three dimensional templates with the other spatial publishing objects selected by the user.

16. The computer-implemented method of claim 1, wherein the three-dimensional user interface presents a user-selectable full screen live video output from any of the first spatial publishing object or the other spatial publishing objects.

17. The computer-implemented method of claim 1, wherein the three-dimensional user interface presents an audio mix of the first spatial publishing object and one of the other spatial publishing objects currently connected to the first spatial publishing object from the communication space, and includes an audio output of a second spatial publishing object within whose audio field the first spatial publishing object is positioned.

18. The computer-implemented method of claim 1, wherein the three-dimensional user interface presents a user-selectable first person view of an animated navigation of the first spatial publishing object within the three-dimensional space, which can include a live or on-demand video output from any of the first spatial publishing object or the other spatial publishing objects for display surrounding the communication space.

19. The computer-implemented method of claim 1, wherein the three-dimensional user interface presents a third person overview of animation and a live or on-demand video output of the first spatial publishing object and a second spatial publishing object within the three dimensional space, which can include the live or on-demand video output from any of the first spatial publishing object or the other spatial publishing objects for display surrounding the communication space.

20. The computer-implemented method of claim 19, wherein

the third person overview requires a sharing permission enabled from the user account or a second user account, and wherein

when the sharing permission is enabled, then the third person overview becomes shared as a public communication space for view from user accounts outside the user account or the second user account.

21. The computer-implemented method of claim 20, wherein

the user account or the second user account, by enabling a mixer panel in the three-dimensional user interface to assist the user in controlling audio outputs from spatial publishing objects, can create an overall combined mix in a variety of formats available for output, including but not limited to mono, stereo, and surround, which can then be audible when the third person overview becomes shared as the public communication space for listening to from user accounts outside the first user account and the second user account.

22. The computer-implemented method of claim 1, wherein

the first spatial publishing object displays different camera angles of a scene or different views through the one or more live inputs, and

one or more of the other spatial publishing objects in the three dimensional space are connected to each other and to the first spatial publishing object in terms of live audiovisual communication.

23. A non-transitory computer-readable medium comprising computer-readable instructions to be executed by a processor, the instructions comprising steps of:

providing, using a user interface software module, a three-dimensional user interface on a display of a client computing device that is in communication with a remote server computing device, the three-dimensional user interface comprising a first spatial publishing object associated with a user presence and a user point-of-view of a user, the user point-of-view separable from the user presence, and the first spatial publishing object comprising an attribute that is searchable, viewable, and listenable, the first spatial publishing object representing a user account created from an internet form-based account sign-up and authentication selection, the first spatial publishing object comprising an interior communication space defined by three dimensions and an exterior surface capable of displaying live or on-demand media;

navigating through other spatial publishing objects represented in a three dimensional space via the first spatial publishing object that is associated with the user presence and the user point-of-view of the user, wherein the other spatial publishing objects have a plurality of properties and states including auditory and visual fields associated with at least one live or on-demand media source;

selecting and saving one or more of the other spatial publishing objects to the user account to create a grouping; and

creating a three dimensional connected communication grouping between the first spatial publishing object and the one or more of the other spatial publishing objects that have been saved to the user account as the grouping.

24. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein user inputs manipulate a property of a second spatial publishing object of the other spatial publishing objects, the user inputs comprising one or more of a three-dimensional gestural input, a voice input, an image input, a proximity input, a touch input, a positional selection, and a remote control input.

25. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein an appearance of the first spatial publishing object or the other spatial publishing objects including a scale, a position, and a rotation can be modified by the user and can take such shapes as furniture, architecture, video screens, billboards, buttons, menus, geometric forms, or anthropomorphic forms.

26. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein location coordinates can be associated with the first spatial publishing object or the other spatial publishing objects to determine their spatial position.

27. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein the user opens a two-way audiovisual communication channel connecting the first spatial publishing object to a second spatial publishing object thereby adding the second spatial publishing object in the three dimensional connected communication grouping.

28. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 27, wherein the first spatial publishing object is connected to the second spatial publishing object by exceeding a preset time threshold for maintaining proximity with the second spatial publishing object.

29. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein the first spatial publishing object further comprises an incrementing counter to represent the number of connections established between the first spatial publishing object and the other spatial publishing objects.

30. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 29, further comprising comparing the incrementing counter with adjustable preset connect count thresholds to instantiate a real-time change of display properties of the first spatial publishing object or the other spatial publishing objects or to generate display-alerting events when any of the adjustable preset connect count thresholds are exceeded.

31. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 30, wherein a display property of the first spatial publishing object or the other spatial publishing objects comprises a scale, and wherein the scale increases to an adjustable preset scale when an adjustable preset connect count threshold is exceeded.

32. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 30, wherein the display-altering event provides a real-time inclusion of an advertisement spatial publishing object, published by an advertiser who pre-bids payments on the adjustable preset connect count thresholds related to specific publishing categories and viewer demographics, within the three-dimensional user interface.

33. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 32, wherein the advertisement spatial publishing object is inserted in a space surrounding a spatial publishing object whose connect count has exceeded an adjustable preset connect count threshold.

34. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 33, wherein the advertisement spatial publishing object has a spatial layout within an advertisement gauntlet and wherein users navigate through the advertisement spatial publishing object on their way in to connect to or view the spatial publishing object whose connect count has exceeded an adjustable preset connect count threshold.

35. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 33, wherein the advertisement spatial publishing object is placed within a communication space of the first spatial publishing object through insertion within audio or video, or as signs and banners on a surface of a three-dimensional interior of the communication space or on an exterior of the spatial publishing object.

36. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, further comprising making a selection to enter a communication space associated with the user presence or one of the other spatial publishing objects.

37. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein users, after entering a communication space, can fill in pre-built three dimensional templates with the other spatial publishing objects selected by the user.

38. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein the three-dimensional user interface presents a user-selectable full screen live video output from any of the first spatial publishing object or the other spatial publishing objects.

39. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein the three-dimensional user interface presents an audio mix of the first spatial publishing object and one of the other spatial publishing objects currently connected to the first spatial publishing object from the communication space, and includes an audio output of a second spatial publishing object within whose audio field the first spatial publishing object is positioned.

40. The non-transitory computer-readable medium comprises the computer-readable instructions of claim 23, wherein the three-dimensional user interface presents a user-selectable first person view of an animated navigation of the first spatial publishing object within the three-dimensional space, which can include a live or on-demand video output from any of the first spatial publishing object or the other spatial publishing objects for display surrounding the communication space.

41. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein the three-dimensional user interface presents a third person overview of animation and a live or on-demand video output of the first spatial publishing object and a second spatial publishing object within the three dimensional space, which can include the live or on-demand video output from any of the first spatial publishing object or the other spatial publishing objects for display surrounding the communication space.

42. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 41, wherein

the third person overview requires a sharing permission enabled from the user account or a second user account, and wherein

when the sharing permission is enabled, then the third person overview becomes shared as a public communication space for view from user accounts outside the user account or the second user account.

43. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 42, wherein

the user account or the second user account, by enabling a mixer panel in the three-dimensional user interface to assist the user in controlling audio outputs from spatial publishing objects, can create an overall combined mix in a variety of formats available for output, including but not limited to mono, stereo, and surround, which can then be audible when the third person overview becomes shared as the public communication space for listening to from user accounts outside the first user account and the second user account.

44. The non-transitory computer-readable medium comprising the computer-readable instructions of claim 23, wherein

the first spatial publishing object displays different camera angles of a scene or different views through the one or more live inputs, and

one or more of the other spatial publishing objects in the three dimensional space are connected to each other and to the first spatial publishing object in terms of live audiovisual communication.