Measurement method

• US 8,854,282 B1
• Filed: 09/06/2011
• Issued: 10/07/2014
• Est. Priority Date: 09/06/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wong, Adrian
• Primary Examiner(s): Leiby, Christopher E
• Application Number:
  US13/225,903
• Time in Patent Office: 1,127 Days
• Field of Search: 345 8- 9
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP

Abstract

A system and method of determining regions of interest via collective gaze of multiple wearable head-mounted displays (HMDs) is disclosed. Each of a plurality of HMDs can be communicatively connected with a communication network, and can transmit respective location-orientation data to the communication network or a server or server system in the network. Location-orientation data from the plurality can be collected and analyzed by the server in order to determine spatial regions that are or appear to be focal points of collective or common gaze of two or more HMDs of the plurality. Information about those regions or about the analysis that identified them may thereafter be provided back to one or more HMDs or other wearable computing devices.

Claims

1. In a device, a computer-implemented method comprising:

receiving respective location-orientation data from each of a first plurality of wearable head-mounted displays (HMDs), the respective location-orientation data including information indicative of a respective actual location and a respective actual field-of-view (FOV) pointing direction of each of the first plurality of wearable HMDs;

making a first determination that the actual FOV pointing directions of the first plurality are within a first threshold distance of intersecting each other within a first region of actual space;

determining a first weighting value of the first region of actual space based on at least one of (i) a number of wearable HMDs in the first plurality, (ii) a size of a geographic region within which the wearable HMDs of the first plurality are located, (iii) a size of the first region of actual space, (iv) a size of the first threshold distance, or (v) a respective distance of the first region of actual space from each wearable HMD of the first plurality;

determining three-dimensional spatial coordinates of a first point indicative of a location of the first region of actual space, and assigning the first weighting value to the coordinates of the first point; and

transmitting information indicative of the first weighting value and the coordinates of the first point.

2. The method of claim 1, wherein the device comprises a server system.

3. The method of claim 1, wherein the device comprises a wearable HMD.

4. The method of claim 1, further comprising:

receiving respective location-orientation data from each of a second plurality of wearable HMDs, the respective location-orientation data including information indicative of a respective actual location and a respective actual FOV pointing direction of each of the second plurality of wearable HMDs;

making a second determination that the actual FOV pointing directions of the second plurality are within a second threshold distance of intersecting each other within a second region of actual space;

determining a second weighting value of the second region of actual space based on at least one of (i) a number of wearable HMDs in the second plurality, (ii) a size of a geographic region within which the wearable HMDs of the second plurality are located, (iii) a size of the second region of actual space, (iv) a size of the second threshold distance, or (iv) a respective distance of the second region of actual space from each wearable HMD of the second plurality; and

determining three-dimensional spatial coordinates of a second point indicative of a location of the second region of actual space, and assigning the second weighting value to the coordinates of the second point.

5. The method of claim 4, wherein the first point and the assigned first weighting value and the second point and the assigned second weighting value comprise two data points of a three-dimensional collective gaze map.

6. The method of claim 5, wherein transmitting information indicative of the first weighting value and the coordinates of the first point comprises transmitting data representative of the three-dimensional collective gaze map to at least one wearable HMD of the first plurality.

7. The method of claim 1, wherein the respective actual location of each of the first plurality of wearable HMDs comprises a geographic location of each of the first plurality of wearable HMDs measured at a respective time,

wherein the respective actual FOV pointing direction of each of the first plurality of wearable HMDs comprises an angular direction of a FOV axis of each of the first plurality of wearable HMDs measured at the respective time,

and wherein receiving the respective location-orientation data from each of the first plurality of wearable HMDs comprises receiving a respective data message transmitted from each respective wearable HMD of the first plurality.

8. The method of claim 7, wherein the geographic location of each of the first plurality of wearable HMDs comprises a Global Position Satellite System location measurement,

and wherein the angular direction of the FOV axis of each of the first plurality of wearable HMDs comprises data corresponding to both an altitude angle measured with respect to a horizontal plane, and an azimuth angle measured with respect to a geographic reference direction.

9. The method of claim 1, wherein making the first determination comprises determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality is within the first threshold distance of respectively intersecting each of the other actual FOV pointing directions of the first plurality.

10. The method of claim 1, wherein making the first determination comprises determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality traverses a path that passes within the first threshold distance of intersecting a common point in actual three-dimensional space,

and wherein determining the three-dimensional spatial coordinates of the first point indicative of the location of the first region of actual space comprises determining three-dimensional spatial coordinates of the common point.

11. The method of claim 1, wherein making the first determination comprises determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality intersects a common voxel in actual three-dimensional space.

12. The method of claim 1, wherein determining the first weighting value of the first region of actual space based on at least the number of wearable HMDs in the first plurality comprises determining the first weighting value as a value that increases as the number of wearable HMDs in the first plurality increases.

13. The method of claim 1, wherein determining the first weighting value of the first region of actual space based on at least the size of the first region of actual space comprises determining the first weighting value as a value that increases as the size of the first region of space decreases.

14. The method of claim 1, wherein determining the first weighting value of the first region of actual space further comprises determining a statistical likelihood of a random occurrence of the actual FOV pointing directions of the first plurality being within a first threshold distance of intersecting each other within the first region of actual space.

15. The method of claim 1, wherein the respective actual FOV pointing direction of at least one wearable HMD of the first plurality includes information indicative of eye pointing direction of a wearer of the at least one wearable HMD.

16. In a wearable head-mounted display (HMD), a computer-implemented method comprising:

determining an actual location and an actual field-of-view (FOV) pointing direction of the wearable HMD;

transmitting a data message to a device, the data message including information indicative of the determined actual location and the determined actual FOV pointing direction;

receiving from the device information indicative of an actual three-dimensional location of one or more actual regions within which respective actual FOV pointing directions of two or more wearable HMDs are within a threshold of intersecting each other; and

displaying information indicative of the location of at least one region of the one or more actual regions.

17. The method of claim 16, wherein determining the actual location and the actual FOV pointing direction of the wearable HMD comprises:

determining a geographic location of the wearable HMD from a Global Position Satellite System location measurement; and

determining an angular direction of a FOV axis corresponding to both an altitude angle measured with respect to a horizontal plane, and an azimuth angle measured with respect to a geographic reference direction.

18. The method of claim 17, wherein determining the actual FOV pointing direction of the wearable HMD further comprises determining an eye pointing direction of a wearer of the wearable HMD.

19. The method of claim 16, wherein transmitting the data message to the device comprises transmitting the data message via a wireless interface to a wireless access network.

20. The method of claim 16, wherein receiving from the device information indicative of the actual three-dimensional location of the one or more actual regions comprises receiving from the device data representative of a three-dimensional collective gaze map of a plurality of wearable HMDs.

21. The method of claim 16, wherein displaying the information indicative of the location of the at least one region comprises displaying a visual indicator in the display at one of two-dimensional display coordinates corresponding to the location as projected onto the display, and three-dimensional coordinates of virtual three-dimensional space.

22. The method of claim 21, wherein displaying the information indicative of the location of the at least one region further comprises displaying information indicative of a physical object located within the at least one region.

23. A device comprising:

one or more processors;

memory; and

machine-readable instructions stored in the memory that, when executed by the one or more processors, cause the device to carry out functions including;

receiving respective location-orientation data from each of a first plurality of wearable head-mounted displays (HMDs), the respective location-orientation data including information indicative of a respective actual location and a respective actual field-of-view (FOV) pointing direction of each of the first plurality of wearable HMDs,

making a first determination that the actual FOV pointing directions of the first plurality are within a first threshold distance of intersecting each other within a first region of actual space,

determining a first weighting value of the first region of actual space based on at least one of (i) a number of wearable HMDs in the first plurality, (ii) a size of a geographic region within which the wearable HMDs of the first plurality are located, (iii) a size of the first region of actual space, (iv) a size of the first threshold distance, or (v) a respective distance of the first region of actual space from each wearable HMD of the first plurality,

determining three-dimensional spatial coordinates of a first point indicative of a location of the first region of actual space, and assigning the first weighting value to the coordinates of the first point, and

transmitting information indicative of the first weighting value and the coordinates of the first point.

24. The device of claim 23, wherein, the machine-readable instructions, when executed by the one or more processors, cause the device to carry out functions further including:

receiving respective location-orientation data from each of a second plurality of wearable HMDs, the respective location-orientation data including information indicative of a respective actual location and a respective actual FOV pointing direction of each of the second plurality of wearable HMDs,

making a second determination that the actual FOV pointing directions of the second plurality are within a second threshold distance of intersecting each other within a second region of space,

determining a second weighting value of the second region of actual space based on at least one of (i) a number of wearable HMDs in the second plurality, (ii) a size of a geographic region within which the wearable HMDs of the second plurality are located, (iii) a size of the second region of actual space, (iv) a size of the second threshold distance, or (iv) a respective distance of the second region of actual space from each wearable HMD of the second plurality, and

determining three-dimensional spatial coordinates of a second point indicative of a location of the second region of actual space, and assigning the second weighting value to the coordinates of the second point.

25. The device of claim 23, wherein causing the device to carry out the function of making a first determination comprises causing the device to carry out at least one of the functions of:

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality is within the first threshold distance of respectively intersecting each of the other actual FOV pointing directions of the first plurality,

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality traverses a path that passes within the first threshold distance of intersecting a common point in actual three-dimensional space, or

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality intersects a common voxel in actual three-dimensional space.

26. The device of claim 23, wherein causing the device to carry out the function of determining the first weighting value of the first region of actual space based on at least one of (i) the number of wearable HMDs in the first plurality, (ii) the size of the geographic region within which the wearable HMDs of the first plurality are located, (iii) the size of the first region of actual space, (iv) the size of the first threshold distance, or (v) the respective distance of the first region of actual space from each wearable HMD of the first plurality comprises causing the device to carry out at least one of the functions of:

determining the first weighting value based on the number of wearable HMDs in the first plurality by determining the first weighting value as a value that increases as the number of wearable HMDs in the first plurality increases, or

determining the first weighting value based on the size of the first region of actual space by determining the first weighting value as a value that increases as the size of the first region of space decreases.

27. The device of claim 23, wherein the device is configured as part of a wearable computing device.

28. A wearable head-mounted display (HMD) comprising:

a display device;

one or more processors;

memory; and

machine-readable instructions stored in the memory that, when executed by the one or more processors, cause the wearable HMD to carry out functions including;

determining an actual location and an actual field-of-view (FOV) pointing direction of the wearable HMD,

transmitting a data message including information indicative of the determined actual location and the determined actual FOV pointing direction,

receiving information indicative of an actual three-dimensional location of one or more regions within which respective actual FOV pointing directions of two or more wearable HMDs are within a threshold of intersecting each other, and

displaying information indicative of the location of at least one region of the one or more actual regions.

29. The wearable HMD of claim 28, wherein causing the HMD to carry out the function of determining the actual location and the actual FOV pointing direction of the wearable HMD comprises causing the wearable HMD to carry out functions including:

determining a geographic location of the wearable HMD from a Global Position Satellite System location measurement, and

determining an angular direction of a FOV axis corresponding to both an altitude angle measured with respect to a horizontal plane, and an azimuth angle measured with respect to a geographic reference direction.

30. The wearable HMD of claim 28, wherein causing the HMD to carry out the function of determining the actual location and the actual FOV pointing direction of the wearable HMD comprises causing the wearable HMD to carry out the functions of:

determining an eye pointing direction of a wearer of the wearable HMD.

31. The wearable HMD of claim 28, wherein causing the HMD to carry out the function of displaying information indicative of the location of the at least one region of the one or more actual regions comprises causing the wearable HMD to carry out the function of:

displaying a visual indicator in the display device at one of two-dimensional display coordinates corresponding to the location as projected onto the display, and three-dimensional coordinates of virtual three-dimensional space.

32. A non-transitory computer-readable medium having instructions stored thereon that, upon execution by one or more processors of a device, cause the device to carry out functions comprising:

receiving respective location-orientation data from each of a first plurality of wearable head-mounted displays (HMDs), the respective location-orientation data including information indicative of a respective actual location and a respective actual field-of-view (FOV) pointing direction of each of the first plurality of wearable HMDs;

making a first determination that the actual FOV pointing directions of the first plurality are within a first threshold distance of intersecting each other within a first region of actual space;

determining a first weighting value of the first region of actual space based on at least one of (i) a number of wearable HMDs in the first plurality, (ii) a size of a geographic region within which the wearable HMDs of the first plurality are located, (iii) a size of the first region of actual space, (iv) a size of the first threshold distance, or (v) a respective distance of the first region of actual space from each wearable HMD of the first plurality;

determining three-dimensional spatial coordinates of a first point indicative of a location of the first region of actual space, and assigning the first weighting value to the coordinates of the first point; and

transmitting information indicative of the first weighting value and the coordinates of the first point.

33. The non-transitory computer-readable medium of claim 32, wherein, the instructions, upon execution by the one or more processors, cause the device to carry out functions further comprising:

receiving respective location-orientation data from each of a second plurality of wearable HMDs, the respective location-orientation data including information indicative of a respective actual location and a respective actual FOV pointing direction of each of the second plurality of wearable HMDs;

making a second determination that the actual FOV pointing directions of the second plurality are within a second threshold distance of intersecting each other within a second region of actual space;

determining a second weighting value of the second region of actual space based on at least one of (i) a number of wearable HMDs in the second plurality, (ii) a size of a geographic region within which the wearable HMDs of the second plurality are located, (iii) a size of the second region of actual space, (iv) a size of the second threshold distance, or (iv) a respective distance of the second region of actual space from each wearable HMD of the second plurality; and

determining three-dimensional spatial coordinates of a second point indicative of a location of the second region of actual space, and assigning the second weighting value to the coordinates of the second point.

34. The non-transitory computer-readable medium of claim 32, wherein making the first determination consists of at least one of:

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality is within the first threshold distance of respectively intersecting each of the other actual FOV pointing directions of the first plurality,

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality traverses a path that passes within the first threshold distance of intersecting a common point in actual three-dimensional space, or

determining that each respective actual FOV pointing direction of each wearable HMD of the first plurality intersects a common voxel in actual three-dimensional space.

35. The non-transitory computer-readable medium of claim 32, wherein determining the first weighting value of the first region of actual space based on at least one of (i) the number of wearable HMDs in the first plurality, (ii) the size of the geographic region within which the wearable HMDs of the first plurality are located, (iii) the size of the first region of actual space, (iv) the size of the first threshold distance, or (v) the respective distance of the first region of actual space from each wearable HMD of the first plurality consists of at least one of:

determining the first weighting value based on the number of wearable HMDs in the first plurality by determining the first weighting value as a value that increases as the number of wearable HMDs in the first plurality increases, or

determining the first weighting value based on the size of the first region of actual space by determining the first weighting value as a value that increases as the size of the first region of space decreases.

36. A non-transitory computer-readable medium having instructions stored thereon that, upon execution by one or more processors of a wearable head-mounted display (HMD), cause the wearable HMD to carry out functions comprising:

determining an actual location and an actual field-of-view (FOV) pointing direction of the wearable HMD;

transmitting a data message including information indicative of the determined actual location and the determined actual FOV pointing direction;

receiving information indicative of an actual three-dimensional location of one or more actual regions within which respective actual FOV pointing directions of two or more wearable HMDs are within a threshold of intersecting each other; and

displaying information indicative of the location of at least one region of the one or more actual regions.

37. The non-transitory computer-readable medium of claim 36, wherein determining the actual location and the actual FOV pointing direction of the wearable HMD comprises:

determining a geographic location of the wearable HMD from a Global Position Satellite System location measurement; and

determining an angular direction of a FOV axis corresponding to both an altitude angle measured with respect to a horizontal plane, and an azimuth angle measured with respect to a geographic reference direction.

38. The non-transitory computer-readable medium of claim 36, wherein determining the actual location and the actual FOV pointing direction of the wearable HMD comprises determining an eye pointing direction of a wearer of the wearable HMD.

39. The non-transitory computer-readable medium of claim 36, wherein displaying information indicative of the location of the at least one region of the one or more actual regions comprises displaying a visual indicator in the display at one of two-dimensional display coordinates corresponding to the location as projected onto the display, and three-dimensional coordinates of virtual three-dimensional space.