Replacing 2D images with 3D images

• US 10,701,346 B2
• Filed: 12/20/2019
• Issued: 06/30/2020
• Est. Priority Date: 04/06/2018
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Chavez, David A., Hosenpud, Jonathan J., Champion, Clifford S., Lelievre, Alexandre R., Berman, Arthur L., Yamada, Kevin S.
• Primary Examiner(s): Bader, Robert
• Application Number:
  US16/722,185
• Publication Number:
  US 20200128227A1
• Time in Patent Office: 193 Days
• Field of Search: None
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

Systems and methods for replacing a 2D image with an equivalent 3D image within a web page. The 2D image displayed within a web page may be identified and a 3D image with substantially equivalent content may also be identified. The 3D image may be integrated into the web page as a replacement to the 2D image. Further, at least one user input manipulating the 3D image within the web page may be received. The at least one user input may include movement of a view point (or point of view) of a user relative to a display displaying the web page and/or detection of a beam projected from an end of a user input device (and/or an end of the user input device) intersecting with the 3D image.

Claims

1. A non-transitory computer readable memory medium storing programming instructions executable by processing circuitry of a user equipment device (UE) to:

display, to a user, a web page that includes at least a two-dimensional (2D) image;

identify a plurality of three-dimensional (3D) images with substantially equivalent content as compared to the content of a 2D image identified within the web page, wherein identifying the plurality of 3D images includes collecting information to assist in selection of a 3D image via questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image and evaluating an online history of the user;

select a 3D image based on the collected information; and

integrate the 3D image into the web page, thereby replacing the 2D image with the 3D image.

2. The non-transitory computer readable memory medium of claim 1,

wherein the programming instructions are further executable to:

receive at least one user input manipulating the 3D image within the web page, wherein the at least one user input comprises movement of a view point of the user, wherein the movement of the view point is determined based, at least in part, on tracking of a head or eyes of the user.

3. The non-transitory computer readable memory medium of claim 1,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

4. The non-transitory computer readable memory medium of claim 1,

wherein, to identify content of the 2D image, the programming instructions are further executable to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

5. The non-transitory computer readable memory medium of claim 1,

wherein, to identify the 3D image, the programming instructions are further executable to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet; and

wherein, to source the 3D image, the programming instructions are further executable to;

search metadata associated with 3D images located on the Internet;

search an index of content of 3D images located on the Internet.

6. The non-transitory computer readable memory medium of claim 1,

wherein, to integrate the 3D image into the web page, the program instructions are further executable to:

flatten the 3D image such that the 3D image appears as in two dimensions, wherein to flatten the 3D image, the program instructions are further executable to decrease a z-scale value associated with a depth of the 3D image.

7. The non-transitory computer readable memory medium of claim 6,

wherein the program instructions are further executable to:

receive at least one user input manipulating the 3D image within the web page; and

un-flatten the 3D image such that the 3D image appears in three dimensions by increasing a z-scale value associated with a depth of the 3D image.

8. The non-transitory computer readable memory medium of claim 1,

wherein, to integrate the 3D image into the web page, the program instructions are further executable to:

replace the 2D image with the 3D image;

determine that a real-world scale and/or size of the 3D image fits within the bounds of a web browser displaying the web page; and

display the 3D image at the real-world scale and/or size.

9. An apparatus, comprising:

at least one processor;

one or more displays, coupled to the at least one processor; and

a memory coupled to the at least one processor, wherein the at least one processor is configured to;

display, to a user, a web page that includes at least a two-dimensional (2D) image;

identify a plurality of three-dimensional (3D) images with substantially equivalent content as compared to the content of a 2D image identified within the web page, wherein identifying the plurality of 3D images includes collecting information to assist in selection of the 3D image via questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image and evaluating an online history of the user;

select a 3D image based on the collected information; and

integrate the 3D image into the web page, thereby replacing the 2D image with the 3D image.

10. The apparatus of claim 9,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

11. The apparatus of claim 9,

wherein, to identify content of the 2D image, the at least one processor is further configured to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

12. The apparatus of claim 9,

wherein, to identify the 3D image, the at least one processor is further configured to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet, wherein, to source the 3D image, the at least one processor is further configured to;

search metadata associated with 3D images located on the Internet; and

search an index of content of 3D images located on the Internet.

13. The apparatus of claim 9,

wherein, to identify the 3D image, the at least one processor is further configured to:

determine that the content of the 3D image is substantially equivalent to content of the 2D image.

14. The apparatus of claim 9,

wherein the at least one processor is further configured to:

receive a user input indicating that the user is no longer interacting with the 3D image within the web page; and

upon receiving the user input, flatten the 3D image such that the image appears in two dimensions, wherein the z-scale value associated with the depth of the 3D image is decreased to flatten the 3D image.

15. The apparatus of claim 9,

wherein the at least one processor is further configured to:

receive a user input indicating that the user is no longer interacting with the 3D image within the web page;

determine that a real-world scale and/or size of the 3D image fits within the bounds of a web browser displaying the web page; and

display the 3D image at the real-world scale and/or size when the 3D image is un-flattened.

16. A user equipment device (UE), comprising:

at least one antenna;

at least one radio in communication with the at least one antenna and configured to perform wireless communication using at least one radio access technology (RAT);

one or more processors coupled to the at least one radio; and

at least one display in communication with the one or more processors;

wherein the one or more processors are configured to cause the wireless device to;

display, to a user, a web page that includes at least a two-dimensional (2D) image;

identify a plurality of three-dimensional (3D) images with substantially equivalent content as compared to the content of a 2D image identified within the web page, wherein identifying the plurality of 3D images includes collecting information to assist in selection of the 3D image via questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image and evaluating an online history of the user;

select a 3D image based on the collected information; and

integrate the 3D image into the web page, thereby replacing the 2D image with the 3D image.

17. The UE of claim 16,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

18. The UE of claim 16,

wherein, to identify content of the 2D image, the one or more processors are further configured to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

19. The UE of claim 16,

wherein, to identify the 3D image, the one or more processors are further configured to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet, wherein, to source the 3D image, the at least one processor is further configured to;

search metadata associated with 3D images located on the Internet; and

search an index of content of 3D images located on the Internet.

20. The UE of claim 16,

wherein, to identify the 3D image, the one or more processors are further configured to:

determine that the content of the 3D image is substantially equivalent to content of the 2D image.