Replacing 2D images with 3D images

• US 10,523,921 B2
• Filed: 04/06/2018
• Issued: 12/31/2019
• Est. Priority Date: 04/06/2018
• Status: Active Grant

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Chavez, David A., Hosenpud, Jonathan J., Champion, Clifford S., Lelievre, Alexandre R., Berman, Arthur L., Yamada, Kevin S.
• Primary Examiner(s): Bader, Robert
• Application Number:
  US15/947,140
• Publication Number:
  US 20190313083A1
• Time in Patent Office: 634 Days
• Field of Search: None
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/366 using viewer tracking

Abstract

Systems and methods for replacing a 2D image with an equivalent 3D image within a web page. The 2D image displayed within a web page may be identified and a 3D image with substantially equivalent content may also be identified. The 3D image may be integrated into the web page as a replacement to the 2D image. Further, at least one user input manipulating the 3D image within the web page may be received. The at least one user input may include movement of a view point (or point of view) of a user relative to a display displaying the web page and/or detection of a beam projected from an end of a user input device (and/or an end of the user input device) intersecting with the 3D image.

Claims

1. A non-transitory computer readable memory medium storing programming instructions executable by processing circuitry of a wireless device to:

identify content of a two-dimensional (2D) image displayed within a web page, wherein the web page is displayed to a user;

identify a plurality of three-dimensional (3D) images with substantially equivalent content as compared to the content of the 2D image, wherein identifying the plurality of 3D images includes

an interrogation of the user of the web page for information to assist in selection of the 3D image, including questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image; and

an evaluation of an online history of the user;

select a 3D image based on the interrogation of the user;

integrate the 3D image into the web page, thereby replacing the 2D image with the 3D image; and

receive at least one user input manipulating the 3D image within the web page, wherein the at least one user input comprises movement of a view point of the user, wherein the movement of the view point is determined based, at least in part, on tracking of a head or eyes of the user.

2. The non-transitory computer readable memory medium of claim 1,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

3. The non-transitory computer readable memory medium of claim 1,

wherein, to identify content of the 2D image, the programming instructions are further executable to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

4. The non-transitory computer readable memory medium of claim 1,

wherein, to identify the 3D image, the programming instructions are further executable to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet.

5. The non-transitory computer readable memory medium of claim 4,

wherein, to source the 3D image, the programming instructions are further executable to:

search metadata associated with 3D images located on the Internet;

search an index of content of 3D images located on the Internet.

6. The non-transitory computer readable memory medium of claim 1,

wherein, to identify the 3D image, the programming instructions are further executable to:

determine that the content of the 3D image is substantially equivalent to content of the 2D image.

7. The non-transitory computer readable memory medium of claim 1,

wherein, to integrate the 3D image into the web page, the program instructions are further executable to:

flatten the 3D image such that the 3D image appears as in two dimensions.

8. The non-transitory computer readable memory medium of claim 7,

wherein to flatten the 3D image, the program instructions are further executable to decrease a z-scale value associated with a depth of the 3D image.

9. The non-transitory computer readable memory medium of claim 7,

wherein, upon receiving the at least one user input manipulating the 3D image within the web page, the program instructions are further executable to:

un-flatten the 3D image such that the 3D image appears in three dimensions.

10. The non-transitory computer readable memory medium of claim 9,

wherein to un-flatten 3D image, the program instructions are further executable to increase a z-scale value associated with a depth of the 3D image.

11. The non-transitory computer readable memory medium of claim 8,

wherein, upon flattening the 3D image, scale of the 3D image is adjusted such that a size of the 3D image corresponds to a size of the 2D image.

12. The non-transitory computer readable memory medium of claim 1,

wherein, to integrate the 3D image into the web page, the program instructions are further executable to:

replace the 2D image with the 3D image; and

display the object represented by the 3D image at a real-world size of the object when the real-world size of the object represented by the 3D image fits within the bounds of a web browser displaying the web page.

13. The non-transitory computer readable memory medium of claim 1,

wherein, to select the 3D image based on the interrogation of the user, the program instructions are further executable to:

rank the plurality of 3D images based on one or more sets of ranking criteria, wherein a first set of ranking criteria includes a number of times a 3D image was previously selected as a replacement 3D image, a number of endorsements that a 3D image received from users that have previously searched for this type of 3D image, and a fee paid by 3D content creators to have 3D imagery ranked higher than lesser paying or non-paying 3D content creators, and wherein a second set of ranking criteria includes a level of detail in a 3D image and a size of the 3D image; and

select a 3D image with a highest-ranking value from the plurality of 3D images ranked.

14. A three-dimensional (3D) stereoscopic display system comprising:

at least one processor;

one or more displays, coupled to the at least one processor; and

a memory coupled to the at least one processor, wherein the at least one processor is configured to;

identify content of a two-dimensional (2D) image displayed within a web page displayed on at least one of the one or more displays, wherein the web page is displayed to a user;

identify, via a search of a web page structure or a search of the Internet and an interrogation of the user of the web page for information to assist in selection of 3D images, including questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image, a 3D image with substantially equivalent content as compared to the content of the 2D image;

replace the 2D image with a flattened 3D image, thereby integrating the 3D image into the web page, wherein a z-scale value associated with a depth of the 3D image is decreased to flatten the 3D image;

receive, from the user, at least one user input manipulating the 3D image within the web page, wherein the at least one user input comprises detection of a beam projected from an end of a user input device intersecting with the 3D image, wherein the detection of the beam intersecting with the 3D image is based, at least in part, on tracking location and orientation of the user input device relative to the at least one display; and

upon receiving, from the user, the at least one user input manipulating the 3D image within the web page, un-flatten the 3D image such that the image appears in three dimensions, wherein the z-scale value associated with the depth of the 3D image is increased to un-flatten the 3D image.

15. The 3D stereoscopic display system of claim 14,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

16. The 3D stereoscopic display system of claim 14,

wherein, to identify content of the 2D image, the at least one processor is further configured to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

17. The 3D stereoscopic display system of claim 14,

wherein, to identify the 3D image, the at least one processor is further configured to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet.

18. The 3D stereoscopic display system of claim 17,

wherein, to source the 3D image, the at least one processor is further configured to:

search metadata associated with 3D images located on the Internet;

search an index of content of 3D images located on the Internet.

19. The 3D stereoscopic display system of claim 14,

wherein, to identify the 3D image, the programming instructions are further executable to:

determine that the content of the 3D image is substantially equivalent to content of the 2D image.

20. The 3D stereoscopic display system of claim 14,

wherein the at least one processor is further configured to:

receive at least one additional user input, wherein the at least one additional user input indicates that the user is no longer interacting with the 3D image within the web page; and

upon receiving the at least one additional user input, flatten the 3D image such that the image appears in two dimensions, wherein the z-scale value associated with the depth of the 3D image is decreased to flatten the 3D image.

21. The 3D stereoscopic display system of claim 14,

wherein, to replace the 2D image, the at least one processor is further configured to:

display the object represented by the 3D image at a real-world size of the object when the real-world size of the object represented by the 3D image fits within the bounds of a web browser displaying the web page.

22. A wireless device, comprising:

at least one antenna;

at least one radio in communication with the at least one antenna and configured to perform wireless communication using at least one radio access technology (RAT);

one or more processors coupled to the at least one radio; and

at least one display in communication with the one or more processors;

wherein the one or more processors are configured to cause the wireless device to;

identify content of a two-dimensional (2D) image displayed within a web page, wherein the web page is displayed to a user;

identify a three-dimensional (3D) image with substantially equivalent content as compared to the content of the 2D image, wherein identifying the 3D image includes

an interrogation of a user of the web page for information to assist in selection of the 3D image, including questioning the user regarding one or more of a type, color, or size of an object represented by the 2D image;

integrate the 3D image into the web page, thereby replacing the 2D image with the 3D image;

receive, from the user, at least one user input manipulating the 3D image within the web page, wherein the at least one user input comprises at least one of;

movement of a point of view of the user relative to the at least one display, wherein the movement of the point of view point is determined based, at least in part, on tracking information of a head or eyes of the user;

or

detection of a beam projected from an end of a user input device intersecting with the 3D image, wherein the detection of the beam intersecting with the 3D image is based, at least in part, on tracking location and orientation of the user input device relative to the at least one display; and

upon receiving the at least one user input manipulating the 3D image within the web page,

display the object represented by the 3D image at a real-world size of the object when the real-world size of the object represented by the 3D image fits within the bounds of a web browser displaying the web page.

23. The wireless device of claim 22,

wherein at least a portion of the content of the 3D image corresponds to at least a portion of the content of the 2D image.

24. The wireless device of claim 22,

wherein, to identify content of the 2D image, the one or more processors are further configured to:

identify metadata associated with the 2D image;

or

analyze a web page structure of the web page.

25. The wireless device of claim 22,

wherein, to identify the 3D image, the one or more processors are further configured to:

determine a location of the 3D image within the web page;

or

source the 3D image via a search of the Internet, wherein, to source the 3D image, the at least one processor is further configured to;

search metadata associated with 3D images located on the Internet;

or

search an index of content of 3D images located on the Internet.

26. The wireless device of claim 22,

wherein, to identify the 3D image, the one or more processors are further configured to:

determine that the content of the 3D image is substantially equivalent to content of the 2D image.

27. The wireless device of claim 22,

wherein, to integrate the 3D image into the web page, one or more processors are further configured to:

flatten the 3D image such that the 3D image appears as in two dimensions, wherein to flatten the 3D image a z-scale value associated with a depth of the 3D image is decreased.

28. The wireless device of claim 27,

wherein, upon receiving the at least one user input manipulating the 3D image within the web page, the one or more processors are further configured to:

un-flatten the 3D image such that the 3D image appears in three dimensions, wherein to un-flatten 3D image the z-scale value associated with a depth of the 3D image is increased.

29. The wireless device of claim 22,

wherein the one or more processors are further configured to:

receive at least one additional user input, wherein the at least one additional user input indicates that the user is no longer interacting with the 3D image within the web page; and

upon receiving the at least one additional user input, scale the 3D image such that a size of the 3D image corresponds to a size of the 2D image.

30. The 3D stereoscopic display system of claim 22,

wherein the at least one processor is further configured to:

receive at least one additional user input, wherein the at least one additional user input indicates that the user is no longer interacting with the 3D image within the web page; and

upon receiving the at least one additional user input, scale the 3D image such that a size of the 3D image corresponds to a size of the 2D image.