Interactivity via mobile image recognition

• US 9,087,270 B2
• Filed: 06/12/2013
• Issued: 07/21/2015
• Est. Priority Date: 11/06/2000
• Status: Expired due to Fees

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Cohen, Ronald H.
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US13/915,731
• Publication Number:
  US 20130272574A1
• Time in Patent Office: 769 Days
• Field of Search: 382/100, 382/165, 382/181, 382/305, 705 261- 272, 725 60- 61
• US Class Current: 1/1
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations

Abstract

Systems and methods of interacting with a virtual space, in which a mobile device is used to electronically capture image data of a real-world object, the image data is used to identify information related to the real-world object, and the information is used to interact with software to control at least one of: (a) an aspect of an electronic game; and (b) a second device local to the mobile device. Contemplated systems and methods can be used to gaming, in which the image data can be used to identify a name of the real-world object, to classify the real-world object, identify the real-world object as a player in the game, to identify the real-world object as a goal object or as having some other value in the game, to use the image data to identify the real-world object as a goal object in the game.

Claims

1. A method of assessing environmental characteristics, the method comprising:

capturing, via a mobile device, video representing a surrounding environment;

analyzing, the surrounding environment represented in the video for at least one identifying object indicative of an environmental characteristic;

identifying, via object recognition, the at least one identifying object indicative of an environmental characteristic within the surrounding environment; and

providing information associated with the environmental characteristic to a user of the mobile device via an information address.

2. The method of claim 1, wherein the mobile device comprises a camera capable of image and/or video capture.

3. The method of claim 1, wherein the video comprises at least one real-world object.

4. The method of claim 1, wherein the at least one identifying object is associated with a real-world object.

5. The method of claim 4, wherein the real-world object is an advertisement.

6. The method of claim 4, wherein the real-world object is a consumer product.

7. The method of claim 1, wherein the at least one identifying object indicative of an environmental characteristic is a cover.

8. The method of claim 7, wherein the cover is a disc cover.

9. The method of claim 1, wherein the at least one identifying object indicative of an environmental characteristic is a goal object.

10. The method of claim 1, wherein the at least one identifying object indicative of an environmental characteristic is a candidate object.

11. The method of claim 1, wherein the information associated with the environmental characteristic is dependent on other information.

12. The method of claim 11, wherein the other information comprises at least one of the following:

a current time, an observed state of the object, and a location of the user.

13. The method of claim 1, wherein the information associated with the environmental characteristic is presented to the user on a display of the mobile device as a virtual object superimposed on the video representing the surrounding environment.

14. The method of claim 13, wherein the virtual object is interactive.

15. The method of claim 13, wherein the virtual object provides information associated with a consumer product.

16. The method of claim 1, further comprising presenting the user with an option to purchase at least one of a product or a service.

17. The method of claim 1, further comprising enabling a transaction based on the content information.

18. The method of claim 1, further comprising conducting a transaction that includes the step of transmitting purchase information from the mobile device.

19. A mobile device configured to assess environmental characteristics comprising:

a camera configured to capture video representing a surrounding environment;

an image processing platform configured to;

analyze the video to determine whether the video includes at least one indentifying object indicative of an environmental characteristic;

identify, via object recognition, the at least one identifying object indicative of an environmental characteristic within the surrounding environment; and

provide to the user of the mobile device, via the mobile device through an information address, information associated with the environmental characteristic.

20. The mobile device of claim 19, wherein the camera is configured to capture at least one of an image and the video.

21. The mobile device of claim 19, wherein the video comprises at least one real-world object.

22. The mobile device of claim 19, wherein the at least one identifying object is associated with a real-world object.

23. The mobile device of claim 22, wherein the real-world object is an advertisement.

24. The mobile device of claim 22, wherein the real-world object is a consumer product.

25. The mobile device of claim 22, wherein the at least one identifying object indicative of an environmental characteristic is a cover.

26. The mobile device of claim 25, wherein the cover is a disc cover.

27. The mobile device of claim 19, wherein the at least one identifying object indicative of an environmental characteristic is a goal object.

28. The mobile device of claim 19, wherein the at least one identifying object indicative of an environmental characteristic is a candidate object.

29. The mobile device of claim 19, wherein the information associated with the environmental characteristic is dependent on other information.

30. The mobile device of claim 29, wherein the other information comprises at least one of the current time, the observed state of the object, or the location of the user.

31. The mobile device of claim 19, wherein the information associated with the environmental characteristic is presented to the user on a display of the mobile device as a virtual object superimposed on the video representing the surrounding environment.

32. The mobile device of claim 31, wherein the virtual object is interactive.

33. The mobile device of claim 31, wherein the virtual object provides information associated with a consumer product.

34. The mobile device of claim 19, further comprising presenting the user with an option to purchase at least one of a product or a service.

35. The mobile device of claim 19, further comprising enabling a transaction based on the content information.

36. The mobile device of claim 19, further comprising conducting a transaction that includes the step of transmitting purchase information from the mobile device.

37. A computer program product comprising non-transitory computer readable medium storing instructions that cause a processor to execute the steps of:

capturing, via a mobile device, video representing a surrounding environment;

analyzing, the surrounding environment represented in the video for at least one identifying object indicative of an environmental characteristic;

identifying, via object recognition, the least one identifying object indicative of an environmental characteristic within the surrounding environment; and

providing information associated with the environmental characteristic to a user of the mobile device via an information address.

38. The computer program product of claim 37, wherein the mobile device comprises a camera configured to capture at least one of an image and the video.

39. The computer program product of claim 37, wherein the video comprises at least one real-world object.

40. The computer program product of claim 37, wherein the at least one identifying object is associated with a real-world object.

41. The computer program product of claim 40, wherein the real-world object is an advertisement.

42. The computer program product of claim 41, wherein the real-world object is a consumer product.

43. The computer program product of claim 40, wherein the at least one identifying object indicative of an environmental characteristic is a cover

44. The computer program product of claim 43, wherein the cover is a disc cover.

45. The computer program product of claim 43, wherein the target object represents at least one of the following:

a building, a store, a vehicle, a media box, a person, a passport, a driver'"'"'s license, a poster, an advertisement, on-screen content, a product, a beverage, a sign, a newspaper, a magazine, a radio frequency, and a machine part.

46. The computer program product of claim 37, wherein the at least one identifying object indicative of an environmental characteristic is a goal object.

47. The computer program product of claim 37, further comprising the step of obtaining other information associated with the environmental characteristic.

48. The computer program product of claim 47, wherein the other information comprises at least one of the current time, the observed state of the object, or the location of the user.

49. The computer program product of claim 37, wherein the information associated with the environmental characteristic is presented to the user on a display of the mobile device as a virtual object superimposed on the video representing the surrounding environment.

50. The computer program product of claim 38, wherein the virtual object is interactive.

51. The computer program product of claim 38, wherein the virtual object provides information associated with a consumer product.

52. The computer program product of claim 37, further comprising the step of presenting the user with an option to purchase at least one of a product or a service.

53. The computer program product of claim 37, further comprising the step of enabling a transaction based on the content information.

54. The computer program product of claim 37, further comprising the step of conducting a transaction that includes the step of transmitting purchase information from the mobile device.