Image and augmented reality based networks using mobile devices and intelligent electronic glasses

• US 9,380,177 B1
• Filed: 12/22/2014
• Issued: 06/28/2016
• Est. Priority Date: 01/30/2004
• Status: Expired due to Term

• Current Assignee: Airspace Reality, Co.
• Original Assignee: IP Holdings Incorporated (IP Holdings LLC)
• Inventors: Rao, Sunil K., Rao, Rekha K., Rao, Raman K.
• Primary Examiner(s): Ayotunde, Ayodeji
• Application Number:
  US14/579,497
• Time in Patent Office: 554 Days
• Field of Search: 348/333.05, 455/556.1, 455/566
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 45/306 Route determination based on the nature of the carried application
• CPC Class Codes: H04N 1/00095 Systems or arrangements for the transmission of the picture signal
• CPC Class Codes: H04N 1/00214 details of transmission
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 7/14 Systems for two-way working H04N7/12, H04N7/173 take precedence
• CPC Class Codes: H04W 40/02 Communication route or path selection, e.g. power-based or shortest path routing

Abstract

A mobile communication system based on digital content including images and video that may be acquired, processed, and displayed using a plurality of mobile device, smartphones, tablet computers, stationary computers, intelligent electronic glasses, smart glasses, headsets, watches, smart devices, vehicles, and servers. Content may be acquired continuously by input and output devices and displayed in a social network. Images may have associated additional properties including nature of voice, audio, data and other information. This content may be displayed and acted upon in an augmented reality or virtual reality system. The imaged base network system may have the ability to learn and form intelligent association between aspects, people and other entities; between images and the associated data relating to both animate and inanimate entities for intelligent image based communication in a network. Acquired content may be processed for predictive analytics. Individuals may be imaged for targeted and personalized advertising.

Claims

1. A wireless communications system for image networks comprising:

a wireless device configured to communicate via at least one communications network, the wireless device further configurable to perform one or more of computing functions or voice communication functions, the wireless device configured with a display, and configurable to acquire and process a plurality of images, wherein the plurality of images comprise still images and video images, and wherein at least one image of the plurality of images is displayed on the wireless device display, and wherein a function is configured to automatically display information which corresponds to an object represented in the at least one image, wherein the object is a physical object separate from the wireless device, wherein the image is linked through a relationship to related data defined by a user or past user behavior stored on a server or the wireless device, wherein the wireless device acquires the images via a camera or image sensor in real time, wherein the images are compressed to enable efficient communication, wherein a set of functions or a menu of functions are overlayed or displayed on top on the image display; and

at least one server connected to a network, wherein the at least one server connected to the network is configurable to store the plurality of images, wherein the at least one server connected to the network is configured to receive an image from the wireless device, wherein the at least one server connected to the network is configured to associate each received image with a plurality of data comprising one or more of text data, voice data, wireless device profile data, user profile data, or location data.

2. The wireless communications system of claim 1 further comprising a executable function, wherein the executable function is displayed on the display of the wireless device, wherein the executable function comprises a function selected from the group consisting essentially of an application that renders at least one image of the plurality of images, and a routing process that transmits at least one image of the plurality of images through the communications network to a different wireless device or network element, wherein the wireless device comprises electronic wearable glasses.

3. The wireless communications system of claim 1 wherein an executable function comprises a command sequence that is transmitted over a second network of the at least one communication network and that causes a physical change in the physical object corresponding to an object represented in the at least one image of the plurality of images, wherein the wireless device comprises electronic wearable glasses.

4. The wireless communications system of claim 1 wherein each image of the plurality of images is encrypted with a security level and a compression format, wherein the wireless device comprises electronic wearable glasses.

5. The wireless communications system of claim 1 wherein the plurality of images comprises a hierarchically organized set of images that are linked through the relationship to form an image network, wherein the wireless device comprises electronic wearable glasses.

6. The wireless communications system of claim 5 wherein each image of the hierarchically organized set of images is encrypted with a respective security level, the security level configured to define a class of users privileged to access an associated image.

7. The wireless communications system of claim 5 wherein the relationship is learned by a software process based on past user behavior patterns and potential user behavior patterns.

8. The wireless communications system of claim 2 wherein the at least one server connected to the network is configurable to analyze the object recognized within the at least one image and determine an embedded executable function from a plurality of possible embedded executable functions associated with the at least one image to automatically actuate the routing process.

9. The wireless communications system of claim 2 wherein the object within the at least one image includes a user of the system, and further wherein the at least one server connected to the network is configurable to correlate the at least one image with the user profile data for the user of the system.

10. The wireless communications system of claim 2 further comprising:

a user interface coupled to the at least one server connected to the network, and configurable to enable a user to interact with the at least one image, including launching an application to perform a function that physically affects the object represented in the at least one image.

11. The wireless communications system of claim 10 wherein the object is physically affected through a network connected intelligent device that is proximate to the object.

12. The wireless communications system of claim 10 wherein the user interface comprises a touch screen display device of a client computer coupled to the at least one server over a third network of the at least one communications network.

13. The wireless communications system of claim 2 further comprising an image switch and router component configured to perform a pattern recognition process on at least one object in the at least one image and transmit the at least one image of the plurality of images through the communications network to a different wireless device or network element.

14. The wireless communications system of claim 13 wherein the embedded executable function includes a function to retrieve a corresponding image from a data source coupled to the at least one communications network, wherein the corresponding image contains one of the at least one object and an object related to the at least one object through the relationship.

30. The wireless communications system of claim 1, wherein the wireless device includes a touch screen display, wherein at least a portion or subset of an image on the touch screen display can be touched to launch one or more applications, control functions or actuation functions related to other intelligent devices, wherein the wireless device comprises electronic wearable glasses.

31. The wireless communications system of claim 1, wherein an image network is propagated across a network of users based upon a profile of the wireless device user and one or more social, professional, personal or other network of the wireless device user is updated, wherein the wireless device comprises electronic wearable glasses.

15. A wireless communications system for image networks comprising:

a wireless device having a communications component to process voice or digital data and communicate over at least one communications network, and an image acquisition component to acquire and process a plurality of images, wherein each image of the plurality of images is associated with multimedia information including image data, associated text data, audio data, or security information and a function that automatically displays information which corresponds to an object represented in the plurality of images, wherein the object is a physical object separate from the wireless device, wherein the wireless device acquires the plurality of images via a camera or image sensor in real time, wherein the plurality of images are compressed to enable efficient communication, wherein a set of functions or a menu of functions are overlayed or displayed on top of a display used for viewing images; and

a server connected to a network, the server connected to the network configurable to store the plurality of images, the server connected to the network configured to receive an image from the wireless device, the server connected to the network configured to associate each received image with wireless device profile data, user profile data, or location data, the server connected to the network further configurable to link the image through a relationship to related data defined by a user or past user behavior stored on the server connected to the network or the wireless device.

16. The wireless communications system of claim 15 wherein sub images of an image network comprise a plurality of images organized by a relationship based on the presence of an object represented in at least the first image or the one or more sub images, wherein the wireless device comprises electronic wearable glasses.

17. The wireless communications system of claim 16 wherein the relationship is provided by a process resident on the wireless device and is stored in a look-up table to enable dynamic look-up by a routing process that transmits the image network through the at least one communications network to one or more other network entities.

18. The wireless communications system of claim 16 wherein the relationship is provided by a process resident on the server connected to the network and is stored in a look-up table to enable dynamic look-up by a routing process that transmits the image network through the at least one communications network to one or more other network entities.

19. The wireless communications system of claim 16 wherein the security information for each image of the image network defines user access privileges, and wherein the hierarchy provided by the relationship associates a different security level to at least some images of the image network.

20. The wireless communications system of claim 15 wherein an application renders at least one image of the plurality of images and a routing process transmits at least one image of the plurality of images through the at least one communications network to a different wireless device or network element, wherein the wireless device comprises electronic wearable glasses.

21. The wireless communications system of claim 15 wherein a function overlayed or displayed on the display used for viewing images comprises a command sequence that is transmitted over a second network of the at least one communications network and that causes a physical change in an object represented by at least one image of the plurality of images, wherein the wireless device comprises electronic wearable glasses.

22. The wireless communications system of claim 21 wherein the server connected to the network comprises an image recognition component to recognize an object in the at least one image and execute a function to automatically actuate an image switch and router component configured to perform a pattern recognition process on the object in the at least one image and transmit the plurality of images through the communications network to a different network element.

23. The wireless communications system of claim 21 wherein multimedia information and the plurality of images are processed through a software process and associated databases resident on one of the wireless device and the server connected to the network.

24. The wireless communications system of claim 15 wherein the image acquisition component comprises a digital camera resident within the wireless device, and wherein the plurality of images comprise at least one of still images and video images, wherein the wireless device comprises electronic wearable glasses.

25. A wireless communications system for image networks comprising:

a wireless device having a communications component to process voice or digital data and communicate over at least one communications network, and an image acquisition component to acquire and process a plurality of images, wherein each image in the plurality of images is associated with multimedia information including image data, associated text data, audio data, or security information, and a function that automatically displays information which corresponds to an object represented in an image entity, wherein the object is a physical object separate from the wireless device, and wherein an image is linked through a relationship to related data defined by a user or past user behavior stored on a server or the wireless device, wherein the wireless device acquires the plurality of images via a camera or image sensor in real time, wherein the plurality of images are compressed to enable efficient communication, wherein a set of functions or a menu of functions are overlayed or displayed on top of a display of the wireless device used for viewing images; and

a server connected to a network wherein the server connected to the network is configured to store the plurality of images, wherein the server connected to the network is configured to receive an image from the wireless device, and wherein the server connected to the network further having an image switch and router component is configured to perform a pattern recognition process on at least one object in the at least one image and transmit the at least one image of an image network through the communications network to a different network element.

26. The wireless communications system of claim 25 wherein the server is further configured to link the plurality of images with one another according to a hierarchical relationship of a first image and one or more sub images to form the image network, and wherein the sub images of the image network comprise a plurality of images organized by a relationship based on the presence of an object represented in at least the first image or the one or more sub images, wherein the wireless device comprises electronic wearable glasses.

27. The wireless communications system of claim 26 wherein the relationship is provided by a process resident on either of the server connected to the network or the wireless device and is stored in a look-up table to enable dynamic look-up by the image switch and router component that transmits the image network through the at least one communications network to one or more other network entities.

28. The wireless communications system of claim 27 wherein an application renders at least one image of the plurality of images and a routing process transmits at least one image of the plurality of images through the communications network to a different wireless device or network element.

29. The wireless communications system of claim 27 wherein a command sequence is transmitted over a second network of the at least one communication network and causes a physical change in an object represented by at least one image of the plurality of images.

32. A wireless communications system for images comprising:

a wireless device comprising a processor, a camera, and a wireless transmit and receive

component;

wherein the wireless device is configured to communicate via at least one wireless communications network;

wherein the wireless device acquires an image using the camera or an image sensor;

wherein the image is compressed to enable efficient communication;

wherein the image is displayed on a display screen of the wireless device;

wherein an object represented in the image is identified by the wireless device;

wherein the object is recognized using an image recognition program and comprises a recognized object;

wherein the recognized object corresponds to a physical object separate from the wireless device;

wherein the wireless device displays text related to the recognized object on the display of the wireless device; and

wherein a set of functions or a menu of functions are overlayed or displayed on top of the display screen.

33. The system of claim 32, wherein the text related to the recognized object is received from a server, wherein the wireless device comprises electronic wearable glasses.

34. The system of claim 32, wherein the image recognition is performed on a server, wherein the wireless device comprises electronic wearable glasses.

35. The system of claim 32, wherein the image recognition is performed by a processor on the wireless device, wherein the wireless device comprises electronic wearable glasses.

36. The system of claim 32, wherein the text displayed on the wireless device display is dynamically generated by a server in response to the image being acquired by the camera or the image sensor and the object being identified by the wireless device, wherein the wireless device comprises electronic wearable glasses.

37. The system of claim 32, wherein the wireless device executes a function in response to the selection of a specified image area corresponding to the object identified by the wireless device and wherein the function controls the physical object, wherein the wireless device comprises electronic wearable glasses.

38. The system of claim 37, wherein the function comprises controlling an on/off command.

39. The system of claim 37, wherein the recognized object is a garage door and wherein the function controls a physical garage door.

40. The system of claim 37, wherein a pattern matching software is executed to identify objects in an image sent by the wireless device to a server.

41. A system for interaction with virtual and physical items using a wireless device comprising:

a wireless device further comprising a display used for viewing images and a camera, wherein the wireless device acquires an image using the camera, wherein the image acquired by the camera is compressed to enable efficient communication, wherein the wireless device presents the image acquired by the camera on the display used for viewing images in the wireless device, wherein a function sends to a server connected to the Internet the image acquired by the camera, wherein the server connected to the Internet performs image recognition on the image acquired by the camera, wherein the server connected to the Internet sends to the wireless device information related to the image acquired by the camera, wherein the wireless device displays the information and the image acquired by the camera on the display of the wireless device; and

wherein a set of functions or a menu of functions are overlayed or displayed on top of the image acquired by the camera viewable on the display used for viewing images in the wireless device.

42. The system of claim 41, wherein a user may touch a selected image area, wherein the wireless device executes a function in response to the selection of the specified image area, wherein the wireless device comprises electronic wearable glasses.

43. A wireless image recognition system comprising:

a wireless device, wherein the wireless device has a touch enabled display and a processor;

a display area on the wireless device;

wherein the wireless device acquires a plurality of images via a camera and performs pattern recognition using the processor on the plurality of images to create a function, wherein the plurality of images are compressed to enable efficient communication, wherein the wireless device sends the plurality of images to a server, wherein a set of functions or a menu of functions are overlayed or displayed on top of an image in the display area;

a server, wherein the server receives the plurality of images from the wireless device, wherein the server performs pattern recognition of the plurality of images; and

an interface on the wireless device which interacts with the display area through light, wherein the act of pointing at an image on the display area launches an application on the wireless device.

44. The system of claim 43, wherein the wireless device comprises electronic wearable glasses.