Camera system API for third-party integrations

• US 9,788,039 B2
• Filed: 01/04/2016
• Issued: 10/10/2017
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Karp, Igor, Stesin, Lev
• Primary Examiner(s): Mudrick, Timothy A
• Application Number:
  US14/987,679
• Publication Number:
  US 20160134932A1
• Time in Patent Office: 645 Days
• Field of Search: 719328
• CPC Class Codes: G05B 13/04 involving the use of models or simulators
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05D 23/1904 variable in time
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: G08B 13/1672 using sonic detecting means, e.g. a microphone operating in the audio frequency range
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 17/10 Actuation by presence of smoke or gases , e.g. automatic alarm devices for analysing flowing fluid materials by the use of optical means
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2829 involving user profiles according to which the execution of a home appliance functionality is automatically triggered
• CPC Class Codes: H04L 63/04 for providing a confidential data exchange among entities communicating through data packet networks
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/125 : involving control of end-device applications over a network
• CPC Class Codes: H04L 67/303 : Terminal profiles
• CPC Class Codes: H04L 67/53 : using third party service providers
• CPC Class Codes: H04L 67/535 : Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 69/24 : Negotiation of communication capabilities
• CPC Class Codes: H04N 21/2187 : Live feed
• CPC Class Codes: H04N 21/231 : Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
• CPC Class Codes: H04N 21/25816 : involving client authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; cryptographic authentication protocols H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/2743 : Video hosting of uploaded data from client
• CPC Class Codes: H04N 21/4223 : Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/4431 : characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/6811 : based on the image signal
• CPC Class Codes: H04N 7/186 : Video door telephones
• CPC Class Codes: H04W 12/033 : of the user plane, e.g. user's traffic
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

In embodiments of a camera system application program interface (API) for third-party integrations, a camera device captures images as a video stream and communicates the video stream to a cloud-based service. The cloud-based service implements a service application that processes video data received as the video stream. The cloud-based service exposes the camera system API that can be invoked by a third-party application running on a client device to request the video data and camera data that is associated with the camera device. The API permits access by the third-party application to the video data and the camera data from the cloud-based service. The API is exposed for the third-party application to communicate with the cloud-based service via a network connection, and the camera device communicates with the cloud-based service via a secure connection to provide the requested camera data and communicate the video stream to the cloud-based service.

Claims

1. A cloud-based service, comprising:

a server computing device configured for executing a service application configured to process video data received as a video stream from a camera device of a user, the camera device captures images and communicates the video stream to the cloud-based service; and

an application program interface (API), hosted on the server computing device, that can be invoked by a third-party application, running on a client device of the user or hosted on a remote service associated with the client device, to request the video data and camera data associated with the camera device, and to access settings of the camera device, the API permitting access by the third-party application on behalf of the user to the video data and the camera data from the cloud-based service.

2. The cloud-based service as recited in claim 1, wherein:

the API is exposed for the third-party application to communicate with the cloud based service via a network connection to request the video data and the camera data; and

the camera device communicates with the cloud-based service via a secure connection to provide the requested camera data and communicate the video stream to the cloud-based service.

3. The cloud-based service as recited in claim 1, wherein:

the camera data includes at least an operational status of the camera device; and

the third-party application is configured to receive the operational status of the camera device from the cloud-based service via the API when the cloud based service receives the operational status from the camera device.

4. The cloud-based service as recited in claim 1, wherein:

the video data that is generated by the service application from the video stream received from the camera device is accessible at the cloud-based service; and

the API is exposed for the third-party application to receive the video data from the cloud based service.

5. The cloud-based service as recited in claim 1, wherein the video data that is generated by the service application from the video stream includes at least one of:

the images of a motion event associated with a motion detected by the camera device; and

the previous images that correspond to a noise event associated with a sound detected by the camera device.

6. The cloud-based service as recited in claim 5, wherein:

the motion event and the noise event are reported by the service application as cue point events to the third-party application via the API as the video stream is received from the camera device and processed by the service application.

7. The cloud-based service as recited in claim 1, wherein:

the API is exposed to the third-party application to grant permissions to request different forms of the camera data that is associated with the camera device and to access the settings of the camera device; and

one or more of the permissions are defined in a camera data scope that identifies the camera data the third-party application is permitted to access.

8. The cloud-based service as recited in claim 7, wherein the camera data scope is configured to be updated to include additional permissions or fewer permissions to permit access to the different forms of the camera data that is associated with the camera device via the API.

9. The cloud-based service as recited in claim 7, wherein the camera data scope is one of:

a camera read scope that includes the one or more permissions allowing the third-party application to access the settings of the camera device, motion events associated with detected motions, and noise events associated with detected sounds;

or

a camera write scope that includes the one or more permissions of the camera read scope as well as an additional permission allowing the third-party application to initiate turning the camera device on and off via the API.

10. The cloud-based service as recited in claim 7, wherein the camera data scope is one of:

a camera images read scope that includes the one or more permissions allowing the third-party application to access settings of the camera device, the images of motion events associated with detected motions, and the images that correspond to noise events associated with detected sounds;

or

a camera images write scope that includes the one or more permissions of the camera images read scope as well as an additional permission allowing the third-party application to initiate turning the camera device on and off via the API.

11. The cloud-based service as recited in claim 1, wherein:

the camera data is structured as a data model of data fields, one or more of the data fields being accessible by the third-party application via the API;

the data fields including a Web URL to identify an addressable location of the camera device;

the data fields further including an application URL to an interface application of the camera device; and

the Web URL and the application URL effective to allow the third-party application running on the client device to link to the interface application of the camera device.

12. The cloud-based service as recited in claim 1, wherein:

the camera data is structured as a data model of data fields, one or more of the data fields being accessible via the API; and

the data model is implemented as a JavaScript Object Notation (JSON) document that describes the camera device, the camera data that is associated with the camera device, and an environment structure of the camera device.

13. A system, comprising:

a service application executable by a server computing device implemented at a cloud-based service, the service application configured to process video data received as a video stream; and

an application program interface (API) implemented at the cloud-based service, the API invokable by a third-party application, running on a client device or hosted on a remote service associated with the client device, to request the video data and camera data associated with a camera device, and to access settings of the camera device, the API permitting remote access by the third-party application to the video data and the camera data from the cloud-based service.

14. The system as recited in claim 13, further comprising:

an API server configured to host the API that is exposed for the third-party application to communicate with the cloud based service via a network connection to request the video data and the camera data; and

the camera device configured to communicate with the cloud-based service via a secure connection to provide the requested camera data and communicate the video stream to the cloud-based service.

15. The system as recited in claim 14, wherein:

the camera data includes at least an operational status of the camera device; and

the third-party application is configured to receive the operational status of the camera device from the cloud-based service via the API when the cloud based service receives the operational status from the camera device.

16. The system as recited in claim 14, wherein:

the video data processed by the service application from the video stream received from the camera device is accessible at the cloud-based service; and

the API is exposed for the third-party application to receive the video data from the cloud based service.

17. The system as recited in claim 14, wherein the video data processed by the service application from the video stream includes at least one of:

images of a motion event associated with a motion detected by the camera device; and

the images that correspond to a noise event associated with a sound detected by the camera device; and

the motion event and the noise event are reported by the service application as cue point events to the third-party application via the API as the video stream is received from the camera device and processed by the service application.

18. The system as recited in claim 14, wherein:

the third-party application is associated with a garage door company;

the camera device is configured to capture images of at least one of a motion event and a sound event when a garage door opens; and

the images that correspond to at least one of the motion event and the sound event of the garage door opening are communicated to the third-party application via the API for display on the client device.

19. The system as recited in claim 14, wherein:

the client device is a television device configured to communicate with the cloud-based service via the API;

the camera device is configured to capture images of at least one of a motion event and a sound event of a detected motion; and

the images that correspond to at least one of the motion event and the sound event are communicated to the television device via the API for display.

20. The system as recited in claim 14, wherein:

the third-party application is associated with a door lock company;

the camera device is configured to capture images of at least one of a motion event and a sound event when a door lock unlocks; and

the images that correspond to at least one of the motion event and the sound event of the door lock unlocking are communicated to the third-party application via the API for display on the client device.

21. The system as recited in claim 14, wherein:

the third-party application is associated with a doorbell company;

the camera device is configured to capture images of at least one of a motion event and a sound event when one of a doorbell rings or motion is detected; and

the images that correspond to at least one of the motion event and the sound event of the doorbell ring are communicated to the third-party application for display on the client device.

22. The system as recited in claim 14, wherein:

the third-party application is associated with baby monitoring;

the camera device is utilized for the baby monitoring and configured to capture images of at least one of a motion event and a sound event of a detected motion; and

the images that correspond to at least one of the motion event and the sound event of the detected motion are communicated to the third-party application via the API for display on the client device.

23. The system as recited in claim 14, wherein:

the third-party application is associated with animal activity monitoring;

the camera device is utilized for the animal activity monitoring and configured to capture images of at least one of a motion event and a sound event of a detected motion; and

the images that correspond to at least one of the motion event and the sound event of the detected motion are communicated to the third-party application via the API for display on the client device.

24. The system as recited in claim 14, wherein:

the client device is a lighting fixture configured to communicate with the cloud-based service via the API;

the camera device is configured to detect motion and capture images of at least one of a motion event and a sound event of the detected motion; and

the service application is configured to initiate turning on a light of the lighting fixture responsive to the motion being detected.

25. The system as recited in claim 14, wherein:

the client device is a wearable device configured to communicate with the cloud-based service via the API;

the camera device is configured to detect motion and capture images of at least one of a motion event and a sound event of the detected motion; and

the service application is configured to initiate a notification of the detected motion to the wearable device.

26. A method to permit access to video data and camera data associated with a camera device via an application program interface (API) at a cloud-based service, the method comprising:

processing a video stream to generate the video data, the video stream being received from a camera device of a user, the camera device captures images and communicates the video stream to the cloud-based service; and

exposing the API that is invoked by a third-party application, running on a client device of the user or hosted on a remote service associated with the client device, to request the video data and the camera data associated with the camera device, and to access settings of the camera device, the API permitting access by the third-party application on behalf of the user to the video data and the camera data from the cloud-based service.

27. The method as recited in claim 26, wherein:

said exposing the API for the third-party application to communicate with the cloud based service via a network connection to request the video data and the camera data; and

the method further comprising;

communicating with the camera device via a secure connection to receive the requested camera data and the video stream from the camera device.

28. The method as recited in claim 26, wherein:

the camera data includes at least an operational status of the camera device; and

the method further comprising;

forwarding the request for the operational status to the camera device;

communicating the operational status of the camera device to the third-party application via the API when the cloud based service receives the operational status from the camera device.

29. The method as recited in claim 26, wherein the video data that is generated from the video stream includes at least one of:

the images of a motion event associated with a motion detected by the camera device; and

the images that correspond to a noise event associated with a sound detected by the camera device;

the method further comprising;

reporting the motion event and the noise event as cue point events to the third party application via the API as the video stream is received from the camera device and processed at the cloud-based service.

30. The method as recited in claim 26, wherein the camera data is structured as a data model of data fields, one or more of the data fields being accessible via the API.