Combine harvester control information for a remote user with visual feed

• US 10,412,889 B2
• Filed: 12/05/2017
• Issued: 09/17/2019
• Est. Priority Date: 12/05/2017
• Status: Active Grant

• Current Assignee: Deere & Company
• Original Assignee: Deere & Company
• Inventors: Palla, Bhanu Kiran Reddy, Karedla, Narendar Reddy
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US15/832,056
• Publication Number:
  US 20190166760A1
• Time in Patent Office: 651 Days
• CPC Class Codes: A01D 41/06 Combines with headers
• CPC Class Codes: A01D 41/127 Control or measuring arrangements specially adapted for combines
• CPC Class Codes: A01D 41/1274 for drives
• CPC Class Codes: H04L 67/00 Network arrangements or protocols for supporting network services or applications user-to-user messaging H04L51/00; network arrangements, protocols or services for supporting real-time applications in data packet communications networks H04L65/00
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

A user device, that is remote from a combine harvester, communicates with the remote harvester to receive contextual information indicative of machine settings on the combine harvester. Remote view and control logic receives the contextual information from the combine harvester, along with image or video display information generated from an image capture device (such as a video camera or other image capture device) on the combine harvester. The contextual information is displayed, along with the video or image information on the remote user device.

Claims

1. An agricultural harvesting machine, comprising:

a crop processing system configured to process a crop that is gathered by the agricultural harvesting machine;

a visual information capture mechanism that captures visual information;

context information correlation logic that identifies context information corresponding to the visual information;

output generator logic configured to generate an output including the context information and corresponding visual information; and

a transmission system that transmits the output to a remote computing system, remote from the agricultural harvesting machine, for display of the context information and the corresponding visual information.

2. The agricultural harvesting machine of claim 1 wherein the visual information capture mechanism comprises:

a plurality of video cameras mounted on the agricultural harvesting machine and each capturing video information.

3. The agricultural harvesting machine of claim 2 wherein first context information corresponds to the video information from a first of the plurality of video cameras and second context information corresponds to the video information from a second of the plurality of video cameras, the first context information being different from the second context information.

4. The agricultural harvesting machine of claim 3 and further comprising:

a plurality of sensors each generating a sensor signal indicative of a sensed variable; and

a control system generating the context information based on the sensor signals.

5. The agricultural harvesting machine of claim 4 and, further comprising:

trigger detection logic configured to detect a trigger indicative of when the output including the context information and corresponding video information is to be transmitted to the remote computing system and generating a trigger signal identifying the video information based on detection of the trigger.

6. The agricultural harvesting machine of claim 5 wherein the trigger detection logic comprises:

a request detector detecting a request from the remote computing system, the request identifying which video information is to be transmitted.

7. The agricultural harvesting machine of claim 1, wherein the captured video information represents the crop in a crop processing system of the agricultural harvesting machine, and wherein the correlation logic is configured to identify the corresponding context information, that corresponds to the identified video information, for transmission with the identified video information.

8. The agricultural harvesting machine of claim 5 wherein the trigger detection logic comprises:

a continuous transmission detector generating the trigger signal indicating that the video information and corresponding context information is to be sent continuously, the transmission system being configured to continuously transmit the identified video information and corresponding context information to the remote computing system based on the trigger signal.

9. The agricultural harvesting machine of claim 5 wherein the trigger detection logic comprises:

an event detector detecting an event and generating the trigger signal indicating that the video information and corresponding context information is to be sent to the remote computing system based on the detected event, the transmission system being configured to continuously transmit the identified video information and corresponding context information to the remote computing system based on the trigger signal.

10. A method of controlling an agricultural harvesting machine, the method comprising:

performing, in a crop processing system of the agricultural harvesting machine, a crop processing operation on a crop gathered by the agricultural harvesting machine;

capturing video information using a video capture mechanism on the agricultural harvesting machine, the captured video information representing the crop in the crop processing system;

generating a sensor signal indicative of a sensed variable;

identifying context information, based on the sensor signal, corresponding to the video information;

generating an output including the context information and corresponding video information; and

transmitting the output to a remote computing system, remote from the agricultural harvesting machine, for display of the context information and the corresponding video information.

11. The method of claim 10 wherein capturing video information comprises:

capturing video information from each of a plurality of different video cameras mounted on the agricultural harvesting machine.

12. The method of claim 11 wherein identifying context information comprises:

identifying first context information corresponding to the video information from a first of the plurality of different video cameras; and

identifying second context information corresponding to the video information from a second of the plurality of video capture mechanisms, the first context information being different from the second context information.

13. The method, of claim 12 wherein generating a sensor signal comprises:

generating a plurality of different, sensor signals, each indicative of a different sensed variable; and

generating the context information based on the plurality of different sensor signals.

14. The method of claim 13 and further comprising:

detecting a request from the remote computing system; and

identifying, based on the request which video information is to be transmitted, wherein identifying context information includes identifying the corresponding context information, that corresponds to the identified video information for transmission with the identified video information.

15. The method of claim 14 wherein transmitting the output to the remote computing system comprises:

continuously transmitting the identified video information and corresponding context information to the remote computing system.

16. The method of claim 13 and further comprising:

detecting an event; and

identifying which video information and corresponding context information is to be sent to the remote computing system based on the detected event, wherein transmitting comprises continuously transmitting the identified video information and corresponding context information to the remote computing system based on the detected event.

17. A mobile device, comprising:

machine selector logic that displays a user actuatable machine selection element that is actuatable to select a remote agricultural harvesting machine, that is remote from the mobile device;

a communication system that communicates with a selected remote agricultural harvesting machine;

video information display logic that displays near real time video information received from a video camera on the selected remote agricultural harvesting machine; and

context information display logic that displays near real time context information, corresponding to the video information, received from the selected agricultural harvesting machine and generated based on sensor signals generated by sensors on the selected agricultural harvesting machine.

18. The mobile device of claim 17 wherein the machine selector logic is configured to display a plurality of different machine selector elements that are each actuatable to select a different remote agricultural harvesting machine, and further comprising:

a user interaction detector configured to detect user actuation of a plurality of the different machine selector elements to select a plurality of different remote agricultural harvesting machines.

19. The mobile device of claim 18 wherein the video information display logic is configured to display near real time video information received from a video camera on each of the plurality of selected remote agricultural harvesting machines and wherein the context information display logic is configured to display near real time context information, corresponding to the video information, received from each of the selected agricultural harvesting machines and generated based on sensor signals generated by sensors on each of the selected remote agricultural harvesting machines.

20. The mobile device of claim 17 and further comprising:

remote control generator logic configured to generate a user actuatable remote control element that is actuated to send a control signal to remotely control the selected remote agricultural harvesting machine.