Method and apparatus for simulating a physical environment to facilitate vehicle operation and task completion

• US 8,508,590 B2
• Filed: 03/02/2010
• Issued: 08/13/2013
• Est. Priority Date: 03/02/2010
• Status: Active Grant

• Current Assignee: Crown Equipment Corporation
• Original Assignee: Crown Equipment Corporation
• Inventors: Laws, Matthew Edwin, Goode, Christopher W., Collett, Toby Hartnoll Joshua, Graham, Andrew Evan
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Ren, Zhubing
• Application Number:
  US12/660,616
• Publication Number:
  US 20110216185A1
• Time in Patent Office: 1,260 Days
• Field of Search: None
• US Class Current: 348/121
• CPC Class Codes: B66F 9/063 Automatically guided
• CPC Class Codes: B66F 9/20 Means for actuating or controlling masts, platforms, or forks power take-off from vehicle transmissions, combined with vehicle propulsion systems B60K
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and apparatus for simulating a physical environment to facilitate vehicle operation and task completion is described. In one embodiment, the method includes processing data that is transmitted from a sensor array comprising a plurality of devices, wherein the plurality of devices are placed throughout a physical environment that comprises a plurality of objects, accessing model information associated with the plurality of objects, wherein the model information indicates at least one physical attribute associated with each of the plurality of objects, correlating the model information with the transmitted data from the sensor array to produce a correlation result, generating at least one visual representation of the physical environment from at least one viewpoint based on the correlation result and identifying at least one obstruction, within the physical environment, using the at least one visual representation.

Claims

1. A method of operating a vehicle by simulating a physical environment, the method comprising:

collecting data from a sensor array, wherein the collected data relates to various physical attributes, including size and shape, of objects within the physical environment;

using the collected data to construct digital, three-dimensional models of the objects within the physical environment;

accessing the three-dimensional models associated with respective ones of the plurality of objects, wherein the three-dimensional models enable creation of visual representations of respective ones of the plurality of objects from any perspective;

correlating the three-dimensional models with the transmitted data from the sensor array to produce a correlation result;

generating at least one visual representation of the physical environment from at least one viewpoint based on the correlation of the three-dimensional models with the transmitted data from the sensor array;

identifying at least one obstruction represented by the three-dimensional models, within the physical environment, using the at least one visual representation generated based on the correlation of the three-dimensional models with the transmitted data from the sensor array; and

operating the vehicle to complete a task based on the obstruction identified using the visual representation generated based on the correlation of the three-dimensional models with the transmitted data from the sensor array.

2. The method of claim 1, wherein the vehicle is operated by transporting objects along defined paths and the identified obstruction is used to change one of the defined paths.

3. The method of claim 1, wherein:

the visual representation of the physical environment illustrates an obstructed view of the physical environment that is obstructed by the obstruction; and

the method comprises operating the vehicle by removing or making transparent the obstruction in the visual representation of the physical environment.

4. The method of claim 3, wherein the visual representation of the physical environment, with the obstruction removed or made transparent, is displayed on a mobile computer attached to the vehicle, to a central computer coupled to the sensor array through a network, or both.

5. The method of claim 3, wherein:

the task comprises transporting a pallet load; and

the obstruction removed or made transparent is the transported pallet load.

6. The method of claim 3, wherein:

the vehicle comprises a forklift;

the task comprises moving the forklift along a defined path; and

the obstruction removed or made transparent is selected from a lift carriage, forks, lifting poles of the forklift, or combinations thereof.

7. The method of claim 1, wherein data transmitted from the sensor array comprises data relating to various physical attributes associated with each of the plurality of objects.

8. The method of claim 1, wherein the vehicle comprises a mobile computer and the method comprises:

storing the data transmitted from the sensor array on the mobile computer;

utilizing the mobile computer to instruct the vehicle to complete the task.

9. The method of claim 8, wherein the visual representation of the physical environment is displayed on the mobile computer.

10. The method of claim 1, wherein:

the vehicle comprises a mobile computer attached to the vehicle;

the mobile computer is coupled to the sensor array and a central computer through a network; and

the method comprises displaying the visual representation of the physical environment on the central computer.

11. The method of claim 1, wherein:

the vehicle comprises a mobile computer attached to the vehicle;

the mobile computer is coupled to the sensor array and a central computer through a network; and

the method comprises using software modules within the mobile computer , the central computer, or both, to render the three-dimensional model of the plurality of objects within the physical environment and generate visual representations of the physical environment at any selected viewpoint.

12. The method of claim 1, wherein the visual representation of the physical environment is generated by further steps comprising:

selecting a viewpoint of the at least one viewpoint;

generating a visual representation from the selected viewpoint of the at least one visual representation;

selecting another viewpoint of the at least one viewpoint; and

generating another visual representation of the physical environment from the other selected viewpoint of the at least one visual representation of the physical environment.

13. The method of claim 1, wherein:

the collected data comprises a point cloud of geometric samples on a surface of an object within the physical environment; and

,

the method comprises extrapolating object shape from the point cloud.

14. The method of claim 1, wherein:

data from the sensor array is collected from multiple scans of an object within the physical environment from different perspectives in order to determine linear and angular displacement values on surface of the object;

object surface data produced from the multiple scans is normalized along a common reference system; and

the three-dimensional models are constructed by merging the normalized object surface data produced from the multiple scans.

15. The method of claim 1, wherein:

data from the sensor array is collected from multiple scans of an object within the physical environment from different perspectives; and

the three-dimensional models are constructed by merging the object surface data produced from the multiple scans.

16. A system for simulating a physical environment to facilitate vehicle operation and task completion, the system comprising a mobile computer coupled to a vehicle, a central computer, and a sensor array, wherein:

the mobile computer, the central computer, and the sensor array are coupled to each other through a network; and

software modules within the central computer, the mobile computer, or both, render three-dimensional models of a plurality of objects within a physical environment in which the vehicle operates and, based on the three-dimensional models, generate visual representations of the physical environment at any selected viewpoint to facilitate vehicle operation by

collecting data from a sensor array, wherein the collected data relates to various physical attributes, including size and shape, of objects within the physical environment,

using the collected data to construct digital, three-dimensional models of the objects within the physical environment,

accessing the three-dimensional models associated with respective ones of the plurality of objects, wherein the three-dimensional models enable creation of visual representations of respective ones of the plurality of objects from any perspective, correlating the three-dimensional models with the transmitted data from the sensor array to produce a correlation result,

generating at least one visual representation of the physical environment from at least one viewpoint based on the correlation of the three-dimensional models with the transmitted data from the sensor array,

identifying at least one obstruction represented by the three-dimensional models, within the physical environment, using the at least one visual representation generated based on the correlation of the three-dimensional models with the transmitted data from the sensor array, and

operating the vehicle to complete a task based on the obstruction.

17. The system of claim 16, wherein the sensor array is fixed to the vehicle.

18. The system of claim 16, wherein the sensor array is distributed throughout the physical environment at fixed positions.