System and method for unmanned aerial vehicle navigation for inventory management

• US 10,217,180 B2
• Filed: 07/27/2017
• Issued: 02/26/2019
• Est. Priority Date: 07/29/2016
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: Gubbi Lakshminarasimha, Jayavardhana Rama, Kandaswamy, Gopi, Purushothaman, Balamuralidhar, Anilkumar, Achanna, Nelliamkunnath, Sandeep, Reddy, Pavan Kumar
• Primary Examiner(s): Tissot, Adam D
• Application Number:
  US15/661,526
• Publication Number:
  US 20180033111A1
• Time in Patent Office: 579 Days
• Field of Search: None
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06K 19/06037 multi-dimensional coding
• CPC Class Codes: G06K 19/06075 the marking containing means for error correction
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

This disclosure relates generally to inventory management, and more particularly to system and method for airborne shelf inspection and inventor management. When a UAV has to be navigated based on navigation information embedded in visual markers on different items in the inventory, and if one or more of the visual markers are not completely visible due to occlusion, then the UAV automatically recovers data that is missing due to the occlusion, and accordingly navigates the UAV.

Claims

1. A processor-implemented method, comprising:

capturing, via one or more hardware processors, at least one image of visual marker on a target, by an image capturing module of the UAV;

determining, via the one or more hardware processors, whether entire area of the visual marker is covered in the captured image, by an image processing module of the UAV;

recovering, via the one or more hardware processors, missing area of the visual marker, if the entire area of the visual marker is found to be not visible in the captured image, by said image processing module; and

navigating the UAV automatically, via the one or more hardware processors, based on at least one data decoded from the visual marker after recovering the missing area of the visual marker, by a navigation module of the UAV.

2. The method as claimed in claim 1, wherein said navigation of the UAV involves localization of the UAV within an environment where the UAV is present, wherein said localization comprises of:

generating a map of the environment, by an exploratory module of the UAV, by traversing entire area of the environment, wherein the map is generated in terms of position a plurality of visual markers in the environment; and

identifying location of the UAV, with respect to at least one visual marker in the proximity of the UAV and the map, by a localization module of the UAV.

3. The method as claimed in claim 1, wherein recovering missing area of the visual marker further comprises of:

extracting parity codes corresponding to data being encoded in the visual marker, from the captured image, by the image processing module; and

calculating values of the missing area of the visual marker, based on the extracted parity codes, by the image processing module.

4. The method as claimed in claim 1, wherein the visual marker is polygonal in shape, wherein the visual marker comprises of:

a binary code representing information pertaining to at least the target, within a plurality of borders of the polygon; and

a set of parity codes corresponding to the binary code, wherein said binary code is reconstructed from said set of parity codes when needed, said set of parity codes are encoded in blocks surrounding the binary code, within all borders of the polygon, wherein the parity codes adjacent to all borders of the polygon represent same values.

5. An Unmanned Aerial Vehicle (UAV), said UAV comprising:

a hardware processor; and

a storage medium comprising a plurality of instructions, said plurality of instructions causing the hardware processor to;

capture at least one image of visual marker on a target, by an image capturing module of the UAV;

determine whether entire area of the visual marker is covered in the captured image, by an image processing module of the UAV;

recover missing area of the visual marker, if the entire area of the visual marker is found to be not visible in the captured image, by said image processing module; and

navigate the UAV automatically, based on at least one data decoded from the visual marker after recovering the missing area of the visual marker, by a navigation module of the UAV.

6. The UAV as claimed in claim 5, wherein said navigation module is configured to navigate the UAV based on localization of the UAV within an environment where the UAV is present, by:

generating a map of the environment, by an exploratory module of the UAV, by traversing entire area of the environment, wherein the map is generated in terms of position a plurality of visual markers in the environment; and

identifying location of the UAV, with respect to at least one visual marker in the proximity of the UAV and the map, by a localization module of the UAV.

7. The UAV as claimed in claim 5, wherein the image processing module is configured to recover missing area of the visual marker by:

extract parity codes corresponding to data being encoded in the visual marker, from the captured image; and

calculate values of the missing area of the visual marker, based on the extracted parity codes, by the image processing module.

8. A non-transitory computer readable medium embodying a program executable in a computing device, the program comprising:

a program code for capturing, via one or more hardware processors, at least one image of visual marker on a target, by an image capturing module of the UAV;

determining, via the one or more hardware processors, whether entire area of the visual marker is covered in the captured image, by an image processing module of the UAV;

recovering, via the one or more hardware processors, missing area of the visual marker, if the entire area of the visual marker is found to be not visible in the captured image, by said image processing module; and

navigating the UAV automatically, via the one or more hardware processors, based on at least one data decoded from the visual marker after recovering the missing area of the visual marker, by a navigation module of the UAV.