Providing a location and item identification data to visually impaired shoppers in a site having barcode labels

• US 6,497,367 B2
• Filed: 04/26/2001
• Issued: 12/24/2002
• Est. Priority Date: 04/26/2001
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Cox, Aaron Roger, Conzola, Vincent Charles, Sluchak, Thomas John, Ortega, Kerry A
• Primary Examiner(s): Le, Thien M.
• Assistant Examiner(s): St. Cyr, Daniel
• Application Number:
  US09/842,954
• Publication Number:
  US 20020158133A1
• Time in Patent Office: 607 Days
• Field of Search: 235/462.45, 235/462.01, 235/462.08, 235/462.17, 235/462.31, 235/472.01
• US Class Current: 235/462.45
• CPC Class Codes: G06Q 20/4015 using location information
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G07G 1/00 Cash registers alarm indicators G07G3/00
• CPC Class Codes: G07G 1/0045 with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
• CPC Class Codes: G09B 21/006 using audible presentation of the information

Abstract

A portable unit assists a visually impaired user within a store by providing an output, using speech synthesis, of his location based on reading various barcode labels. The location of each barcode label is determined from data stored within the portable unit. The portable unit also determines a path between the user'"'"'s location and an item he selects to find, describing the path using speech synthesis. The user can select, by speech or by depressing a button, items for a target list. Preferably, some barcode labels identify an end of an aisle, which cause the portable unit to describe, using speech synthesis, items on the aisle and items in the target list on the aisle.

Claims

1. A system for providing information within a site, wherein said system comprises:

a plurality of barcode labels placed at a plurality of locations within said site, wherein each barcode label within said plurality of barcode labels includes a pattern of parallel markings representing a coded value, and

a portable device including;

a barcode reader producing electrical signals in response to patterns of said parallel markings;

an audio amplifier;

an audio speaker driven by said audio amplifier;

data storage including a barcode location data structure having a first field containing data representing a plurality of coded values, a second field containing data representing a plurality of values of a first location coordinate, a third field containing data representing a plurality of values of a second location coordinate, wherein each coded value in said plurality of coded values is represented by a pattern of parallel markings on a barcode label within said plurality of barcode labels, wherein each first location coordinate within said second field and each corresponding second location coordinate within said third field form a set of location coordinates representing a location of a label having a corresponding coded value in said first field; and

a processor programmed to generate a most recently read coded value represented by a pattern of parallel markings read by said barcode reader in response to said electrical signals produced by said barcode reader, to find a set of location coordinates within said second and third fields of said first data structure corresponding to said coded value, to generate a pattern of synthetic speech describing a location represented by said set of location coordinates, and to cause said audio amplifier to drive said speaker in response to a generated pattern of synthetic speech.

2. The system of claim 1, wherein

said system additionally comprises a plurality of items placed at a plurality of locations within said site,

said barcode location data structure additionally has a fourth field containing a plurality of descriptions representing said items, wherein each description in said plurality of descriptions is associated with a set of location coordinates in said second and third fields, and wherein said set of location coordinates describes a location of an item described by said description, and

said processor is additionally programmed to accept an input describing an item in said plurality of items, and to determine a location of said item by finding a set of location coordinates associated with a description of said item in said first data structure.

3. The system of claim 2, wherein said processor is additionally programmed to determine a path between a location represented by said set of location coordinates corresponding to said coded value in said data structure and said location of said item, and to generate a pattern of synthetic speech describing said path.

4. The system of claim 2, wherein said processor is additionally programmed to form a list of descriptions of items at a location represented by said set of location coordinates corresponding to said coded value in said data structure, and to generate a pattern of synthetic speech describing said list of descriptions of said items.

5. The system of claim 2, wherein

said portable device additionally includes a microphone, and

said input describing an item in said plurality of items is spoken into said microphone.

6. The system of claim 1, wherein

said system additionally comprises a plurality of items placed at a plurality of locations within said site,

said first data structure additionally has a fourth field containing a plurality of descriptions representing said items, wherein each description in said plurality of descriptions is associated with a set of location coordinates in said second field, and wherein said set of location coordinates describes a location of an item described by said description, and

said processor is additionally programmed to generate a first list of descriptions of items from said fourth field having sets of location coordinates equal to said location coordinates corresponding to said coded value, and to generate a pattern of synthetic speech describing said first list of descriptions.

7. The system of claim 1, wherein

said system additionally comprises a plurality of items placed at a plurality of locations within said site,

each item within said plurality of items includes a barcode label representing a coded value,

said barcode location data structure additionally has a fourth field containing a plurality of descriptions representing said items, wherein each description in said plurality of descriptions is associated with a set of location coordinates in said second and third fields, and wherein said set of location coordinates describes a location of an item described by said description,

said data storage additionally includes a target data structure containing a plurality of coded values, and

said processor is additionally programmed to accept an input describing an item in said plurality of items, to determine a target description corresponding to said input, to find said target description within said plurality of descriptions in said fourth field of said location data structure, and to add a coded value, from said first field of said location data structure corresponding to a description within said plurality of descriptions including said target description, to said target data structure.

8. The system of claim 7, wherein

said plurality of barcode labels include a number of aisle identifying barcode labels placed to identify aisles within said site,

each said first coordinate represents an aisle within said site,

said barcode location data structure additionally includes a fifth field containing data representing a type of barcode label corresponding to each value represented in said first field of said barcode location data structure,

said processor is additionally programmed to determine that said most recently read coded value is from an aisle identifying barcode label by finding said most recently read coded value among data contained in said first field corresponding to data in said fifth field representing an aisle identifying label, then to determine whether said target data field contains data representing a coded value found within said first field of said barcode location data structure corresponding to a location coordinate in second field representing an aisle identified by said aisle identifying bar code label, and then to generate a pattern of synthetic speech describing each item in said plurality of items represented by data in said target data structure.

9. The system of claim 7, wherein said processor is additionally programmed to determine that said most recently read coded value is from a UPC barcode printed on a surface of an item within said plurality of items by finding said most recently read coded value among data contained in said first field corresponding to data in said fifth field representing UPC barcode labels, then to determine that a coded value represented by data in said target data field is equivalent to said most read coded value, and then to delete said coded value represented by data in said target data field from said target data field.

10. The system of claim 1, wherein:

said system additionally comprises a site computer accessing a database containing data representing said coded values and said location coordinates of said barcode labels, and a data transmission link connecting said site computer with said portable device;

said site computer transfers said data over said data transmission link; and

said processor is additionally programmed to receive data from said data transmission link and to store said data within said barcode location data structure.

11. The system of claim 10, wherein said data link includes a transmitter transmitting a radio frequency signal modulated with data from said site computer and a receiver receiving said radio frequency signal at said portable unit.

12. The system of claim 10, wherein said data link includes a transmitter transmitting an infrared beam modulated with data from said site computer and a receiver receiving said infrared beam at said portable unit.

13. The system of claim 10, wherein said data link includes:

a switched telephone network;

a connection between said site computer and said switched telephone network;

a personal computer downloading data from said site computer from said switched telephone network;

a channel transmitting said data from said personal computer to said portable device.

14. A method performed within a portable device for providing information for providing information within a site based on a plurality of barcode labels within said site, wherein said method comprises:

downloading information describing coded values of barcode labels within said site and locations of said barcode labels within said site;

storing said information in a barcode location data structure;

reading a barcode label;

generating a most recently read coded value from said barcode label;

determining a location of said barcode label from said information stored within said barcode location data structure;

generating a pattern of synthetic speech describing said location of said barcode label; and

driving an audio speaker in response to said pattern of synthetic speech describing said location of said barcode label.

15. The method of claim 14, wherein said information downloaded and stored in said barcode location data structure additionally includes data indicating types of said barcode labels and information describing items on which certain of said barcode labels are place, and wherein said method additionally comprises:

storing said location of said barcode label in a user location data structure;

receiving a user input choosing an item;

determining a location of said item from data stored within said barcode location data structure;

storing said location of said item in said user location data structure;

determining a path within said site between said location of said barcode label and said location of said item;

generating a pattern of synthetic speech describing said path; and

driving an audio speaker in response to said pattern of synthetic speech describing said path.

16. The method of claim 14, wherein said information downloaded and stored in said barcode location data structure additionally includes data indicating types of said barcode labels and information describing items on which certain of said barcode labels are place, and wherein said method additionally comprises:

determining that said barcode label identifies an end of an aisle within said site;

determining each item located on said aisle from information stored in said barcode location data structure;

generating a pattern of synthetic speech describing each item determined to be located on said aisle; and

driving an audio speaker in response to said pattern of synthetic speech describing each item determined to be located on said aisle.

17. The method of claim 16, additionally comprising:

accepting a user input selecting an item after driving said audio speaker in response to said pattern of synthetic speech describing each item determined to be located on said aisle; and

storing information describing each item selected in a target data structure.

18. The method of claim 17, additionally comprising, after determining that said barcode label identifies an end of an aisle within said site:

determining each item described by information stored within said target data structure and located on said aisle;

generating a pattern of synthetic speech describing each item described by information stored within said target data structure and located on said aisle; and

driving an audio speaker in response to said pattern of synthetic speech describing each item determined to be described by information within said target data structure and located on said aisle.

19. A method performed within a portable device for providing information within a site based on a plurality of barcode labels within said site, wherein said method comprises:

storing, within a barcode location data structure, a plurality of coded values in a first field, a plurality of first coordinate values in a second field, and a plurality of second coordinate values in a third field, wherein each coded value in said plurality of coded values is represented by a pattern of parallel markings on a barcode label in said plurality of barcode labels, and wherein each first location coordinate within said second field and each corresponding second location coordinate within said third field form a set of location coordinates representing a location of a label having a corresponding coded value in said first field;

reading a barcode label within said plurality of barcode labels;

generating a most recently read coded value from said barcode label most recently read;

finding a stored coded value matching said most recently read coded value from a plurality of coded values stored within said first field of said barcode location data structure;

finding a first location coordinate matching said most recently read coded value within data stored in a second field of said barcode location data structure at a location corresponding to said stored coded value matching said most recently read coded value;

finding a second location coordinate matching said most recently read coded value within data stored in a third field of said barcode location data structure corresponding to said stored coded value matching said most recently read coded value;

generating a pattern of synthetic speech describing a location represented by said first and second location coordinates matching said most recently read coded value; and

driving an audio speaker in response to said pattern of synthetic speech.

20. The method of claim 19, additionally comprising storing said first and second location coordinates matching said most recently read coded value in a first field within a user location data structure.

21. The method of claim 20, additionally comprising:

storing, within a fourth field of said barcode location data structure, a plurality of descriptions of items, wherein each description within said fourth field is descriptive of an item found at location within said site described by coordinates in said second and third fields corresponding to said description within said fourth field;

receiving an input describing an item;

finding a target description corresponding to said input describing an item within data stored in a fourth field of said barcode location data structure;

finding a third location coordinate within data stored in said second field of said barcode location data structure corresponding to said target description;

finding a fourth location coordinate within data stored in said third field of said barcode location data structure corresponding to said target description;

storing said third and fourth location coordinates in a second field of said user location data structure;

setting a flag bit in storage within said portable device;

determining a path within said site between a location described by first and second location coordinates and a location described by said third and fourth location coordinates;

generating a pattern of synthetic speech describing said path; and

driving an audio speaker in response to said pattern of synthetic speech describing said path.

22. The method of claim 21, wherein receiving an input describing an item includes recognizing an input spoken into a microphone of said portable device.

23. The method of claim 21, additionally comprising:

determining that said flag bit is set;

comparing said first and second coordinates found within data stored in said second and third fields of said barcode location data structure corresponding to said most recently read coded value, after reading said barcode label, with location coordinates previously stored in said first field and second fields within said user location data structure;

if said first and second coordinates found within data stored in said second and third fields of said barcode location data structure corresponding to said most recently read coded value are identical to said location coordinates previously stored in said second field within said user location data, generating a pattern of synthetic speech indicating that the location of said item has been reached; and

if said first and second coordinates found within data stored in said second and third fields of said barcode location data structure corresponding to said most recently read coded value are different from said location coordinates previously stored in said first and second fields of said user location data structure, determining a new path within said site between a location described by first and second location coordinates corresponding to said most recently read coded value and a location described by said third and fourth location coordinates;

generating a new pattern of synthetic speech describing said new path; and

driving an audio speaker in response to said new pattern of synthetic speech describing said new path, and storing said first and second coordinates found within data stored in said second and third fields of said barcode location data structure corresponding to said most recently read coded value in said first field of said user location data structure.

24. The method of claim 20, additionally comprising:

storing, within a fourth field of said barcode location data structure, a plurality of descriptions of items, wherein each description within said fourth field is descriptive of an item found at location within said site described by coordinates in said second and third fields corresponding to said description within said fourth field;

receiving an input describing an item;

finding a target description corresponding to said input describing an item within data stored in a fourth field of said barcode location data structure;

finding a coded value within data stored in said first field of said barcode location data structure corresponding to said target description;

finding a third location coordinate within data stored in said second field of said barcode location data structure corresponding to said target description;

finding a fourth location coordinate within data stored in said third field of said barcode location data structure corresponding to said target description;

storing said coded value in said first field of a target data structure;

storing said third location coordinate in a second field of said target data structure; and

storing said fourth location coordinate in a third field of said target data structure.

25. The method of claim 24, wherein receiving an input describing an item includes recognizing an input spoken into a microphone of said portable device.

26. The method of claim 21, additionally comprising:

storing, within a fifth field of said barcode location data structure, a plurality of barcode type identifiers, wherein each barcode type of identifier within said fifth field is descriptive of a type of barcode label corresponding to a value represented in said first field of said barcode location data structure, wherein each coordinate value stored in said second field of said barcode location data structure identifies an aisle in which a barcode label corresponding to a value represented in said first field of said barcode location data structure is found, and wherein an end of aisle identifier stored within said fifth field describes a barcode label placed at an end of an aisle,

determining that said most recently read coded value is from a barcode label placed at an end of an aisle by finding an end of aisle identifier in said fifth field corresponding to said most recently read coded value in said first field of said barcode location data structure;

finding each target coded value within said first field of said target data structure corresponding to a coordinate value within said second field of said target data structure identifying said aisle in which said most recently read barcode label is placed, as determined by a coordinate value within said second field of said barcode location data structure;

for each said target coded value, finding an equivalent coded value in said first field of said barcode location data structure;

for each said equivalent coded value, finding a corresponding target description in said fourth field of said barcode location data structure;

generating a pattern of synthetic speech describing each said target description; and

driving an audio speaker in response to said pattern of synthetic speech.

27. The method of claim 26, additionally comprising, after determining that said most recently read coded value is from a barcode label placed at an end of an aisle by finding an end of aisle identifier in said fifth field corresponding to said most recently read coded value in said first field of said barcode location data structure:

finding a listed coded value within said first field of said barcode location data structure corresponding to a coordinate value within said second field of said barcode location data structure corresponding to a coordinate value within said second field of said barcode location data structure identifying said aisle in which said most recently read barcode label is placed;

for each said listed coded value, finding a corresponding listed description in said fourth field of said barcode location data structure;

generating a pattern of synthetic speech describing each said listed description; and

driving said audio speaker in response to said pattern of synthetic speech describing each said listed description.

28. The method of claim 26, additionally comprising, after driving said audio speaker in response to said pattern of synthetic speech describing a listed description:

accepting a user input selecting said listed description; and

adding a coded value and location coordinates corresponding to said listed description in said barcode data structure to said target data structure.

29. The method of claim 27, additionally comprising:

determining that said most recently read coded value is from a UPC barcode label on an item;

determining that data representing said item is listed in said target data structure; and

erasing said data representing said item from said target data structure.