Methods and systems for object identification and interaction

• US 7,178,718 B2
• Filed: 04/09/2003
• Issued: 02/20/2007
• Est. Priority Date: 05/25/1999
• Status: Expired due to Fees

• Current Assignee: Silverbrook Research Pty Limited
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Silverbrook, Kia, Lapstun, Paul, Walmsley, Simon Robert
• Primary Examiner(s): Nguyen, Kimberly D.
• Application Number:
  US10/409,848
• Publication Number:
  US 20030195820A1
• Time in Patent Office: 1,413 Days
• Field of Search: 235/375, 235/381, 235/383, 235/385, 705/23, 705/10, 705/14, 705/17, 345/179, 345/175, 345/166, 345/574, 178/18.09, 178 1901- 1905
• US Class Current: 235/375
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

A method of facilitating an interaction between a user and a product item, the product item having an identity and the method including a number of steps. The first step involves providing the user with an interface surface associated with the product item and containing information relating to the product item. The interface surface includes coded data indicative of the identity of the product item and of a number of reference points of the interface surface. The next step involves receiving, in a computer system, indicating data from a sensing device regarding the identity of the product item and a position of the sensing device relative to the interface surface. The sensing device, when placed in an operative position relative to the interface surface, sensing the coded data in the vicinity of the sensing device and generating the indicating data using the sensed coded data. The next step involves facilitating, in the computer system and with reference to the indicating data, the interaction between the user and the product item.

Claims

1. A method of facilitating an interaction between a user and a product item, the product item having an identity and the method including the steps of:

providing the user with an interface surface associated with the product item and containing information relating to the product item, the interface surface having disposed thereon coded data indicative of;

(i) a product ID identifying said product item, (ii) a serial number uniquely identifying said product item within said class, and (iii) a plurality of map references of the interface surface;

receiving, in a computer system, indicating data from a sensing device regarding the class of product item, the serial number of said product item and a position of the sensing device relative to the interface surface, the sensing device, when placed in an operative position relative to the interface surface, the sensing at least some of the coded data in the vicinity of the sensing device and generating the indicating data using at least some of the sensed coded data; and

facilitating, in the computer system and with reference to the indicating data, the interaction between the user and the product item.

2. The method of claim 1 in which the interaction is associated with at least one zone of the interface surface and in which the method includes identifying, in the computer system and from the zone relative to which the sensing device is located, the interaction.

3. The method of claim 2 which includes:

receiving, in the computer system, data regarding movement of the sensing device relative to the interface surface, the sensing device sensing its movement relative to the interface surface using at least some of the coded data; and

identifying, in the computer system and from the movement being at least partially within the at least one zone, the interaction.

4. The method of claim 1, wherein said coded data is further indicative of a parameter of the interaction, and said method includes the steps of:

receiving, in the computer system and from the sensing device, indicating data regarding the parameter of the interaction and movement data regarding movement of the sensing device relative to the interface surface, the sensing device, when moved relative to the interface surface, sensing at least some of the coded data and generating the indicating data and the movement data using at least some of the sensed coded data; and

interpreting, in the computer system, the indicating data and the movement data as it relates to the interaction.

5. The method of claim 1, further comprising the step of:

receiving, in the computer system, user data from a sensing device regarding an identity of the user, the sensing device being programmed with the user data; and

facilitating, in the computer system and with reference to the user data and the indicating data, the interaction between the user and to product item.

6. The method of any one of claims 1, 4 or 5 in which the interaction is selected from to group comprising:

(a) providing product information about the product item to the user;

(b) recording a purchase transaction relating to the product item;

(c) recording a potential purchase transaction relating to the product item;

(d) providing comparison information to the user, the comparison information comparing the product information about the product item with product information about another product item;

(e) playing a game associated with the product item; and

(f) conducting a competition in relation to the product item.

7. A method according to claim 6 wherein the product information comprises information relating to any one of the product item'"'"'s:

(a) cost;

(b) contents;

(c) weight;

(d) place of origin;

(e) manufacturer;

(f) date of manufacture;

(g) date of packaging;

(h) use-by date;

(i) currant owner; and

(j) dimensions.

8. A method according to any one of claims 1, 4 or 5 wherein the interface surface is selected from the group comprising:

(a) a label;

(b) a package; and

(c) a surface of the product item itself.

9. A method according to any one of claims 1, 4 or 5 wherein the coded data is substantially invisible to the average unaided human.

10. A method according to any one of claims 1, 4 or 5 wherein the coded data is disposed over a substantial portion of the interface surface.

11. A method according to claim 10 wherein the coded data is disposed over more than 20% of the interface surface.

12. A method according to claim 10 wherein the coded data is disposed over more than 90% of the interface surface.

13. The method of any one of claims 1, 4 or 5 wherein the method further comprises identifying, in the computer system, that the user has entered handwritten text data by means of the sensing device and effecting, in the computer system, an operation associated with the handwritten text data.

14. The method of claim 13 which includes converting, in the computer system, the handwritten text data to computer text.

15. The method of any one of claims 1, 4 or 5 wherein the method further comprises identifying, in the computer system, that the user has entered a handwritten signature by means of the sensing device and effecting, in to computer system, an operation associated with the handwritten signature.

16. The method of claim 15 which includes verifying, in the computer system, that the signature is that of the user.

17. The method of claim 15 in which the operation associated with the handwritten signature is associated with payment authorization.

18. The method of any one of claims 1, 4 or 5 wherein the method further comprises identifying, in the computer system, that the user has entered a hand-drawn picture by means of the sensing device and effecting, in the computer system, an operation associated with the hand-drawn picture.

19. The method of any one of claims 1, 4 or 5 in which:

a portion of to coded data is superimposed with a visual graphic, the visual graphic being at least partially indicative, to the user, of the interaction.

20. The method of claim 1 or claim 4 in which the sensing device contains an identifier which imparts a unique identity to the sensing device and identifies it as belonging to the user and in which the method includes monitoring, in the computer system, the identifier.

21. The method of any one of claims 1, 4 or 5 which includes providing all required information relating to the interaction in the interface surface to eliminate the need for a separate display device.

22. The method of any one of claims 1, 4 or 5 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout encoding an orientation codeword comprising a sequence of an integer multiple m of n symbols, where m is one or more, each encoded symbol being distributed at n locations about a center of rotational symmetry of the layout such that decoding the symbols at each of the n orientations of the layout produces n representations of the orientation codeword, each representation comprising a different cyclic shift of the orientation codeword and being indicative of the degree of rotation of the layout, and wherein the orientation codeword is fault tolerant.

23. The method of any one of claims 1, 4 or 5 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout including n identical sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each sub-layout including rotation-indicating data that distinguishes the rotation of that sub-layout from the rotation of at least one other sub-layout within the layout.

24. The method of any one of claims 1, 4 or 5 wherein the coded data is disposed on the interface surface in accordance with a layout having n-fold rotational symmetry, where n is at least two, the layout including n identical first sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each first sub-layout including rotation-indicating data that distinguishes the rotation of that first sub-layout from the rotation of at least one other first sub-layout within the layout.

25. A system for facilitating an interaction between a user and a product item, the product item having an identity and the system including:

a interface surface associated with the product item and containing information relating to the product item, the interface suite having disposed thereon coded data indicative of;

(i) a product ID identifying a class of said product item, (ii) a serial number uniquely identifying said product item within said class, (iii) a plurality of map references of the interface surface; and

a computer system adapted to facilitate the interaction in response to receiving indicating data from a sensing device, the indicating data being indicative of the class of product item, the serial number of said product item and a position of the sensing device relative to the interface surface, the sensing device, when placed in an operative position relative to the interface surface, sensing at least some of the coded data in the vicinity of the sensing device and generating the indicating data using at least some of the sensed coded data.

26. The system of claim 25 in which the interaction is associated with at least one zone of the interface surface.

27. The system of claim 25 which includes the sensing device, the sensing device sensing its movement relative to the interface surface using at least some of the coded data.

28. The system of claim 25, wherein said coded data is further indicative of a parameter of the interaction, and said computer system is adapted to:

receive, from the sensing device, indicating data regarding the parameter of the interaction and movement data regarding movement of the sensing device relative to the interface surface; and

interpret the movement of the sensing device as it relates to the interaction, the sensing device, when moved relative to the interface surface, sensing at least some of the coded data and generating the indicating data and the movement data using at least some of the sensed coded data.

29. The system of claim 25, further wherein:

the computer system is adapted to receive from the sensing device user data regarding an identity of the user, and for facilitating, with reference to the user data and the indicating data, the interaction between the user and the product item, the sensing device being programmed with the user data.

30. The system of any one of claims 25, 28 or 29 in which the interaction is selected from the group comprising:

(a) providing product information about the product item to the user;

(b) recording a purchase transaction relating to the product item;

(c) recording a potential purchase transaction relating to the product item;

(d) providing comparison information to the user, the comparison information comparing the product information about the product item with product information about another product item;

(e) playing a game associated with the product item; and

(f) conducting a competition in relation to the product item.

31. The system of claim 30 wherein the product information comprises information relating to any one of the product item'"'"'s:

(a) cost;

(b) contents;

(c) weight;

(d) place of origin;

(e) manufacturer;

(f) date of manufacture;

(g) date of packaging;

(h) use-by date;

(i) current owner; and

(j) dimensions.

32. The system of any one of claims 25, 28 or 29 wherein the interface surface is selected from the group comprising:

(a) a label;

(b) a package; and

(c) a surface of the product item itself.

33. The system of any one of claims 25, 28 or 29 wherein the coded data is substantially invisible to the average unaided human.

34. The system of any one of claims 25, 28 or 29 wherein the coded data is disposed over a substantial portion of the interface surface.

35. The system of claim 34 wherein the coded data is disposed over more than 20% of the interface surface.

36. The system of claim 34 wherein the coded data is disposed over more than 90% of the interface surface.

37. The system of any one of claims 25, 28 and 29 in which the sensing device includes a marking nib.

38. The system of any one of claims 25, 28 and 29 in which the sensing device contains an identifier which imparts a unique identity to the sensing device and identifies it as belonging to the user.

39. The system of any one of claims 25, 28 or 29 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout encoding an orientation codeword comprising a sequence of an integer multiple m of n symbols, where m is one or more, each encoded symbol being distributed at a locations about a center of rotational symmetry of the layout such that decoding the symbols at each of then orientations of the layout produces n representations of the orientation codeword, each representation comprising a different cyclic shift of the orientation codeword and being indicative of the degree of rotation of the layout, and wherein the orientation codeword is fault tolerant.

40. The system of any one of claims 25, 28 or 29 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout including a identical sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each sub-layout including rotation-indicating data that distinguishes the rotation of that sub-layout from the rotation of at least one other sub-layout within the layout.

41. The system of any one of claims 25, 28 or 29 wherein the coded data is disposed on the interface surface in accordance with a layout having n-fold rotational symmetry, where n is at least two, the layout including n identical first sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each first sub-layout including rotation-indicating data that distinguishes the rotation of that first sub-layout from the rotation of at least one other first sub-layout within the layout.

42. An interactive product item adapted for interaction with a user via a sensing device and a computer system, and comprising:

a product item having an identity;

an interface surface associated with the product item and having disposed thereon information relating to the product item and coded data indicative of;

(i) a product ID identifying a class of said product item, (ii) a serial number uniquely identifying said product item within said class, (iii) and a plurality of map references of the interface surface; and

receiving, in a computer system, indicating data from a sensing device regarding the class of product item, the serial number of said product item and a position of the sensing device relative to the interface surface, the sensing device, when placed in an operative position relative to the interface surface, sensing at least some of the coded data in the vicinity of the sensing device and generating the indicating data using at least some of the sensed coded data.

43. The interactive product item of claim 42 wherein the coded data is further indicative of a parameter of an interaction.

44. The interactive product item of claim 42 wherein the interface surface is selected from the group comprising:

(a) a label;

(b) a package; and

(c) a surface of the product item itself.

45. The interactive product item of claim 44 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout encoding an orientation codeword comprising a sequence of an integer multiple m of n symbols, where m is one or more, each encoded symbol being distributed at n locations about a center of rotational symmetry of the layout such that decoding the symbols at each of the n orientations of the layout produces n representations of the orientation codeword, each representation comprising a different cyclic shift of the orientation codeword and being indicative of the degree of rotation of the layout, and wherein the orientation codeword is fault tolerant.

46. The interactive product item of claim 44 wherein the coded data is disposed on the interface surface in accordance with a layout, the layout having at least order n rotational symmetry, where n is at least two, the layout including n identical sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each sub-layout including rotation-indicating data that distinguishes the rotation of that sub-layout from the rotation of at least one other sub-layout within the layout.

47. The interactive product item of claim 44 wherein the coded data is disposed on the interface surface in accordance with a layout having n-fold rotational symmetry, where n is at least two, the layout including n identical first sub-layouts rotated 1/n revolutions apart about a center of rotational symmetry of the layout, the coded data disposed in accordance with each first sub-layout including rotation-indicating data that distinguishes the rotation of that first sub-layout from the rotation of at least one other first sub-layout within the layout.