Method and arrangement for identifying and processing commands in digital images, where the user marks the command, for example by encircling it

• US 7,289,110 B2
• Filed: 07/16/2001
• Issued: 10/30/2007
• Est. Priority Date: 07/17/2000
• Status: Expired due to Fees

• Current Assignee: Human Messaging AB
• Original Assignee: Human Messaging AB
• Inventors: Hansson, Mårten
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Shapiro; Leonid
• Application Number:
  US10/333,257
• Publication Number:
  US 20040100508A1
• Time in Patent Office: 2,297 Days
• Field of Search: 345/173, 345/179, 345/522, 178/18, 178/19, 382/314
• US Class Current: 345/173
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: H04M 1/2755 by optical scanning
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/7243 with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72439 for image or video messaging
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00355 Mark-sheet input sensing record carriers G06K7/00; character recognition G06V30/10; document-oriented image-based pattern recognition G06V30/40
• CPC Class Codes: H04N 1/00366 Marks in boxes or the like, e.g. crosses or blacking out H04N1/0036 takes precedence
• CPC Class Codes: H04N 1/00371 on a separate sheet
• CPC Class Codes: H04N 1/00379 Means for enabling correct scanning of a mark sheet or area, e.g. registration or timing marks
• CPC Class Codes: H04N 2101/00 Still video cameras

Abstract

A user captures a digital image of a paper where the user has marked one or several objects or commands with a pen. The marking is done according to a predefined set of marking rules, which eases the separation of objects and commands later on. Examples of rules for marking objects or commands are encircling, underlining and highlighting. The embedded commands in the digital image can be associated with one or several objects in the digital image. The association is done according to a predefined set of association rules. The image is then transmitted to a computing device for processing. The transmission is using a cable, IR or RF technologies like Bluetooth or cellular telephony. The computing device is called “command-server”. It identifies and separates objects and commands by analysing the digital image using the predefined set of marking rules. The command-server also identifies, interprets and associates commands with objects by analysing the digital image according to the predefined sets of marking rules and association rules. Finally the command-server initiate services related to the commands on the objects.

Claims

1. A method for processing information present in a digital image of an original picture, on which at least one graphical object and at least one processing command is marked on the digital image, the at least one processing command is pre-recorded on the digital image prior to processing the information, said method comprising the steps of:

identifying, by using a predefined set of marking rules, the at least one graphical object in the digital image,

identifying, by using the predefined set of marking rules, the at least one pre-recorded processing command in the digital image,

identifying a relation between the at least one pre-recorded processing command and the at least one graphical object wherein the identification is performed by using a predefined set of association rules, and

performing processing relating to the digital image, said processing being dependent on the at least one pre-recorded processing command.

2. A method according to claim 1, wherein the predefined set of marking rules contains at least one of the following marking rules:

encircling the object or command,

drawing a square box around the object or command,

underlining the object or command, and

high-lighting the object or command with a marker pen.

3. A method according to claim 1, wherein the predefined set of association rules contains at least one of the following association rules:

encircling of both the command and object,

drawing a line between the command and the object or their marking graphics, and

drawing a line between the command and the circle or square containing the object.

4. A method according to claim 1, further comprising the step of customizing any of the set of marking rules and the set of association rules.

5. A method according to claim 1, wherein the processing involves initiating at least one of the services:

handwriting recognition of at least one object,

transform at least one object into a selected file format,

send email,

send fax,

print and send letter,

store at least one object for future retrieval,

enter at least one object in to, or synchronize with, a calendar database, an address book database or a “

to-do”

list database,

perform a financial transaction,

dial a phone number,

store a phone number,

send a request to the user to confirm the interpretation of commands or object before any service is initiated, and

fetch one or several digital images from a mass-storage for digital images, for further processing.

16. A computer program, comprising software instructions to perform a method according to claim 1.

6. An arrangement for processing information present in a digital image of an original picture, on which at least one graphical object and at least one processing command is marked on the digital image, the at least one processing command is pre-recorded on the digital image prior to processing the information, said arrangement comprising:

a device for identifying, by using a predefined set of marking rules, the at least one graphical object in the digital image,

a device for identifying, by using the predefined set of marking rules, the at least one pre-recorded processing command in the digital image,

a device for identifying a relation between the at least one pre-recorded processing command and the at least one graphical object wherein the identification is performed by using a predefined set of association rules, and

a device for performing processing relating to the digital image, said processing being dependent on the at least one pre-recorded processing command.

7. An arrangement according to claim 6, wherein the predefined set of marking rules contains at least one of the following marking rules:

encircling the object or command

drawing a square box around the object or command

underlining the object or command, and

high-lighting the object or command with a marker pen.

8. An arrangement according to claim 6, wherein the predefined set of association rules contains at least one of the following association rules:

encircling of both the command and object,

drawing a line between the command and the object or their marking graphics, and

drawing a line between the command and the circle or square containing the object.

9. An arrangement according to claim 6, further comprising a device for customizing any of the set of marking rules and the set of association rules.

10. An arrangement according to claim 6, wherein the device for processing of the object comprises a device for initiating at least one of the services:

handwriting recognition of at least one object,

transform at least one object into a selected file format,

send email,

send fax,

print and send letter,

store at least one object for future retrieval,

enter at least one object in to, or synchronise with, a calendar database, an address book database or a “

to-do”

list database,

perform a financial transaction,

dial a phone number,

store a phone number,

send a request to the user to confirm the interpretation of commands or object before any service is initiated, and

fetch one or several digital images from a mass-storage for digital images, for further processing.

11. A mobile communication terminal capable of processing information present in a digital image of an original picture, on which at least one graphical object and at least one processing command is marked on the digital image, the at least one processing command is pre-recorded on the digital image prior to processing the information, comprising:

a device for obtaining the digital image of the picture,

a device for identifying, by using a predefined set of marking rules, the at least one graphical object in the digital image,

a device for identifying, by using the predefined set of marking rules, the at least one pre-recorded processing command in the digital image,

a device for identifying a relation between the at least one pre-recorded processing command and the at least one graphical object wherein the identification is performed by using a predefined set of association rules, and

a device for performing processing relating to the digital image, said processing being dependent on the at least one pre-recorded processing command.

12. A mobile communication terminal according to claim 11, wherein the predefined set of marking rules contains at least one of the following marking rules:

encircling the object or command,

drawing a square box around the object or command,

underlining the object or command, and

high-lighting the object or command with a marker pen.

13. A mobile communication terminal according to claim 11, wherein the predefined set of association rules contains at least one of the following association rules:

encircling of both the command and object,

drawing a line between the command and the object or their marking graphics, and

drawing a line between the command and the circle or square containing the object.

14. A mobile communication terminal according to claim 11, further comprising a device for customizing any of the set of marking rules and the set of association rules.

15. A mobile communication terminal according to claim 11, wherein a device for processing of the object comprises means for initiating at least one of the services:

handwriting recognition of at least one object,

transform at least one object into a selected file format,

send email,

send fax,

print and send letter,

store at least one object for future retrieval,

enter at least one object in to, or synchronise with, a calendar database, an address book database or a “

to-do”

list database,

perform a financial transaction,

dial a phone number,

store a phone number,

send a request to the user to confirm the interpretation of commands or object before any service is initiated, and

fetch one or several digital images from a mass-storage for digital images, for further processing.