Authentication and Validation of Smartphone Imagery

• US 20140049653A1
• Filed: 08/20/2013
• Published: 02/20/2014
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Tautachrome, Inc. (The Cianbro Cos.)
• Original Assignee: Tautachrome, Inc. (The Cianbro Cos.)
• Inventors: Leonard, Jon N., Staker, Matthew W., Gille, Robert P., Sercel, Joel C., Davis, Jeffery S.
• Granted Patent: US 9,582,843 B2
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G06T 1/0042 Fragile watermarking, e.g. so as to detect tampering
• CPC Class Codes: G06T 1/005 Robust watermarking, e.g. average attack or collusion attack resistant
• CPC Class Codes: G06T 2201/0064 for copy protection or copy management, e.g. CGMS, copy only once, one-time copy
• CPC Class Codes: H04N 1/00106 using land mobile radio networks, e.g. mobile telephone
• CPC Class Codes: H04N 1/00119 of sound information only
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 1/32272 Encryption or ciphering
• CPC Class Codes: H04N 1/32331 Fragile embedding or watermarking
• CPC Class Codes: H04N 2201/3214 of a date
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3233 of authentication information, e.g. digital signature, watermark
• CPC Class Codes: H04N 2201/3235 Checking or certification of the authentication information, e.g. by comparison with data stored independently
• CPC Class Codes: H04N 2201/3236 Details of authentication information generation
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/3254 Orientation, e.g. landscape or portrait; Location or order of the image data, e.g. in memory
• CPC Class Codes: H04N 2201/3281 Encryption; Ciphering
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence
• CPC Class Codes: H04W 48/16 : Discovering, processing access restriction or access information

Abstract

A method, system, and processor-executable software for processing images or video captured by a smartphone or other digital image capture device for subsequent validation and authentication captures not only the image, but also metadata relevant to the authenticity of validity of the image, such as position, time, camera movement and orientation, image parameters, and so forth. The image is watermarked and both the watermarked image and the metadata, as well as a symmetric key used in the watermarking, are digitally signed and transmitted or uploaded to a web server or authentication centric entity for authentication and storage. When a third party submits an image to the web server or authentication centric entity, the submitted image is compared with the stored watermarked image for authentication and validation of the submitted image, and the metadata is then retrieved to enable analysis of the contents of authenticated and validated image.

Claims

1. A method of processing images or video captured by a digital image capture device for subsequent validation and authentication, comprising the steps of:

the digital still or video image capture device capturing a still or video image and metadata associated with the still or video image;

the digital image capture device generating a symmetric key;

the digital image capture device generating a unique identifier and embedding the unique identifier in the captured image using the symmetric key to form a watermarked image;

the digital image capture device digitally signing the watermarked image, the symmetric key, and the metadata; and

the digital image capture device transmitting the digitally signed watermarked image, symmetric key, and metadata to an authentication centric entity for authentication and validation.

2. A method as claimed in claim 1, wherein the digital image capture device is a smartphone.

3. A method as claimed in claim 1, wherein the metadata includes data on a location at which the image was captured, a time when the image was captured, and additional information associated with the image capture.

4. A method as claimed in claim 3, wherein the additional information includes information obtained from sensors in the image capture device, said information including at least one of velocity and/or direction of movement;

high frequency movements arising from jostling, handling or dropping the device.

5. A method as claimed in claim 3, wherein the additional information includes audio information.

6. A method as claimed in claim 3, wherein the additional information includes network tower and nearby WiFi identification.

7. A method as claimed in claim 3, wherein the additional information includes information on camera settings at the time of image capture.

8. A method as claimed in claim 7, wherein the information on camera settings is in Exif format.

9. A method as claimed in claim 7, wherein the information on camera settings includes at least one of an exposure time and aperture.

10. A method as claimed in claim 3, wherein the additional information includes at least two of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

11. A method as claimed in claim 3, wherein the additional information includes at least three of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

12. A method as claimed in claim 3, wherein the position information is obtained from GPS and/or other location determining sensors or the communications network.

13. A method as claimed in claim 3, wherein the position information is taken from a cell network, from the GPS satellite data, from NIST'"'"'s FM signal, from any of several internet sites, or if no connectivity is available to access these services, then from the smartphone'"'"'s internal clock.

14. A method as claimed in claim 3, wherein the step of creating the watermark comprises the step of embedding the unique identifier in hidden form throughout the captured imagery or of using the unique identifier as a key for finding the hidden watermark through the captured imagery.

15. A method as claimed in claim 3, wherein the symmetric key is a random number, or derived from a random number.

16. A method as claimed in claim 3, wherein the step of digitally signing the watermarked image, the symmetric key, and the metadata comprises the step of encrypting the image using a private key of a public key/private key cryptosystem to generate the digital signature.

17. A method of authenticating and validating an image by an authentication centric entity, comprising the steps of:

receiving from a digital image capture device a digitally signed watermarked still or video image, symmetric key, and metadata;

verifying, by a digital computing device, the authenticity of the watermarked image, symmetric key, and metadata by decrypting the digital signature;

storing the watermarked image, symmetric key, and metadata in at least one data storage device;

upon receiving a submitted image, comparing the submitted image with the stored watermarked image;

if the submitted image matches the stored watermarked image or if differences between the submitted image and the stored watermarked images are acceptable differences, retrieving at least the stored metadata; and

notifying a sender of the submitted image that the image is deemed to be at least potentially authentic and valid, and sending the retrieved metadata to the sender for further analysis.

18. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, wherein acceptable differences include scaling and cropping of the watermarked image.

19. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, further comprising the step of retrieving the stored symmetric key to extract the unique identifier from the stored and submitted images.

20. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, further comprising the step of carrying out additional authentication by comparing Exif-like metadata extracted from the submitted image with stored Exif-like metadata.

21. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, further comprising the step of, following authentication of the digitally signed watermarked image received from the digital image capture device, forwarding the watermarked images to one or more designated recipients or websites.

22. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, wherein the step of authenticating the digitally signed watermarked image is carried out by retrieving a public key and decrypting a digital signature that was encrypted by a private key stored on the digital image capture device and corresponding to the public key.

23. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, further comprising the step of retrieving the stored watermarked image and corresponding metadata by extracting an image identifier from the submitted image.

24. A method of authenticating and validating an image by an authentication centric entity as claimed in claim 17, wherein the stored metadata includes data on a location at which the image was captured, a time when the image was captured, and additional information associated with the image capture.

25. A method of processing images or video captured by a digital image capture device for subsequent validation and authentication, comprising the steps of:

the digital still or video image capture device capturing a still or video image and metadata associated with the still or video image, the metadata includes data on a location at which the image was captured, a time when the image was captured, and additional information associated with the image capture;

the digital image capture device digitally signing the captured image and the metadata; and

the digital image capture device transmitting the digitally signed image and metadata to an authentication centric entity for authentication and validation.

26. A method as claimed in claim 25, wherein the digital image capture device is a smartphone.

27. A method as claimed in claim 25, wherein the additional information includes information obtained from sensors in the image capture device, said information including at least one of velocity and/or direction of movement;

high frequency movements arising from jostling, handling or dropping the device.

28. A method as claimed in claim 25, wherein the additional information includes audio information.

29. A method as claimed in claim 25, wherein the additional information includes network tower and nearby WiFi identification.

30. A method as claimed in claim 25, wherein the additional information includes information on camera settings at the time of image capture.

31. A method as claimed in claim 30, wherein the information on camera settings is in Exif format.

32. A method as claimed in claim 30, wherein the information on camera settings includes at least one of an exposure time and aperture.

33. A method as claimed in claim 25, wherein the additional information includes at least two of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

34. A method as claimed in claim 25, wherein the additional information includes at least three of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

35. A method as claimed in claim 25, wherein the position information is obtained from GPS and/or other location determining sensors or the communications network.

36. A method as claimed in claim 25, wherein the position information is taken from a cell network, from the GPS satellite data, from NIST'"'"'s FM signal, from any of several internet sites, or if no connectivity is available to access these services, then from the smartphone'"'"'s internal clock.

37. Processor-executed software including instructions for executing a method of processing images or video captured by a digital image capture device for subsequent validation and authentication, the method executed by the software including the steps of:

the digital still or video image capture device capturing a still or video image and metadata associated with the still or video image;

the digital image capture device generating a symmetric key;

the digital image capture device generating a unique identifier and embedding the unique identifier in the captured image using the symmetric key to form a watermarked image;

the digital image capture device digitally signing the watermarked image, the symmetric key, and the metadata; and

the digital image capture device transmitting the digitally signed watermarked image, symmetric key, and metadata to an authentication centric entity for authentication and validation.

38. Processor-executed software as claimed in claim 37, wherein the digital image capture device is a smartphone.

39. Processor-executed software as claimed in claim 37, wherein the metadata includes data on a location at which the image was captured, a time when the image was captured, and additional information associated with the image capture.

40. Processor-executed software as claimed in claim 39, wherein the additional information includes information obtained from sensors in the image capture device, said information including at least one of velocity and/or direction of movement;

high frequency movements arising from jostling, handling or dropping the device.

41. Processor-executed software as claimed in claim 39, wherein the additional information includes audio information.

42. Processor-executed software as claimed in claim 39, wherein the additional information includes network tower and nearby WiFi identification.

43. Processor-executed software as claimed in claim 39, wherein the additional information includes information on camera settings at the time of image capture.

44. Processor-executed software as claimed in claim 43, wherein the information on camera settings is in Exif format.

45. Processor-executed software as claimed in claim 43, wherein the information on camera settings includes at least one of an exposure time and aperture.

46. Processor-executed software as claimed in claim 39, wherein the additional information includes at least two of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

47. Processor-executed software as claimed in claim 39, wherein the additional information includes at least three of the following information types of:

information on orientation or movement of the camera device, network tower or WiFi information, audio, and camera settings.

48. Processor-executed software as claimed in claim 39, wherein the position information is obtained from GPS and/or other location determining sensors or the communications network.

49. Processor-executed software as claimed in claim 39, wherein the position information is taken from a cell network, from the GPS satellite data, from NIST'"'"'s FM signal, from any of several internet sites, or if no connectivity is available to access these services, then from the smartphone'"'"'s internal clock.

50. Processor-executed software as claimed in claim 39, wherein the step of creating the watermark comprises the step of embedding the unique identifier in hidden form throughout the captured image or of using the unique identifier as a key for finding the hidden watermark through the captured imagery.

51. Processor-executed software as claimed in claim 39, wherein the symmetric key is a random number, or derived from a random number.

52. Processor-executed software as claimed in claim 39, wherein the step of digitally signing the watermarked image, the symmetric key, and the metadata comprises the step of encrypting the image using a private key of a public key/private key cryptosystem to generate the digital signature.

53. An authentication centric entity including at least one server, the at least one server being connected to receive and authenticate digitally-signed watermarked images from any of a plurality of registered image capture devices, and to receive submitted image authentication and validation requests from users, the authentication centric entity including hardware and processor-executed software for carrying out the following steps:

receiving from a digital image capture device a digitally signed watermarked still or video image, symmetric key, and metadata;

verifying, by a digital computing device, the authenticity of the watermarked image, symmetric key, and metadata by decrypting the digital signature;

storing the watermarked image, symmetric key, and metadata in at least one data storage device;

upon receiving a submitted image, comparing the submitted image with the stored watermarked image;

if the submitted image matches the stored watermarked image or if differences between the submitted image and the stored watermarked images are acceptable differences, retrieving at least the stored metadata; and

notifying a sender of the submitted image that the image is deemed to be at least potentially authentic and valid, and sending the retrieved metadata to the sender for further analysis.

54. An authentication centric entity as claimed in claim 53, wherein acceptable differences include scaling and cropping of the watermarked image.

55. An authentication centric entity as claimed in claim 53, further comprising the step of retrieving the stored symmetric key to extract the unique identifier from the stored and submitted images.

56. An authentication centric entity as claimed in claim 53, further comprising the step of carrying out additional authentication by comparing Exif-like metadata extracted from the submitted image with stored Exif-like metadata.

57. An authentication centric entity as claimed in claim 53, further comprising the step of, following authentication of the digitally signed watermarked image received from the digital image capture device, forwarding the watermarked images to one or more designated recipients or websites.

58. An authentication centric entity as claimed in claim 53, wherein the step of authenticating the digitally signed watermarked image is carried out by retrieving a public key and decrypting a digital signature that was encrypted by a private key stored on the digital image capture device and corresponding to the public key.

59. An authentication centric entity as claimed in claim 53, further comprising the step of retrieving the stored watermarked image and corresponding metadata by extracting an image identifier from the submitted image.

60. An authentication centric entity as claimed in claim 53, wherein the stored metadata includes data on a location at which the image was captured, a time when the image was captured, and additional information associated with the image capture.