MIGRATION PROJECT AUTOMATION, E.G., AUTOMATED SELLING, PLANNING, MIGRATION AND CONFIGURATION OF EMAIL SYSTEMS

• US 20130212200A1
• Filed: 02/13/2013
• Published: 08/15/2013
• Est. Priority Date: 02/13/2012
• Status: Active Grant

• Current Assignee: SkyKick, Inc.
• Original Assignee: SkyKick, Inc.
• Inventors: Dennis, John, Richman, Evan, Schwartz, Todd, Schwartz, Trent, Tett, Rick, Younge, Brad
• Granted Patent: US 10,965,742 B2
• Time in Patent Office: Days
• US Class Current: 709/206
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 51/42 Mailbox-related aspects, e.g. synchronisation of mailboxes
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Technology is generally described for automating the project management and execution of data migration from a source email system to a destination email system. In some examples, the technology can include receiving a domain name of a second computing system; obtaining domain name system (DNS) records for the received domain name; determining an email hosting provider for the second computing system; determining, by the processor, based on the obtained DNS records, an email system service type of the source email system on the second computing system; discovering mailboxes and message delivery rules of the source email system; displaying customization options for migrating discovered source email system mailboxes; migrating data items from the source email system to the destination email system; and managing migration of data from source email system client computing devices to the destination email system.

Claims

1. A computer-readable storage medium storing computer-executable instructions for causing a first computing system having a processor to perform a method for automating a project of data migration from a source email system to a destination email system, comprising:

receiving a domain name of a second computing system;

obtaining domain name system (DNS) records for the received domain name;

determining an email hosting provider for the second computing system;

determining, by the processor, based on the obtained DNS records, an email system service type of the source email system on the second computing system;

discovering mailboxes and message delivery rules of the source email system;

displaying customization options for migrating discovered source email system mailboxes;

migrating data items from the source email system to the destination email system; and

managing migration of data from source email system client computing devices to the destination email system.

2. A method performed by a first computing system having a processor to determine email hosting information about a second computing system, comprising:

receiving a domain name of the second computing system;

querying a database that maps domain names to domain registrars;

receiving, in response to the querying, identification of the domain registrar for the second computing system;

requesting, from the identified domain registrar, domain name system (DNS) records for the received domain name;

receiving, in response to the requesting, DNS records for the received domain name;

accessing a database of known email hosting provider DNS records;

comparing, by the processor, the received DNS records and the database of known email hosting provider DNS records; and

determining, based on the comparing, an email hosting provider for the second computing system.

3. The method of claim 2, wherein receiving a domain name of the second computing system comprises receiving an email address of the second computing system.

4. The method of claim 2, wherein receiving DNS records for the received domain name comprises receiving one or more of an MX record, an Autodiscover record, or an SPF record.

5. The method of claim 2, wherein:

comparing the received DNS records and the database of known email hosting provider DNS records comprises;

searching DNS fields in a predetermined order; and

identifying at least one received DNS record that matches a corresponding known email hosting provider DNS record; and

determining an email hosting provider for the second computing system comprises selecting a known email hosting provider with the first matching record.

6. The method of claim 5, wherein the predetermined order is MX, MX IP Address, Autodiscover Record, SPF Record.

7. The method of claim 2, wherein:

comparing the received DNS records and the database of known email hosting provider DNS records comprises scoring received DNS records that match corresponding known email hosting provider DNS records; and

determining an email hosting provider for the second computing system comprises selecting a known email hosting provider based on the scoring.

8. A method performed by a first computing system having a processor to identify sufficient email system login credentials for an email system having an email system service type, comprising:

obtaining an email address and an email password for the email system;

identifying a second computing system that hosts the email system;

obtaining domain name system (DNS) records for the second computing system;

determining, by the processor, based on the obtained DNS records, an email system service type of the second computing system email system;

deriving, based on the obtained email address and email password and the determined email system service type, email system login credentials; and

connecting to the email system using the derived email system login credentials.

9. The method of claim 8, wherein obtaining an email address and an email password for the email system comprises obtaining no other email system login credentials.

10. The method of claim 8, wherein determining the email system service type of the second computing system email system comprises identifying a DNS record associated with an email system type;

11. The method of claim 10, wherein the DNS record is an Exchange Autodiscover record;

12. The method of claim 8, wherein determining the email system service type of the second computing system email system comprises:

accessing a database of known email hosting provider DNS records and associated known email hosting provider email system service types;

comparing, by the processor, the obtained DNS records and the database of known email hosting provider DNS records;

determining, based on the comparing, an email hosting provider for the second computing system; and

determining an email system service type associated with the email hosting provider.

13. The method of claim 12, wherein determining an email hosting provider for the second computing system comprises selecting a known email hosting provider for which at least one obtained DNS record matches the corresponding known email hosting provider DNS record.

14. The method of claim 12, wherein:

comparing the obtained DNS records and the database of known email hosting provider DNS records comprises scoring obtained DNS records that match corresponding known email hosting provider DNS records; and

determining an email hosting provider for the second computing system comprises selecting a known email hosting provider based on the scoring.

15. The method of claim 8, wherein determining an email system service type comprises selecting a most likely email system service type.

16. The method of claim 8, wherein deriving email system login credentials and connecting to the email system using the derived email system login credentials comprises executing concurrent Internet input/output (I/O) calls;

17. The method of claim 16, further comprising:

assigning a ranking to connections; and

when a successful connection is made, canceling currently executing and scheduled I/O calls for connections with a same or lower ranking.

18. The method of claim 8, further comprising, if the derived email system login credentials are insufficient to connect to the email system:

selecting, based on the email system service type, email system login credentials to request; and

prompting for the selected email system login credentials.

19. The method of claim 18, wherein prompting for the selected email system login credentials comprises requesting a username.

20. The method of claim 18, wherein prompting for the selected email system login credentials comprises requesting a host computer name.

21. The method of claim 8, further comprising:

creating a list of possible email system hosts using the obtained DNS records; and

for each host in the list of hosts;

attempting to connect to the host on a port associated with email systems; and

for each successful connection;

determining a supported communication protocol.

22. The method of claim 21, wherein determining a supported communication protocol comprises:

attempting to log in to the email system using protocol-specific login methods in a predetermined order of communication protocols; and

identifying the protocol associated with the first successful login attempt.

23. A method performed by a computing system having a processor to discover substantially all mailboxes and message delivery rules in an email system, comprising:

obtaining a list of email addresses on the email system; and

classifying, by the processor, each of substantially all the obtained addresses as a mailbox, alias, forwarding rule, or distribution list.

24. The method of claim 23, wherein obtaining a list of email addresses comprises:

receiving administrative credentials for logging in to a control panel for the email system hoster; and

executing a web automation script that logs into the email system hoster control panel, instructs the email system hoster to provide email address data, and parses the email system address data.

25. The method of claim 23, wherein obtaining a list of email addresses comprises:

making Web service calls to the email system for email address resolution using every letter of the location-specific alphabet and digits 0-9;

consolidating returned addresses into a list of unique addresses; and

expanding distribution list addresses.

26. The method of claim 23, wherein obtaining a list of email addresses comprises:

logging into an email account with messages on the email system;

scanning the headers of messages to parse address information from the to;

, from;

, cc;

, and bcc;

fields in each message header; and

identifying addresses that are on the same domain as the email account.

27. The method of claim 23, wherein classifying each of substantially all the obtained addresses as a mailbox, alias, forwarding rule, or distribution list comprises:

attempting to log in to one or more returned address; and

using the login attempt failure response code to distinguish mailbox addresses from distribution group addresses.

28. The method of claim 23, wherein classifying each of substantially all the obtained addresses as a mailbox, alias, forwarding rule, or distribution list comprises:

creating a social graph data structure containing obtained email addresses; and

using, by the processor, the social graph data structure to classify one or more addresses as a mailbox, alias, forwarding rule, or distribution list.

29. The method of claim 28, wherein creating a social graph data structure containing obtained email addresses comprises creating a bi-directed graph in which vertices represent email addresses and/or messages, edges represent relationships based on existence of an address in a message header field, and directionality represents whether a message is going from or coming to a particular email address.

30. The method of claim 29, wherein using the social graph data structure to classify one or more addresses comprises classifying a yet-unclassified address as a mailbox if a number of incoming edges above a certain threshold and a number of outgoing edges above a certain, possibly different, threshold exist to a particular email address vertex, the email address corresponding to that vertex is classified as a mailbox.

31. The method of claim 29, wherein using the social graph data structure to classify one or more addresses comprises classifying a yet-unclassified address as an alias to a mailbox address if at least a threshold number of messages in the mailbox are not connected by any incoming or outgoing edge to the mailbox address, and have exactly one outgoing edge to the yet-unclassified address.

32. The method of claim 29, wherein using the social graph data structure to classify one or more addresses comprises classifying a yet-unclassified address as a distribution list if at least a threshold number of messages in a plurality of mailboxes are not connected by any incoming or outgoing edge to the addresses of the mailboxes they are in, and have exactly one outgoing edge to the yet-unclassified address.

33. The method of claim 29, wherein using the social graph data structure to classify one or more addresses comprises reclassifying a mailbox-classified address as a forwarding rule if at least a threshold number of messages in the mailbox are not connected by any incoming or outgoing edge to the mailbox address, and have exactly one outgoing edge to the yet-unclassified address.

34. A method performed by a computing system having a processor to enable a user to customize an email system migration from a source email system to a destination email system, comprising:

displaying a list of all source email system mailboxes that may be migrated;

for each source email system mailbox in the list;

providing a set of options for routing email addressed to the mailbox after migration to the destination email system; and

providing a set of options for migrating mailbox data to the destination email system; and

ensuring, by the processor, that selected options do not conflict.

35. The method of claim 34, wherein the set of options for routing email addressed to the mailbox after migration to the destination email system comprises routing to the mailbox, an alias, a distribution group, or a shared mailbox, or deleting the mailbox address.

36. The method of claim 34, wherein the set of options for migrating mailbox data to the destination email system comprises migrating data to the mailbox, a different mailbox, or a shared mailbox, or not migrating the mailbox data.

37. A method performed by a computing system having a processor to migrate data items from a source email system to a destination email system, comprising:

creating a data structure for recording items copied to the destination email system;

performing a first migration pass, comprising;

dividing source email system data into batches of item identifiers;

retrieving item metadata from the source email system;

for each item in a batch to be transferred;

assigning a unique item ID to the item;

retrieving item content;

creating an identification hash for the item;

translating the item to an item type compatible with the destination email system;

copying the item to the destination email system;

saving the assigned unique item ID in the item metadata on the destination email system; and

saving in the data structure the identification hash and a reference to the item in the destination email system; and

performing a second migration pass, comprising;

determining, for each item, whether identical items exist on the source email system and the destination email system; and

for each item for which it is determined that identical items do not exist on the source email system and the destination email system, resolving the inconsistency by updating the destination email system.

38. The method of claim 37, wherein the basis for the identification hash includes the item folder path, flags, file length, and at least a portion of the item'"'"'s content.

39. The method of claim 37, wherein the length of the identification hash is defined so that the hash may be assumed to be a unique identifier for each item and its source state.

40. The method of claim 37, wherein determining, for each item, whether identical items exist on the source email system and the destination email system comprises determining the hash for the item, and if it is already in the data structure, updating the data structure to show that the item is still present in the source email system in the previously copied state.

41. The method of claim 37, wherein for each item for which it is determined that identical items do not exist on the source email system and the destination email system, resolving the inconsistency comprises:

for an item whose identification hash is not found in the data structure, copying the item to the destination email system and adding the identification hash to the data structure; and

for an identification hash in the data structure for which the corresponding item is not found in the source email system, deleting the item from the destination email system using the reference to the item contained in the data structure and removing the identification hash from the data structure;

42. A computer-readable storage medium storing computer-executable instructions for causing a computing system having a processor to perform a method for domain name system (DNS) record discovery for an identified domain, comprising:

creating a list of potential DNS label records;

creating a set of discovered DNS records;

for each DNS label in the list;

performing, by the processor, DNS requests for records matching the label;

for each request that returns an alias to a record on the identified domain, recursively performing DNS requests; and

for each request that returns an IP address, adding the record to the set of discovered DNS records;

identifying wildcard DNS records; and

removing DNS records having the same IP address or alias as a wildcard DNS record from the set of discovered DNS records.

43. The computer-readable storage medium of claim 42, further comprising, for each DNS label in the list, for each request that returns an alias to a record on a domain different than the identified domain, adding the record to the set of discovered DNS records.

44. The computer-readable storage medium of claim 42, wherein identifying wildcard DNS records comprises:

creating a string of characters that is extremely unlikely to be a domain name label;

performing a DNS request for a record matching the created string in the identified domain; and

determining, if the performed DNS request returns a successful result, that the returned result is the value of a wildcard DNS record.

45. The computer-readable storage medium of claim 44, wherein the requested record is an A record.

46. The computer-readable storage medium of claim 44, wherein the requested record is a CNAME record.

47. The computer-readable storage medium of claim 44, further comprising executing a web automation script that logs into an identified domain hoster control panel, instructs the domain hoster to provide DNS data for the identified domain, and parses the DNS data.

48. A computer-readable storage medium storing instructions for causing a computing system having a processor to perform a method for managing email system migration from a plurality of source email system client computing devices to a destination email system, comprising:

receiving information about source email system client computing devices;

creating, by the processor, a migration plan for each source email system client computing device; and

transmitting the migration plan to each source email system client computing device, such that the source email system client computing devices copy email system data to the destination email system according to the migration plan.

49. The computer-readable storage medium of claim 48, wherein a migration plan comprises an instruction to copy email system data to the destination email system at a specified time.

50. A computer-readable storage medium storing instructions for causing a source email system client computing device having a processor to perform a method for email system migration to a destination email system, comprising:

transmitting information about the source email system client computing device to a migration coordination system;

receiving a command from the migration coordination system to migrate email system data from the source email system client computing device to the destination email system; and

copying data from a data store on the source email system client computing device to a cached data store that is configured to automatically synchronize data with the destination email system.

51. The computer-readable storage medium of claim 50, wherein the synchronization of the cached data store with the destination email system is performed in a background process.

52. The computer-readable storage medium of claim 50, further comprising:

receiving an indication of a maximum overall size for data to be migrated to the destination email system;

determining, by the processor, the total size of all data to be migrated from the distinguished source email system client computing device; and

if the total size of all data to be migrated from the distinguished source email system client computing device exceeds the maximum overall size for data to be migrated to the destination email system, migrating a portion of the data according to a schedule prioritizing data to be migrated.

53. The computer-readable storage medium of claim 52, wherein determining the total size of all data to be migrated from the distinguished source email system client computing device comprises determining the size of data that matches scope and date criteria for the migration.

54. The computer-readable storage medium of claim 52, wherein, for an email system having Inbox and Sent Items folders, migrating a portion of the data according to a schedule prioritizing data to be migrated comprises:

if the combined size of the Inbox and Sent Items folders is less than the maximum overall size, migrating only the Inbox and Sent Items data to the destination email system; and

if the combined size of the Inbox and Sent Items folders is greater than the maximum overall size, migrating items from the Inbox and Sent Items until the size limit is reached.

55. The computer-readable storage medium of claim 52, further comprising:

creating an archive data store;

copying data exceeding the maximum overall size that is not migrated to the destination email system to the archive data store; and

configuring the destination email system to make the archive data store accessible from the destination email system.

56. A computer-readable storage medium storing computer-executable instructions for causing a computing system having a processor and a network connection to perform a method for managing migration data transfer bandwidth, comprising:

testing, by the processor, the network connection outbound bandwidth;

determining a migration data transfer bandwidth limitation; and

managing migration data transfer bandwidth over the network connection according to the determined migration data transfer bandwidth limitation.

57. The computer-readable storage medium of claim 56, wherein testing the network connection outbound bandwidth comprises uploading a known amount of data to a specific endpoint on the Internet.

58. The computer-readable storage medium of claim 56, wherein determining a migration data transfer bandwidth limitation comprises:

establishing a baseline network connection outbound bandwidth for Internet communication other than migration data transfer; and

subtracting the baseline network connection outbound bandwidth from the tested network connection outbound bandwidth.

59. The computer-readable storage medium of claim 58, wherein the baseline network connection bandwidth is time-dependent.

60. The computer-readable storage medium of claim 56, wherein managing migration data transfer bandwidth comprises establishing no more than a restricted number of concurrent persistent protocol-specific TCP connections for transferring data over the network connection.

61. The computer-readable storage medium of claim 60, wherein the restricted number of connections is time-dependent.

62. The computer-readable storage medium of claim 56, wherein managing migration data transfer bandwidth comprises using thread communication and blocking techniques to throttle network calls when messages over a threshold size are encountered.

63. The computer-readable storage medium of claim 56, wherein managing migration data transfer bandwidth comprises establishing no more than a restricted number of threads for processing migration data to be transferred over the network connection.

64. The computer-readable storage medium of claim 56, wherein managing migration data transfer bandwidth comprises controlling the timing of migration data transfer operations.

65. The computer-readable storage medium of claim 64, wherein controlling the timing of migration data transfer operations comprises scheduling at least one operation to occur only during off-peak hours.

66. The computer-readable storage medium of claim 64, wherein controlling the timing of migration data transfer operations comprises distributing individual account migrations across the period leading up to a specified migration date.

67. The computer-readable storage medium of claim 64, wherein controlling the timing of migration data transfer operations comprises adjusting the specified migration date.