Electronic notebook system

• US 10,484,845 B2
• Filed: 01/17/2019
• Issued: 11/19/2019
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: The Notebook, LLC
• Original Assignee: The Notebook, LLC
• Inventors: Khaleghi, Karen Elaine
• Primary Examiner(s): Chuang, Jung-Mu T
• Application Number:
  US16/250,279
• Publication Number:
  US 20190149962A1
• Time in Patent Office: 306 Days
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/72 for transmitting results of analysis
• CPC Class Codes: G16H 20/00 ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
• CPC Class Codes: G16H 40/20 for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: G16H 80/00 ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 4/14 Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]

Abstract

An electronic notebook system is described that comprises a housing, a computing device, wireless interfaces, antennas, sensors, a touch display configured to receive input via a stylus and/or human digit input, the stylus comprising a pressure and/or an inclination sensor, a microphone, camera, the notebook system configured to provide a user condition interface, receive a user selection of a first user condition, provide an interface configured to receive user details, receive audible user details via the microphone, convert the audible user details received via the microphone to text, perform natural language processing to identify text keywords utilizing sentence segmentation, part-of-speech tagging, paraphrase recognition, and/or co-reference resolution, identify a condition based at least in part on the identified one or more keywords, dynamically generate an alert based at least in part on the identified condition, wirelessly transmit the generated alert to one or more destinations via at least a first wireless interface and antenna.

Claims

1. A communication system, the communication system comprising:

a processor device;

non-transitory computer readable medium that stores instructions configured to cause the processor device to perform operations comprising;

establishing an encrypted transmission over a wireless network between the communication system and a cloud system using a secure transfer mechanism;

authenticating a user using an authentication system, the authentication system configured to;

capture an image of the user using a camera,

extract features of the user from the image,

determine whether the extracted features correspond to features of an authorized user, and if so, identify the user as the authorized user;

or

scan of a fingerprint of the user using a fingerprint scanner,

determine whether the fingerprint of the user using the communication system corresponds to an authorized user, and if so, identify the user as the authorized user;

wirelessly accessing, using the secure transfer mechanism, user medical data from a plurality of sources stored on the cloud system at least partly in response to authenticating that the user is the authorized user;

generating a user interface rendering at least a portion of the user medical data from the plurality of sources using a zoomable patient health timeline configured to be zoomed in response to a gesture made using two fingers, the zoomable patient health timeline comprising;

medical appointment entries,

medical diagnosis entries, and

treatment entries;

detecting a user selection of a first medical appointment on the generated zoomable patient health timeline;

in response to detecting the user selection of the first medical appointment on the generated zoomable patient health timeline, opening and displaying a corresponding calendar entry, including time, date, and location information;

in response to receiving a user zoom instruction on to the zoomable patient health timeline to select a first portion of the timeline, the user zoom instruction comprising a first gesture, causing the first portion of the zoomable patient health timeline to be displayed at a first resolution in a first region together with a display of the original zoomable patient health timeline at a second resolution displayed in a second region separate from the first region;

receiving new medical data;

utilizing the secure transfer mechanism to wirelessly synchronize the received new medical data with user medical data stored on the cloud system at partly in response to detecting a first event;

enabling the zoomable patient health timeline to be updated with the new medical data, wherein the new medical data is visually emphasized relative to other health timeline data for a first period of time;

storing a first calendar entry, including a second medical appointment timing, associated with a second medical appointment;

receiving one or more questions from the user as a voice input for the second medical appointment;

using a voice-to-text application hosted by the cloud system to convert the voice input from the user to text, the text comprising questions;

receiving, over the wireless network from the cloud system, the text comprising questions, corresponding to the voice input from the user;

populating one or more user interface fields in a first user interface using the text received from the cloud system;

providing an interface enabling the user to share the text received from the cloud system with a first recipient;

detecting an occurrence of the second medical appointment timing associated with the second medical appointment; and

at least partly in response to detecting the occurrence of the second medical appointment timing associated with the second medical appointment, displaying at least a portion of the text comprising questions received from the cloud system via a display.

2. The communication system as defined in claim 1, the operations further comprising:

generating a data sharing user interface configured to share information from a plurality of sources regarding medical matters with other users;

the data sharing user interface comprising;

a search field configured to receive a user search query enabling the user to search through postings regarding the medical matters;

a search filter user interface comprising predefined search filters, the search filters including one or more medical diagnosis or medical conditions;

a posting field configured to receive information to be posted;

a tag field configured to receive a tag to be associated with the received information to be posted;

a favorite control that enables the user to designate a posting or a posting user as a favorite;

conducting, using the cloud system, a search using the user search query and one or more user-selected search filters; and

providing search results filtered in accordance with the user-selected one or more search filters for display.

3. The communication system as defined in claim 1, the operations further comprising:

encrypting user data;

storing the encrypted user data in a first computer system; and

enabling a key associated with the encrypted user data to be stored on a computer system physically separate from the first computer system.

4. The communication system as defined in claim 1, the operations further comprising:

converting, using the cloud system, the voice input from voice to text using natural language processing,

the natural language processing comprising determining a probability of certain words or sounds following on from one another to thereby speed up word recognition and improve recognition accuracy.

5. The communication system as defined in claim 1, the operations further comprising:

converting, using the cloud system, the voice input from voice to text via a natural language processing system comprising a neural networking trained to recognize patterns comprising word sounds to improve the natural language processing system'"'"'s ability to understand the voice input.

6. The communication system as defined in claim 1, the operations further comprising:

providing a team user interface that enables;

a medical team comprising a plurality of team members to be defined for the user;

respective tasks to be associated with respective members of the medical team;

a designation of a team leader;

receiving, via the team user interface, a medical team definition and a team leader designation;

using a second voice input from the user in detecting a first user condition;

enabling an alert, including text corresponding to at least a portion of the voice input from the user, to be transmitted to at least the designated team leader; and

enabling members of the defined medical team to communicate with each other and access data related to the user.

7. The communication system as defined in claim 1, the operations further comprising:

accessing user data comprising;

diagnosis dates,

medical intervention dates,

generating the zoomable patient health timeline comprising a plurality of items using the accessed user data;

detecting a user selection of an item from the plurality of items on the generated zoomable patient health timeline; and

causing additional information regarding the zoomable patient health timeline item selection to be presented.

8. The communication system as defined in claim 1, the operations further comprising:

generating a calendar entry for a third medical appointment;

recording audio content related to the third medical appointment using a microphone of the communication system;

transmitting the audio content to the cloud system using the secure transfer mechanism;

using the cloud system to perform natural language processing comprising language modeling and statistical analysis to convert the audio content to text; and

adding the text, one or more medical test results, and one or more prescriptions to the calendar entry for the third medical appointment.

9. The communication system as defined in claim 1, the operations further comprising:

recording audio content of a medical professional at a third medical appointment;

transmitting the audio content to the cloud system using the secure transfer mechanism;

using the cloud system to perform natural language processing comprising language modeling and statistical analysis to convert the audio content of the medical professional to text;

determining, using the cloud system, if the text includes a recommendation; and

at least partly in response to a determination that the text includes the recommendation, adding the recommendation to an electronic to-do list.

10. The communication system as defined in claim 1, the operations further comprising:

generating a calendar entry for a third medical appointment;

recording video and audio content related to the third medical appointment using the communication system camera and microphone; and

associating a timestamp with the recorded video and audio content; and

adding the video and audio content to the calendar entry for the third medical appointment.

11. The communication system as defined in claim 1, the operations further comprising:

receiving referral contact information for a first medical provider;

automatically augmenting at least one calendar entry with at least a portion of the referral contact information.

12. A computer implemented method, the method comprising:

establishing, using a network interface of a mobile computer system, an encrypted transmission over a wireless network between the mobile computer system and a cloud system using a secure transfer mechanism;

authenticating a user by using an authentication system of the mobile computer system configured to;

capture an image of the user using a mobile computer system camera,

extract features of the user from the image,

determine whether the extracted features correspond to features of

an authorized user, and if so, identify the user as the authorized user;

or

scan of a fingerprint of the user using a fingerprint scanner,

determine whether the fingerprint of the user using the mobile computer system corresponds to an authorized user, and if so, identify the user as the authorized user;

enabling the mobile computer system to wirelessly access, using the secure transfer mechanism, user medical data from a plurality of sources stored on the cloud system at least partly in response to authenticating that the user is the authorized user;

generating a user interface rendering at least a portion of the user medical data from the plurality of sources using a zoomable timeline configured to provide a quick overview of the user'"'"'s medical history , the zoomable timeline comprising;

medical appointment entries,

medical diagnosis entries, and

treatment entries;

detecting a user selection of a first medical appointment on the generated zoomable timeline;

in response to detecting the user selection of the first medical appointment on the generated zoomable timeline, opening and displaying a corresponding calendar entry, including time, date, and location information;

in response to receiving a user zoom instruction on the generated zoomable timeline to select a first portion of the generated zoomable timeline, the user zoom instruction comprising a first gesture, causing the first portion of the generated zoomable timeline to be displayed at a first resolution in a first region together with a display of the original generated zoomable timeline at a second resolution displayed in a second region separate from the first region;

receiving new medical data via the mobile computer system;

utilizing the secure transfer mechanism to wirelessly synchronize the received new medical data on the mobile computer system with data stored on the cloud system at partly in response to detecting a first event;

enabling the zoomable timeline to be updated with the new medical data, wherein the new medical data is visually emphasized relative to other zoomable timeline data for a first period of time;

storing a first calendar entry associated with a second medical appointment associated with a second medical appointment timing;

receiving a voice input from the user for the second medical appointment;

using a voice-to-text application hosted by the cloud system to convert the voice input from the user to text;

receiving, over the wireless network from the cloud system, text corresponding to the voice input from the user;

populating one or more user interface fields in a first user interface using the text received from the cloud system;

providing an interface enabling the user to share the text received from the cloud system with a first recipient;

detecting an occurrence of the second medical appointment timing associated with the second medical appointment; and

at least partly in response to detecting the occurrence of the second medical appointment timing associated with the second medical appointment, enabling at least a portion of the text to be displayed.

13. The method as defined in claim 12, the method further comprising:

at least partly in response to detecting the occurrence of the second medical appointment timing associated with the second medical appointment, displaying at least a portion of the text, comprising user questions, received from the cloud system via a display of the mobile computer system.

14. The method as defined in claim 12, the method further comprising:

receiving at the cloud system a task;

identifying a source of data associated with the task; and

assigning the task to a cloud node based at least in part on a geo proximity of the cloud node to the source of data associated with the task.

15. The method as defined in claim 12, the method further comprising:

encrypting user data;

storing the encrypted user data in a first computer system; and

storing a key associated with the encrypted user data on a computer system physically separate from the first computer system.

16. The method as defined in claim 12, the method further comprising:

converting, using the cloud system, the voice input from voice to text using natural language processing,

the natural language processing comprising determining a probability of certain words or sounds following one from one another to thereby speed up word recognition and improve recognition accuracy.

17. The method as defined in claim 12, the method further comprising:

converting, using the cloud system, the voice input from voice to text via a natural language processing system comprising a neural networking trained to recognize patterns comprising word sounds to improve the natural language processing system'"'"'s ability to understand the voice input.

18. The method as defined in claim 12, the method further comprising:

providing a team user interface that enables;

a medical team comprising a plurality of team members to be defined for the user;

respective tasks to be associated with respective members of the medical team;

a designation of a team leader;

receiving, via the team user interface, a medical team definition and a team leader designation;

detecting, using a second voice input from the user, a first user condition;

transmitting an alert, including text corresponding to at least a portion of the voice input from the user to at least the designated team leader; and

enabling members of the defined medical team to communicate with each other and access data related to the user.

19. The method as defined in claim 12, the method further comprising:

accessing user data comprising;

diagnosis dates, and

medical intervention dates,

generating the zoomable timeline comprising a plurality of items using the accessed user data;

detecting a user selection of an item on the generated zoomable timeline; and

causing additional information regarding the timeline item selection to be presented.

20. The method as defined in claim 12, the method further comprising:

generating a calendar entry for a third medical appointment;

recording audio content related to the third medical appointment using a microphone of the mobile computer system;

transmitting the audio content from the mobile computer system using the network interface to the cloud system using the secure transfer mechanism;

using a trainable computer model hosted on the cloud system to perform natural language processing comprising language modeling and statistical analysis to convert the audio content to text; and

adding the text, one or more medical test results, and one or more prescriptions to the calendar entry for the third medical appointment.

21. The method as defined in claim 12, the method further comprising:

recording audio content of a medical professional at a third medical appointment using a microphone of the mobile computer system;

transmitting the audio content from the mobile computer system using the network interface to the cloud system using the secure transfer mechanism;

using the cloud system to perform natural language processing comprising language modeling and statistical analysis to convert the audio content of the medical professional to text;

determining, using the cloud system, if the text includes a recommendation; and

at least partly in response to a determination that the text includes the recommendation, adding the recommendation to an electronic to-do list.

22. The method as defined in claim 12, the method further comprising:

generating a calendar entry for a third medical appointment;

recording video and audio content related to the third medical appointment using the mobile computer system camera and microphone; and

associating a timestamp with the recorded video and audio content; and

adding the video and audio content to the calendar entry for the third medical appointment.

23. The method as defined in claim 12, the method further comprising:

receiving referral contact information for a first medical provider;

automatically augmenting at least one calendar entry with at least a portion of the referral contact information.

24. A computer implemented method, the method comprising:

initiating, using a network interface of a computer system, an encrypted transmission over a wireless network between the computer system and a cloud system using a secure transfer mechanism;

authenticating a user;

enabling the computer system to wirelessly access, using the secure transfer mechanism, user medical data from a plurality of sources stored on the cloud system at least partly in response to authenticating that the user is the authorized user;

generating a user interface rendering at least a portion of the user medical data from the plurality of sources using a zoomable timeline having a first resolution, the zoomable timeline configured to provide an overview of the user'"'"'s medical history, the zoomable timeline comprising;

medical appointment entries,

detecting a user selection of a first medical appointment on the generated zoomable timeline;

in response to detecting the user selection of the first medical appointment on the generated zoomable timeline, opening and displaying a corresponding calendar entry, including time, date, and location information;

receiving new medical data via the computer system;

utilizing the secure transfer mechanism to wirelessly synchronize the received new medical data on the computer system with data stored on the cloud system at partly in response to detecting a first event;

enabling the zoomable timeline to be updated with the new medical data, wherein the new medical data is visually emphasized relative to other timeline data for a first period of time;

storing a first calendar entry, including a second medical appointment timing, associated with a second medical appointment;

receiving a voice input from the user for the second medical appointment;

using a voice-to-text application hosted by the cloud system to convert the voice input from the user to text;

receiving, over the wireless network from the cloud system, text corresponding to the voice input from the user;

populating one or more user interface fields in a first user interface using the text received from the cloud system;

providing an interface enabling the user to share the text received from the cloud system with a first recipient;

detecting an occurrence of the second medical appointment timing associated with the second medical appointment; and

at least partly in response to detecting the occurrence of the second medical appointment timing associated with the second medical appointment, enabling at least a portion of the text to be displayed;

causing the user to be prompted, using a first prompt, to provide medical status information;

receiving, via a microphone on the computer system, a first spoken response to the first prompt from the user, the first spoken response comprising a first sequence of words;

transmitting the first spoken response to the first prompt from the computer system over the network to the cloud system using the secure transfer mechanism;

performing, using the cloud system, natural language processing, comprising language modeling and statistical analysis, on the received first spoken response comprising the first sequence of words to convert the first spoken response to text;

identifying, using the cloud system, one or more keywords in the first sequence of words to at least determine if the first sequence of words includes patient concerns;

generating, using the cloud system, a criticality score using the keywords identified in the first sequence of words and respective criticality keyword weightings, wherein a first keyword in the first sequence of words is associated with a first criticality weight and a second keyword in the first sequence of words is associated with a second criticality weight;

identifying, using the cloud system, an urgent user condition based at least in part on the criticality score generated using the keywords identified in the first sequence of words and the respective keyword criticality weightings;

dynamically generating, using the cloud system, an alert based at least in part on the identified user urgent condition, wherein the dynamically generated alert comprises one or more of the keywords identified in the first sequence of words; and

transmitting, using the cloud system, the dynamically generated alert, including the one or more of the keywords identified in the first sequence of words, using the network interface to one or more destinations.

25. Non-transitory memory that stores instructions that when executed by a computer system are configured to cause the computer system to perform operation comprising:

establishing an encrypted transmission over a wireless network between the computer system and a remote system using a secure transfer mechanism;

authenticating a user using an authentication system, the authentication system configured to;

capture an image of the user using a camera,

extract features of the user from the image,

determine whether the extracted features correspond to features of an authorized user, and if so, identify the user as the authorized user;

or

scan of a fingerprint of the user using a fingerprint scanner,

determine whether the fingerprint of the user using the computer system corresponds to an authorized user, and if so, identify the user as the authorized user;

wirelessly accessing, using the secure transfer mechanism, user medical data from a plurality of sources stored on the remote system at least partly in response to authenticating that the user is the authorized user;

generating a user interface rendering at least a portion of the user medical data from the plurality of sources using a zoomable timeline configured to be zoomed in response to a gesture made using two fingers, the zoomable timeline comprising;

medical appointment entries,

medical diagnosis entries, and

treatment entries;

detecting a user selection of a first medical appointment on the generated zoomable timeline;

in response to detecting the user selection of the first medical appointment on the generated zoomable timeline, opening and displaying a corresponding calendar entry, including time, date, and location information;

in response to receiving a zoom instruction on the generated zoomable timeline to select a first portion of the generated zoomable timeline, the user zoom instruction comprising a first gesture, causing the first portion of the zoomable timeline to be displayed in a first region at a first resolution together with a display of the original zoomable timeline at a second resolution displayed in a second region separate from the first region;

receiving new medical data;

utilizing the secure transfer mechanism to wirelessly synchronize the received new medical data with data stored on the remote system at partly in response to detecting a first event;

enabling the zoomable timeline to be updated with the new medical data, wherein the new medical data is visually emphasized relative to other timeline data for a first period of time;

storing a first calendar entry, including a second medical appointment timing, associated with a second medical appointment;

receiving a voice input from the user for the second medical appointment;

using a voice-to-text application hosted by the remote system to convert the voice input from the user to text;

receiving, over the wireless network from the remote system, text corresponding to the voice input from the user;

populating one or more user interface fields in a first user interface using the text received from the remote system;

detecting an occurrence of the second medical appointment timing associated with the second medical appointment; and

at least partly in response to detecting the occurrence of the second medical appointment timing associated with the second medical appointment, displaying at least a portion of the text received from the remote system via a display.

26. The non-transitory memory as defined in claim 25, the operations further comprising:

generating a data sharing user interface configured to share information from a plurality of sources regarding medical matters with other users;

the sharing user interface comprising;

a search field configured to receive a user search query enabling a user to search through postings regarding the medical matters;

a search filter user interface comprising predefined search filters, the search filters including one or more medical diagnosis or medical conditions;

a posting field configured to receive information to be posted;

a tag field configured to receive a tag to be associated with the received information to be posted;

a favorite control that enables the user to designate a posting or a posting user as a favorite;

initiating a search using the user search query and one or more user-selected search filters; and

providing search results filtered in accordance with the user-selected one or more search filters for display.

27. The non-transitory memory as defined in claim 25, the operations further comprising:

converting, using the remote system, the voice input from voice to text using natural language processing,

the natural language processing comprising determining a probability of certain words or sounds following on from one another to thereby speed up word recognition and improve recognition accuracy.

28. The non-transitory memory as defined in claim 25, the operations further comprising:

accessing user data comprising;

diagnosis dates, and

medical intervention dates,

generating the zoomable timeline comprising a plurality of items using the accessed user data;

detecting a user selection of an item on the generated zoomable timeline; and

causing additional information regarding the timeline item selection to be presented.

29. The non-transitory memory as defined in claim 25, the operations further comprising:

recording audio content of a medical professional at a third medical appointment;

transmitting the audio content to the remote system using the secure transfer mechanism;

using the remote system to perform natural language processing comprising language modeling and statistical analysis to convert the audio content of the medical professional to text;

determining, using the remote system, if the text includes a recommendation; and

at least partly in response to a determination that the text includes the recommendation, adding the recommendation to an electronic to-do list.