U.S. patent application number 14/488979 was filed with the patent office on 2015-07-02 for unified communication device.
This patent application is currently assigned to T-Mobile USA, Inc.. The applicant listed for this patent is T-Mobile USA, Inc.. Invention is credited to Kanakrai Gajendra Chauhan, Omar Hassan.
Application Number | 20150188956 14/488979 |
Document ID | / |
Family ID | 53482250 |
Filed Date | 2015-07-02 |
United States Patent
Application |
20150188956 |
Kind Code |
A1 |
Chauhan; Kanakrai Gajendra ;
et al. |
July 2, 2015 |
Unified Communication Device
Abstract
In some implementations, a computing device may receive a
user-selection to initiate a real-time communication session with a
second computing device. The first computing device may
authenticate user credentials. The computing device may send a
request to the second computing device to initiate the real-time
communication session using a web browser application. The
real-time communication session may be initiated in response to
receiving an acceptance message to the request. The real-time
communication session may include at least one of an audio stream
or a video stream.
Inventors: |
Chauhan; Kanakrai Gajendra;
(Bellevue, WA) ; Hassan; Omar; (Kirkland,
WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
T-Mobile USA, Inc. |
Bellevue |
WA |
US |
|
|
Assignee: |
T-Mobile USA, Inc.
|
Family ID: |
53482250 |
Appl. No.: |
14/488979 |
Filed: |
September 17, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61921215 |
Dec 27, 2013 |
|
|
|
Current U.S.
Class: |
726/7 ;
709/204 |
Current CPC
Class: |
G06Q 30/08 20130101;
H04L 63/0861 20130101; H04L 29/06 20130101; H04L 65/607 20130101;
H04L 65/403 20130101; G06Q 30/016 20130101; H04L 29/08072 20130101;
H04L 63/08 20130101; G06F 13/10 20130101; G06Q 30/012 20130101;
H04L 67/22 20130101; H04L 69/329 20130101; G06Q 10/087 20130101;
G06Q 30/0281 20130101 |
International
Class: |
H04L 29/06 20060101
H04L029/06 |
Claims
1. A method performed by one or more processors configured with
specific instructions, the method comprising: receiving, at a first
computing device, a user-selection to initiate a real-time
communication session with a second computing device;
authenticating, at the first computing device, user credentials;
sending a request to the second computing device to initiate the
real-time communication session using a web browser application;
initiating the real-time communication session in response to
receiving an acceptance message to the request, the acceptance
message received from the second computing device; and securely
displaying a document that is associated with the real-time
communication session.
2. The method of claim 1, wherein the real-time communication
session includes at least one of: a first video stream being sent
from the first computing device to the second computing device
substantially in real-time; a video audio stream being sent from
the second computing device to the first computing device
substantially in real-time; a first audio stream being sent from
the first computing device to the second computing device
substantially in real-time; or a second audio stream being sent
from the second computing device to the first computing device
substantially in real-time.
3. The method of claim 1, wherein: a first user is associated with
the first computing device; a second user is associated with the
second computing device; and the document includes confidential
information associated with at least one of the first user or the
second user.
4. The method of claim 3, wherein: the second user is capable of
viewing the document; and the first user is incapable of viewing
the document.
5. The method of claim 4, wherein: the second user is capable of
viewing and annotating the document; and the first user is capable
of viewing but not annotating the document.
6. The method of claim 1, wherein the real-time communication
session is established without using a browser plugin.
7. The method of claim 1, wherein the real-time communication
session comprises a WebRTC session.
8. One or more computer readable media storing instructions that
are executable by one or more processors to perform acts
comprising: authenticating, using biometric authentication, one or
more user credentials; requesting a real-time communication session
between a first computing devices and a second computing device
using a web browser without using browser plugins; initiating the
real-time communication session; retrieving a document associated
with the communication session, the document including confidential
information associated with a first user associated with the first
computing device; and displaying the document on the second
computing device.
9. The one or more computer readable media of claim 8, wherein the
user credentials include at least one of a username, a password, a
passcode, or a biometric input.
10. The one or more computer readable media of claim 8, wherein the
first user is incapable of viewing the document.
11. The one or more computer readable media of claim 8, the acts
further comprising: receiving, from the second computing device,
annotations associated with the document.
12. The one or more computer readable media of claim 11, wherein
the first user is capable of viewing the document but incapable of
viewing the annotations.
13. The one or more computer readable media of claim 8, wherein the
real-time communication session includes a security layer to
encrypt an outgoing media stream and to decrypt an incoming media
stream.
14. A tablet computing device, comprising: one or more processors;
and one or more computer readable media to store instructions that
are executable by the one or more processors to display a user
interface comprising: a real-time communication application to
initiate a real-time communication session; a viewing window to
view multiple media streams associated with the real-time
communication session; a document retrieval window to display one
or more documents associated with the real-time communication
session; a document sharing window to display documents that are
securely shared between the tablet computing device and other
computing devices participating in the real-time session; a
collaboration application to enable two or more of the other
computing devices participating in the real-time communication
session to modify a document; and a calendar application to
schedule appointments and provide reminders of the scheduled
appointments.
15. The tablet computing device of claim 14, the user interface
further comprising: a recording application to: record the multiple
media streams associated with the real-time communication session
to create a recorded session; and store the recorded session for
playback at a later point in time.
16. The tablet computing device of claim 15, further comprising: an
imaging device to generate a video stream for transmission to the
second computing device and to the third computing device.
17. The tablet computing device of claim 15, further comprising: a
microphone to generate an audio stream for transmission to the
second computing device and to the third computing device.
18. The tablet computing device of claim 14, wherein the real-time
communication session uses peer-to-peer networking for
communications between the first computing device, the second
computing device, and the third computing device.
19. The tablet computing device of claim 18, wherein the
peer-to-peer networking is achieved using at least one of:
Interactive Connectivity Establishment (ICE); Session Traversal
Utilities for Network address translation (STUN); or Traversal
Using Relays around Network address translation (TURN).
20. The tablet computing device of claim 14, the user interface
further comprising: a customer relationship manager to display a
plurality of views of contacts.
Description
RELATED APPLICATIONS
[0001] This application claims priority to U.S. provisional patent
application No. 61/921,215, entitled "WebRTC" and filed on Dec. 27,
2013. Application No. 61/921,215 is fully incorporated herein by
this reference.
BACKGROUND
[0002] Despite the proliferation of computing devices, such as
tablets and wireless phones, current Business-to-business (B2B) and
business-to-consumer (B2C) communications are still relatively
primitive. For example, most B2B or B2C communications take place
in-person, over the phone, or using electronic mail ("email").
BRIEF DESCRIPTION OF THE DRAWINGS
[0003] The detailed description is set forth with reference to the
accompanying figures. In the figures, the left-most digit(s) of a
reference number identifies the figure in which the reference
number first appears. The use of the same reference numbers in
different figures indicates similar or identical items.
[0004] FIG. 1 is a block diagram illustrating a system that
includes a device with a real-time communications application
according to some implementations.
[0005] FIG. 2 is a block diagram illustrating a computing device
architecture that includes a real-time communications application
according to some implementations.
[0006] FIG. 3 is a block diagram illustrating a system in which two
or more devices are participating in a real-time communication
session according to some implementations.
[0007] FIG. 4 is a flow diagram of an example process that includes
sending a request to initiate a real-time communication session
according to some implementations.
[0008] FIG. 5 is a flow diagram of an example process that includes
receiving a request to initiate a real-time communication session
according to some implementations.
[0009] FIG. 6 is a block diagram illustrating a computing device
participating in a real-time communication session according to
some implementations.
DETAILED DESCRIPTION
[0010] The system and techniques described herein enable devices to
communicate with each other in real-time. For example, a real-time
communications (RTC) application, such as WebRTC, may enable
multiple devices to communicate with each other in real-time,
thereby facilitating more meaningful B2B and B2C
communications.
[0011] An RTC application may be installed or integrated into
devices, such as wireless phones, tablets, and other computing
devices, to provide a secure application for communication that
includes voice, video, chat, document access, and other types of
communication. For example, an RTC application may be customized to
provide communications between patients and healthcare
professionals, such as doctors and nurses. To illustrate, a patient
may be provided with a patient device that includes a customized
RTC application. The user interface (UI) may enable the patient to
initiate a communication (e.g., audio and/or video) session with a
healthcare professional, such as a doctor or a nurse. The RTC
application may request the patient's credentials (e.g., username
and password), authenticate the patient's identity, retrieve a set
of document(s) associated with the patient, and display the
document(s) to the healthcare professional. The document(s) that
are displayed may include a document that includes confidential
information, e.g., information that may be shared with a select
number of individuals but may not be shared with other individuals.
In some cases, the doctor may view at least one document that the
patient is not capable of viewing. For example, the doctor may view
a scan (e.g., an ultrasound, or a radiological scan) that includes
annotations from another physician (e.g., radiologist). The doctor
may share a portion of the document with the patient. For example,
the doctor may share the scan but not the physician annotations
with the patient. The documents that are displayed may be retrieved
using various criteria. For example, the patient may use the RTC
application at a first point in time. When the patient initiates a
call to the healthcare professional at a second point in time, the
RTC application may identify and retrieve patient data (e.g., blood
test results, results from an x-ray or ultrasound, etc.) associated
with the patient that were created after the first point in time.
Thus, when the patient initiates a communication session with the
healthcare professional, the RTC application may determine an
identity of the patient, and retrieve and display patient data
associated with the patient to the healthcare professional.
[0012] The healthcare professional may view the patient data on a
device in which the RTC application (e.g., WebRTC) has been
installed. For example, the healthcare professional may receive a
request from a patient to initiate an audio or video communication
session. When the healthcare professional accepts the request, the
audio or video communication session may begin. After the
communication session begins, the RTC application may display the
patient data on the device. In addition, the RTC application may
enable the healthcare professional to retrieve additional patient
data. For example, after the RTC application displays a current
test result, the doctor may use the RTC application to retrieve and
display a previous test result. The patient data that the doctor
can view using the RTC application may include patient data (e.g.,
test results, x-ray images, ultrasound videos, etc.) stored in a
database and real-time patient data. For example, the patient may
wear a monitoring device that, substantially in real-time,
periodically (e.g., at a pre-determined interval) measures various
physiological signs and transmits them securely to the patient's
device. The physiological signs that are measured and transmitted
may include body temperature, blood pressure level, blood sugar
level, heart rate, another physiological measurement associated
with a human, or any combination thereof. When the patient
initiates a communication session with the doctor, the RTC
application may enable the doctor to view the physiological signs
being received by the patient device. The physiological signs
transmitted from the monitoring device to the patient device may be
stored on the patient device. In some cases, the patient device may
periodically transmit the stored physiological signs to a remote
server, such as a hospital database.
[0013] The RTC application may enable multi-user communication
sessions (e.g., audio or video conference call) between two or more
people. For example, while communicating with a patient, a doctor
may use the RTC application to create a three-way communication
session with a specialist (e.g., cardiologist, nephrologist, or
other medical specialist) by initiating a communications request to
the specialist. As another example, a nurse may use the RTC
application to create a three-way communication session with a
doctor. As yet another example, a nurse may create a 3-way
communication session with a doctor and the doctor may create a
4-way communication session by initiating communications with a
specialist. The RTC application may enable a subset of the parties
included in the communication session to temporarily engage in a
discussion while excluding one or more of the parties. For example,
during a 3-way communication between the patient, the nurse, and
the doctor, the doctor may temporarily exclude the patient while
the doctor discusses a matter with the nurse. As another example,
during a 4-way communication between the patient, the nurse, the
doctor, and the specialist, the doctor and the specialist may
temporarily exclude the patient and the nurse from the discussion.
For example, the doctor and the specialist may discuss treatment
options while temporarily excluding the patient and the nurse from
the discussion.
[0014] Of course, other types of communications besides
doctor-patient communications are possible. Other examples of
communication sessions that may be facilitated with the RTC
application include customer-salesperson communications,
teacher-student communications, lawyer-client communications, etc.
For example, in a customer-salesperson communication session, the
RTC application may enable the salesperson to view the customer's
purchase history, information about previous purchases, etc. while
communicating with the customer. In a teacher-student communication
session, the teacher may view a lesson plan along with the student,
review test results, conference in another person (e.g., a guidance
counselor, another teacher, the school principal, the student's
patent(s), etc.).
[0015] The RTC application may also provide additional features,
such as capture (e.g., recording) and playback of communication
sessions. For example, a patient may record a communication session
with a doctor when the doctor is providing advice (e.g., "Take one
pill in the morning and another pill at night") to enable the
patient to playback the session and refresh the patient's memory
regarding the doctor's advice. The RTC application may also enable
signature capture, image capture, and other features to capture and
store audio, video, images, or any combination thereof.
[0016] To provide secure communications, a layer of security may be
added to access the RTC application. For example, a token-based
security system may require a caller (e.g., patient) to enter a
code before a communication session can be initiated. An Internet
Protocol Multimedia Subsystem (IMS) stack may provide
authentication, provisioning, and other services. In some cases, a
vendor may provide a customized IMS stack for a specific usage
scenario. For example, a first customized IMS stack may be provided
for medical usage (e.g., doctor-patient communications), a second
customized IMS stack may be provided for sales usage (e.g.,
customer-salesperson communications), a third customized IMS stack
may be provided for student-teacher usage, and so on.
[0017] In some implementations, a vendor, such as a network service
provider, may provide a customized RTC application for download. In
other implementations, the vendor may hard code a customized RTC
application into devices. For example, the customized RTC
application may be integrated into a firmware of the device or an
operating system of the device such that the customized RTC
application can be modified by authorized personnel (or an
authorized software agent) but cannot be modified by an end user.
As another example, the customized RTC application may be stored in
a protected portion of memory that can be modified by authorized
personnel (or an authorized software agent) but cannot be modified
by an end user.
[0018] Thus, a vendor (e.g., a network service provider) may
provide an RTC application for download or integrate the RTC
application into a computing device to enable two or more people to
participate in a communication session. The vendor may customize
the RTC application for a particular type of business, a particular
type of situation, particular types of participants, etc. The
communication session may enable a participant to present various
types of data to other participants, including one or more of audio
data, video data, still images, stored data, real-time data, and/or
other types of data. The communication session may be secured
(e.g., securely transmitted) such that non-participants cannot
access the data that is being exchanged or accessed.
[0019] FIG. 1 is a block diagram illustrating a system 100 that
includes a device with a real-time communications application
according to some implementations. The system 100 includes one or
more computing devices 102(1) to 102(N) (where N>0) that are
communicatively coupled to a server 104 via a network 106. The
network 106 may include one or more wireless networks and/or wired
networks. The wireless networks may use various protocols, such as
one or more of global system for mobile (GSM), code division
multiple access (CDMA), long term evolution (LTE), general packet
radio system (GPRS), enhanced data rates for GSM evolution (EDGE),
802.11, Bluetooth, and the like. The wired networks may use various
protocols, such as one or more of data over cable service interface
specification (DOCSIS), digital subscriber line (DSL), fiber
optics, Ethernet, or similar protocols.
[0020] Each of the computing devices 102 may include one or more
processors 108 and one or more computer readable media 110. The one
or more processors 108 may include one or more hardware devices,
such as integrated circuits, capable of executing software
instructions. The computing devices 102 may include a wireless
phone, a tablet computing device, a media playback device, a still
and/or video camera, another type of computing device or any
combination thereof. The one or more computer readable media 110
may include various types of non-transitory memory storage devices,
such as one or more of read-only memory (ROM), random access memory
(RAM), solid state drives (SSD), disk-based storage, etc.
[0021] The computer readable media 110 may be used to store
instructions 112 that are executable by the processors 108 to
perform various functions, such as, for example, establishing a
secure communications session between two or more of the computing
devices 102. The computer readable media 110 may also include a
communications application 114, an authentication layer 116, a
security layer 118, a real-time communication (RTC) application
120, a browser 122 (e.g., web browser application), and an internet
protocol (IP) multimedia subsystem (IMS) 124. The communications
application 114 may be used to establish secure communications
between two or more of the computing devices 102. The secure
communications may include both real-time communications and
accessing stored data.
[0022] The authentication layer 116 may authenticate credentials of
a user associated with the computing device. In some cases, based
on an identity of the user, particular features and/or applications
may be available while other features and/or applications may be
unavailable. For example, after authenticating a doctor's
credentials, the authentication layer 116 of the computing device
102(1) may enable the doctor to access patient data associated with
multiple patients (e.g., the doctor's patients). After
authenticating a patient's credentials, the authentication layer
116 of the computing device 102(2) may enable the patient to access
the patient's own data but not patient data associated with the
doctor's other patients. Credentials may be provided by a user
using an input device such as (i) a keyboard (e.g., by entering a
username, password, etc.) or (ii) a biometric input device, such as
a fingerprint scanner, retinal or iris scanner, or an imaging
device (e.g., for use with facial recognition). Of course, other
types of input devices may be used for providing input for
authentication of a user.
[0023] The security layer 118 may provide secure communications by
encrypting data that is sent from the computing devices 102 and
decrypting data received by the computing devices. For example,
during a communications session between computing devices 102(1)
and 102(2), the security layer 118 of the computing device 102(1)
may encrypt data being transmitted from the computing device 102(1)
and decrypt data received from the computing device 102(2). The
security layer 118 may provide security using one or more
techniques, including token-based security.
[0024] The RTC application 120 may enable a user to establish a
real-time communication session with one or more other users. The
RTC application 120 may support browser-to-browser applications for
voice calling, video chat, and peer-to-peer (P2P) file sharing
without the use of browser plug-ins. The RTC application 120 may
use the browser 122 to access a camera, a microphone, a keyboard, a
stylus, or other input device associated with the computing devices
102 to capture images, audio, and/or video media. The RTC
application 120 may use the browser 122 to share data between two
or more of the computing devices 102 via peer-to-peer
networking.
[0025] The server 104 may include one or more processors 126 and
one or more computer readable media 128. The computer readable
media 128 may store instructions 130, one or more databases 132, a
content manager 134, a directory 136, a session initiation protocol
(SIP) proxy 138, and a module 140 for interactive connectivity
establishment (ICE), Session Traversal Utilities for Network
address translation (STUN), and Traversal Using Relays around
Network address translation (TURN). The instructions 130 may be
executable by the one or more processors 126 to perform various
functions, including enabling a secure communications session
between two or more of the computing devices 102. The database 132
may be used to store (e.g., record) data associated with RTC
sessions, user data (e.g., patient data, sales data, test data,
test results data, etc.), or other types of data. The content
manager 134 may manage the content associated with an RTC session,
including managing the audio streams, managing video streams, etc.
The directory 136 may enable accessing and maintaining distributed
directory information services over an Internet Protocol (IP)
network. For example, when a patient, using the computing device
102(1), requests (e.g., initiates) a communications session with a
doctor, the directory 136 may be used to identify an IP address
associated with the computing device 102(N) associated with the
doctor.
[0026] ICE may enable a browser (e.g., the browser 122) to connect
with peers, e.g., by bypassing firewalls that would prevent opening
connections, providing a unique address if one of the computing
devices 102 does not have a public internet protocol (IP) address,
and relaying data if a router does not permit direct connection
with peers. STUN may discover a public IP address and determine any
restrictions that would prevent a direct connection with a peer.
Network address transaction (NAT) may be used to provide one of the
computing devices 102 with a public IP address. For example, a
router may have a public IP address while devices connected to the
router may have private IP addresses. When a P2P connection is
requested, the request may be translated from the computing
device's private IP address to the router's public IP along with a
unique port identifier. In this way, a device is discoverable on
the network 106 without having a unique public IP address.
[0027] Some routers using NAT employ a restriction called
`Symmetric NAT` where the router will only accept connections from
peers with which a computing device has previously connected. TURN
may enable a computing device to bypass a Symmetric NAT restriction
by opening a connection with a TURN server and relaying all
information through TURN.
[0028] One or more monitoring devices 142 may be connected to the
network 106. For example, in a medical setting, the monitoring
devices 142 may include a device that monitors a patient's
physiological conditions (e.g., body temperature, blood pressure
level, blood sugar level, heart rate, etc.). The monitoring devices
142 may generate real-time data 144 that is capable of being viewed
using the RTC application 120. For example, during a communication
session between a doctor and a patient, the doctor may access and
view the real-time data 144 associated with one or more of the
monitoring devices 142 that are monitoring physiological conditions
of the patient.
[0029] One or more additional databases 146 may be connected to the
network 106. For example, the additional databases 146 may include
one or more of patient information databases, sales information
databases, student information databases, customer/client
information databases, or the like.
[0030] When two or more of the computing devices 102 establish a
RTC session using the RTC application 120, the computing devices
102 may exchange data substantially in real-time. For example, the
computing device 102(1) may send first data 148 substantially in
real-time to the computing device 102(N). The computing device
102(N) may send second data 150 substantially in real-time to the
computing device 102(1). The first data 148 and the second data 150
may include audio data (e.g., provided by an audio input device,
such as a microphone), video data (e.g., provided by an imaging
sensor, such as a camera), still frames (e.g., provided by the
imaging sensor), data from the monitoring devices 142, data from
the database 132, data from the additional databases 146, data from
input devices (e.g., mouse, trackball, keyboard, stylus, and the
like) associated with the computing devices 102, other types of
data, or any combination thereof. The first data 148 and the second
data 150 may include data sent substantially in real-time,
non-real-time data (e.g., data stored in the databases 132 and/or
the additional databases 146), or both.
[0031] Thus, a vendor, such as a network service provider, may
customize the RTC application 120 for RTC sessions with multiple
users in various situations, including doctor-patient
communications, student-teacher communications, client-salesperson
communications, or attorney-client communications. The RTC
application 120 may be downloadable or may be pre-loaded and
pre-configured before the computing devices 102 are provided to
users. The RTC application 120 may be loaded onto the computing
devices 102 in such a way that the RTC application 120 can be
modified by authorized personnel or authorized software
applications but cannot be modified (or tampered with) by
unauthorized personnel or by unauthorized software
applications.
[0032] FIG. 2 is a block diagram illustrating a computing device
architecture 200 that includes a real-time communications
application according to some implementations. The computer
readable media 110 of the computing device 102(N) (where N>0)
may include the RTC application 120, the browser 122, a web
application programming interface (API) 202, an RTC API 204, a
session management module 206, an audio engine 208, a video engine
210, a transport engine 212, and audio capture/render module 214, a
video capture/render module 216, and a network input/output (I/O)
module 218.
[0033] The web API 202 may provide an application programming
interface for a web server, a web browser, or both. The RTC API 204
may provide an application programming interface for real-time
communications applications, such as the RTC application 120. The
session management module 206 may manage multiple communications
sessions, with each communications session including two or more
devices.
[0034] The audio engine 208 may provide functions associated with
the audio included in RTC sessions. For example, a speech
coder/decoder (CODEC) 220 may be used to compress/decompress
digital audio signals that include speech. For example, the digital
audio signals may be compressed by the computing device 102(1)
prior to transmission and decompressed by the computing device
102(N) after the transmission is received and prior to playback.
The audio engine 208 may include an equalization module 222 to
provide multiple bands of frequency equalization. For example, the
equalization module 222 may boost and/or cut certain frequency
bands to improve the intelligibility of speech. The audio engine
208 may include noise reduction module 224 to reduce noise present
in the digital audio streams that are included in an RTC session.
The noise reduction module 224 may provide echo suppression, echo
cancellation, extraneous noise removal, another type of noise
reduction, or any combination thereof.
[0035] The video engine 210 may include a video CODEC 226, a video
buffer 228, and an image enhancement module 230. The video CODEC
226 may be used to compress/decompress digital video signals
included in RTC sessions. For example, the computing device 102(1)
may include an imaging device, such as a camera, that generates a
video signal. The video CODEC 226 may compress the video signal
from the imaging device prior to transmission. The computing device
102(N) may receive the compressed video signal from the computing
device 102(1). The video CODEC 226 of the computing device 102(N)
may decompress the received video signal prior to playback of the
video signal.
[0036] The transport engine 212 may include a Secure Real-time
Transport Protocol (SRTP) module 232, a multiplexer 234, and a
peer-to-peer (P2P) module 236. The SRTP module 232 may enable SRTP
compliant communications, including providing encryption, message
authentication and integrity, and replay protection to the data
streams in the RTC sessions. For example, the SRTP module 232 may
encrypt the audio data, the video data, and other types of data
that are exchanged in an RTC session to prevent non-participants
from accessing the audio data, the video data, and other types of
data. The SRTP module 232 may authenticate data that is received in
an RTC session and provide data integrity by compensating for data
corruption or data loss (e.g., using checksums or another type of
data integrity mechanism). The SRTP module 232 may provide
protection from a replay attack, in which a valid data transmission
is maliciously or fraudulently repeated or delayed. The P2P module
236 may provide various services associated with peer-to-peer
connectivity, such as, for example, STUN, TURN, ICE, etc.
[0037] The audio capture/render module 214 may capture audio (e.g.,
using a microphone connected to the computing device 102(N)),
render digital audio data received during an RTC session, or both.
The video capture/render module 216 may capture video (e.g., using
an imaging device, such as a video camera, that is connected to the
computing device 102(N)), render digital video data received during
an RTC session, or both. The network I/O module 218 may enable the
computing device 102(N) to initiate or engage in RTC sessions with
other computing devices via a network, such as the network 106 of
FIG. 1.
[0038] Thus, a user of the computing device 102(N) may use the RTC
application 120 to participate in RTC sessions in which data,
including audio data and/or video data may be sent and received
substantially in real-time.
[0039] FIG. 3 is a block diagram illustrating a system 300 in which
two or more devices are participating in a real-time communication
session according to some implementations. Each of the computing
devices 102 may be connected to one or more display devices 302 and
one or more input devices 304. The input devices 304 may include a
keyboard, a mouse, a trackball, an imaging device (e.g., camera),
an audio transducer (e.g., microphone), a gesture recognition
device, a biometric input device (e.g., fingerprint scanner, iris
scanner, retinal scanner, imaging device for use with facial
recognition software, or the like), another type of input device,
or any combination thereof.
[0040] The computing device 102(1) may initiate an RTC session 306
with the computing device 102(2) using a real-time communications
application, such as the RTC app 120 of FIG. 1. For example, a
patient may use the computing device 102(1) to initiate the RTC
session 306 with a doctor that is associated with the computing
device 102(2). The patient or the doctor may add an additional
participant, e.g., associated with the computing device 102(3), to
the RTC session 306.
[0041] The RTC session 306 may enable each participant to see (and
hear) video data associated with other participants in the RTC
session 306. For example, a first participant associated with the
computing device 102(1) may view a window 308(1) associated with a
second participant (e.g., associated with the computing device
102(2)) and view a window 308(2) associated with a third
participant (e.g., associated with the computing device 102(3)).
The second participant may view the first participant using the
window 308(3) and the third participant using the window 308(4).
The third participant may view the first participant using the
window 308(5) and the second participant using the window 308(6).
The windows 308 may be used to play an audio stream, a video
stream, a graphics stream, an image stream, a document stream,
another type of media stream, or any combination thereof.
[0042] The RTC session 306 may enable the second participant to
view additional data, such as, for example, the real-time data 144,
history 310, and documents 312. For example, a doctor talking to a
patient using the RTC session 306 may view confidential data, such
as the patient's history 310 and the patient's documents 312 (e.g.,
blood tests, scans, and other documents associated with the
patient). The doctor may view the real-time data 144 from a
monitoring device (e.g., blood pressure monitor, blood glucose
monitor, heart rate monitor, or other monitoring device) associated
with the patient.
[0043] The RTC session 306 may provide remote access 314 to
additional data (include confidential data). For example, a
participant may use the remote access 314 to access data stored in
the databases 132, the additional databases 146, or other data that
is not stored on the computing device 102(2).
[0044] Of course, the RTC session 306 may be configured according
to the type of services being offered. For example, in a financial
advisor-client RTC session, the real-time data 144 may include
real-time stock information, the history 310 may include a trading
history associated with a client, the documents 312 may include
monthly or quarterly statements, a prospectus associated with an
investment vehicle, etc. In an attorney-client RTC session, the
real-time data 144 may include real-time legal events, the history
310 may include a history of the client (e.g., when the client has
spoken to the attorney, for how long, regarding which matters,
etc.), and the documents 312 may include court documents,
affidavits, and the like. In a salesperson-client RTC session, the
real-time data 144 may include real-time information relevant to a
potential sale, the history 310 may include a history of the client
(e.g., when the client acquired items, how much the client has
paid, etc), and the documents 312 may include invoices, sales
contracts, and the like. The RTC session may provide authentication
and access restrictions. For example, at least some of the data
being accessed in an RTC session, such as confidential data, may
have restricted access such that some, but not necessarily all, of
the participants in the RTC session may access certain types of
data. To illustrate, the identity of a user of a computing device
may be authenticated and the type of data that the user is
permitted to access may be determined based on the user's identity.
For example, after a doctor's identity has been authenticated, the
RTC session may provide the doctor access to multiple patient
records. In contrast, after a patient's identity has been
authenticated, the RTC session may provide the patient access to
the patient's own records but not the records of other patients. In
addition, the patient may be provided with access to a subset of
the patient's records, e.g., only those patient records that the
doctor designates that the patient can access. Thus, the doctor may
determine which of the patient's own records the patient can access
and which records the patient cannot access. As another example, in
a financial setting, after the identity of a financial advisor has
been authenticated, the financial advisor may have access to the
records associated with clients of the financial advisor. After the
identity of a particular client has been authenticated, the client
may access only those records associated with the client that the
financial advisor has designated as shareable documents. As a
further example, in a legal setting, after the identity of an
attorney has been authenticated, the attorney may have access to
the records associated with clients of the attorney. After the
identity of a particular client has been authenticated, the client
may access only those records associated with the client that the
attorney desires to share with the client. As a further example,
for collaboration purposes, the RTC session may (i) permit a first
set of participants to view but not create, modify, or delete a
file, (ii) permit a second set of participants to view and modify
but not create or delete the file, and (iii) permit a third set of
participants to create, view, modify, and delete the file.
[0045] The authentication of each participant in an RTC session may
be performed using a number of different techniques. For example, a
user of a computing device may be prompted to enter a username, a
password, a passcode, another input, or any combination thereof. As
another example, the user may be prompted to provide biometric
input, such as a fingerprint scan, an iris scan, a retinal scan, a
facial scan, another type of biometric input, or any combination
thereof. The input may be authenticated to verify an identity of
the user before enabling the user to participate in the RTC
session. In addition, the identity of each participant may
determine a level of access to data items. Depending on the level
of access the RTC session provides, a participant may be (i) unable
to access a file (e.g., the participant may not be able to see that
the file exists or the participant may be to see the file but
unable to view the contents of the file), (ii) able to create a
file, (iii) able to view a file, (iv) able to modify a file, (v)
able to delete a file, or any combination thereof.
[0046] Thus, two or more users may participate in the RTC session
306. Each participant may be capable of viewing audio and/or video
from other computing devices associated with other participants.
The RTC session 306 may enable each participant to view various
types of data, including real-time data and stored data. The data
that a particular participant can view in the RTC session 306 may
be determined by the particular participant's credentials. For
example, some types of participants, such as patients or clients,
may be able to view data associated with the participant but may
not be able to view other data associated with other users.
However, some participants, such as doctors, attorneys,
salespersons etc., may be able to view data associated with all
their patients or all their clients.
[0047] In the flow diagrams of FIGS. 4 and 5, each block represents
one or more operations that can be implemented in hardware,
software, or a combination thereof. In the context of software, the
blocks represent computer-executable instructions that, when
executed by one or more processors, cause the processors to perform
the recited operations. Generally, computer-executable instructions
include routines, programs, objects, modules, components, data
structures, and the like that perform particular functions or
implement particular abstract data types. The order in which the
blocks are described is not intended to be construed as a
limitation, and any number of the described operations can be
combined in any order and/or in parallel to implement the
processes. For discussion purposes, the processes 400 and 500 are
described with reference to the systems 100, 200, and 300, as
described above, although other models, frameworks, systems and
environments may implement these processes.
[0048] FIG. 4 is a flow diagram of an example process 400 that
includes sending a request to initiate a real-time communication
session according to some implementations. The process 400 may be
performed by a computing device, such as the computing devices 102
of FIGS. 1, 2, and 3.
[0049] At 402, a user-selection to initiate a real-time
communication session with a second computing device may be
received. For example, in FIG. 3, a user may instruct the computing
device 102(1) to initiate the RTC session 306 with the computing
device 102(2).
[0050] At 404, user credentials may be authenticated. For example,
in FIG. 3, in response to receiving the instruction to initiate the
RTC session 306, the computing device 102(1) may prompt the user to
enter user credentials (e.g., type in a username and a password,
scan a fingerprint, scan an iris or a retina, provide an image of
the user's face for facial recognition, or the like). The computing
device 102(1) may authenticate the user credentials after the user
has entered the user credentials to verify the user's identity. For
example, authenticating the user credentials may prevent
unauthorized personnel from requesting a real-time communication
session with the second computing device. To illustrate, during
authentication, the user of the computing device 102(1) may be
verified as being a patient of a doctor prior to sending a request
to initiate the RTC session 306.
[0051] At 406, a request to initiate the real-time communication
session may be sent to the second computing device. For example, in
FIG. 3, the computing device 102(1) may send a request to establish
the RTC session 306 with the computing device 102(2).
[0052] At 408, the real-time communication session may be
initiated. For example, in FIG. 3, if the computing device 102(2)
sends a response accepting the request to establish the RTC session
306, the computing device 102(1) may initiate the RTC session 306
with the computing device 102(2).
[0053] At 410, at least one of an audio stream or a video stream
may be included in the real-time communication session. For
example, in FIG. 1, the first data 148 and the second data 150 may
include at least one of an audio stream or a video stream.
[0054] At 412, a file hosted by the second computing device may be
accessed by the first computing device using peer-to-peer file
sharing included in the real-time communication session. For
example, the computing device 102(1) may retrieve a document hosted
by the computing device 102(2) using a peer-to-peer file sharing
feature of the RTC session 306.
[0055] Thus, when a user instructs a first computing device to
initiate an RTC session with a second computing device, the first
computing device may prompt the user for credentials, authenticate
the credentials to verify an identity of the user, and then send a
request to the second computing device to initiate the RTC session
after verifying that the user is authorized to communicate with a
second user associated with the second computing device. The RTC
session may include one or more media streams, such as a video
stream and/or an audio stream. A device participating in the RTC
session may access files hosted by another device that is
participating in the RTC session using a peer-to-peer file sharing
feature of the RTC session.
[0056] FIG. 5 is a flow diagram of an example process that includes
receiving a request to initiate a real-time communication session
according to some implementations. The process 500 may be performed
by a computing device, such as the computing devices 102 of FIGS.
1, 2, and 3.
[0057] At 502, a request to initiate a real-time communication
session between a first computing device and a second computing
device may be received. For example, in FIG. 3, the computing
device 102(1) may send a request to establish the RTC session 306
with the computing device 102(2).
[0058] At 504, user credentials may be authenticated. For example,
in FIG. 3, in response to receiving a request from the computing
device 102(1) to initiate the RTC session 306, the computing device
102(2) may prompt the user to enter user credentials (e.g.,
username and password). The computing device 102(2) may
authenticate the user credentials after the user has entered the
user credentials to verify the user's identity. For example,
authenticating the user credentials may prevent unauthorized
personnel from accepting a request to initiate a real-time
communication session. To illustrate, during authentication, the
user of the computing device 102(2) may be verified as being a
doctor of a patient that is requesting the RTC session prior to
accepting the request to initiate the RTC session 306.
[0059] At 506, the real-time communication session may be
initiated. For example, in FIG. 3, if the computing device 102(2)
sends a response accepting the request to establish the RTC session
306, the computing device 102(1) may initiate the RTC session 306
with the computing device 102(2).
[0060] At 508, a request may be sent (e.g., either from the first
computing device or the second computing device) to a third
computing device to include the third computing device in the RTC
session. For example, in FIG. 3, the computing device 102(1) or the
computing device 102(2) may send a request to the computing device
102(3) asking whether a user of the computing device 102(3) desires
to join the RTC session 306.
[0061] At 510, the third computing device may be included in the
RTC session. For example, if a user of the computing device 102(3)
accepts the request to join the RTC session 306, the computing
device 102(3) may be included in the RTC session 306. The RTC
session 306 may include one or more media streams (e.g., audio
and/or video streams) that are exchanged between the computing
device 102(1), the computing device 102(2), and the computing
device 102(3).
[0062] At 512, a file hosted by the second computing device may be
retrieved. For example, in FIG. 3, the computing device 102(1) or
the computing device 102(2) may retrieve a document hosted by the
computing device 102(2) using a peer-to-peer file sharing feature
of the RTC session 306.
[0063] Thus, when a user instructs a first computing device to
initiate an RTC session with a second computing device, the first
computing device may prompt the user for credentials, authenticate
the credentials to verify an identity of the user, and then send a
request to the second computing device to initiate the RTC session
after verifying that the user is authorized to communicate with a
second user associated with the second computing device. The second
computing device may request and authenticate second credentials
received from the second user to verify that the second user is
authorized to communicate with the first user.
[0064] In some cases, a request may be sent (e.g., either by the
first computing device or the second computing device) to the third
computing device requesting the third computing device to join the
RTC session. In response to the third computing device accepting
the request, the third computing device may join the RTC session
that includes the first computing device and the second computing
device. The first computing device, the second computing device,
and the third computing device may exchange one or more media
streams with each other. A device participating in the RTC session
may access files hosted by another device that is participating in
the RTC session using a peer-to-peer file sharing feature of the
RTC session.
[0065] FIG. 6 is a block diagram 600 illustrating a computing
device participating in a real-time communication session according
to some implementations. When a computing device, such as the
computing device 102(n), is participating in an RTC session (e.g.,
the RTC session 306), one or more windows, such as the windows
308(1), 308(2), and 308(3) may be displayed. Each of the windows
308(1), 308(2), and 308(3) may display a media stream (e.g.,
including audio and/or video) associated with each of the
participants in the RTC session 306. The RTC session 306 may
provide functions 602, such as, for example, a record session
function 604, a secure file sharing function 606, and other
functions 608. The record session function 604 may enable the RTC
session 306 to be recorded and stored. For example, in a medical
setting, the RTC session 306 may be recorded for regulatory
compliance, such as compliance with the Health Insurance
Portability and Accountability Act. As another example, a doctor
may record an RTC session with a patient in which (i) the doctor
reviews the risks involved with a medical procedure and/or (ii) the
patient provides consent for the medical procedure. The secure file
sharing 606 may enable files to be securely shared between two or
more participants in the RTC session 306. The secure file sharing
606 may enable non-persistent file streaming. The other functions
608 may include toggling between a full screen view of the RTC
session 306 and a smaller sized view, a mute function, a dialing
function to initiate a request to include an additional participant
in the RTC session 306, a search engine, etc. For example the
search engine may search a local (e.g., stored on the computing
device 102(N)) directory of contacts, such as contacts 610, a
directory of contacts stored at a hospital server with which the
computing device 102(n) is wirelessly communicating, an internet
search, another type of search, or any combination thereof.
[0066] The contacts 610 may provide a directory of contacts. For
example, in a medical context, the contacts 610 may include the
names and contact information associated with medical professionals
(doctors, nurses, administrative staff, lab technicians, etc.)
and/or patients. The contacts 610 may display presence information,
e.g., indicating which of the contacts is currently available to
participate in the RTC session 306.
[0067] A messaging window 612 may enable a user of the computing
device 102(N) to send and receive messages substantially in
real-time ("instant messaging") with one or more of the contacts
610. A customer relationship manager (CRM) 614 may include business
process integration and the ability to organize and view
customer-related data. In a medical context, the customer-related
data may include data associated with other medical professionals,
data associated with patients, etc. For example, the CRM 614 may
provide tabbed views of data, such as viewing medical professionals
by specialization, viewing patients by a type of illness (e.g.,
patients with heart-related issues, patients with cancer, etc.), or
other types of views. The CRM 614 may display customer details 616
associated with the information managed by the CRM 614.
[0068] A document collaboration application 618 may enable one or
more participants in the RTC session 306 to modify a document, such
as the document 622. A document retrieval window 620 may enable the
retrieval of documents, such as the document 624. In some cases,
the document 624 may be automatically retrieved and displayed on
the computing device 102(N) after the RTC session 306 is
established. For example, in a medical setting, a most recent
document associated with a patient may be automatically retrieved
when the RTC session 306 is established. The document retrieval
window 620 may enable a user of the computing device 102(N) to
retrieve documents (e.g., including documents with confidential
information) associated with one or more of the other participants.
For example, a doctor may use the document retrieval window 620 to
retrieve an additional document 624, such as a previous blood
test.
[0069] From a user's perspective, the computing device 102(N) may
provide an integrated experience in which the computing device
102(N) includes the functionality of a phone, a video conferencing
application, a contacts directory, a messaging application, a CRM
application, etc. Thus, the user can carry a single device that
provides an integrated environment instead of carrying multiple
devices and use a single application instead of multiple
applications. The computing device 102(N) may provide an integrated
corporate communications portal (e.g., web-based or using a tablet
application). The computing devices 102 may be sold as a turnkey
solution where a company, such as a hospital, purchases
pre-configured tablet computers, customized applications, service,
and support from a single vendor. The computing devices 102 may
provide single-device access to a variety of enterprise
communications and collaboration tools including voice, video,
instant messaging (IM), text messaging, file access and exchange,
etc. The computing device 102(N) may enable a user to interact with
internal customers (e.g., medical professionals) and external
customers (e.g., patients). The computing device 102(N) may enable
everything from 1-to-1 communications to 1-to-many communications,
corporate directory access, presence awareness, integration with
business tools (e.g., CRM, etc), etc. The vendor may provide a
turnkey solution that is customized for each client. For example,
the customization may include customization for each type of
business (e.g., medical services, financial services, legal
services, etc.) and for the way in which the business operates.
[0070] The type of companies that may purchase a customized turnkey
solution may include medical services providers (e.g., hospitals,
clinics, and the like), corporations, schools, legal services,
financial services, and other types of services. For example, the
turnkey solution for a hospital may include patient record
management, patient interaction with medical staff at each hospital
bed (e.g., including two-way video and audio capability). Medical
staff may be provided with the ability to remotely view (e.g. via a
computing device) real-time data provided by equipment monitoring
various physiological functions of the patient. For a doctor
receiving a call from a patient, the doctor's computing device may
access and display the patient's most recent records and provide
the doctor access to the patient's older records. Patients who are
no longer staying in a medical facility (e.g., hospital) may be
provided with a downloadable application that enables the patient
to communication with the doctor using the patient's computing
device (e.g., smartphone, tablet, etc.). The vendor and the
hospital may use metered billing, in which a patient is charged
based on a length (e.g., in minutes, in seconds, etc.) of their
communication session. For example, an RTC session between a
patient and a doctor may be billed at $M dollars per rounded-up
minute. The money received from the patient may be divided between
one or more of (i) the doctor, (ii) the hospital, and (iii) the
vendor. For example, the doctor may receive 60% of the money paid
by the patient, the hospital 30%, and the vendor 10%. The vendor
may provide billing services to companies that purchase a turnkey
solution.
[0071] Corporations may purchase a customized turnkey solution to
enable more effective and efficient communications between
employees to facilitate collaboration and efficient access to data.
The turnkey solution may be sold or leased as a package that
includes computing devices (e.g., tablet devices) configured with
custom software, along with services (e.g., customization services,
installation services, etc.), and technical support. The turnkey
solution may provide single-screen access to a host of enterprise
communications and collaboration tools including voice, video, IM
and text messaging to facilitate interact with internal and
external customers.
[0072] Schools, colleges, and universities may purchase a
customized turnkey solution to enable interaction between teachers
and students, to facilitate collaboration between students, etc.
Legal advisors and financial advisors may purchase a customized
turnkey solution to enable efficient and secure communications with
clients.
[0073] Thus, a customized turnkey solution may provide users with
single-screen access to a host of enterprise communications and
collaboration tools, including voice, video, IM and text messaging,
access to corporate directories and client files, presence
awareness, etc. The integration of CRM applications may enable
efficient management of clients, customers, patients, or the like.
For example, when a call is received, relevant client data may be
identified and retrieved from a CRM application, corporate
databases, etc.
[0074] The various techniques described above are assumed in the
given examples to be implemented in the general context of
computer-executable instructions or software, such as program
modules, that are stored in computer-readable storage and executed
by the processor(s) of one or more computers or other devices such
as those illustrated in the figures. Generally, program modules
include routines, programs, objects, components, data structures,
etc., and define operating logic for performing particular tasks or
implement particular abstract data types.
[0075] Other architectures may be used to implement the described
functionality, and are intended to be within the scope of this
disclosure. Furthermore, although specific distributions of
responsibilities are defined above for purposes of discussion, the
various functions and responsibilities might be distributed and
divided in different ways, depending on particular
circumstances.
[0076] Similarly, software may be stored and distributed in various
ways and using different means, and the particular software storage
and execution configurations described above may be varied in many
different ways. Thus, software implementing the techniques
described above may be distributed on various types of
computer-readable media, not limited to the forms of memory that
are specifically described.
[0077] Furthermore, although the subject matter has been described
in language specific to structural features and/or methodological
acts, it is to be understood that the subject matter defined in the
appended claims is not necessarily limited to the specific features
or acts described. Rather, the specific features and acts are
disclosed as exemplary forms of implementing the claims.
* * * * *