U.S. patent application number 14/341818 was filed with the patent office on 2015-03-19 for system for a virtual multipoint control unit for unified communications.
The applicant listed for this patent is ClearOne Inc.. Invention is credited to Avishay Ben Natan, Derek Graham.
Application Number | 20150077509 14/341818 |
Document ID | / |
Family ID | 52667581 |
Filed Date | 2015-03-19 |
United States Patent
Application |
20150077509 |
Kind Code |
A1 |
Ben Natan; Avishay ; et
al. |
March 19, 2015 |
System for a Virtual Multipoint Control Unit for Unified
Communications
Abstract
This disclosure describes a virtual multipoint control unit
(112, 300) for unified communications. The virtual multipoint
control unit (112, 300) communicates with multiple devices (102,
104) over a network (106). The control unit (112, 300) includes
multiple unified communication (UC) applications (208, 314) being
executed on the devices (102, 104). The UC applications (208, 314)
decode received audio data stream and video data stream. A virtual
imaging device (214) and a virtual audio device (216) are mapped to
the UC applications (208, 314). A virtual video mixer (210)
receives the decoded video data stream from the UC applications
(208, 314) via the virtual imaging device (214). A virtual audio
mixer (212) receives the decoded audio data stream from the UC
applications (208, 314) via the virtual audio device (216).
Inventors: |
Ben Natan; Avishay; (Hod
Hasharon, IL) ; Graham; Derek; (Salt Lake City,
UT) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
ClearOne Inc. |
Salt Lake City |
UT |
US |
|
|
Family ID: |
52667581 |
Appl. No.: |
14/341818 |
Filed: |
July 27, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61859358 |
Jul 29, 2013 |
|
|
|
Current U.S.
Class: |
348/14.09 |
Current CPC
Class: |
G10L 19/173 20130101;
H04N 7/152 20130101; H04N 7/147 20130101 |
Class at
Publication: |
348/14.09 |
International
Class: |
H04N 7/15 20060101
H04N007/15 |
Claims
1. A virtual multipoint control unit in communication with a
plurality of devices over a network, comprising: a plurality of
unified communication (UC) applications being executed on at least
one of the plurality of devices, each of the plurality of UC
applications receiving an encoded audio data stream and an encoded
video data stream, wherein the plurality of UC applications decode
the received audio and the video data streams; at least one virtual
imaging device mapped to each of the plurality of UC applications;
at least one virtual audio device mapped to each of the plurality
of UC applications; a virtual video mixer in communication with a
physical imaging device, wherein the virtual video mixer receives
the decoded video data stream from each of the plurality of UC
applications via the at least one virtual imaging device; and a
virtual audio mixer in communication with a physical audio device,
wherein the virtual audio mixer receives the decoded audio data
stream from each of the plurality of UC applications via the at
least one virtual audio device.
2. The claim according to claim 1, wherein the plurality of UC
applications comprises at least one of Microsoft Lync, IBM
Sametime, Cisco IP Communicator, and Skype.
3. A system for managing communication among a plurality of
devices, the system being employed within a communication network,
comprising: a communication device receiving encoded audio and
video data streams via a plurality of unified communication (UC)
applications, wherein the plurality of UC applications decode the
received encoded audio data stream and the received encoded video
data stream; and a virtual multipoint control unit in communication
with the communication device, wherein the virtual multipoint
control unit comprises: at least one virtual imaging device mapped
to each of the plurality of UC applications; at least one virtual
audio device mapped to each of the plurality of UC applications; a
virtual video mixer in communication with a physical imaging
device, wherein the virtual video mixer receives the decoded video
data stream from each of the plurality of UC applications via the
at least one virtual imaging device; and a virtual audio mixer in
communication with a physical audio device, wherein the virtual
audio mixer receives the decoded audio data stream from each of the
plurality of UC applications via the at least one virtual audio
device.
4. The claim according to claim 3, wherein the plurality of UC
applications comprises at least one of Microsoft Lync, IBM
Sametime, Cisco IP Communicator, and Skype.
5. A non-transitory computer readable medium storing a program of
instructions executable by a computing device to perform a method
for employing a virtual multipoint control unit for unified
communications, comprising: receiving, from a plurality of
terminals, an encoded audio data stream and an encoded video data
stream; decoding, using a plurality of unified communication (UC)
applications, the received encoded audio data stream and the
received encoded video data stream; mapping, using a virtual
multipoint control unit, at least one virtual imaging device and at
least one virtual audio device to each of the plurality of UC
applications; communicating, using a virtual video mixer, the
decoded video data stream to a physical imaging device via the at
least one virtual imaging device; and communicating, using a
virtual audio mixer, the decoded audio data stream to a physical
audio device via the at least one virtual audio device.
6. The claim according to claim 5, wherein the plurality of UC
applications comprise at least one of Microsoft Lync, IBM Sametime,
Cisco IP Communicator, and Skype.
7. A method for managing communication among a plurality of
devices, the method comprising: receiving, from a plurality of
terminals, an encoded audio data stream and an encoded video data
stream; decoding, using a plurality of unified communication (UC)
applications, the received encoded audio data stream and the
received encoded video data stream; mapping, using a virtual
multipoint control unit, at least one virtual imaging device and at
least one virtual audio device to each of the plurality of UC
applications; communicating, using a virtual video mixer, the
decoded video data stream to a physical imaging device via the at
least one virtual imaging device; and communicating, using a
virtual audio mixer, the decoded audio data stream to a physical
audio device via the at least one virtual audio device.
8. The claim according to claim 8, wherein the plurality of UC
applications comprises at least one of Microsoft Lync, IBM
Sametime, Cisco IP Communicator, and Skype.
9. A method for manufacturing a system for managing communication
among a plurality of devices, the system being employed within a
communication network, comprising: providing a communication device
receiving encoded audio and video data streams via a plurality of
unified communication (UC) applications, wherein the plurality of
UC applications decode the received encoded audio data stream and
the received encoded video data stream; and providing a virtual
multipoint control unit in communication with the communication
device, wherein the virtual multipoint control unit comprises: at
least one virtual imaging device mapped to each of the plurality of
UC applications; at least one virtual audio device mapped to each
of the plurality of UC applications; a virtual video mixer in
communication with a physical imaging device, wherein the virtual
video mixer receives the decoded video data stream from each of the
plurality of UC applications via the at least one virtual imaging
device; and a virtual audio mixer in communication with a physical
audio device, wherein the virtual audio mixer receives the decoded
audio data stream from each of the plurality of UC applications via
the at least one virtual audio device.
10. The claim according to claim 9, wherein the plurality of UC
applications comprises at least one of Microsoft Lync, IBM
Sametime, Cisco IP Communicator, and Skype.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority and the benefits of the
earlier filed Provisional U.S. Application No. 61/859,358, filed
Jul. 29, 2013, which is incorporated by reference for all purposes
into this specification.
TECHNICAL FIELD
[0002] This disclosure relates to unified communication systems.
More specifically, this disclosure invention relates to a system
for a virtual multipoint control unit for unified
communications.
BACKGROUND ART
[0003] Several Unified Communications (UC) software applications
have proliferated into the enterprise communications market. These
applications allow end-users to communicate using voice, video,
application and data sharing, instant messaging, etc. A few
examples of these applications include Microsoft Lync, Skype, IBM
Sametime, Cisco IP Communicator, etc.
[0004] Most of these applications use different call control
protocols and media formats (i.e. video/audio encoding and decoding
formats), making them incompatible with each other. For example,
Microsoft Lync uses a proprietary implementation of the Session
Initiation Protocol (SIP) that is different from that being
implemented by Skype for call negotiation. In addition, various UC
applications typically operate with different media formats,
thereby causing these applications to become incompatible with each
other for communicating audio, video, or data streams. As a result,
the communication among these applications is facilitated by a
multipoint control unit, which may be implemented as standalone
hardware or a web-based service via a central host performing call
signaling and media transcoding. The standalone hardware is
extremely costly whereas the web-based central host does not
support ad-hoc communication among the UC applications running on
terminals that are not connected to the same central host.
[0005] Further, the multipoint control unit typically allows
multiple UC applications to access local hardware devices in a
mutually exclusive manner. For e.g., video data streams from one UC
application are temporarily restricted from being played on a local
hardware device such as a display device while the device is in use
by another UC application. As a result, the multipoint control unit
induces an unwanted delay during execution of simultaneously
received data streams from different UC applications
[0006] Therefore, there exists a need for a system that allows
multiple UC applications to access the local hardware devices
simultaneously for ad-hoc unified communications.
SUMMARY OF INVENTION
[0007] This disclosure describes a system for a virtual multipoint
control unit for unified communications.
[0008] In one embodiment, a virtual multipoint control unit is in
communication with a plurality of devices over a network. The
virtual multipoint control unit comprises a plurality of unified
communication (UC) applications, at least one virtual imaging
device, at least one virtual audio device, a virtual video mixer,
and a virtual audio mixer. The plurality of unified communication
(UC) applications is executed on at least one of the plurality of
devices. Each of the plurality of UC applications receives an
encoded audio data stream and an encoded video data stream. The
plurality of UC applications decodes the received audio data and
the video data streams. The at least one virtual imaging device is
mapped to each of the plurality of UC applications. The at least
one virtual audio device is mapped to each of the plurality of UC
applications. The virtual video mixer is in communication with a
physical imaging device. The virtual video mixer receives the
decoded video data stream from each of the plurality of UC
applications via the at least one virtual imaging device. The
virtual audio mixer is in communication with a physical audio
device. The virtual audio mixer receives the decoded audio data
stream from each of the plurality of UC applications via the at
least one virtual audio device.
[0009] In another embodiment, a system for managing communication
among a plurality of devices is disclosed. The system is employed
within a communication network. The system comprises a
communication device and a virtual multipoint control unit. The
communication device receives encoded audio and video data streams
via a plurality of unified communication (UC) applications. The
plurality of UC applications decodes the received encoded audio
data stream and the received encoded video data stream. The virtual
multipoint control unit is in communication with the communication
device. The virtual multipoint control unit comprises at least one
virtual imaging device, at least one virtual audio device, a
virtual video mixer, and a virtual audio mixer. The at least one
virtual imaging device is mapped to each of the plurality of UC
applications. The at least one virtual audio device is mapped to
each of the plurality of UC applications. The virtual video mixer
is in communication with a physical imaging device. The virtual
video mixer receives the decoded video data stream from each of the
plurality of UC applications via the at least one virtual imaging
device. The virtual audio mixer is in communication with a physical
audio device. The virtual audio mixer receives the decoded audio
data stream from each of the plurality of UC applications via the
at least one virtual audio device.
[0010] In yet another embodiment, a non-transitory computer
readable medium storing a program of instructions executable by the
computing device to perform a method for employing a virtual
multipoint control unit for unified communications is disclosed.
The method comprises receiving, from a plurality of terminals, an
encoded audio data stream and an encoded video data stream. The
method also comprises decoding, using a plurality of unified
communication (UC) applications, the received encoded audio data
stream and the received encoded video data stream. The method
further comprises mapping, using a virtual multipoint control unit,
at least one virtual imaging device and at least one virtual audio
device to each of the plurality of UC applications. The method also
comprises communicating, using a virtual video mixer, the decoded
video data stream to a physical imaging device via the at least one
virtual imaging device. Furthermore, the method comprises
communicating, using a virtual audio mixer, the decoded audio data
stream to a physical audio device via the at least one virtual
audio device.
[0011] In still another embodiment, a method for managing
communication among a plurality of devices is disclosed. The method
comprises receiving, from a plurality of terminals, an encoded
audio data stream and an encoded video data stream. The method also
comprises decoding, using a plurality of unified communication (UC)
applications, the received encoded audio data stream and the
received encoded video data stream. The method further comprises
mapping, using a virtual multipoint control unit, at least one
virtual imaging device and at least one virtual audio device to
each of the plurality of UC applications. The method also comprises
communicating, using a virtual video mixer, the decoded video data
stream to a physical imaging device via the at least one virtual
imaging device. Furthermore, the method comprises communicating,
using a virtual audio mixer, the decoded audio data stream to a
physical audio device via the at least one virtual audio
device.
[0012] Other and further aspects and features of the disclosure
will be evident from reading the following detailed description of
the embodiments, which are intended to illustrate, and not limit,
the present disclosure.
BRIEF DESCRIPTION OF DRAWINGS
[0013] To further aid in understanding the disclosure, the attached
drawings help illustrate specific features of the disclosure and
the following is a brief description of the attached drawings:
[0014] FIG. 1A is a schematic that illustrates a first environment
for implementing an exemplary virtual multipoint control unit,
according to an embodiment of the present disclosure.
[0015] FIG. 1B is a schematic that illustrates a second environment
for implementing the exemplary virtual multipoint control unit of
FIG. 1A, according to an embodiment of the present disclosure.
[0016] FIG. 2A is a schematic that illustrates the exemplary
virtual multipoint control unit of FIG. 1A, according to an
embodiment of the present disclosure.
[0017] FIG. 2B is a schematic that illustrates an exemplary unified
communication application included in the virtual multipoint
control unit of FIG. 1A, according to an embodiment of the present
disclosure.
[0018] FIG. 3 is a schematic that illustrates a virtual multipoint
control unit 300 managing a multipoint video conference, according
to an embodiment of the present disclosure.
[0019] FIG. 4 is a schematic illustrating implementation of a
device driver module of the virtual multipoint control unit for
communicating with hardware devices for conducting a video
conference, according to an embodiment of the present
disclosure.
DISCLOSURE OF EMBODIMENTS
[0020] This disclosure describes a system for a virtual multipoint
control unit for unified communications. This disclosure describes
numerous specific details in order to provide a thorough
understanding of the present invention. One skilled in the art will
appreciate that one may practice the present invention without
these specific details. Additionally, this disclosure does not
describe some well known items in detail in order not to obscure
the present invention.
[0021] In various embodiments of the present disclosure,
definitions of one or more terms that will be used in the document
are provided below.
[0022] An endpoint refers to one or more computing devices capable
of establishing a communication channel for exchange of audio,
video, textual, or symbolic data in a communication session.
Examples of the computing devices may include, but are not limited
to, a desktop PC, a personal digital assistant (PDA), a server, a
mainframe computer, a mobile computing device (for e.g., mobile
phones, laptops, tablets, etc.), an internet appliance, calling
devices (for e.g., a telephone, an internet phone, video telephone,
etc.).
[0023] The numerous references in the disclosure to a system for a
virtual multipoint control unit are intended to cover any and/or
all devices capable of performing respective operations on
endpoints in a unified communication application-based conferencing
environment relevant to the applicable context, regardless of
whether or not the same are specifically provided.
[0024] FIG. 1A is a schematic that illustrates a first environment
for implementing an exemplary virtual multipoint control unit,
according to an embodiment of the present disclosure. Embodiments
are disclosed in the context of environments that represent a
multipoint video conference among multiple users via respective
endpoints capable of executing one or more computer applications
for unified communications in the same communication session.
However, other embodiments may be applied in the context of other
scenarios (e.g., an audio conference, a webinar, a multiplayer
online game, etc.) involving at least one of audio, video, textual,
or symbolic (e.g., emoticons, images, etc.) data being communicated
among various endpoints in the same communication session. In some
embodiments, at least one of the endpoints may execute a unified
communication application (UC application) during the session.
[0025] The first network environment 100 may comprise multiple
endpoints including a communication device 102 configured to
communicate with terminals 104-1, 104-2, 104-3, 104-4, and 104-5
(collectively, terminals 104) via a network 106. The network 106
may comprise, for example, one or more of the Internet, Wide Area
Networks (WANs), Local Area Networks (LANs), analog or digital
wired and wireless telephone networks (e.g., a PSTN, Integrated
Services Digital Network (ISDN), a cellular network, and Digital
Subscriber Line (xDSL)), radio, television, cable, satellite,
and/or any other delivery or tunneling mechanism for carrying
data.
[0026] The network 106 may comprise multiple networks or
sub-networks, each of which may comprise, for example, a wired or
wireless data pathway. The network 106 may comprise a
circuit-switched voice network, a packet-switched data network, or
any other network that is able to carry electronic communications.
For example, the network 106 may comprise networks based on the
Internet protocol (IP) or asynchronous transfer mode (ATM), and may
support voice using, for example, VoIP, Voice-over-ATM, or other
comparable protocols used for voice data communications. In some
embodiments, the network 106 may comprise a cellular telephone
network configured to enable exchange of textual data, audio data,
video data, or any combination thereof between the communication
device 102 and at least one of the terminals 104.
[0027] The communication device 102 may comprise or be coupled with
one or more hardware devices either wirelessly or in a wired
fashion for enabling a user to dynamically interact with other
users via the endpoints connected to the network 106. For example,
the communication device 102 may be coupled with an imaging device
108 (including, but not limited to, a video camera, a webcam, a
scanner, or any combination thereof) and an audio device 110
(including, but not limited to, a speaker, a microphone, or any
combination thereof). The communication device 102 may be
compatible with any other device (not shown) connected to the
network 106 to exchange audio, video, textual or symbolic data
streams with each other or any other compatible devices.
[0028] In one embodiment, the communication device 102 may comprise
a virtual multipoint control unit 112 configured to, at least one
of: (1) create a logical representation of one or more hardware
devices in communication with the communication device 102; (2)
establish a communication bridge or channel between the UC
applications executed by the communication device 102, and the
corresponding UC applications being executed by at least one of the
terminals 104; (3) store, manage, and process the multimodal input
data streams received from the endpoints and/or associated devices
such as the imaging device 108 and the audio device 110 connected
to the network 106; and (4) request services from or deliver
services to, or both, various devices connected to the network
106.
[0029] In one embodiment, the virtual multipoint control unit 112
may facilitate integration of real-time and non-real-time
communication services by bridging communication among various UC
applications being simultaneously executed on various endpoints,
such as the communication device 102 and the terminals 104.
Examples of such real-time services may include, but are not
limited to, instant messaging, internet protocol (IP) telephony,
video conferencing, desktop sharing, data sharing, call control,
and speech recognition. Examples of these non-real-time services
may include, but are not limited to, voicemail, E-mail, SMS, and
fax. Further, examples of UC applications may include, but are not
limited to, Microsoft Lync, Skype, IBM Sametime, and Cisco IP
Communicator. Each of the UC applications may operate with same or
different communication protocols and media formats.
[0030] In another embodiment, the virtual multipoint control unit
112 may be implemented as a standalone and dedicated "black box"
including hardware and installed software, where the hardware is
closely matched to the requirements and/or functionality of the
software. Alternatively, the virtual multipoint control unit 112
may be implemented as a software application or a device driver.
The virtual multipoint control unit 112 may enhance or increase the
functionality and/or capacity of the network 106 to which it is
connected. The virtual multipoint control unit 112 may be further
configured, for example, to perform e-mail tasks, security tasks,
network management tasks including IP address management, and other
tasks.
[0031] In yet another embodiment, the virtual multipoint control
unit 112 may be configured to expose its computing environment or
operating code to the user, and may comprise related art I/O
devices, such as camera, speaker, scanner, keyboard or display. The
virtual multipoint control unit 112 may, however, comprise
software, firmware or other resources that support remote
administration and/or maintenance of the virtual multipoint control
unit.
[0032] In a further embodiment, the virtual multipoint control unit
112 may comprise at least one processor (not shown) executing
machine readable program instructions for performing various
operations, such as those discussed above, on the received
multimodal input audio, video, textual, or symbolic data stream.
The virtual multipoint control unit 112 may comprise, in whole or
in part, a software application working alone or in conjunction
with one or more hardware resources. Such software applications may
be executed by the processor on different hardware platforms or
emulated in a virtual environment, discussed below in greater
detail. Aspects of the virtual multipoint control unit 112 may
leverage known, related art, or later developed off-the-shelf
software.
[0033] Now turning to FIG. 1B, a second network environment 150 is
provided. In this embodiment, the virtual multipoint control unit
112 may be integrated with, or installed on, a network appliance
152 that is associated with or used to establish the network 106.
The network appliance 152 may be capable of operating as an
interface device to assist exchange of program instructions and
data between the communication device 102 and the terminals 104. In
some embodiments, the network appliance 152 may be preconfigured or
dynamically configured to comprise the virtual multipoint control
unit 112 integrated with other devices. For example, the virtual
multipoint control unit 112 may be integrated with the
communication device 102 or any other device, such as at least one
of the terminals 104 connected to the network 106. The
communication device 102 may comprise a module (not shown), which
introduces the communication device 102 to the network appliance
152, thereby enabling the network appliance 152 to invoke the
virtual multipoint control unit 112 as a service. Examples of the
network appliance 152 may include, but are not limited to, a DSL
modem, a wireless access point, a router, a base station, and a
gateway having a predetermined computing power sufficient for
implementing the virtual multipoint control unit 112.
[0034] The virtual multipoint control unit 112 and the
communication device 102 may collectively constitute a unified
communication system, which may reside in a single device or may be
distributed across multiple devices. The unified communication
system may be implemented in hardware or a suitable combination of
hardware and software, and may comprise one or more software
systems operating on a digital signal processing platform. The
"hardware" may comprise a combination of discrete components, an
integrated circuit, an application-specific integrated circuit, a
field programmable gate array, a digital signal processor, or other
suitable hardware. The "software" may comprise one or more objects,
agents, threads, lines of code, subroutines, separate software
applications, two or more lines of code or other suitable software
structures operating in one or more software applications or on one
or more processors. Embodiments may comprise the unified
communication system operating as or in a mobile switching center,
network gateway system, Internet access node, application server,
IMS core, service node, or some other communication system,
including any combination thereof.
[0035] Similar to the communication device 102, each of the
terminals 104 may be associated with various devices which may
include, but are not limited to, a camera, display device,
microphone, speakers, and one or more codecs, or any other type of
conferencing hardware, or in any combination thereof. The terminals
104 may comprise video, voice and data communications capabilities
(for e.g., videoconferencing capabilities) by being coupled to or
including, various audio devices (e.g., microphones, audio input
devices, speakers, audio output devices, telephones, speaker
telephones, etc.), various video devices (e.g., monitors,
projectors, displays, televisions, video output devices, video
input devices, cameras, etc.), various networks (IP, PSTN, etc.) or
any combination thereof. Each of the terminals 104 may comprise or
implement one or more real time protocols, e.g., session initiation
protocol (SIP), H.261, H.263, H.264, H.323, among others.
[0036] FIG. 2A illustrates the exemplary virtual multipoint control
unit of FIG. 1A, according to an embodiment of the present
disclosure. The virtual multipoint control unit 112 may comprise
one or more processor(s) 202, one or more interface(s) 204, and a
memory module 206. The processor(s) 202 may execute a machine
readable program comprising instructions for manipulating the
received video signal. The processor(s) 202 may comprise, for
example, microprocessors, microcomputers, microcontrollers, digital
signal processors, central processing units, state machines, logic
circuits, and/or any devices that manipulate signals based on
operational instructions. The virtual multipoint control unit 112
may further comprise, in whole or in part, a software application
working alone or in conjunction with one or more hardware
resources. Such software applications may be executed by one or
more processors on different hardware platforms or emulated in a
virtual environment. Aspects of the virtual multipoint control unit
112 may leverage known, related art, or later developed
off-the-shelf software. Among other capabilities, the processor(s)
202 may be configured to fetch and execute instructions in computer
readable memory module 206.
[0037] The interface(s) 204 may coordinate interactions of the
virtual multipoint control unit 112 with at least one of the
communication device 102 and the terminals 104 over the network
106. The interface(s) 204 may comprise a variety of known, related
art, or later developed interfaces, such as (1) software
interfaces, for example, an application programming interface, a
graphical user interface, etc.; (2) hardware interfaces, for
example, cable connectors, keyboards, touchscreen, scanners,
display screens, etc.; or both. The interface(s) 204 facilitate
receiving of the audio, video, or data signals, and reliable
broadcast or multicast transmissions of one or more encoded output
signals.
[0038] The memory module 206 may comprise any computer-readable
medium known in the art, comprising, for example, volatile memory
(e.g., RAM) and/or non-volatile memory (e.g., flash, etc.). The
memory module 206 may comprise UC applications 208-1, . . . , 208-N
(collectively referred to as UC applications 208), a virtual video
mixer 210, a virtual audio mixer 212, virtual imaging devices
214-1, . . . 214-N (collectively, virtual imaging devices 214),
virtual audio devices 216-1, . . . , 216-N (collectively, virtual
audio devices 216), and a device driver module 218.
[0039] The virtual multipoint control unit 112 may be configured to
implement various real-time and non-real time communication
protocols for rendering and transmitting various audio, video,
textual, or symbolic communication signals from the UC applications
208. In some embodiments, the virtual multipoint control unit 112
may be configured to determine various characteristics of the
endpoints such as the communication device 102 and the terminals
104 for handling the respective received signals. The
characteristics may include, but are not limited to, type of
endpoint (for e.g., a mobile phone, a laptop, an IP television,
etc.), supported video resolution, supported codecs, network
connection speed, and so on.
[0040] The virtual video mixer 210 may be configured to receive a
decoded video stream from each of the UC applications 208 or a
physical local imaging device such as the imaging device 108, which
is associated with the communication device 102. The virtual video
mixer 210 may be further configured to switch between various video
streams received from different UC applications 208 to either
continuously display a received video stream while operating in a
continuous presence mode, or tile a video stream received from each
of the UC applications 208 from which a respective audio stream is
actively received while operating in a voice-switched mode.
[0041] The virtual audio mixer 212 may be configured to receive an
input audio stream either from a physical local audio device such
as the audio device 110 (for e.g., a microphone) or a virtual audio
device 216 (for e.g., a virtual sound card) in communication with
each of the UC applications 208. The virtual audio mixer may use
the received audio streams to create a mixed and re-encoded audio
stream for being played or transmitted to one or more UC
applications 208 simultaneously. The re-encoded audio stream
excludes the input audio stream received from its corresponding
virtual source, discussed below in greater detail.
[0042] The virtual imaging devices 214 and the virtual audio
devices 216 may be logical representations of an imaging device
(for e.g., a camera) and an audio device (for e.g., a speaker, a
microphone, etc.) with no physical counterparts. The virtual
imaging device 214 may be an image derived from the virtual video
mixer 210. The virtual imaging device 214 may provide either a
tiled image of active participants, or an image of the active
speaker, received from the respective UC applications, depending on
whether the virtual video mixer 210 is operating in voice-switched
or mixer (i.e., continuous presence) mode.
[0043] The virtual audio devices 216 may be virtual representations
of an audio device, which may be emulated by each of the UC
applications 208. Similar to the virtual imaging device 214, the
virtual audio device 216 has no physical counterpart and derives
its input audio stream from the virtual audio mixer 212 as well as
outputs audio streams to the virtual audio mixer 212. Each of the
virtual imaging devices 214 and the virtual audio devices 216 may
be created for each of the UC applications 208 for independently
supporting the audio and video data streams being received from or
sent to the endpoints or any other unified communication system
simultaneously.
[0044] The device driver module 218 may comprise device drivers,
which are software programs that introduce hardware devices such as
endpoints (e.g., the communication device 102 and the terminals
104) and associated devices (e.g., the imaging device 108 and the
audio device 110) to the virtual multipoint control unit 112. The
device drivers handle software instructions received from the UC
applications 208 for accessing the hardware devices and associated
resources (e.g., attached peripheral devices, hardware memory,
etc.) without causing conflicts.
[0045] Further, as shown in FIG. 2B, at least one of the UC
applications such as the UC application 208-1 may comprise an audio
encoder/decoder 220, a video encoder/decoder 222, a display window
224, and a virtual source 226. The audio encoder/decoder 220 may be
configured to decode and encode audio streams received from or sent
to the virtual audio mixer 212 respectively using known, related
art, or later developed encoding/decoding protocols and standards.
Similarly, the video encoder/decoder 222 may be configured to
decode and encode the video streams received from or sent to the
virtual video mixer 210 respectively using known, related art, or
later developed encoding/decoding protocols and standards. The
display window 224 refers to an output software interface
configured to display the video streams/signals. The virtual source
226 may be configured to provide decoded video data stream to the
virtual video mixer 210 via the display window 224.
[0046] FIG. 3 is a schematic that illustrates a virtual multipoint
control unit 300 managing a multipoint video conference, according
to an embodiment of the present disclosure. In one embodiment, the
virtual multipoint control unit 300 may comprise a virtual video
mixer 302, a virtual audio mixer 304, and a device driver module
306. The virtual video mixer 302 may be in communication with one
or more physical local imaging devices such as a camera 308 and the
device driver module 306 may be in communication with one or more
physical local audio devices such as a local speaker 310 and a
local microphone 312. The device driver module 306 comprises device
drivers for operating or managing the respective one or more
physical hardware devices such as the camera 308, the local speaker
310, and the microphone 312.
[0047] The virtual multipoint control unit 300 may further comprise
various videoconferencing software applications such as UC
applications 314-1, 314-2, and 314-3 (collectively, UC applications
314) for conducting a video conference among multiple endpoints,
for e.g., the communication device 102 and the terminals 104. The
first UC application 314-1 comprises a first audio encoder/decoder
316-1, a first video encoder 318-1, a first video decoder 320-1, a
first display window 322-1, and a first virtual source 324-1.
Similarly, the second UC application 314-2 comprises a second audio
encoder/decoder 316-2, a second video encoder 318-2, a second video
decoder 320-2, a second display window 322-2, and a second virtual
source 324-2. The third UC application 314-3 comprises a third
audio encoder/decoder 316-3, a third video encoder 318-3, a third
video decoder 320-3, a third display window 322-3, and a third
virtual source 324-3.
[0048] The display windows 322-1, 322-2, 322-3 (collectively,
display windows 322) may be configured to receive the decoded video
streams from the respective video decoders 320-1, 320-2, 320-3
(collectively, video decoders 320) for displaying the video streams
on a local display device such as an HDTV display.
[0049] The virtual sources 324-1, 324-2, 324-3 (collectively,
virtual sources 324) may be configured to receive the decoded video
data streams via the respective display windows 322 for being sent
to the virtual video mixer 302, which combines the received video
streams and renders a single video stream (e.g., to display images
from each of the video streams in a tiled format) to reduce the
required network bandwidth for display or recording.
[0050] Each of the UC applications 314 may communicate with a
respective set of UC virtual devices to allow the UC applications
314 to access the physical hardware devices, which are in
communication with the virtual multipoint control unit 300
simultaneously. For example, the UC application 314-1 may
communicate with a first set of UC virtual devices including a
first UC virtual camera 326-1 and a first UC virtual sound card
328-1, and the UC application 314-2 may communicate with a second
set of UC virtual devices including a second UC virtual camera
326-2 and a second UC virtual sound card 328-2. Similarly, the UC
application 314-3 may communicate with a third set of UC virtual
devices including a third UC virtual camera 326-3 and a third UC
virtual sound card 328-3.
[0051] Further, each set of the UC virtual devices may emulate a
physical device by defining a software configuration and a related
hardware configuration, both being compatible with the respective
UC applications. Correspondingly, each of the UC virtual cameras
326-1, 326-2, 326-3 (collectively, UC virtual cameras 326) may be a
logical representation or an image of an imaging device (e.g., a
camera) without being mapped or corresponding to a physical camera.
Similarly, each of the UC virtual sound cards 328-1, 328-2, 328-3
(collectively, UC virtual sound cards 328) may refer to a logical
representation or an image of an audio device (e.g., a speaker or a
microphone) without being mapped or corresponding to a physical
audio device. The UC virtual cameras 326 may be derived from the
virtual video mixer 302 and the UC virtual sound cards 328 may be
derived from the virtual audio mixer 302 by the respective UC
applications 314.
[0052] During a video conference, multiple endpoints such as the
communication device 102 and the terminals 104 may execute various
videoconferencing software applications similar to the UC
applications 314 in the virtual multipoint control unit 300. Each
of these videoconferencing software applications may interact with
the corresponding UC applications 314 in the virtual multipoint
control unit 300 during a single communication session to conduct
the video conference, since the videoconferencing software
applications may not be compatible with each other due to different
control protocols and media formats being used. The virtual
multipoint control unit 300 may operate in four distinct modes,
namely, inbound audio mode, inbound video mode, outbound audio
mode, and outbound video mode, to handle any audio, video, or data
streams to and from these UC applications 314, while simultaneously
providing one or more local hardware devices to each of the
participating UC applications 314.
[0053] In the inbound audio mode, the endpoints such as the
communication device 102 and the terminals 104 may capture local
audio data stream and provide encoded audio streams via UC
applications 314 (for e.g., Microsoft Lync, Skype, Cisco IP
communicator, etc.) executing on each of these endpoints. The
provided encoded audio streams may be received by the audio
encoder/decoders 316-1, 316-2, 316-3 (collectively, audio
encoders/decoders 316) of the same UC applications 314 running in
the virtual multipoint control unit 300. The received audio streams
may be encoded in any of the known, related art, or later developed
real-time audio protocols such as RTP (real-time transport
protocol). The audio encoder/decoders 316 may be configured to
decode the received audio streams and send the decoded audio
streams to the respective UC virtual sound cards 328, which may
route the decoded audio streams to the virtual audio mixer 304 via
the virtual video mixer 302. The UC virtual sound cards 328
facilitate to provide the decoded audio streams to the virtual
audio mixer 304 from multiple UC applications 314
simultaneously.
[0054] The decoded audio streams may facilitate the operation of
the virtual video mixer 302 being configured to operate in the
voice-switched mode. The received audio streams may be summed at
the virtual audio mixer 304 may be configured to perform a
summation of the received audio streams to generate a single audio
stream so as to minimize audio interferences or background noise as
known in the art. The generated audio stream may be sent to one or
more physical audio devices such the local speaker 310. When the
generated audio stream is played by the local speaker 310, the
virtual audio mixer 304 may send a notification to the virtual
video mixer 302 to allow synchronization between the audio and the
video streams. In some embodiments, the virtual video mixer 302 may
add delay to a received video stream to synchronize it with the
audio stream.
[0055] In the outbound audio mode, the local microphone 312 may
capture audio data stream of the local participants and send the
captured audio data stream to the virtual audio mixer 304. The
virtual audio mixer 304 may send the captured audio stream to the
audio encoder/decoders, such as the audio encoder/decoder 316-1,
via the respective UC virtual sound card 328-1 of a selected UC
application, such as the UC application 314-1, among the available
UC applications 314. Such selection of the UC application may be
performed on-the-fly by a user or may be preprogrammed. The audio
encoder/decoder 316-1 may encode the received audio data stream
using a predetermined real-time protocol known in the art, related
art or later developed, and transmit the encoded audio data stream
to a predetermined endpoint such as the communication device 102 or
the terminals 104. The audio streams may be encoded using any of
the known, related art, or later developed real-time audio
protocols such as RTP (real-time transport protocol). The audio
encoder/decoders 316-2 and 316-3 may also operate in a similar
manner.
[0056] In the inbound video mode, the endpoints such as the
communication device 102 and the terminals 104 may capture video
data stream of participants local to the endpoints and provide
encoded video streams via the videoconferencing software
applications being executed on each of these endpoints. The
provided encoded audio streams may be received by the video
decoders 320-1, 320-2, 320-3 (collectively, video decoders 320) of
the respective UC applications 314, which were used at the
endpoints to send the video streams to the virtual multipoint
control unit 300. The received video streams may be encoded in any
of the known, related art, or later developed real-time video
protocols such as H.264, H.261, Scalable video coding (SVC), of any
other protocol known in the art, related art, or developed later.
The video decoders 320 may be configured to decode the received
video streams and send the decoded video streams to the respective
UC virtual cameras 326-1, 326-2, 326-3 (collectively, UC virtual
cameras 326), which may route the decoded video streams to the
virtual video mixer 302.
[0057] The UC virtual cameras 326 facilitate to provide the decoded
video streams for use from multiple UC applications 314
simultaneously. The virtual video mixer 302 may perform a summation
of the received video streams to generate a single video stream for
at least one of (1) minimizing video interferences or background
noise; and (2) represent the video images from different decoded
video streams as tiles for display. The decoded video streams may
be displayed on a display device (not shown) (e.g., an interactive
display, HDTV display, etc.) on the basis of the virtual video
mixer 302 being preset to operate in continuous presence mode or
the voice-switched mode.
[0058] In the outbound video mode, the local camera 308 may capture
the video data stream of the local participants and send the
captured video data stream to the virtual video mixer 302. The
virtual video mixer 302 sends the captured video stream to a video
encoder such as the video encoder 318-1 via the UC virtual camera
326-1 of a selected UC application, such as the UC application
314-1, among the available UC applications 314. Such selection of
the UC application 314-1 may be performed on-the-fly by a user or
may be preprogrammed. The video encoders 320 may encode the
received video data stream using a predetermined real-time protocol
known in the art, related art or later developed and transmit the
encoded video data stream to a predetermined endpoint such as the
communication device 102 or the terminals 104. The video streams
may be encoded using any of the known in the art, related art, or
later developed real time protocols, e.g., session initiation
protocol (SIP), H.261, H.263, H.264, H.323, etc.
[0059] FIG. 4 is a schematic implementation of a device driver
module of the virtual multipoint control unit for communicating
with hardware devices, according to an embodiment of the present
disclosure. Illustrated embodiments are disclosed in the context of
a video conference environment 400 including (1) a
videoconferencing system 402 (e.g., CLEARONE Collaborate Room)
comprising a video encoder 404 and a video decoder 406; (2) a UC
application 408 comprising a UC video encoder 410 and a UC decoder
412, (3) a physical camera 414, (4) a display device 416, and (5)
the device driver module 306 of the virtual multipoint control unit
300. The device driver module 306 comprises a video mixer driver
418 and a video display mixer driver 420. The physical camera 414
may be in communication with the video conferencing system 402 and
the UC application 408 via the video display mixer driver 418. The
display device 416 may be in communication with the
videoconferencing system 402 and the UC application 408 via the
video display mixer driver 420.
[0060] The operation of the virtual multipoint control unit 300 may
be described in four different modes. In the first mode, the
physical camera 414 may capture a local image of the participants
and provide a video stream for transmission. The video mixer driver
418 may be configured to transmit the captured video stream to a
physical hardware such as the video conferencing system 402 or to
the UC application 408 based on a transmission mode selected by a
user. When the transmission mode indicates for a hardware to be
used for transmission, the video mixer driver 418 guides the
captured video stream to the video encoder 404, which may encode
the video stream using any of the known in the art, related art, or
later developed encoding algorithms for real-time network
transmission using a predetermined network protocol such as H.263,
SIP, etc.
[0061] When the transmission mode indicates for the UC application
408 to be used for transmission, the video mixer driver 418 guides
the captured video stream to the UC video encoder 410, which may
encode the video stream using any of the known in the art, related
art, or later developed encoding algorithms for real-time network
transmission using a predetermined network protocol such as H.263,
SIP, etc. In some embodiments, the captured video stream may be
simultaneously sent to the video encoder 404 and the UC video
encoder 410 for encoding and network transmission.
[0062] Similarly, the video display mixer driver 420 may be
configured to manage an encoded video stream received from a
network such as the network 106 via the videoconferencing system
402. The received video stream may be encoded using any of the
known in the art, related art, or later developed algorithms
according to predetermined network protocols. The video decoder 406
may receive and decode the encoded video stream to generate a
decoded video stream, which may be sent to the video mixer driver
418. The video mixer driver 418 may be configured to guide the
decoded video stream to the video display mixer driver 420, which
may transmit the decoded video stream to the display device 416 for
display. Alternatively, the encoded video stream may be received by
the UC video decoder 412, which may be configured to decode the
encoded video stream and send a generated decoded video stream to
the display device 416 via the video display mixer driver 420 for
display.
[0063] In one embodiment, a virtual multipoint control unit 112,
300 is in communication with a plurality of devices over a network
106. The virtual multipoint control unit 112, 300 comprises a
plurality of UC applications 208, 314, at least one virtual imaging
device 214, at least one virtual audio device 216, a virtual video
mixer 210, and a virtual audio mixer 212. The plurality of UC
applications 208, 314 is executed on at least one of the plurality
of devices 102, 104. Each of the plurality of UC applications 208,
314 receives an encoded audio data stream and an encoded video data
stream. The plurality of UC applications 208, 314 decodes the
received audio data and the video data streams. The at least one
virtual imaging device 214 is mapped to each of the plurality of UC
applications 208, 314. The at least one virtual audio device 216 is
mapped to each of the plurality of UC applications 208, 314. The
virtual video mixer 210 is in communication with a physical imaging
device 108. The virtual video mixer 210 receives the decoded video
data stream from each of the plurality of UC applications 208, 314
via the at least one virtual imaging device 214. The virtual audio
mixer 212 is in communication with a physical audio device 110. The
virtual audio mixer 212 receives the decoded audio data stream from
each of the plurality of UC applications 208, 314 via the at least
one virtual audio device 216.
[0064] In another embodiment, a system for managing communication
among a plurality of devices is disclosed. The system is employed
within a communication network 106. The system comprises a
communication device 102 and a virtual multipoint control unit 112,
300. The communication device 102 receives encoded audio and video
data streams via a plurality of UC applications 208, 314. The
plurality of UC applications 208, 314 decodes the received encoded
audio data stream and the received encoded video data stream. The
virtual multipoint control unit 112, 300 is in communication with
the communication device 102. The virtual multipoint control unit
112, 300 comprises at least one virtual imaging device 214, at
least one virtual audio device 216, a virtual video mixer 210, and
a virtual audio mixer 212. The at least one virtual imaging device
214 is mapped to each of the plurality of UC applications 208, 314.
The at least one virtual audio device 216 is mapped to each of the
plurality of UC applications 208, 314. The virtual video mixer 210
is in communication with a physical imaging device 108. The virtual
video mixer 210 receives the decoded video data stream from each of
the plurality of UC applications 208, 314 via the at least one
virtual imaging device 214. The virtual audio mixer 212 is in
communication with a physical audio device 110. The virtual audio
mixer 212 receives the decoded audio data stream from each of the
plurality of UC applications 208, 314 via the at least one virtual
audio device 216.
[0065] In yet another embodiment, a non-transitory computer
readable medium storing a program of instructions executable by the
computing device to perform a method for employing a virtual
multipoint control unit 112, 300 for unified communications is
disclosed. The method comprises receiving, from a plurality of
terminals 104, an encoded audio data stream and an encoded video
data stream. The method also comprises decoding, using a plurality
of UC applications 208, 314, the received encoded audio data stream
and the received encoded video data stream. The method further
comprises mapping, using a virtual multipoint control unit 112,
300, at least one virtual imaging device 214 and at least one
virtual audio device 216 to each of the plurality of UC
applications 208, 314. The method also comprises communicating,
using a virtual video mixer 210, the decoded video data stream to a
physical imaging device 108 via the at least one virtual imaging
device 214. Furthermore, the method comprises communicating, using
a virtual audio mixer 212, the decoded audio data stream to a
physical audio device 110 via the at least one virtual audio device
216.
[0066] In still another embodiment, a method for managing
communication among a plurality of devices is disclosed. The method
comprises receiving, from a plurality of terminals 104, an encoded
audio data stream and an encoded video data stream. The method also
comprises decoding, using a plurality of UC applications 208, 314,
the received encoded audio data stream and the received encoded
video data stream. The method further comprises mapping, using a
virtual multipoint control unit 112, 300, at least one virtual
imaging device 214 and at least one virtual audio device 216 to
each of the plurality of UC applications 208, 314. The method also
comprises communicating, using a virtual video mixer 210, the
decoded video data stream to a physical imaging device 108 via the
at least one virtual imaging device 214. Furthermore, the method
comprises communicating, using a virtual audio mixer 212, the
decoded audio data stream to a physical audio device 110 via the at
least one virtual audio device 216.
[0067] Other embodiments of the present invention will be apparent
to those skilled in the art after considering this disclosure or
practicing the disclosed invention. The specification and examples
above are exemplary only, with the true scope of the present
invention being determined by the following claims.
* * * * *