U.S. patent application number 15/405201 was filed with the patent office on 2017-07-20 for architecture for a media system.
The applicant listed for this patent is Arria Live Media, Inc.. Invention is credited to David B. ANDERSEN, Edward D. ARRINGTON, Carol BELL.
Application Number | 20170208112 15/405201 |
Document ID | / |
Family ID | 59315319 |
Filed Date | 2017-07-20 |
United States Patent
Application |
20170208112 |
Kind Code |
A1 |
ARRINGTON; Edward D. ; et
al. |
July 20, 2017 |
ARCHITECTURE FOR A MEDIA SYSTEM
Abstract
A media system that includes one or more smart devices.
Inventors: |
ARRINGTON; Edward D.;
(Hillsboro, OR) ; BELL; Carol; (Beaverton, OR)
; ANDERSEN; David B.; (Forest Grove, OR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Arria Live Media, Inc. |
Hillsboro |
OR |
US |
|
|
Family ID: |
59315319 |
Appl. No.: |
15/405201 |
Filed: |
January 12, 2017 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62308650 |
Mar 15, 2016 |
|
|
|
62280249 |
Jan 19, 2016 |
|
|
|
62280235 |
Jan 19, 2016 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04L 12/10 20130101;
H04L 65/605 20130101; H04L 65/608 20130101; H04W 84/12 20130101;
H04L 65/4076 20130101; H04L 65/607 20130101 |
International
Class: |
H04L 29/06 20060101
H04L029/06; H04L 12/10 20060101 H04L012/10 |
Claims
1. A media system comprising: (a) a smart audio device including at
least one of: (i) said smart audio device capable of sending an
audio stream and associated configuration data together with said
sent audio stream on a network; and (ii) said smart audio device
capable of receiving an audio stream and associated configuration
data together with said received audio stream on said network.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional App.
No. 62/308,650, filed Mar. 15, 2016, U.S. Provisional App. No.
62/280,249, filed Jan. 19, 2016 and U.S. Provisional App. No.
62/280,235, filed Jan. 19, 2016.
FIELD OF THE INVENTION
[0002] The present invention relates to a media system.
BACKGROUND OF THE INVENTION
[0003] Media systems receive audio and/or video media streams from
one or more sources, process the media streams in some manner, and
then distribute the one or more resulting media streams to one or
more output devices which may include speakers, video monitors, and
recording devices.
[0004] A mixing console or audio mixer, generally referred to as a
sound board is an electronic device for combining audio signals,
routing the received and/or combined audio signals, and changing
the level, timbre, and/or dynamics of the audio signals. The
modified signals are combined together to produce combined output
signals.
[0005] Multiple mixers may be used where the mixers perform
sub-mixing. The mixing of the audio signals occurs in a
hierarchical fashion, with groups of signals being pre-mixed in one
mixer, and the result of that pre-mix being fed into another mixer
where it is combined with other individual signals or other
pre-mixes coming from other sub-mixers.
[0006] Referring to FIG. 1, traditionally media systems are
implemented using a central mixing console that is responsible for
combining, processing, and distributing the various media streams.
The system's connectivity is configured with the central device as
a hub and point-to-point connections extending between the central
device and each input or output device.
[0007] Furthermore, in most instances, these point-to-point
connections between the central device and endpoints usually
accommodate only the media signal itself. The central device does
not typically have a manner of determining what endpoint devices
are connected to it, nor a manner of directly controlling the
endpoint devices.
[0008] For example, in the case of a sound reinforcement system for
live performance, the central device is the audio mixing console.
The endpoint devices are microphones, instruments, and speakers,
and the connectivity between each of these endpoints and the mixing
console is an analog cable.
[0009] The mixing console cannot determine by itself which of its
ports have endpoint devices connected, nor can it determine what
endpoint device is connected to a given port, nor can it directly
control endpoint devices. As a result, signal routing is often very
complex and it is very common for errors to occur when setting up
the many signal paths required in a typical sound system.
[0010] Because the mixing console cannot determine how many of its
ports have endpoint devices connected, it must always present the
user with control capabilities for all possible ports. So even if
there is only one microphone and one speaker connected, the user
must still cope with a complicated control interface that may
support dozens of endpoint devices. Also, the inability to control
endpoints often makes it necessary for a system operator to
physically go to where the endpoint devices are located in order to
adjust endpoint device settings such as power on/off, gain,
frequency, etc.
[0011] Referring to FIG. 2, in the case of a consumer entertainment
system, the central device is an A/V receiver, and the endpoints
are devices such as cable boxes, DVRs, media streamers, speakers,
and video monitors. Connectivity between the endpoints and the
central device is usually implemented either with one or more
analog cables, optical cables, or via digital HDMI cables, all of
which are strictly point-to-point connections.
[0012] While HDMI cables may provide for exchange of some limited
device identification and control information, analog and optical
cables do not. So, in the general case, the A/V receiver does not
necessarily know which of its ports have devices connected, what
the connected devices are, or have a way to control those devices.
This gives rise to the alarmingly large collection of remote
control units needed to operate a typical consumer entertainment
system, which in turn makes such systems so very difficult to
fathom and vexing to use.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] FIG. 1 illustrates a media system.
[0014] FIG. 2 illustrates a consumer entertainment system.
[0015] FIG. 3 illustrates a set of smart devices in a peer to peer
network configuration.
[0016] FIG. 4 illustrates a set of smart devices in a peer to peer
configuration together with a network.
[0017] FIG. 5 illustrates a set of smart devices in a peer to peer
configuration together with a network and a control panel.
[0018] FIG. 6 illustrates another set of smart devices in a peer to
peer configuration together with a network and a control panel.
[0019] FIG. 7 illustrates an audio input converter.
[0020] FIG. 8 illustrates an audio output converter.
[0021] FIG. 9 illustrates a smart output converter.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0022] Architecting media systems around a sophisticated central
device has been the prevailing practice for many decades. This is
because media systems, by their very nature, require
synchronization and control coordination of all audio and video
streams. Historically, the only technically viable and
cost-effective way to implement the needed synchronization,
control, and functionality has been to incorporate all of the
"intelligence" in a sophisticated central device and utilize
point-to-point connections that carry only a media stream, to
relatively less sophisticated end points.
[0023] However, when media systems utilize this central device
architecture, the intrinsic feature set and capacities of the
central device imposes constraints on the media system as a whole.
In particular the central device determines the media system's
capacity, as measured by the number of endpoints (both input and
output devices) that can be accommodated. In particular, the
central device also determines the media system's set of processing
features. In particular, the central device may further determine
the media system's control mechanisms and methodologies.
[0024] Expanding either the system capacity or the feature set or
changing the system control mechanisms (for example to provide
remote control via a tablet) generally means replacing an existing
central device with a more capable one. Furthermore, connecting a
sophisticated central device to the endpoint devices using
point-to-point links that carry no information other than the media
stream itself results in media systems being very complex to
configure, being subject to frequent configuration errors that are
difficult to find, and being very complicated to operate. In
general, sound reinforcement systems built around audio mixing
consoles or consumer entertainment systems built around A/V
receivers are difficult and complicated to configure and
operate.
[0025] High capacity digital networking may be used as a
communication backbone to facilitate re-architected media systems
in ways that facilitate many compelling advantages. One of the
resulting advantages, with a suitably re-architected media system
is to greatly simplify the tasks of configuring and setting up a
media system. Another of the resulting advantages, with a suitably
re-architected media system is allowing media devices to be
dynamically inserted into and removed from a functioning media
system with plug and play simplicity. Another of the resulting
advantages, with a suitably re-architected media system is to
significantly improving ease of operation. Yet another of the
resulting advantages, with a suitably re-architected media system
is enabling a media system's capacity to scale incrementally
without obsoleting or needing to replace other components. Yet
another of the resulting advantages, with a suitably re-architected
media system is allowing additional functionality to be introduced
without obsoleting or needing to replace other components. Moreover
one of the resulting advantages, with a suitably re-architected
media system is reducing the number of components needed to
implement a media system.
[0026] Referring to FIG. 3, it is desirable to make each endpoint
device of a media system smart by directly incorporating media
processing capabilities appropriate to the endpoint type (input or
output) and connecting each smart endpoint device via a
packet-switched digital network communication system. A system
control protocol may be implemented in a distributed fashion across
all of the smart endpoint devices. Collectively, the smart endpoint
devices implement all of the required functionality while also
enabling additional functionality. In general, each of the smart
devices include a processor and memory, together with input and/or
output to the network, and input and/or output to an audio source
(as an example) and/or a speaker source (as an example).
[0027] The intelligence and functionality that used to be
instantiated within a sophisticated central device is thus moved
out, at least in part, to the smart endpoint devices which operate
in a peer-to-peer fashion among other smart endpoint devices. This
peer to peer approach eliminates the need for a sophisticated
central device and the attending limitations imposed by such
devices.
[0028] Referring to FIG. 4, the digital network enables each smart
endpoint device to directly communicate with every other device
using both point-to-point and point-to-multipoint transmission
methods. The connection topology is peer-to-peer. The network
simultaneously carries multiple synchronized media streams,
metadata associated with media streams, and the messages needed for
the endpoint devices to collectively implement the system control
protocol.
[0029] The system control protocol allows endpoint devices to be
dynamically inserted or removed from the media system, using any
available network port, with plug and play simplicity. Adding an
endpoint device to the system may be as simple as connecting a USB
mouse to a personal computer. Upon adding an endpoint device to the
network, it just shows up and is ready to be used. Thus no central
panel needs to be configured to incorporate a new endpoint
device.
[0030] The system control protocol also ensures that all media
streams are properly synchronized and automatically routed from
input devices to output devices with no operator intervention
required and with very low latency. It maintains overall system
state in a cohesive and robust manner. It also provides all of the
information needed for a user employing a control application,
typically (though not necessarily) running on a mobile device, to
see all of the connected components and easily operate the system
as desired, as illustrated in FIG. 5.
[0031] While the media system is operating, each smart input device
multicasts its media streams on the network to all smart output
devices, preferably including itself. System control messages also
broadcast on the network instructing each smart output device as to
how it should combine and enhance the received audio streams or
select from amongst (and then possibly also enhance) the various
video streams in order to render the specific output (sound or
video image) that is needed from it.
[0032] Referring to FIG. 6, a sound reinforcement system is
illustrated. Each of the various components are connected to the
digital network 80 using any convenient port. Since this system
preferably utilizes Ethernet for the digital network, devices would
be connected to ports on one or more Ethernet switches. Unlike
conventional sound reinforcement systems, it does not matter which
port each component is plugged into.
[0033] For example, the sound to be reinforced may originate with
two smart vocal microphones 10 and 20 and a smart electric guitar
30. Each of these input devices multicast their corresponding input
audio stream to each of the smart output devices. The sound heard
by the audience is as a stereo sound image produced by the
combination of smart speakers 40 and 50. The performers use smart
stage monitors 60 and 70, each of which produces a separate mono
sound image, to help them hear better and thus perform better.
[0034] A WiFi adaptor 90 is also connected to the digital network
80, to allow a wifi enabled tablet device 100, running a system
control application 110, to act as the system control device. The
various media streams preferably do not flow over the WiFi link in
order to avoid a significant increase in end-to-end system latency,
and to avoid overwhelming the WiFi link with high traffic
levels.
[0035] As it may be observed, no central mixing console or mixing
engine is needed since all of the media processing may happen
directly in the various endpoint devices. End-to-end system latency
remains at a low value (approximately 2 ms) because each media
stream is transmitted through the network exactly once.
[0036] Furthermore, because the system is controlled via a system
control protocol, multiple instances of the control application can
be run simultaneously on separate mobile devices. Performers could
use their personal smart phones to control their own monitor mixes,
while a sound engineer uses a tablet device to control the sound
the audience hears. If desired, a hardware audio control surface
with multiple faders, knobs, and switches could also be used to
control the system. In this case software running on the control
surface would translate between hardware control settings and
system control protocol messages.
[0037] There is functionality that is preferably common to all
smart audio endpoints. In the description provided herein,
"endpoints" and "devices" are used interchangeably to describe
devices that are used for input and/or output. One of the
characteristics of all devices described herein is that each device
provides either audio input and/or audio output, though preferably
in most cases not both (although in limited cases, such as an
intercom headset, both input and output may exist in the same
enclosure, though they remain functionally independent. Input
devices and output devices may be combined into a single package,
but each side acts as an input or output device separately. There
is preferably no "short-cut" connection between input and output of
a particular device. In this manner the output is provided to the
network from a device and the input is received from the network
for the same device. As described the input devices and output
devices--which primarily convert audio between the analog and
digital domains--network connectivity, audio sample rate
coordination, and implementation of the system control protocol are
consistent for all devices.
[0038] With respect to network connectivity, all devices have a
connection to a digital (normally packet-switched) network such as
an Ethernet network. This Ethernet connection is based on industry
standards, and may use both layer 2 (Data Link) and layer 3 (IP
Network) protocols for various purposes. Data rates are preferably
at least 100 Mbs, but can be gigabit or faster. Because the network
connections use industry standards, virtually all commercially
available network equipment (such as network switches) may also be
used. Power for endpoints can (optionally) be provided by using
Power Over Ethernet (POE). POE may be required for devices that do
not have another power source. Physical Ethernet connections may be
based on industry-standard RJ-45 connections, but may also be made
using more robust Ethercon.TM. connectors, which are also fully
compatible with RJ-45 connectors.
[0039] With respect to system wide clocking, all system devices are
preferably synchronized to a common digital clock. This may be done
through an implementation of the industry standard IEEE 1588-2008
protocol, often referred to as Precision Timing Protocol (PTP). PTP
requires one device to act as the clock master, while all other
devices follow. As an industry standard, the IEEE 1588-2008
specification provides information on how the best master clock is
selected among available devices. Such a master-clock mechanism is
used in a peer-to-peer environment, where devices may join or leave
the network at any point in time. When a device that is acting as
master clock is removed from the network, another device then
provides the master clock service. IEEE 1588-2008 also allows for
others clocks, such as clocks that are highly precise (GPS-based,
for example) to provide master clock services.
[0040] With respect to audio sample rate coordination, every device
on the network using network timing provided by PIP, the sample
rate used to convert analog signals to digital, or to convert from
digital signals to analog--a capability used by all smart audio
devices, may be tightly coordinated. In fact, the sample rates on
all smart devices on the network are preferably aligned with one
another. Accordingly, the sampling rate should be the same for all
the smart devices, and if a particular device has more than one
potential sampling rate it should select a sampling rate that is
common to all the other devices on the network. Even minor changes
in audio sample rates may result in undesirable audible effects
including pops, clicks, and jitter. All smart devices may use an
aligned audio sampling rate to maintain synchronization of audio
sampling across all devices on the network. Each device may be
periodically checking sample rates and, as needed, making
relatively minor adjustments in its sampling rate to maintain
precision. This audio timing mechanism may use the capabilities of
a system control protocol to maintain precision and minimize
jitter.
[0041] With respect to the system control layer, distributed
implementation of the system control protocol across all of the
smart input and output devices provides added functionality. The
distributed nature of the functionality permits independent and
disparate media devices to act cohesively and collectively as one
system, even as any device may be dynamically removed from or
inserted into the system. To accomplish this, the system control
protocol uses characteristics of digital networks including both
point-to-point and multipoint transmission modes, and the ability
to simultaneously carry multiple high bit rate, uncompressed media
streams, as well as metadata, control commands, and status
information. The system control protocol may be a coordinated set
of instructions designed to make each device respond and act in the
manner desired. The control protocol may have two layers--the
hardware control layer and the application control layer.
[0042] With respect to the hardware control layer of the system
control protocol, it is used to keep all devices and endpoints
coordinated. Hardware control instructions are transmitted and
received by endpoint devices only. No centralized processor is used
for the hardware control layer. In that sense, the system is a true
peer-to-peer system.
[0043] To make this system operate more efficiently, each device
may be a master of itself only. This may be referred to as a single
mater rule. Each input device maintains the settings for itself as
an input, and each output device maintains the settings for itself
as an output. If another device needs to know something about one
of the other devices, it gets that information from the other
device directly. The various devices preferably communicate their
master information to many other devices frequently without
necessarily receiving a request so that all devices can maintain
updated information.
[0044] The hardware control layer provides low-level functionality
by communicating settings to various devices on a need-to-know
basis. For example, an audio input device may, as single master,
maintain settings for volume. That information, however, is
utilized on an audio output device. The input device, as single
master, may communicate to the audio output device what that volume
setting is, and update the output device whenever it changes.
Because of the single master rule, many output devices are able to
track the volume for each individual audio input device, and
maintain control synchronization. The hardware control layer is
normally implemented at the data link layer of the packet-switched
network. Other data may be provided by the input device that is
then used by the output device or other input devices.
[0045] The application control layer provides a mechanism for
applications external to the device to control the parameters of
the various devices. The application control layer is normally
implemented on the network layer of the packet-switched network
using standard Internet protocols such as UDP and TCP/IP. Using the
application control layer, applications can query current settings
and command new settings on the various endpoint devices. For
example, if an application desires to change the volume for a
specific device, the application control layer is used to make the
request of the device (which is the single master) for the new
value. The requested device responds when the change has been
successful.
[0046] With respect to a capability discovery mechanism, the
application control layer is dependent upon a description of the
capabilities, present (and potentially unique) in each device. This
description is referred to as a "schema". Each device has a schema
that describes the functions, settings, attributes, and
capabilities of that device. Each device can have a different
schema. While many schema entries are common between devices (such
as volume), some devices have schema entries for functions or
capabilities that are unique to that device. For example, a speaker
might have the capability of changing the crossover frequency.
Control applications utilize schema information to know how to
properly present the control capabilities of each device.
[0047] With respect to discovery and admission control, as smart
endpoints are connected to the digital network they implement a
discovery protocol to detect already connected system components
and determine which component is currently acting as the master
with respect to admission control. Devices then report in with the
master and seek admission to the system. At this point, without any
operator intervention, devices just appear on the control
application.
[0048] Based on operator preferences, the master appropriately
facilitates several admittance scenarios. One admittance scenario
may be clean start--a device with all default settings is connected
to the network and seeking to be admitted. Another admittance
scenario may be transfer in--a device that still contains settings
and rnetadata from its use in a previous performance seeks to be
admitted. A further admittance scenario may be re-admittance--a
device that had been operating in this system but went offline,
due, say, to a brief power failure, is seeking to be
readmitted.
[0049] Admission policies makes it possible for devices being
re-admitted to quickly reappear on the operator's display without
intervention, while also allowing the operator to decide whether
other devices will be automatically admitted or admitted only after
being re-initialized and only when the operator is ready. If at any
time the device that is currently acting as master for admission
control goes off line, the remaining devices will readily select a
successor. In this eventuality no loss of state occurs, because the
master device keeps other devices constantly updated and ready to
step in if needed.
[0050] With respect to routing and disseminating input media
streams to all output devices, as part of the admission process,
input devices may be provided with network addresses to be used to
multicast their input streams and corresponding mix-specific
metadata. Once admission has taken place, input streams for unmuted
devices are sent continuously to the designated network addresses.
This mechanism eliminates the need for an operator to be involved
in configuring and mapping signal paths. The input streams from all
input devices are simultaneously available for consumption by all
output devices. It also ensures a very low and constant end-to-end
latency, since audio streams are sent across the network exactly
one time.
[0051] With respect to grouping, another capability of the system
is the ability of each device to be "grouped" with other devices.
For example, a group of microphones that are used for backup
vocalists, can be grouped together with a common volume or mute
control. Grouping may be based upon tight coordination between
devices at the hardware control layer, as well as at the
application control layer. Groups create new virtual objects, which
act like a device, but are not actually a physical implementation
of such. Information about the virtual object resides in all group
members, however to maintain the single master rule, only one
device acts as the group master. Groups may be added or removed.
Grouping may also be hierarchical, meaning a group can be a member
of another group. Grouping is useful in reducing the complexity
presented to a system operator. Instead of seeing faders for all 8
mics used on a drum kit, for example, the operator can see just one
for the entire group.
[0052] With respect to robustly maintaining system state, the
device acting as admission control master may also have the job of
maintaining overall system state. This consists of a number of
settings, policies, and assigned values that all components,
including system control applications, may need to access. When a
change in system state is made by, say, an operator using a system
control application, the new value is sent to the master device
which in turn makes it available to all other devices. Redundant
copies of system state information is maintained in other devices
so that "instant" failover can occur should the master device go
offline.
[0053] With respect to persistent storage, all devices may include
non-volatile memory for remembering hardware control settings,
application control settings, and group membership information even
when powered off. This allows devices to be removed from the
network, then come up again as they were previously. Maintaining
non-volatile memory across a distributed peer-to-peer system is
facilitated as a result of the single master rule and coordination
at the hardware control layer.
[0054] As illustrated in FIG. 6, devices 10, 20 and 30 are
considered smart input devices. Examples of smart input devices
include microphones, musical instruments, and audio playback
devices. By way of example, the term "smart" may reference, that at
a minimum each devices includes one or more of the following. The
device may communicate via a digital network 80, which may be for
example, an Ethernet connection. The device may synchronize itself
to a system-wide clocking signal transmitted via the network. The
device may utilize the system-wide clock to synchronously convert
audio signals from the analog to the digital domain. The device may
use multicasting of one or more digital audio streams to smart
output devices. The device may make real-time input level metering
data available to all instances of system controllers. The device
may send device status information and receive commands to set
device modes and parameters. The device may retain operating
parameters and metadata in non-volatile storage. The device may
implement system control protocols. The device may include a
firmware update mechanism, error logging, and direct device
interrogation via network protocols.
[0055] In addition, each smart input device may also keep track of
a comprehensive set of parameters that instruct smart output
devices regarding how the input device's media stream is to be
processed when creating the various output mixes. This includes
input fader level, multiband equalization settings and/or effect
send levels to adjust the amounts of effects such as reverb or echo
to be applied. These mix-specific parameters are transmitted
throughout the system as metadata that is associated with the
device's media stream.
[0056] Implementing the smart input device functionality directly
within the device itself enables delivery of all the features in an
error-free basis, together with true plug and play simplicity for
both system setup and subsequent operation.
[0057] Referring to FIG. 7, to accommodate legacy microphones,
instruments, and other sound sources that do not incorporate the
requisite intelligence, a sound reinforcement system may also
employ one or more audio input converters that allow such devices
to supply an analog audio signal to the system via a legacy analog
cable. Preferably, the audio input converter includes 2 or more
connectors: a combo connector that accepts either a standard XLR
audio cable(s) or a 1/4'' audio cable(s) (such as would typically
be used for an electric guitar), and an Ethernet connector. The
audio input converter implements the smart functionality enumerated
above for smart input devices, while also receiving the legacy
audio input. Other versions of audio input converters can also be
constructed that accommodate other types of audio connectors and
signals including, for example, a converter that accepts line level
stereo signals such as would come from a personal media player.
[0058] Because control settings and metadata are stored within the
converter on behalf of the associated legacy audio source, it is
preferable to preserve a one-to-one relationship between each
legacy audio source and its corresponding audio input
converter.
[0059] In an alternative instantiation, one may gang together
multiple audio converters into a single physical device with
multiple analog input connectors and a single, shared, network
connector. In order to avoid the configuration errors that could
otherwise easily occur with such an instantiation (for example,
mistakenly plugging a guitar into an input port where a drum was
expected) it is preferable that a means be provided for the system
to automatically determine which analog source device is connected
to each input port.
[0060] This can be accomplished by embedding a tiny digital
integrated circuit chip inside the analog source device (for
example a microphone or guitar) when it is manufactured or, in the
case of an existing device, within the device's existing analog
connector. This integrated circuit chip receives power through and
communicates digitally over the existing analog cabling. The
presence of this chip does not in any way alter or degrade the
functionality of the analog endpoint device. Further, circuitry
within the audio input converter interacts, via the analog cabling,
with the digital chip added to the analog source device, and
thereby retrieves from it a unique digital identifier. This unique
identifier is then used to access the set of operating parameters
and metadata that is to be associated with the connected analog
endpoint device.
[0061] The smart audio output devices 40, 50, 60, and 70 will most
often be instantiated as a powered speaker, an audio amplifier that
drives a passive speaker, a network-connected pair of headphones,
and/or an audio recording device. Smart output devices are
preferably capable of one or more of the following. One capability
of the smart output device is communicating via a digital network
80. Another capability of the smart output device is synchronizing
to a system-wide clocking signal transmitted via the network. A
further capability of the smart output device is receiving one or
more multicast digital audio streams along with mix-specific
metadata from other system components. Yet another capability of
the smart output device is implementing mix-specific instructions
associated with each incoming media stream to combine and enhance
the received audio streams, producing a digital "mix" that is
specific to this particular output device. Another capability of
the smart output device is providing real-time output level
metering data to all instances of system controllers. Another
capability of the smart output device is utilizing the system-wide
clock to synchronously convert the digital mix signal into sound
emanating from the associated speaker. Another capability of the
smart output device is sending device status information and
receiving commands to set device modes and parameters. Another
capability of the smart output device is retaining operating
parameters and metadata in non-volatile storage. Another capability
of the smart output device is implementing speaker management
functions. Another capability of the smart output device is
implementing the system control protocols. Another capability of
the smart output device is providing firmware update mechanisms,
error logging, and direct device interrogation via standard
Internet and worldwide web protocols.
[0062] As a convenience to system designers and installers, smart
speakers may also include speaker management functionality. Since
many of these speaker management parameters are set according to a
speaker's installed location within a venue and the speaker's
physical characteristics, provision is included to lock these
settings so that they are not changed inadvertently. Speaker
management functionality may include one or more of the following:
crossover settings, feedback suppression, delay, pink noise
generation, tone generation, and/or level adjust.
[0063] As with smart input devices, the benefits are preferably
implemented directly within each smart audio output device. Since
speakers and amplifiers are usually physically larger and more
expensive devices, embedding this functionality is usually quite
feasible.
[0064] Referring to FIG. 8, where there is a need to use existing,
non-smart sound output devices, such as speakers that are already
installed within a venue, it may be desirable to utilize one or
more smart output converters. This smart output converter may
provide 3 connection points: a short audio cable with a male XLR
audio connector, an Ethernet connector, and a female XLR connector.
The male XLR cable provides a line level audio output that can be
plugged directly into either a conventional powered speaker or into
an audio amplifier. The inclusion of a female XLR connector, which
accepts a line level analog input signal, allows the converter to
be inserted in parallel with an existing legacy sound reinforcement
system.
[0065] Referring to FIG. 9, an exemplary operation of the smart
output converter is illustrated. When the smart output converter is
not powered, the pass through relay is latched in the A position,
so the legacy analog signal passes straight through. When powered
up the relay switches to the B position, and the mix engine
combines both network input and the analog input, with a unity gain
for the analog input.
[0066] The smart output converter of FIG. 9 implements all of the
smart functionality previously enumerated for smart output devices
in general, and smart speakers in particular. Other instantiations
of audio output converters can also be constructed that accommodate
other types of audio connectors and audio signals including, for
example, a converter that delivers audio intended for headphone
monitoring through a standard headphone jack. It is desirable to
maintain a one-to-one relationship between a sound reproduction
device (or recording device) and its associated converter. Doing so
minimizes the potential to introduce configuration errors into the
sound reinforcement system.
[0067] With respect to a system control software development kit,
as previously noted, the system control protocol facilitates
multiple instances of a control application to be used to operate
the system. To make it easier to implement such control
applications the system control software development kit (SDK) may
also be used. The SDK encapsulates the protocol details and
provides a programmatic interface for control applications to use.
The SDK is preferably implemented as a software module that
executes on the same platform that the control application is
implemented on.
[0068] The availability of the system control SDK simplifies the
implementation of different versions of a system control
application. For example, a control application to be used by
performers in controlling their own monitor mix would not provide
access to control other mixes, including the house mix. It could
also be optimized for use on the smaller sized screen of a mobile
phone. A different version of the control application could be made
available for non-technical persons who are renting a venue to be
able to easily adjust the house mix without allowing overall volume
levels to be too high and without exposing all of the detailed
control capabilities that a professional sound engineer might
utilize.
[0069] The system control SDK can also operate in a device
emulation mode so that a sound engineer can pre-configure a show
without needing to be connected to any of the actual devices. Using
this capability the engineer can instantiate all of the various
endpoint devices that will be needed, name the devices, and
establish a set of initial operating parameters. This information
can then be saved to a file and recalled when the actual system is
being configured at the venue. Device emulation mode also provides
a very convenient and safe way for new operators to become familiar
with the various functions and capabilities of the sound system
control application.
[0070] An exemplary type of system is a sound reinforcement system
for live performance where audio streams from one or more sources
(e.g. microphones, musical instruments and devices containing
pre-recorded audio) are combined and aesthetically enhanced in
various ways before being sent to one or more speakers, where the
several speakers serve different needs, as well as to one or more
recording devices. A paging system serving the needs of one or
multiple buildings where audible messages from one or several
sources must be able to be dynamically routed to specific areas of
a building or a collection of buildings (a campus), or to every
location within the building or campus. Such a system supports
coordination of message delivery such that messages from the
various sources do not collide with one another, and so that
emergency and life-safety messages are always delivered regardless
of what other messages are currently being distributed. A consumer
entertainment system where several sources of video entertainment
(e.g. cable TV channels, digital video recorder, Blu-ray disc,
video programming streamed via the Internet) and several sources of
audio entertainment (e.g. broadcast radio, audio CD, audio media
files and audio programming received via the Internet or via a
personal mobile device) are simultaneously available for
consumption using one or more video displays and speaker systems
which may be located in one or more rooms throughout the consumer's
home. A broadcast production facility where sources of audio and
video (e.g. microphones, cameras and media playback devices) must
be routed to a variety of different media processing stations, and
the resulting processed media then sent on to a variety of
destinations including monitoring equipment, recording devices and
transmission head ends.
[0071] The terms and expressions that have been employed in the
foregoing specification are used as terms of description and not of
limitation, and there is no intention, in the use of such terms and
expressions, of excluding equivalents of the features shown and
described or portions thereof, it being recognized that the scope
of the invention is defined and limited only by the claims that
follow.
* * * * *