U.S. patent number RE43,462 [Application Number 09/301,656] was granted by the patent office on 2012-06-12 for video monitoring and conferencing system.
This patent grant is currently assigned to Kinya (Ken) Washino. Invention is credited to Barry H. Schwab, Kinya Washino.
United States Patent |
RE43,462 |
Washino , et al. |
June 12, 2012 |
**Please see images for:
( Certificate of Correction ) ** |
Video monitoring and conferencing system
Abstract
A PC-based system for monitoring and storing representative
images from video cameras may be utilized for security or other
monitoring applications. Camera inputs from digital or analog
sources are individually and independently digitized and displayed
at a first set of image sizes, sampling rates, and frame rates, and
may be stored in digital form on various recording media at a
second set of image sizes, sampling rates, and frame rates, and
these two sets of sizes and rates may or may not be identical.
Provisions are included for adding detection or alarm systems which
will automatically alter image size, sampling rate and/or frame
rate of an individual input source, or activate other physical
responses. In addition to security system monitoring, further
applications of the invention are disclosed for process monitoring
in manufacturing environments and also for applications in
videoconferencing.
Inventors: |
Washino; Kinya (Dumont, NJ),
Schwab; Barry H. (West Bloomfield, MI) |
Assignee: |
Washino; Kinya (Ken) (Little
Ferry, NJ)
|
Family
ID: |
46177960 |
Appl.
No.: |
09/301,656 |
Filed: |
April 28, 1999 |
Related U.S. Patent Documents
|
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
Issue Date |
|
|
08050861 |
Apr 21, 1993 |
5450140 |
|
|
|
08298104 |
Aug 30, 1994 |
5537157 |
|
|
Reissue of: |
08418823 |
Apr 7, 1995 |
5625410 |
Apr 29, 1997 |
|
|
Current U.S.
Class: |
348/154;
348/E9.038; 348/E7.016; 348/E7.049; 348/E5.111; 348/E5.043;
348/552; 386/E9.013; G9B/27.01; 348/E9.039; 348/E7.086; 348/E5.108;
375/240.11; 348/E5.042; 348/E7.083; 386/E5.002; 348/E5.022;
G9B/27.051; 386/355; 348/E7.081; 348/E7.079 |
Current CPC
Class: |
G09G
5/14 (20130101); H04N 9/8042 (20130101); G06T
9/007 (20130101); H04N 5/23203 (20130101); H04N
5/765 (20130101); H04N 21/4334 (20130101); G06F
3/14 (20130101); H04N 5/247 (20130101); H04N
7/104 (20130101); H04N 5/222 (20130101); H04N
21/4316 (20130101); H04N 21/440263 (20130101); H04N
21/47 (20130101); H04N 5/77 (20130101); H04N
21/440281 (20130101); H04N 21/4622 (20130101); G11B
27/032 (20130101); H04N 5/23293 (20130101); H04N
5/23206 (20130101); H04N 7/181 (20130101); H04N
21/4223 (20130101); H04N 21/4143 (20130101); G11B
27/34 (20130101); G11B 27/031 (20130101); G11B
2220/2516 (20130101); G09G 2340/0435 (20130101); H04N
5/772 (20130101); G09G 2340/0442 (20130101); G11B
27/034 (20130101); H04N 5/915 (20130101); G11B
2220/90 (20130101); H04N 9/896 (20130101); G11B
2220/41 (20130101); G11B 2220/455 (20130101); H04N
5/781 (20130101); G11B 27/024 (20130101); G11B
2220/20 (20130101); G09G 2340/0407 (20130101); H04N
5/85 (20130101); H04N 21/4788 (20130101); G06F
3/1454 (20130101); H04N 9/641 (20130101); G11B
2220/913 (20130101); H04N 9/642 (20130101); G11B
27/002 (20130101); H04N 9/8063 (20130101); G09G
2340/02 (20130101); G11B 2220/2525 (20130101); G11B
2220/61 (20130101); G06F 3/1423 (20130101) |
Current International
Class: |
H04N
7/18 (20060101) |
Field of
Search: |
;348/154,552,398
;375/240.11 ;386/112 ;358/342 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
0494752 |
|
Jul 1992 |
|
EP |
|
4184761 |
|
Jul 1992 |
|
JP |
|
4373386 |
|
Dec 1992 |
|
JP |
|
5168760 |
|
Jul 1993 |
|
JP |
|
5227532 |
|
Sep 1993 |
|
JP |
|
5346773 |
|
Dec 1993 |
|
JP |
|
6004731 |
|
Jan 1994 |
|
JP |
|
6006804 |
|
Jan 1994 |
|
JP |
|
6044470 |
|
Feb 1994 |
|
JP |
|
686289 |
|
Mar 1994 |
|
JP |
|
6070040 |
|
Mar 1994 |
|
JP |
|
6078269 |
|
Mar 1994 |
|
JP |
|
H670277 |
|
Mar 1994 |
|
JP |
|
6096378 |
|
Apr 1994 |
|
JP |
|
6141310 |
|
May 1994 |
|
JP |
|
6266774 |
|
Sep 1994 |
|
JP |
|
H6303651 |
|
Oct 1994 |
|
JP |
|
7067098 |
|
Mar 1995 |
|
JP |
|
WO-96/25710 |
|
Aug 1996 |
|
WO |
|
Other References
McLeod, Automated Video Surveillance--Teaching an Old Dog New
Tricks, S.P.I.E., Dec. 17, 1993. cited by other .
Hasegawa et al., Development and Picture Quality Evaluation of a
Prototype Hi-Vision Coding System for Facility Monitoring,
S.P.I.E., Sep. 16, 1994. cited by other .
Darg, a New High-Speed Video System for Motion Analysis, I.E.E.E.,
Jun. 26-28, 1991. cited by other .
Chang et al., A Remote Multi-Camera Visual Surveillance System
Using Frame-Switching Technology, I.E.E.E., Oct. 5-7, 1988. cited
by other .
Cudworth, Modern Video Surveillance and Control Systems Using
Digital Image Processing Methods, I.E.E.E., Mar. 31, 1992-Apr. 2,
1992. cited by other .
Murtoviita et al., Visual Aids for Substation Monitoring and
Security Control, I.E.E.E., Jun. 26-28, 1991. cited by other .
Rangan et al., Designing An On-Demand Multimedia Service, Jul. 1,
1992. cited by other .
Vicon Industries, Vicon Introduces The V5000DVM Digital Video
Multiplexer, Vicon Industries, Apr. 15, 1992. cited by other .
Pappageorge, The Secrets of CCTV, Security Management, Aug. 1,
1993. cited by other .
Hayter et al., The Desk Area Network, ACM SIGOPS Operating Systems
Review, Oct. 1991. cited by other .
Point-of-sale monitoring downsizes for small venues, May 1993.
cited by other .
Newton, Picturing the Future of CCTV, Security Management, Nov. 1,
1994. cited by other .
Rangan et al., Efficient Storage Techniques for Digital Continuous
Multimedia, I.E.E.E., Aug. 1, 1993. cited by other .
Barthel, Digital Storage Weighed for ATM Surveillance, American
Banker, Feb. 22, 1994. cited by other .
Carvahlo et al., Automated Detection of Intruders Using a Neural
Network, S.P.I.E., Apr. 21, 1992. cited by other .
Intellution's Plant TV Brochure Date unknown. cited by
other.
|
Primary Examiner: Diep; Nhon
Attorney, Agent or Firm: Gifford, Krass, Sprinkle, Anderson
& Citkowski, P.C.
Parent Case Text
REFERENCE TO RELATED APPLICATIONS
This application is a continuation-in-part of U.S. Ser. No.
08/050,861, filed Apr. 21, 1993 now U.S. Pat. No. 5,450,140 and
Ser. No. 08/298,104, filed Aug. 30, 1994 now U.S. Pat. No.
5,537,157.
Claims
We claim:
1. A video storage and display system, comprising: .[.a plurality
of.]. .Iadd.one or more .Iaddend.video cameras, each outputting a
signal representative of a video image; means to receive the
signals from each camera and digitally compress the images; two
forms of high-capacity storage media, one being randomly searchable
while the other continues to store the digitally compressed image;
and a computer configured to receive the digitally compressed
images, the computer being interfaced to the following devices: a
display screen, means to receive externally derived operator
commands, and the high-capacity storage media, and wherein the
computer is programmed to perform the following functions: display
the digitally compressed images from the cameras in different
windows on the display screen, each window being associated with an
update rate and dimensions in pixels, vary .[.the dimensions and
the rate.]. .Iadd.spatial parameters and temporal parameters
.Iaddend.at which a particular image is updated in its window in
accordance with one of the externally derived commands, store the
digitally compressed images in the high-capacity storage
.[.medium.]. .Iadd.media.Iaddend., and vary the .[.dimensions and
the rate.]. .Iadd.spatial parameters and temporal parameters
.Iaddend.at which a particular image is stored in accordance with
one of the externally derived commands.
2. The video storage and display system of claim 1, further
including means associated with the computer for controlling the
operation of one or more of the video cameras.
3. The video storage and display system of claim 1, wherein the
means to digitally compress the image from a particular camera is
disposed at the location of the camera.
4. The video storage and display system of claim 1, wherein the
means to digitally compress the image from a particular camera is
disposed at the location of the computer.
5. The video storage and display system of claim 1, further
including a separate computer associated with each camera, the
computers being networked together over a common communication bus,
enabling an operator situated at a particular computer to display
the images gathered by other cameras in separate windows on that
operator's display screen.
6. The video storage and display system of claim 1, wherein
.Iadd.one or both of .Iaddend.the high-capacity storage
.[.medium.]. .Iadd.media .Iaddend.comprises a magnetic tape.
7. The video storage and display system of claim 1, wherein
.Iadd.one or both of .Iaddend.the high-capacity storage
.[.medium.]. .Iadd.media .Iaddend.comprises a magnetic disk.
.[.8. The method of simultaneously displaying and storing multiple
video images, comprising the steps of: receiving video images from
a plurality of sources; digitizing one or more of the images if not
already in digital form; displaying at least certain of the
digitized images in separate windows on a display device, using a
first, predetermined frame rate and resolution associated with each
window; and simultaneously storing the displayed images using a
second, predetermined frame rate and resolution associated with
each image..].
9. The method of claim .[.8.]. .Iadd.12.Iaddend., further including
the step of receiving a command to set the frame rate and
resolution associated with the display and storage of a particular
image.
10. The method of claim 9, wherein the command is based upon an
operator input.
11. The method of claim 9, wherein the command is based upon an
external stimulus.
12. The method of simultaneously displaying and storing multiple
video images, comprising the steps of: receiving video images
.Iadd.at a personal computer based system .Iaddend.from .[.a
plurality of.]. .Iadd.one or more .Iaddend.sources; digitizing
.[.one or more.]. .Iadd.any .Iaddend.of the images .[.if.]. not
already in digital form .Iadd.using an analog-to-digital
converter.Iaddend.; displaying at least certain of the digitized
images in separate windows on a .Iadd.personal computer based
.Iaddend.display device, using a first set of temporal and spatial
parameters associated with each image in each window;
.[.simultaneously storing the displayed images.]. .Iadd.converting
one or more of the video source images into a data storage format
.Iaddend.using a second set of temporal and spatial parameters
associated with each image.Iadd.; and simultaneously storing the
converted images in a storage device.Iaddend..
13. The method of claim 12, the temporal parameters including frame
rate.
14. The method of claim 12, the spatial parameters including image
dimension in pixels.
15. A video storage and display system, comprising: .[.a plurality
of.]. .Iadd.one or more .Iaddend.video cameras, each outputting a
signal representative of a video image; means to receive the
signals from each camera and digitally compress the images; and a
computer configured to receive the digitally compressed images, the
computer being interfaced to the following devices: a display
screen, means to receive externally derived operator commands
including means for sensing a deviation from the normal-state image
scene associated with at least one of the video cameras, the
existence of the deviation being used as the basis for generating
an externally derived command, and a high-capacity storage medium,
and programmed to perform the following functions: display the
digitally compressed images from the cameras in different windows
on the display screen, each window being associated with an update
rate and dimensions in pixels, vary .[.the dimensions and the
rate.]. .Iadd.spatial parameters and temporal parameters
.Iaddend.at which a particular image is updated in its window in
accordance with one of the externally derived commands, store the
digitally compressed images in the high-capacity storage medium,
and vary the .[.dimensions and the rate.]. .Iadd.spatial parameters
and temporal parameters .Iaddend.at which a particular image is
stored in accordance with one of the externally derived
commands.
.Iadd.16. The video storage and display system of claim 1, wherein
one or more video images or camera control signals are received
through a network connection..Iaddend.
.Iadd.17. The video storage and display system of claim 1, wherein
one or more of the high-capacity storage media comprises a
removable or permanent magnetic disk, a removable or permanent
magneto-optical disc, a removable optical disc or a removable or
permanent semiconductor-based device..Iaddend.
.Iadd.18. The video storage and display system of claim 1, wherein
the temporal parameters include frame rate and the spatial
parameters include image dimension in pixels..Iaddend.
.Iadd.19. The method of claim 12, wherein one or more video images
or camera control signals are received through a network
connection..Iaddend.
.Iadd.20. The method of claim 12, wherein the high-capacity storage
medium comprises a removable or permanent magnetic disk, a
removable or permanent magneto-optical disc, a removable optical
disc or a removable or permanent semiconductor-based
device..Iaddend.
.Iadd.21. The method of claim 12, including a display device
associated with each source and a communication capability enabling
an operator situated at the display for one source to view images,
in separate windows, gathered by one or more different
source..Iaddend.
.Iadd.22. The video storage and display system of claim 15, further
including a device for remotely controlling the operation of one or
more of the video cameras..Iaddend.
.Iadd.23. The video storage and display system of claim 15, wherein
one or more video images or camera control signals are received
through a network connection..Iaddend.
.Iadd.24. The video storage and display system of claim 15, wherein
the high-capacity storage medium comprises a removable or permanent
magnetic disk, a removable or permanent magneto-optical disc, a
removable optical disc or a removable or permanent
semiconductor-based device..Iaddend.
.Iadd.25. The video storage and display system of claim 15, further
including a memory for storing the sensed deviation information in
conjunction with the image data..Iaddend.
.Iadd.26. The video storage and display system of claim 15, further
including: a computer and display device associated with each video
camera; and communication capability enabling an operator situated
at the display device associated with one camera to view images, in
separate windows, gathered by one or more different
cameras..Iaddend.
.Iadd.27. The video storage and display system of claim 15, wherein
the temporal parameters include frame rate and the spatial
parameters include image dimension in pixels..Iaddend.
.Iadd.28. A video storage system, comprising: one or more video
sources, each outputting a signal representative of a video image;
means to receive the signals from each source and digitally
compress the images; two forms of a high-capacity video storage
media; and a computer interfaced to the following devices: an input
to receive externally derived operator commands, and the
high-capacity storage media, and wherein the computer is programmed
to perform the following functions: store the digitally compressed
images in the high-capacity storage media, and vary the spatial
parameters and temporal parameters at which a particular image is
stored in accordance with one of the externally derived
commands..Iaddend.
.Iadd.29. The video storage system of claim 28, wherein the
high-capacity storage media include one medium being randomly
searchable, and with the other being serially
searchable..Iaddend.
.Iadd.30. The video storage system of claim 28, wherein one or more
video images is received through a network connection..Iaddend.
.Iadd.31. The video storage system of claim 28, wherein the
high-capacity storage media comprises a removable or permanent
magnetic disk, a removable or permanent magneto-optical disc, a
removable optical disc or a removable or permanent
semiconductor-based device..Iaddend.
.Iadd.32. The video storage system of claim 28, wherein the
temporal parameters include frame rate and the spatial parameters
include image dimension in pixels..Iaddend.
.Iadd.33. A digital video recording and monitoring system
configured for use with a display device, comprising: one or more
inputs for receiving video material characterized by having spatial
parameters and temporal parameters; circuitry for digitally
compressing the video material; a first video storage medium which
is randomly addressable; a second video storage medium which is
serially addressable; an output for delivering the video material
to the display device; a user control; and processing hardware or
software operative to perform the following functions under user
control: a) store the digitally compressed video material in one or
both of the first and second video storage media, and b) output the
video material for monitoring to the display device..Iaddend.
.Iadd.34. The digital video recording and monitoring system of
claim 33, wherein the processing circuitry is operative to
simultaneously store the digitally compressed video material in the
first and second video storage media..Iaddend.
.Iadd.35. The digital video recording and monitoring system of
claim 33, wherein the first video storage medium is a magnetic
disk..Iaddend.
.Iadd.36. The digital video recording and monitoring system of
claim 33, wherein the second video storage medium is a magnetic
tape..Iaddend.
.Iadd.37. The digital video recording and monitoring system of
claim 33, wherein the processing circuitry further permits
searching of the video material previously recorded on the first
storage medium while continuing to store the material on the second
storage medium..Iaddend.
.Iadd.38. The digital video recording and monitoring system of
claim 33, wherein the spatial parameters and temporal parameters of
the video material in the first or second storage medium are
different from the spatial parameters and temporal parameters of
the video material delivered to the display device..Iaddend.
.Iadd.39. The digital video recording and monitoring system of
claim 33, further including one or more video cameras interfaced to
the one or more inputs..Iaddend.
.Iadd.40. The digital video recording and monitoring system of
claim 33, further including: a plurality of video cameras
interfaced to the one or more inputs; and the video material from
different cameras is visible in different windows on the display
device..Iaddend.
.Iadd.41. The digital video recording and monitoring system of
claim 33, wherein the temporal parameters include frame rate and
the spatial parameters include image dimension in pixels..Iaddend.
Description
FIELD OF THE INVENTION
This invention relates generally to video monitoring, and, more
particularly, to such systems employing means for digitizing camera
images for display at a first image size, sampling rate, and frame
rate, and for digital storage at a second image size, sampling
rate, and frame rate.
BACKGROUND OF THE INVENTION
Existing systems for monitoring video signals from multiple sources
either have relied upon switching means to sequence through each of
the sources in a predetermined pattern, or else implement some form
of "split screen" display, in which a conventional video monitor
displays, for example, the output of four cameras, each occupying a
portion of the display screen. Often, the video output is also
supplied to signal storage means such as a VCR which is operated in
a single-frame recording mode, providing a time-lapse record of the
camera outputs. These types of systems are in common usage, and are
well known in the art.
More recently, the availability of digital techniques for video
signal processing and data compression has opened new video
monitoring alternatives. Mathisen, for example, U.S. Pat. No.
4,198,656, describes a system employing a signal selector which is
controlled by signal storage means, including a VCR functioning as
a time-lapse frame recorder, and disc storage means functioning as
a frame-store type device which provides continuity in the display
while the selector is advancing through its sequence to the next
signal source, or to the next recorded signal from one particular
source. Each image is further provided with a digital code, which
enables the image source to be identified upon playback.
Katz (U.S. Pat. No. 5,216,502) discloses a system for monitoring
and recording transactions at multiple cashier lanes. According to
the preferred embodiment, output signals from four cameras are fed
to a four-quadrant multiplexer, which contains a frame store having
provisions for reducing each camera image to one-quarter size and
then displaying it in one of the four quadrants of the video
monitor. The combined output signal is then recorded on a Video
Cassette Recorder (VCR) to provide a more permanent record of the
transaction. No description is given of the size-reduction method,
but this kind of display may be implemented by employing a
time-compression scheme for the horizontal lines, and eliminating
alternate scan lines from the vertical dimension of the video
frame. Data from the individual cash registers is encoded
digitally, and then either recorded on lines of video in the
vertical-interval or else recorded on the audio tracks of the VCR.
The monitoring provisions include a video overlay generator, so
that transaction information may be displayed concurrently with
images of the event. Provisions are also included for selective
recording of representative frames of video, as triggered by
transaction events at individual cashier lanes. This further
reduces the number of frames recorded, and extends the recording
duration of the video cassette.
Neither of these two inventions disclose the use of image data
compression schemes, nor of image resizing by digital means. Katz
does disclose selective recording of frames of video, but only in
analog form, and using conventional video recording means, such as
a VCR.
Blum et al. (U.S. Pat. No. 5,237,408) disclose a system employing
multiple interface circuit boards containing provisions for
capturing and digitizing images from a plurality of cameras,
further storing a slow-scan sequence of images for each camera in
active buffer memory means as RAM on each of the interface boards.
The display of these image sequences is preferably controlled by a
PC with disk storage capabilities, and individual images
selectively may be recorded on the hard disk, under operator
control. Storage capacity is limited, however, because no image
data compression is provided, and it is not practical to store
large quantities or sequences of images. There are no provisions
for automatic recording of images, nor any provision for storing
these images in a data form differing from the display format image
size.
Gormley (U.S. Pat. No. 5,258,837) discloses video compression means
as a method for fitting multiple camera images within a single
video display screen. However, Gormley clearly indicates that the
system relies on fixed image-size compression of the digitized
incoming video signal, and does not perform any kind of bandwidth
compression. Provisions are included for storing images on a video
recorder, but, as in other systems, these images are stored by
taking the screen display as a whole, rather than storing
independently constituted representations of the individual camera
outputs.
In general, these conventional systems, regardless of the
subsequent signal processing employed, accept analog video signals
from the camera sources for input to the monitoring system. As
such, these signals are susceptible to disturbances such as RF
interference, ground-loop noise, and high-frequency signal loss due
to long runs of coaxial cables. Automatic video switcher units
sequence the source signals at a relatively slow rate, requiring as
much as 16 seconds delay between successive viewings of a
particular source when using a one source-per-second rate to
sequence through 16 sources. When such a system is equipped with a
time-lapse video recorder, the recording medium is expensive, has
low resolution, and is limited in recording capacity.
Other systems, such as video conferencing arrangements, use
multiple camera images and audio sources which are coordinated
through electronic switching means and image display means, with
interconnections between conferencing rooms locally or at remote
sites via telephone lines or other communications links. Fabris et
al. (U.S. Pat. No. 4,516,156) discloses a sophisticated
implementation of such a system, employing satellite links for
long-distance communications, a full range of camera remote
controls (including lens controls and pan-tilt-head controls), and
touch-screen monitor facilities for system controls of cameras and
signal switching. This system, however, has no provisions for image
storage, or any application of image data compression means coupled
with display means or image storage means.
SUMMARY OF THE INVENTION
The present invention implements an automated video monitoring
system by way of a PC-based platform employing display windowing
software, with camera sources being interfaced to an input circuit
board which includes provisions for image data compression. Using a
basic image size in pixels of 320.times.240, and optionally
including color processing employing a Y-U-V 4:1:1 or 4:2:2
sampling technique, a range of performance standards are
established. An economical simultaneous display of 4 sources in a
2.times.2 configuration on a conventional 10'' VGA-format
(640.times.480 pixels) monitor may be upgraded to a more elaborate
24-source (plus one utility window, which also may be used as a
graphical-based input source device for transmitting control
commands to the individual cameras) display in a 5.times.5
configuration on a high-resolution 20'' (1600.times.1200 pixels)
monitor. Other combinations are possible, including arrays of
images of size 640.times.480 pixels or even 800.times.600 pixels,
depending on the screen size of the display monitor and the
capabilities of the video display adapter circuit card. In
addition, not all image windows need to be of the same size, nor
updated at the same rate, but rather they may be mixed and combined
based on particular applications. Remote controls for the
individual cameras may be implemented by way of windowing software
and/or monitor touch-screen devices. Automatic sensing of
particular events (representing security alarms, equipment or
process disturbances, or a change in the person speaking in a
videoconferencing environment) may be employed to cause
reconfiguration by way of resizing the image or modifying the
update rate of individual windows on the display screen, or by
modifying the data format of recording images on a storage
device.
Storage of images may be implemented by way of a tape back-up
device, such as a DAT or 8-mm tape recorder, which are capable of
storing as much as 960 hours of monitoring images, or by way of
disk storage devices, preferably including removable disk drives
such as magneto-optical disks or PCMCIA-compatible disk-drive
modules. Images are preferably stored as a succession of
data-compressed representations, corresponding to various window
sizes sampled at diverse update rates; however, though the image
representations need not be identical to the sizes and rates used
for video monitors displaying the various images. In an alternative
embodiment, cameras are provided with data compression means at the
camera location, with a bi-directional data link providing control
signals from the PC, and accepting data-compressed images from the
cameras. Depending on the data rates selected, the transmission of
these signals may be by means of coaxial cables, twisted-pair
lines, or fiber-optic cables. These signals may alternatively be
transmitted over internal PBX telephone lines, or over greater
distances, including satellite links, as would be the case for
remote monitoring of facilities or videoconferencing. The use of
digital images allows the application of various useful techniques,
such as digital noise reduction and motion-sensing, to trigger
security alarms, for example.
It is an object of the invention to provide a more efficient method
for monitoring camera outputs by means of a multiple-window display
system implemented on a computer platform.
It is another object of the invention to provide an improved
recording system for event-logging or security applications, by
means of storing data-compressed digital images, including
identifying information, which are more representative of events
than typical analog time-lapse recordings, and further to implement
this system on a computer platform.
It is a further object of the invention to provide extended
recording time for event-logging or security applications by means
of either tape-based, or disk-based, or both tape-based and
disk-based data recording means.
It is yet another object of the invention to provide a system for
remote monitoring of cameras by means of transmission of
data-compressed digital images over communication links.
It is yet a further object of the invention to provide a convenient
method of implementing videoconferencing facilities by means of a
PC-based platform.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGS. 1-6 show various possible screen display configurations
according to the invention;
FIG. 7 is a functional diagram of a PC-based system wherein plug-in
printed-circuit boards implement digital processing for analog
input signals;
FIG. 8 is a functional diagram of a plug-in printed-circuit board
introduced with reference to FIG. 7;
FIG. 9 is a functional diagram of a universal camera adapter having
digital output signals;
FIG. 10 is a diagram of an integrated camera system;
FIG. 11 is a functional diagram of a PC-based monitoring system
adapted for videoconferencing at multiple remote locations;
FIG. 12 is an overhead view of a physical layout of the system of
FIG. 11;
FIGS. 13A and 13B are drawings of a multiple-camera monitoring unit
for table-top use in videoconferencing;
FIG. 14 is a side-view of an expanded multiple-camera monitoring
unit implemented for videoconferencing;
FIG. 15 is a table listing a variety of possible operating modes,
depending on computer monitor display facilities, in either a
monitoring or a videoconferencing version of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
The present invention implements an automated video monitoring
system by way of a PC-based platform employing display windowing
software, with camera sources being interfaced to an input circuit
board which includes provisions for image data compression. The
basic video window size of 320.times.240 pixels from each camera
source can be displayed on a variety of video monitors, in a number
of formats, depending on system complexity. The preferred recording
medium is a 4-mm helical-scan data cartridge, commonly referred to
as a digital audio tape (DAT). Each tape cartridge is capable of
storing 10 GB (gigabytes) of data. All recording times in the
explanation below are based a data-compression ratio of 100:1,
utilizing a 4:2:2 Y/U/V sampling method for color images. Other
higher capacity media, such as 8-mm tapes capable of 20 GB of data
storage, may be employed when longer times are desired.
FIG. 15 shows a variety of possible operating modes, depending on
the particular implementation of the PC-based monitoring system.
These six representative modes will be explained in detail with the
understanding that the invention is by no means limited to the
specific examples shown, and that many different alternatives are
possible. It should be noted also that in certain of the
implementations shown, the entire screen may not be occupied by the
various windows, and any unused areas at the top, bottom, or sides
may be utilized to display warning messages, control buttons, or
other such facilities.
FIG. 1 shows a computer monitor display for a system configured in
accordance with Mode I as listed in FIG. 15. Using a commonly
available 14'' VGA-format computer monitor with a dimension in
pixels of 640.times.480, four windows having a dimension in pixels
of 320.times.240 may be displayed simultaneously. Recording this
data at the rate of one frame per second (1 fps) in black and white
(B/W) and with 100:1 data-compression on a 10 GB DAT tape provides
approximately 960 hours of images per tape, or alternatively more
than 480 hours of color images. A second monitor optionally may be
used to display utility windows including various system operating
controls, as, for example, camera pan, tilt, zoom, focus, and so
forth.
FIG. 2 shows a computer monitor display for a system configured in
accordance with Mode II as listed in FIG. 15. Using a 15'' to 17''
SVGA-format computer monitor with a dimension in pixels of
1024.times.768, nine windows, each having a dimension in pixels of
320.times.240, may be displayed simultaneously. Eight windows may
be implemented as camera displays, and the ninth window may be used
for the continuous display of the utility window, as described
above. Recording of all eight windows at 1 fps will allow 480 hours
of B/W images in this configuration per 10 GB DAT tape, or
alternatively more than 240 hours of color images.
FIG. 3 shows a computer monitor display for a system configured in
accordance with Mode III as listed in FIG. 15. Using a 17'' to 20''
high-resolution computer monitor with a dimension in pixels of
1240.times.1024, eight windows having a dimension in pixels of
320.times.240 may be displayed, in addition to two windows having a
dimension in pixels of 640.times.480. One of these larger windows
may be implemented as a utility window, and the other larger window
may be implemented for sequential display of the eight camera
inputs at full camera resolution having a dimension in pixels of
640.times.480. Recording all eight camera image windows, and also
the input-scan window, at 1 fps will allow 360 hours of B/W images
in this configuration per 10 GB DAT tape, or alternatively more
than 180 hours of images in color.
FIG. 4 shows a computer monitor display for a system configured in
accordance with Mode IV as listed in FIG. 15. Using a 20''
high-resolution computer monitor with a dimension in pixels of
1600.times.1200, 25 windows with a dimension in pixels of
320.times.240 may be displayed simultaneously. Twenty-four windows
may be implemented as camera displays, and one is implemented as a
.[.the.]. utility window. Recording all twenty-four windows, and
also the utility window, at 1 fps will allow 180 hours of B/W
images in this configuration per 10 GB DAT tape. If color recording
is desired, a 20 GB 8-mm data cartridge will allow 180 hours of
color images.
FIG. 5 shows a computer monitor display for a system configured in
accordance with Mode V as listed in FIG. 15. Using a 20''
high-resolution computer monitor with a dimension in pixels of
1600.times.1200, twelve larger windows having a dimension in pixels
of 400.times.300 may be displayed, as well as one large
high-resolution window, with a dimension in pixels of
800.times.600, to display the sequentially scanned output of the
camera images. Recording in this format at 1 fps will provide 180
hours of B/W images in this configuration per 10 GB DAT tape. If
color recording is desired, a 20 GB 8-mm data cartridge will allow
180 hours of color images.
FIG. 6 shows a computer monitor display for a system configured in
accordance with Mode VI as listed in FIG. 15. Using a 14'' to 17''
SVGA computer monitor with a dimension in pixels of
1240.times.1024, and implementing a smaller window size with a
dimension in pixels of 240.times.180, eight windows are available
for the camera image displays, and two large windows of
480.times.360 are available for sequentially-scanned camera images.
Recording in this screen format at 1 fps allows 480 hours of B/W
images in this configuration on a 20 GB 8-mm data cartridge, and
allows 240 hours of images in color.
In any of the display configurations just described, it is also
possible to use a video overlay technique to display the utility
controls superimposed on the camera video. Regardless of which
display or recording format is used, the PC based monitoring system
includes various facilities and features, which will now be
described in further detail.
In operation, the user need only observe the various display
windows on the monitor screen, rather than concentrate on many
monitors simultaneously, thereby reducing the risk of missing an
important event. Since no video switcher is used in recording, and
images from all sources are continuously recorded at the selected
frame rate for each source, more information is recorded than in
conventional analog systems, wherein events may be missed due to
the sequential switching of input images. The digital format also
improves the picture quality, as the signal-to-noise ratio will be
higher than for analog systems. In addition, there is no loss of
quality during recording or playback, and because the recording
technique is digital, other types of information optionally may
.Iadd.be .Iaddend.recorded along with the camera data, such as
audio, time, date, location, etc.
An additional feature is the capability to implement a
dual-recording-media option. This facility provides the ability to
record simultaneously both on a tape (for high capacity, long term
storage) and also on a removable media, such as a removable hard
disk (i.e. PCMCIA) or magneto-optical disk (for short-term storage
of up to 24 hours of images). These disks facilitate high-speed
searching of recorded information without interrupting the tape, as
well as providing a back up for the recording on the tape. In
addition, the system is capable of simultaneously searching the
recorded images on the disk storage unit while continuing to store
images on the tape storage unit.
It should be noted that there is no requirement that the image
sizes or frame rates utilized for the video display match those
utilized for the storage media. In practice, these two
specifications may not agree, and will be determined by other
factors, such as operator manipulation of the displayed image sizes
or changes resulting from the detection of alarm conditions.
FIG. 7 shows a functional diagram of an analog input-digital
processing card installed in the PC which allows the use of
existing analog cameras and cables with the PC-based monitoring
system. This card, which may be obtained from such manufacturers as
Nova, Model No. V-SW, features four video inputs 2a-2d having
balanced or differential input circuitry for good noise immunity
(or optically-coupled inputs for fiber-optic cabling), four video
amplifiers 4a-4d, a 4.times.1 video switcher 6 by which any one of
the four video inputs may be selected as a signal source, and an
analog-to-digital (A/D) converter 8. The recording of camera inputs
is digital, however, there is no provision for remote control of
camera functions such as pan, tilt, zoom, and so forth. If
additional inputs are desired, multiple cards may be installed in
the PC. The output of the A/D converter 8 is supplied to the
graphics processor and image data-compression engine 10. This unit
performs the various functions required to configure the image
sizes and frame rates as specified by the equipment operator.
Because the overall data recording bandwidth of the system is a
factor in the selection of the individual picture data rates,
computer software is provided to implement menu-driven management
of the data bandwidth allocation to the various image sources.
Based on this configuration, signals are provided to the video
switcher 4 to facilitate the selection of the particular image
sources as required. The data-compressed images are provided in
digital form to the microprocessor data bus 12. The microprocessor
unit (not shown) in turn provides control signals to the graphics
processor 10 by way of this data bus, in accordance with the image
data allocation configuration selected.
FIG. 8 is a functional diagram of a digital input/output-digital
processing card which implements bi-directional digital
communications to and from the camera/adapter unit over a single
transmission line, thereby greatly reducing the cost of cable and
installation, and allowing existing wiring to be used. Various
implementations of this printed-circuit card are required,
depending on the type of network interconnection media selected. In
addition, system performance will depend on the type of network
interconnection media. As an example, a fiber optic network will
have a higher transmission rate and better signal quality than a
telephone network. Interfacing technology for these communication
methods are in common usage and well known in the art. This
printed-circuit card also serves four cameras through inputs
52a-52d, and additional cards can be implemented if required. The
bi-directional digital video and camera control data switcher 54
serves to select the individual image sources as explained in
reference to FIG. 7. These source signals are then provided to the
graphics processor and image data-compression engine 56, which
provides similar functions to those provided by the graphics
processor 10, discussed in reference to FIG. 7. The processed
data-compressed images are then provided to the microprocessor data
bus 60. However, in this implementation, the bi-directional nature
of the data switcher 54 is exploited to provide full functional
control of the camera facilities through the bi-directional
controller 58. These functional controls include the cameras
themselves (lens and exposure control) and the camera's physical
mounting provisions (pan and tilt). The various control signals for
these functions are those traditionally provided to these kinds of
equipment, and these concepts are well known in the art. In an
alternative embodiment, the individual cameras are each equipped
with separate image data-compression facilities, utilizing such
techniques as motion-JPEG or MPEG compression. In this case,
control signals for these additional functions are provided from
the microprocessor bus 60 through the bi-directional controller 58
and the data switcher 54.
FIG. 9 is a functional diagram of a digital output universal camera
adapter. This is a fully digital version of the camera adapter as
described in co-pending U.S. patent application Ser. No.
08/050,861, titled "PERSONAL-COMPUTER BASED VIDEO PRODUCTION
SYSTEM" now U.S. Pat. No. 5,450,140. The A/D-converter, digital
image data-compressor, and bi-directional interface camera adapter
100 accepts analog audio and video signals from the camera 102, and
converts them to digital signals in anticipation of the
transmission of these signals over the interconnection network 104.
The camera adapter also receives camera control commands from the
PC by means of the interconnection network, and translates them
into the appropriate pan, tilt, zoom, focus and iris control
signals for the particular camera equipment, including the camera
lens 106 and the pan/tilt mounting facilities 108. In addition, the
camera adapter also has inputs for several "alarm system" type
sensors 110, as, for example, motion detectors, photocell
detectors, or simple switches. These alarm signals are digitized,
encoded, and then transmitted to the main PC by means of the
.[.interconnection,network.]. .Iadd.interconnection
network.Iaddend.. Power is supplied for this equipment from a local
source 112. This camera adapter is implemented to provide a
full-function system, adaptable to all existing types of cameras
and control equipment. However, because of the large number of
interconnections involved in the adapter, camera, pan/tilt unit,
and sensors, the installation process may be somewhat
complicated.
By equipping an individual camera input with digital frame store
capabilities, it is possible to detect an alarm condition based on
deviations from the normal-state image scene. For example, if a
camera is monitoring a door exit or an area of a warehouse aisle
that is not utilized during evening hours, any change in the image
would represent an alarm condition. To allow camera movement, this
sensing process would be disabled during pan, scan, tilt, zoom, and
other camera positioning controls, and the sensing would be
re-enabled after camera positioning ceased, or by manual operator
control.
FIG. 10 is a functional diagram of an integrated camera system
according to the invention including means to overcome problems
associated with having a physically separate camera, pan/tilt unit,
and adapter unit. The integrated system includes a camera 150 with
pan/tilt control, A/D converter, digital data-compression
circuitry, and requisite interfacing circuitry. Such provisions are
preferably included in the camera base structure 152, which
simplifies installation, as no separate interconnecting cables are
required. The integrated camera system optionally may include
provisions for supplying electrical operating power from a network
coaxial cable, and thereby may be implemented with only a single
interconnecting cable. An integrated camera system of this type may
be implemented in a physical package which is as small as the
camera itself, since most of the electronics may be highly
miniaturized by developing custom integrated circuits, including
LSI, ASIC, DSP, and mixed-signal processing.
It will be appreciated that in any of these implementations, alarm
or sensor signals may be utilized to automatically re-configure the
system operating mode, as, for example, increasing the frame rate
or image size for an image source associated with the sensor which
has initiated the alarm signal condition. As explained above, the
displayed windows and image sizes may be reconfigured into an
operating mode different from the reconfiguration of the digital
storage mode. If desired, the operator may choose to allow the
system to automatically adjust the compression ratio utilized for a
particular window in response to alarm signal conditions.
Alternatively, the compression ratio may be adjusted in response to
the selection by the operator of a particular window for closer
monitoring, by switching to an image window having larger
dimensions in pixels. For some applications, the use of a
resolution-independent data compression scheme, such as the
"Fractal Compression" method of Iterated Systems, Inc., will be
preferred, since images compressed by this method may be resized to
fit larger or smaller windows as desired, without loss of apparent
resolution. In addition, automatic switching of the audio signals
associated with a particular window in response to alarm signal
condition will enable the operator to fully investigate such
events, and the audio signals may serve to attract the operate's
attention to the event. Such switching optionally may be coupled
with operator selections for monitoring purposes as well, and in
both of these options may be integrated into the system previously
described for recording the images, with or without any associated
audio signals, onto the mass-storage media.
FIG. 11 illustrates a PC-based monitoring system implemented for
videoconferencing at multiple remote locations. In this application
of the invention, a main control center, shown generally as 200, is
interconnected, by way of a network, with remote locations, shown
generally as 204 to 210, designated as "1" through "N". The network
202 may be implemented by way of telephone lines (slow speed), ISDN
Lines (medium speed), or alternatively coaxial cable or fiber
optics cables (high speed). The higher speed implementations permit
transmission at higher frame rates and higher image resolutions. As
an alternative, wireless networks or microwave links optionally may
be used. Network communication between the main and remote
locations is bi-directional; the main computer receives audio,
video, and sensor data from each of the remote locations, and
transmits controls for camera pan, tilt, focus, zoom, and so forth.
Audio and video communications for the system operators also is
provided, by means of an additional camera and headset at each
location.
The application of these PC-based monitoring techniques also may
.[.he.]. .Iadd.be .Iaddend.implemented in cases in which the
central monitoring area may be located at some distance from the
remote site. In this case, the physical hardware and networking
facilities will follow the example shown in FIG. 11, with the
understanding that "Location 1" through "Location IN" will now
refer to separate monitoring for security applications or other
monitoring purposes.
The purpose of the main control center is to manage multiple
cameras at multiple locations. By monitoring and controlling the
remote locations from a central location 220, the system may
function without an operator at each location. Optionally,
additional computers shown, by way of example, as 222 and 224 at
the main location may be implemented to simplify the monitoring and
control of the remote locations. The main control center stores the
data from remote locations in the main control storage facilities,
shown, by way of example, as 238, 240 and 242, using digital tape
and removable media, including, by way of example, PCMCIA-based
removable media or magneto-optical (MO) media. The images may be
stored at various frame rates and resolutions, depending on the
requirements for the intended use of the stored images. Intercom
provisions as shown as 230, 232 and 234, facilitate the
coordination of activities between the main control center and the
various remote locations, and video camera units are provided at
each of the computers, shown, by way of example, as 244, 246 and
248.
Each remote location includes a PC/monitor 250, with multiple video
cameras, shown, by way of example, as 252 and 254, including
provisions 256 for image and data storage utilizing digital tape
and other removable media. This remote location may be operated
under control from either the main location (as described above),
or a local operator, or by specific software designed for automatic
control. In the case of software-controlled operations, provisions
are included for control to be assumed by the main control center,
if necessary. As in the case of the main control computers, each of
the remote computers is equipped with intercom facilities as 258
and video cameras as 260.
In the traditional videoconferencing situation, only one camera and
monitor are used. Unfortunately, this results in an unnatural scene
which provides the users with only a single viewing perspective.
The use of a wide-angle camera lens which is needed to include many
people seated around a table creates a distorted view, especially
for those seated furthest from the camera. FIG. 12 shows an
overhead view of the typical videoconferencing arrangement,
including conference table 300, camera and monitor 302, and
conference participants, shown as circles as 304.
Using the PC-based monitoring system, it is possible to create a
video conference that presents a much more natural viewing
appearance. As shown in FIG. 13A, a multiple-camera,
multiple-display unit 310 .[.preferably.]. is preferably located
directly on the conference table 312. This reduces the
camera-perspective-distorting effects just described, because
conference members 314 may be seated in a more comfortable and
convenient position. The resulting video image is also much more
natural. Each controller remote site computer-display section 322
(as described herein below) simultaneously shows all of the members
participating in the remote conference, with an individual video
window allocated for each participant, under control from the
PC-based monitoring system operator. In practice, the remote site
equipment operator will select one of the display operating modes
as described in FIG. 15, depending on the number of subjects
(camera views) and the capabilities of the remote site computer
equipment. Optionally, an additional camera 316 fitted with a
wide-angle lens will provide an overall view of the conference
room, in accordance with more traditional systems.
To facilitate the aiming of the individual cameras for the
participants, an arrangement such as that implemented at airport
gates to assist the parking of airplanes (typically an "I" and an
"O" formed of neon light tubes is utilized for airplane gate
parking) optionally may be included. In this case, the two
indicators are mounted so as to overlap only when viewed directly
along the optical axis of the camera lens system. Thus, the camera
position need only be adjusted so as to cause the indicators to
overlap, thereby assuring that the camera correctly is aimed at the
subject. A number of microphones 318 are situated around the
conference table in locations suitable for proper audio coverage of
the participants' speech.
FIG. 13B shows a side-view depicting the physical layout of each of
the four sides of the multiple-camera, multiple-display unit 310.
The top unit 320 houses the individual cameras (2 to 8 units) and
associated electronics, optionally including aiming provisions as
described herein above. An LCD display screen 322 shows the various
camera images of the conference participants. Speakers located at
positions 324 provide audio from the remote sites.
As shown in FIG. 14, the multiple-camera/display unit shown in FIG.
13B preferably is modular in construction including removable side
panels, in order to facilitate expansion along either horizontal
axis so as to accommodate any size or shape conference table, with
each section of the unit serving up to four conference
participants. In the preferred embodiment, the entire unit
preferably is constructed so as to provide a low profile, by
utilizing slanted LCD display panels 330 to create a minimum
obstruction for the local conference participants. If a low profile
design is not required, the display panels optionally could be
larger, and mounted vertically. Until color LCD panels become more
cost-effective, the display unit would be constructed with any of
the currently available small LCD projector systems. Each
conference location may include a designated PC operator, however,
the implementation of the network connection facilitates control of
the system from a remote location.
It should be noted that the implementation of the PC-based
monitoring system is not limited to these examples of security
systems or videoconferencing. Many alternative implementations,
such as workplace, factory, production line, and process
monitoring, would benefit from this system, and these alternative
implementations should be considered to be within the scope of the
invention.
* * * * *