U.S. patent application number 12/397073 was filed with the patent office on 2010-03-04 for methods for extracting identifying information from video and audio content.
Invention is credited to Geoffrey B. Rhoads.
Application Number | 20100054529 12/397073 |
Document ID | / |
Family ID | 26755175 |
Filed Date | 2010-03-04 |
United States Patent
Application |
20100054529 |
Kind Code |
A1 |
Rhoads; Geoffrey B. |
March 4, 2010 |
Methods for Extracting Identifying Information From Video and Audio
Content
Abstract
The presently claimed invention relates generally to extracting
or computing identifying information from media content, e.g., such
as video and audio. One claim recites a method including: obtaining
data representing video or audio associated with an online network
resource; obtaining instructions from the online network resource,
the instructions are associated with the data; processing the data
in accordance with at least some of the instructions; utilizing one
or more processors, computing identifying information from the
data, the identifying information facilitating identification of
the video or audio. Of course, other claims and combinations are
provided as well.
Inventors: |
Rhoads; Geoffrey B.; (West
Linn, OR) |
Correspondence
Address: |
DIGIMARC CORPORATION
9405 SW GEMINI DRIVE
BEAVERTON
OR
97008
US
|
Family ID: |
26755175 |
Appl. No.: |
12/397073 |
Filed: |
March 3, 2009 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
11187783 |
Jul 22, 2005 |
7499566 |
|
|
12397073 |
|
|
|
|
10208735 |
Jul 29, 2002 |
6922480 |
|
|
11187783 |
|
|
|
|
09293601 |
Apr 15, 1999 |
6427020 |
|
|
10208735 |
|
|
|
|
09074034 |
May 6, 1998 |
6449377 |
|
|
09293601 |
|
|
|
|
08438159 |
May 8, 1995 |
5850481 |
|
|
09074034 |
|
|
|
|
09234780 |
Jan 20, 1999 |
|
|
|
09293601 |
|
|
|
|
60082228 |
Apr 16, 1998 |
|
|
|
60071983 |
Jan 20, 1998 |
|
|
|
Current U.S.
Class: |
382/100 ;
725/118 |
Current CPC
Class: |
G06T 2201/0065 20130101;
H04N 1/00005 20130101; G07D 7/0034 20170501; H04N 1/00002 20130101;
H04N 1/32144 20130101; H04N 2201/327 20130101; G06T 1/0028
20130101; G06T 1/0064 20130101; H04N 1/32208 20130101; H04N 1/32309
20130101; G06T 1/0021 20130101; H04N 2201/3205 20130101; H04N
1/32219 20130101; H04N 2201/3249 20130101; H04N 2201/3207 20130101;
G10L 19/018 20130101; H04N 1/32203 20130101; H04N 1/00244 20130101;
H04N 1/0087 20130101; H04N 1/00867 20130101; G06T 1/0071 20130101;
H04N 2201/3246 20130101; B29C 45/1418 20130101; G06T 1/005
20130101; G06T 2201/0051 20130101; H04N 1/00846 20130101; H04N
1/00859 20130101; H04N 1/32261 20130101; H04N 2201/3239 20130101;
H04N 1/32304 20130101; H04N 1/00037 20130101; H04N 1/00092
20130101; H04N 2201/3226 20130101; H04N 1/00875 20130101; G07D
7/004 20130101; G06T 7/0002 20130101; H04N 2201/3233 20130101; H04N
2201/3271 20130101; B42D 25/29 20141001; Y10S 283/901 20130101 |
Class at
Publication: |
382/100 ;
725/118 |
International
Class: |
G06K 9/00 20060101
G06K009/00; H04N 7/173 20060101 H04N007/173 |
Claims
1. A method comprising: obtaining data representing video or audio
associated with an online network resource; obtaining instructions
from the online network resource, the instructions are associated
with the data; processing the data in accordance with at least some
of the instructions; utilizing one or more processors, computing
identifying information of the data, the identifying information
facilitating identification of the video or audio.
2. The method of claim 1 further comprising managing rendering of
the video or audio based at least in part on the identifying
information.
3. The method of claim 2 further comprising accessing usage rights
associated with the identifying information, said act of managing
utilizes the usage rights.
4. The method of claim 1 in which the act of computing identifying
information from the data comprises decoding digital watermarking
hiding in the data.
5. The method of claim 4 in which the identifying information
comprises a plural-bit identifier.
6. A computer readable medium comprising instructions stored
thereon to control an apparatus to perform the method of claim
1.
7. An apparatus comprising: one or more processors to: obtain data
representing video or audio associated with an online network
resource; obtain instructions from the online network resource, the
instructions are associated with the data; process the data in
accordance with at least some of the instructions; and compute
identifying information of the data, the identifying information
facilitating identification of the video or audio.
8. The apparatus of claim 7 in which the one or more processors
further manage rendering of the video or audio based at least in
part on the identifying information.
9. The apparatus of claim 8 in which the one or more processors
further access usage rights associated with the identifying
information, the one or more processors utilize the usage rights
when managing rendering of the video or audio.
10. The apparatus of claim 7 in which the one or more processors
compute identifying information from the data by decoding digital
watermarking hiding in the data.
11. The apparatus of claim 10 in which the identifying information
comprises a plural-bit identifier.
12. A method comprising: obtaining data representing video or audio
by querying an online network resource; obtaining instructions from
the online network resource, the instructions are associated with
the data; manipulating the data by reference to least some of the
instructions; utilizing one or more processors, extracting
identifying information from the data.
Description
RELATED APPLICATION DATA
[0001] This application is a continuation of U.S. patent
application Ser. No. 11/187,783, filed Jul. 22, 2005 (U.S. Pat. No.
7,499,566), which is a continuation of U.S. patent application Ser.
No. 10/208,735, filed Jul. 29, 2002 (U.S. Pat. No. 6,922,480),
which is a continuation of U.S. patent application Ser. No.
09/293,601, filed Apr. 15, 1999 (U.S. Pat. No. 6,427,020) which is
a divisional application of U.S. patent application Ser. No.
09/074,034, filed May 6, 1998 (U.S. Pat. No. 6,449,377), which is a
continuation-in-part of application Ser. No. 08/438,159, filed May
8, 1995 (U.S. Pat. No. 5,850,481), and copending provisional
application 60/082,228, filed Apr. 16, 1998.
[0002] U.S. patent application Ser. No. 09/293,601 is also a
continuation-in-part of application Ser. No. 09/234,780, filed Jan.
20, 1999, which claims priority to provisional application
60/071,983, filed Jan. 20, 1998.
[0003] (The specification of application Ser. No. 08/438,159 is
substantially identical to that of applicant's issued patents U.S.
Pat. Nos. 5,636,292, 5,710,834, 5,748,763, 5,748,783. The
specification of application 60/082,228 is attached hereto as
Appendix A.)
[0004] Patent application Ser. No. 09/293,602 (now U.S. Pat. No.
6,285,776), entitled Methods for Identifying Equipment Used in
Counterfeiting, was filed as a sibling application of parent
application Ser. No. 09/293,601.
[0005] The subject matter of this application is also related to
that of the present assignee's other pending applications, namely
Ser. Nos. 08/508,083 (now U.S. Pat. No. 5,841,978), 08/534,005 (now
U.S. Pat. No. 5,832,119), 08/637,531 (now U.S. Pat. No. 5,822,436),
08/649,419 (now U.S. Pat. No. 5,862,260), 08/746,613 (now U.S. Pat.
No. 6,122,403), 08/951,858 (now U.S. Pat. No. 6,026,193), and
08/969,072 (now U.S. Pat. No. 5,809,160), and the allowed
applications referenced below.
FIELD OF THE INVENTION
[0006] The present invention relates to methods and devices for
recognizing banknotes and responding accordingly (e.g., by
intervening to prevent attempted banknote counterfeiting).
BACKGROUND AND SUMMARY OF THE INVENTION
[0007] Watermarking is a quickly growing field of endeavor, with
several different approaches. The present assignee's work is
reflected in U.S. Pat. Nos. 5,710,834, 5,636,292, 5,721,788,
5,748,763, 5,748,783, and 5,745,604, in allowed U.S. application
Ser. Nos. 08/327,426 (now U.S. Pat. No. 5,768,426), 08/508,083 (now
U.S. Pat. No. 5,841,978), 08/438,159 (now U.S. Pat. No. 5,850,481),
and in laid-open PCT application WO97/43736. Other work is
illustrated by U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659,726,
5,664,018, 5,671,277, 5,687,191, 5,687,236, 5,689,587, 5,568,570,
5,572,247, 5,574,962, 5,579,124, 5,581,500, 5,613,004, 5,629,770,
5,461,426, 5,743,631, 5,488,664, 5,530,759, 5,539,735, 4,943,973,
5,337,361, 5,404,160, 5,404,377, 5,315,098, 5,319,735, 5,337,362,
4,972,471, 5,161,210, 5,243,423, 5,091,966, 5,113,437, 4,939,515,
5,374,976, 4,855,827, 4,876,617, 4,939,515, 4,963,998, 4,969,041,
and published foreign applications WO 98/02864, EP 822,550, WO
97/39410, WO 96/36163, GB 2,196,167, EP 777,197, EP 736,860, EP
705,025, EP 766,468, EP 782,322, WO 95/20291, WO 96/26494, WO
96/36935, WO 96/42151, WO 97/22206, WO 97/26733. Some of the
foregoing patents relate to visible watermarking techniques. Other
visible watermarking techniques (e.g. data glyphs) are described in
U.S. Pat. Nos. 5,706,364, 5,689,620, 5,684,885, 5,680,223,
5,668,636, 5,640,647, 5,594,809.
[0008] Most of the work in watermarking, however, is not in the
patent literature but rather in published research. In addition to
the patentees of the foregoing patents, some of the other workers
in this field (whose watermark-related writings can by found by an
author search in the INSPEC database) include I. Pitas, Eckhard
Koch, Jian Zhao, Norishige Morimoto, Laurence Boney, Kineo Matsui,
A. Z. Tirkel, Fred Mintzer, B. Macq, Ahmed H. Tewfik, Frederic
Jordan, Naohisa Komatsu, and Lawrence O' Gorman.
[0009] The artisan is assumed to be familiar with the foregoing
prior art.
[0010] In the following disclosure it should be understood that
references to watermarking encompass not only the assignee's
watermarking technology, but can likewise be practiced with any
other watermarking technology, such as those indicated above.
[0011] Watermarking can be applied to myriad forms of information.
The present disclosure focuses on its applications to line art
imagery, of the sort typically employed in banknotes, travelers
checks, passports, stock certificates, and the like (hereafter
collectively referred to as "banknotes"). However, it should be
recognized that the principles discussed below can also be applied
outside this particular field.
[0012] Most of the prior art in image watermarking has focused on
pixelated imagery (e.g. bit-mapped images, JPEG/MPEG imagery,
VGA/SVGA display devices, etc.). In most watermarking techniques,
the luminance or color values of component pixels are slightly
changed to effect subliminal encoding of binary data through the
image. (This encoding can be done directly in the pixel domain, or
in another domain, such as the DCT domain.) In some systems,
isolated pixels are changed in accordance with one or more bits of
the binary data; in others, plural domain-related groupings of
pixels (e.g. locally adjoining, or corresponding to a given DCT
component) are so changed. In all cases, however, pixels have
served as the ultimate carriers of the embedded data.
[0013] While pixelated imagery is a relatively recent development,
line art goes back centuries. One familiar example is U.S. paper
currency. On the one dollar banknote, for example, line art is used
in several different ways. One is to form intricate webbing
patterns around the margin of the note (generally comprised of
light lines on dark background). Another is so form grayscale
imagery, such as the portrait of George Washington (generally
comprised of dark lines on a light background).
[0014] In an illustrative embodiment of the present invention, the
line art on a banknote or other security document is slightly
changed to effect the encoding of plural-bit digital data (i.e. the
banknote is digitally watermarked). When such a banknote is
thereafter scanned, the resulting image data can be recognized as
corresponding to a banknote by detection of the encoded data.
(Alternatively, the image data can be recognized as corresponding
to a banknote by machine detection of other forms of watermarking,
or by reference to visible structures characteristic of
banknotes.)
[0015] In response to detection of a banknote, the detecting
apparatus (e.g., a scanner, photocopier, or printer) can intervene
so as to prevent or deter counterfeiting. For example, the
apparatus can interrupt a copying operation, and display a message
reminding the operator that it is illegal to reproduce currency.
Another option is to dial a remote service and report the attempted
reproduction of a banknote. Yet another option is to permit the
copying, but to insert forensic tracer data (e.g.
steganographically encoded binary data) in the resultant copy. The
tracer data can memorialize the serial number of the machine that
made the copy and/or the date and time the copy was made. To
address privacy concerns, such tracer data is not normally inserted
in photocopied output, but is so inserted only when the subject
being photocopied is detected as being a banknote.
[0016] The foregoing features and advantages of the invention will
be more readily apparent from the following detailed description,
which proceeds with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] FIGS. 1A and 1B show prior art techniques for achieving
grayscale effects using line art.
[0018] FIG. 2 shows a virtual array of grid points that can be
imposed on an image according to one embodiment of the present
invention.
[0019] FIG. 3 shows a virtual array of regions that can be imposed
on an image according to the FIG. 2 embodiment.
[0020] FIG. 4 shows an excerpt of FIG. 3 with a line from a line
art image passing therethrough.
[0021] FIG. 5 shows changes to the width of the line of FIG. 3 to
effect watermark encoding according to one embodiment of the
present invention.
[0022] FIG. 6 shows changes to the position of the line of FIG. 3
to effect watermark encoding according to another embodiment of the
present invention.
[0023] FIG. 7 is a block diagram of a photocopier according to
another embodiment of the invention.
DETAILED DESCRIPTION
[0024] Referring to FIG. 2, an illustrative form of the invention
employs a grid 10 of imaginary reference points arrayed over a line
art image. The spacing between points is 250 .mu.m in the
illustrated arrangement, but greater or lesser spacings can of
course be used.
[0025] Associated with each grid point is a surrounding region 12,
shown in FIG. 3. As described below, the luminosity (or
reflectance) of each of these regions 12 is slightly changed to
effect the subliminal encoding of binary data.
[0026] Region 12 can take various shapes; the illustrated
rounded-rectangular shape is representative only. (The illustrated
shape has the advantage of encompassing a fairly large area while
introducing fewer visual artifacts than, e.g., square regions.) In
other embodiments, squares, rectangles, circles, ellipses, etc.,
can alternatively be employed.
[0027] FIG. 4 is a magnified view of an excerpt of FIG. 3, showing
a line 14 passing through the grid of points. The width of the
line, of course, depends on the particular image of which it is a
part. The illustrated line is about 25 .mu.m in width; greater or
lesser widths can naturally be used.
[0028] In a first embodiment of the invention, shown in FIG. 5, the
width of the line is controllably varied so as to change the
luminosity of the regions through which it passes. To increase the
luminosity (or reflectance), the line is made narrower (i.e. less
ink in the region). To decrease the luminosity, the line is made
wider (i.e. more ink).
[0029] Whether the luminance in a given region should be increased
or decreased depends on the particular watermarking algorithm used.
Any algorithm can be used, by changing the luminosity of regions 12
as the algorithm would otherwise change the luminance or colors of
pixels in a pixelated image.
[0030] In an exemplary algorithm, the binary data is represented as
a sequence of -1 s and 1 s, instead of 0 s and 1 s. (The binary
data can comprise a single datum, but more typically comprises
several. In an illustrative embodiment, the data comprises 100
bits.)
[0031] Each element of the binary data sequence is then multiplied
by a corresponding element of a pseudo-random number sequence,
comprised of -1 s and 1 s, to yield an intermediate data signal.
Each element of this intermediate data signal is mapped to a
corresponding sub-part of the image, such as a region 12. The image
in (and optionally around) this region is analyzed to determine its
relative capability to conceal embedded data, and a corresponding
scale factor is produced. Exemplary scale factors may range from 0
to 3. The scale factor for the region is then multiplied by the
element of the intermediate data signal mapped to the region in
order to yield a "tweak" value for the region. In the illustrated
case, the resulting tweaks can range from -3 to 3. The luminosity
of the region is then adjusted in accordance with the tweak value.
A tweak value of -3 may correspond to a -5% change in luminosity;
-2 may correspond to -2% change; -1 may correspond to -1% change; 0
may correspond to no change; 1 may correspond to +1% change; 2 may
correspond to +2% change, and 3 may correspond to +5% change. (This
example follows the basic techniques described in the Real Time
Encoder embodiment disclosed in U.S. Pat. No. 5,710,834.)
[0032] In FIG. 5, the watermarking algorithm determined that the
luminance of region A should be reduced by a certain percentage,
while the luminance of regions C and D should be increased by
certain percentages.
[0033] In region A, the luminance is reduced by increasing the line
width. In region D, the luminance is increased by reducing the line
width; similarly in region C (but to a lesser extent).
[0034] No line passes through region B, so there is no opportunity
to change the region's luminance. This is not fatal to the method,
however, since the watermarking algorithm redundantly encodes each
bit of data in sub-parts spaced throughout the line art image.
[0035] The changes to line widths in regions A and D of FIG. 5 are
exaggerated for purposes of illustration. While the illustrated
variance is possible, most implementations will modulate the line
width 3-50% (increase or decrease).
[0036] (Many watermarking algorithms routinely operate within a
signal margin of about +/-1% changes in luminosity to effect
encoding. That is, the "noise" added by the encoding amounts to
just 1% or so of the underlying signal. Lines typically don't
occupy the full area of a region, so a 10% change to line width may
only effect a 1% change to region luminosity, etc. Banknotes are
different from photographs in that the art need not convey
photorealism. Thus, banknotes can be encoded with higher energy
than is used in watermarking photographs, provided the result is
still aesthetically satisfactory. To illustrate, localized
luminance changes on the order of 10% are possible in banknotes,
while such a level of watermark energy in photographs would
generally be considered unacceptable. In some contexts, localized
luminance changes of 20, 30, 50 or even 100% are acceptable.)
[0037] In the illustrated embodiment, the change to line width is a
function solely of the tweak to be applied to a single region.
Thus, if a line passes through any part of a region to which a
tweak of 2% is to be applied, the line width in that region is
changed to effect the 2% luminance difference. In variant
embodiments, the change in line width is a function of the line's
position in the region. In particular, the change in line width is
a function of the distance between the region's center grid point
and the line's closest approach to that point. If the line passes
through the grid point, the full 2% change is effected. At
successively greater distances, successively less change is
applied. The manner in which the magnitude of the tweak changes as
a function of line position within the region can be determined by
applying one of various interpolation algorithms, such as the
bi-linear, bi-cubic, cubic splines, custom curve, etc.
[0038] In other variant embodiments, the change in line width in a
given region is a weighted function of the tweaks for adjoining or
surrounding regions. Thus, the line width in one region may be
increased or decreased in accordance with a tweak value
corresponding to one or more adjoining regions.
[0039] Combinations of the foregoing embodiments can also be
employed.
[0040] In the foregoing embodiments, it is sometimes necessary to
trade-off the tweak values of adjoining regions. For example, a
line may pass along a border between regions, or pass through the
point equidistant from four grid points ("equidistant zones"). In
such cases, the line may be subject to conflicting tweak
values--one region may want to increase the line width, while
another may want to decrease the line width. (Or both may want to
increase the line width, but differing amounts.) Similarly in cases
where the line does not pass through an equidistant zone, but the
change in line width is a function of a neighborhood of regions
whose tweaks are of different values. Again, known interpolation
functions can be employed to determine the weight to be given the
tweak from each region in determining what change is to be made to
the line width in any given region.
[0041] In the exemplary watermarking algorithm, the average change
in luminosity across the image is zero, so no generalized
lightening or darkening of the image is apparent. The localized
changes in luminosity are so minute in magnitude, and localized in
position, that they are essentially invisible (e.g.
inconspicuous/subliminal) to human viewers.
[0042] An alternative embodiment is shown in FIG. 6, in which line
position is changed rather than line width.
[0043] In FIG. 6 the original position of the line is shown in
dashed form, and the changed position of the line is shown in solid
form. To decrease a region's luminosity, the line is moved slightly
closer to the center of the grid point; to increase a region's
luminosity, the line is moved slightly away. Thus, in region A, the
line is moved towards the center grid point, while in region D it
is moved away.
[0044] It will be noted that the line on the left edge of region A
does not return to its nominal (dashed) position as it exits the
region. This is because the region to the left of region A also is
to have decreased luminosity. Where possible, it is generally
preferable not to return a line to its nominal position, but
instead permit shifted lines to remain shifted as they enter
adjoining regions. So doing permits a greater net line movement
within a region, increasing the embedded signal level.
[0045] Again, the line shifts in FIG. 6 are somewhat exaggerated.
More typical line shifts are on the order of 3-50 .mu.m.
[0046] One way to think of the FIG. 6 embodiment is to employ a
magnetism analogy. The grid point in the center of each region can
be thought of as a magnet. It either attracts or repels lines. A
tweak value of -3, for example, may correspond to a strong-valued
attraction force; a tweak value of +2 may correspond to a
middle-valued repulsion force, etc. In FIG. 6, the grid point in
region A exhibits an attraction force (i.e. a negative tweak
value), and the grid point in region D exhibits a repulsion force
(e.g. a positive tweak value).
[0047] The magnetic analogy is useful because the magnetic effect
exerted on a line depends on the distance between the line and the
grid point. Thus, a line passing near a grid point is shifted more
in position than a line near the periphery of the region.
[0048] Each of the variants discussed above in connection with FIG.
5 is likewise applicable to FIG. 6.
[0049] Combinations of the embodiments of FIGS. 5 and 6 can of
course be used, resulting in increased watermark energy, better
signal-to-noise ratio and, in many cases, less noticeable
changes.
[0050] In still a further embodiment, the luminance in each region
is changed while leaving the line unchanged. This can be effected
by sprinkling tiny dots of ink in the otherwise-vacant parts of the
region. In high quality printing, of the type used with banknotes,
droplets on the order of 3 .mu.m in diameter can be deposited.
(Still larger droplets are still beyond the perception threshold
for most viewers.) Speckling a region with such droplets (either in
a regular array, or random, or according to a desired profile such
as Gaussian), can readily effect a 1% or so change in luminosity.
(Usually dark droplets are added to a region, effecting a decrease
in luminosity. Increases in luminosity can be effected by speckling
with a light colored ink, or by forming light voids in line art
otherwise present in a region.)
[0051] In a variant of the speckling technique, very thin mesh
lines can be inserted in the artwork--again to slightly change the
luminance of one or more regions.
[0052] Although not previously mentioned, it is contemplated that
the banknote will include some manner of calibration information to
facilitate registration of the image for decoding. This calibration
information can be steganographic or overt. Several techniques for
steganographically embedding calibration information are disclosed
in my prior patents and applications. Other techniques can be found
in others of the cited work.
[0053] To decode watermark data, the encoded line art image must be
converted into electronic form for analysis. This conversion is
typically performed by a scanner.
[0054] Scanners are well known, so a detailed description is not
provided here. Suffice it to say that scanners conventionally
employ a line of closely spaced photodetector cells that produce
signals related to the amount of the light reflected from
successive swaths of the image. Most inexpensive consumer scanners
have a resolution of 300 dots per inch (dpi), or a center to center
spacing of component photodetectors of about 84 .mu.m. Higher
quality scanners of the sort found in most professional imaging
equipment and photocopiers have resolutions of 600 dpi (42 .mu.m),
1200 dpi (21.mu.m), or better.
[0055] Taking the example of a 300 dpi scanner (84 .mu.m
photodetector spacing), each 250 .mu.m region 12 on the banknote
will correspond to about a 3.times.3 array of photodetector
samples. Naturally, only in rare instances will a given region be
physically registered with the scanner so that nine photodetector
samples capture the luminance in that region, and nothing else.
More commonly, the line art is skewed with respect to the scanner
photodetectors, or is longitudinally misaligned (i.e. some
photodetectors image sub-parts of two adjoining regions). However,
since the scanner oversamples the regions, the luminance of each
region can unambiguously be determined.
[0056] In one embodiment, the scanned data from the line art is
collected in a two dimensional array and processed--according to
one of the techniques disclosed in my prior patents and
applications--to detect the embedded calibration information. The
array is then processed to effect a virtual re-registration of the
image data. A software program then analyzes the statistics of the
re-registered data (using the techniques disclosed in my prior
writings) to extract the bits of the embedded data.
[0057] (Again, the reference to my earlier watermark decoding
techniques is exemplary only. Once scanning begins and the data is
available in pixel form, it is straightforward to apply any other
watermark decoding technique to extract a correspondingly-encoded
watermark.)
[0058] In a variant embodiment, the scanned data is not assembled
in a complete array prior to the processing. Instead, it is
processed in real-time, as it is generated, in order to detect
embedded watermark data without delay. (Depending on the parameters
of the scanner, it may be necessary to scan a half-inch or so of
the line art image before the statistics of the resulting data
unambiguously indicate the presence of a watermark.)
[0059] In accordance with another aspect of the invention, various
hardware devices are provided with the capability to recognize
embedded watermark data in any line art images they process, and to
respond accordingly.
[0060] One example is a color photocopier. Such devices employ a
color scanner to generate sampled (pixel) data corresponding to an
input media (e.g. a dollar bill). If watermark data associated with
a banknote is detected, the photocopier can take one or more
steps.
[0061] One option is simply to interrupt copying, and display a
message reminding the operator that it is illegal to reproduce
currency.
[0062] Another option is to dial a remote service and report the
attempted reproduction of a banknote. Photocopiers with dial-out
capabilities are known in the art (e.g. U.S. Pat. No. 5,305,199)
and are readily adapted to this purpose. The remote service can be
an independent service, or can be a government agency.
[0063] Yet another option is to permit the copying, but to insert
forensic tracer data in the resultant copy. This tracer data can
take various forms. Steganographically encoded binary data is one
example. An example is shown in U.S. Pat. No. 5,568,268. The tracer
data can memorialize the serial number of the machine that made the
copy and/or the date and time the copy was made. To address privacy
concerns, such tracer data is not normally inserted in photocopied
output, but is so inserted only when the subject being photocopied
is detected as being a banknote. (Such an arrangement is shown in
FIG. 7.)
[0064] Desirably, the scan data is analyzed on a line-by-line basis
in order to identify illicit photocopying with a minimum of delay.
If a banknote is scanned, one or more lines of scanner output data
may be provided to the photocopier's reprographic unit before the
banknote detection decision has been made. In this case the
photocopy will have two regions: a first region that is not
tracer-marked, and a second, subsequent region in which the tracer
data has been inserted.
[0065] Photocopiers with other means to detect not-to-be-copied
documents are known in the art, and employ various response
strategies. Examples are detailed in U.S. Pat. Nos. 5,583,614,
4,723,149, 5,633,952, 5,640,467, and 5,424,807.
[0066] Another hardware device that can employ the foregoing
principles is a standalone scanner. A programmed processor (or
dedicated hardware) inside the scanner analyzes the data being
generated by the device, and responds accordingly.
[0067] Yet another hardware device that can employ the foregoing
principles is a printer. A processor inside the device analyzes
graphical image data to be printed, looking for watermarks
associated with banknotes.
[0068] For both the scanner and printer devices, response
strategies can include disabling operation, or inserting tracer
information. (Such devices typically do not have dial-out
capabilities.)
[0069] Again, it is desirable to process the scanner or printer
data as it becomes available, so as to detect any banknote
processing with a minimum of delay. Again, there will be some lag
time before a detection decision is made. Accordingly, the scanner
or printer output will be comprised of two parts, one without the
tracer data, and another with the tracer data.
[0070] Banknotes presently include various visible structures that
can be used as aids in banknote detection (e.g. the seal of the
issuing central bank, and various geometrical markings). In
accordance with a further aspect of the present invention, a
banknote is analyzed by an integrated system that considers both
the visible structures and watermark-embedded data.
[0071] Visible banknote structures can be sensed using known
pattern recognition techniques. Examples of such techniques are
disclosed in U.S. Pat. Nos. 5,321,773, 5,390,259, 5,533,144,
5,539,841, 5,583,614, 5,633,952, 4,723,149 and 5,424,807 and
laid-open foreign application EP 766,449.
[0072] In photocopiers (and the like) equipped to detect both
visible and watermarked banknote markings, the detection of either
causes one or more of the above-noted banknote responses to be
initiated.
[0073] Again, scanners and printers can be equipped with a similar
capability--analyzing the data for either of these banknote
hallmarks. If either is detected, the software (or hardware)
responds accordingly.
[0074] Identification of banknotes by watermark data provides an
important advantage over recognition by visible structures--it
cannot so easily be defeated. A banknote can be doctored (e.g. by
white-out, scissors, or less crude techniques) to remove/obliterate
the visible structures. Such a document can then be freely copied
on either a visible structure-sensing photocopier or
scanner/printer installation. The removed visible structure can
then be added in via a second printing/photocopying operation. If
the printer is not equipped with banknote-disabling capabilities,
image-editing tools can be used to insert visible structures back
into image data sets scanned from such doctored bills, and the
complete bill freely printed. By additionally including embedded
watermark data in the banknote, and sensing same, such ruses will
not succeed.
[0075] (A similar ruse is to scan a banknote image on a
non-banknote-sensing scanner. The resulting image set can then be
edited by conventional image editing tools to remove/obliterate the
visible structures. Such a data set can then be printed--even on a
printer/photocopier that examines such data for the presence of
visible structures. Again, the missing visible structures can be
inserted by a subsequent printing/photocopying operation.)
[0076] Desirably, the visible structure detector and the watermark
detector are integrated together as a single hardware and/or
software tool. This arrangement provides various economies, e.g.,
in interfacing with the scanner, manipulating pixel data sets for
pattern recognition and watermark extraction, electronically
re-registering the image to facilitate pattern
recognition/watermark extraction, issuing control signals (e.g.
disabling) signals to the photocopier/scanner, etc.
[0077] (To provide a comprehensive disclosure without unduly
lengthening the following specification, applicants incorporate by
reference the patent documents cited above (except application Ser.
No. 09/234,780 and 60/071,983).)
[0078] From the foregoing, it will be recognized that embodiments
according to the present invention allow banknotes to be recognized
based on optical scan data, and responsive actions taken, thereby
reducing the risk of counterfeiting. Other devices, responsive to
authentic banknotes, are enabled.
[0079] Having described and illustrated the principles of my
invention with reference to several illustrative embodiments, it
will be recognized that these embodiments are exemplary only and
should not be taken as limiting the scope of my invention. Guided
by the foregoing teachings, it should be apparent that other
watermarking, decoding, and anti-counterfeiting technologies can be
substituted for, and/or combined with, the elements detailed above
to yield similar effects.
[0080] While the invention has been described with reference to
embodiments employing regular rectangular arrays of grid points,
those skilled in the art will recognize that other arrays of
points--neither rectangular nor regular--can alternatively be
used.
[0081] While the invention has been described with reference to
embodiments that scale the embedded energy in accordance with local
image characteristics, in other embodiments a manually crafted
energy profile can be implemented. That is, a mask defining
embedded signal magnitudes at different parts of the image can be
manually devised, and employed to tailor the change in luminance in
each region.
[0082] In view of the many possible embodiments to which the
principles of the invention may be put, it should be recognized
that the detailed embodiments are illustrative only and should not
be taken as limiting the scope of my invention. Rather, I claim as
my invention all such embodiments as may come within the scope and
spirit of the following claims and equivalents thereto.
* * * * *