U.S. patent application number 10/622079 was filed with the patent office on 2004-05-13 for digital watermarking and fingerprinting applications.
Invention is credited to Levy, Kenneth L., Stager, Reed R..
Application Number | 20040091111 10/622079 |
Document ID | / |
Family ID | 32233282 |
Filed Date | 2004-05-13 |
United States Patent
Application |
20040091111 |
Kind Code |
A1 |
Levy, Kenneth L. ; et
al. |
May 13, 2004 |
Digital watermarking and fingerprinting applications
Abstract
Methods and systems include: (1) steganographically embedding
location information in images, where the location information is
obtained from remote sources like a cell phone network or remote
GPS receiver; (2) steganographically embedding participant IDs in
content to ensure proper billing and royalty tracking; (3)
providing a fingerprint database which is limited to a play list
for a particular broadcast network over a predetermined period; (4)
providing fair-use content management based upon digital
watermark-tracked usage; (5) providing micro-payments based upon
watermarked ID cards for retailers; or (6) providing watermarked
logon cards, such as watermarked hotel room keys, to better provide
internet logon access control.
Inventors: |
Levy, Kenneth L.;
(Stevenson, WA) ; Stager, Reed R.; (Portland,
OR) |
Correspondence
Address: |
DIGIMARC CORPORATION
19801 SW 72ND AVENUE
SUITE 250
TUALATIN
OR
97062
US
|
Family ID: |
32233282 |
Appl. No.: |
10/622079 |
Filed: |
July 16, 2003 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60396893 |
Jul 16, 2002 |
|
|
|
Current U.S.
Class: |
380/202 ;
705/58 |
Current CPC
Class: |
H04N 1/00323 20130101;
H04N 2201/3253 20130101; H04N 2201/0084 20130101; H04N 2201/327
20130101; H04N 1/32144 20130101; H04N 1/00307 20130101 |
Class at
Publication: |
380/202 ;
705/058 |
International
Class: |
H04L 009/00 |
Claims
What is claimed is:
1. A method of steganographically embedding geo-location
information in an image captured by a camera associated with cell
phone, said method comprises steps of: determining geo-location
information based on attributes associated with a cell phone
network; and steganographically embedding the geo-location
information in the image.
2. The method of claim 1, wherein the cell phone network comprises
a signal tower, and the attributes are associated with the
tower.
3. The method of claim 2, wherein the cell phone communicates a
signal, and wherein the attributes comprise strength of the signal
as received by the tower and a direction associated with the
signal.
4. The method of claim 1, wherein the cell phone network comprises
a plurality of towers, and wherein the cell phone communicates a
signal, the attributes comprising an evaluation of the signal as
received by the plurality of towers.
5. The method of claim 4, wherein the evaluation considers relative
reception timing of the signal as received by each of the plurality
of towers.
6. The method of claim 4, wherein the evaluation involves
triangulation.
7. The method of claim 1, wherein the cell phone comprises a
steganographic embedder, and uses the embedder to
steganographically embed the geo-location information in the
image.
8. The method of claim 1, wherein the cell phone network comprises
a steganographic embedder located remotely from the cell phone, and
wherein the embedder steganographically embeds the geo-location
information in the image.
9. The method of claim 8, further comprising communicating the
embedded image to the cell phone.
10. The method of claim 1, wherein the steganographic embedding
comprises digital watermarking.
11. A method of steganographically embedding geo-location
information in an image captured by a camera which is integrated
with cell phone, wherein the cell phone comprises a wireless
interface, said method comprises steps of: communicating with a
global positioning system (GPS) receiver, which is remotely located
from the cell phone via the wireless interface; receiving
geo-location information from the GPS receiver; and
steganographically embedding the geo-location information in the
image.
12. The method of claim 11, wherein the geo-location information
and image are communicated to a cell phone network server which
includes a steganographic embedder, and wherein the cell phone
network server performs said step of steganographically embedding
the geo-location information in the image.
13. A method of providing internet access for a computer user
comprising: issuing the user a digitally watermarked object,
wherein the digitally watermarked object comprises a digital
watermark embedded therein, the digital watermark comprising an
identifier; associating the identifier with the user via a data
repository; receiving optical scan data corresponding to a portion
of the object, the portion comprising the digital watermark;
decoding the digital watermark from the scan data to obtain the
identifier; verifying that the identifier is valid; and enabling
internet access for the user when the identifier is valid.
14. The method of claim 13, wherein the object comprises at least
one of a hotel room key and an object provided by a hotel.
15. The method of claim 14, further comprising associating a bill
for internet access with the user via the identifier.
16. A method of accumulating financial charges attributable to a
customer so as to minimize transaction fees, the customer
possession a digitally watermarked object, the digitally
watermarked object comprising a digital watermark including an
identifier, said method comprising: receiving scan data associated
with the digitally watermarked object; analyzing the scan data to
obtain the identifier from the object; accessing a data record that
is associated with the identifier; updating the data record to
reflect a monetary amount owed for a transaction; accumulating a
plurality of such monetary amounts in the data record; and
forwarding the accumulated amounts for payment at least when one of
the following occur: a predetermined amount for the accumulated
amount is reached, and after a predetermined amount of time.
17. The method of claim 16, wherein the aggregated monetary amounts
are forwarded to the customer for payment.
18. The method of claim 16, wherein the aggregated monetary amounts
are forwarded to a credit agency for payment.
19. The method of claim 18, wherein the credit agency comprises at
least one of a bank and credit card company.
20. The method of claim 16, wherein the identifier comprises
information personal to the customer.
21. The method of claim 20, wherein the identifier comprises a hash
of the personal information.
22. The method of claim 16, wherein the identifier is combined with
information provided by the customer to access the data record that
is associated with the identifier.
23. The method of claim 16, wherein the digital watermark further
comprises a biometric, and said method comprises comparing a
biometric sample of the customer to the biometric carried by the
digital watermark.
24. A method to regulate protected content while allowing fair use
of the content, wherein the content includes a digital watermark
embedded therein, the digital watermark including at least a copy
protection indicator and a time interval indicator, said method
comprising: recognizing the content as protected content by
reference to the copy protection indicator; and upon recognition of
the content as protected content, measuring the amount of content
rendered by reference to the time interval indicator, and disabling
rendering after a predetermined amount of content has been
rendered, the predetermined amount corresponding to fair use of the
content.
25. A method for providing royalty payments for content distributed
via a network, said method comprising: receiving registration
information from a participant who requires royalty payments for
content to be distributed; assigning a unique identifier to the
participant; steganographically embedding the content with the
identifier; and associating a royalty payment action with the
identifier in a data repository.
26. The method of claim 25, wherein the royalty payment action is
initiated when a rendering device decodes the steganographic
embedding and obtains the identifier during a transaction, the
identifier being provided to the data repository and in response,
said method comprises performing the royalty payment action,
wherein the transaction exceeds evaluation of the content.
27. The method of claim 26, wherein the network comprises a
peer-to-peer file-sharing network.
28. The method of claim 27, wherein the royalty payment action
comprises determining a percentage of revenue that corresponds to
an amount of times the content undergoes a transaction.
30. A method of monitoring a content item which is to be broadcast
through a broadcasting network, the content item to be identified
by a fingerprint of the content derived from the content itself,
said method comprising: maintaining a limited list of content
items, the list consisting of those content items that are to be
broadcast by the broadcasting network during a predetermined time
period, the limited list of content items being respectively
associated with one or more fingerprints derived from the content
items themselves; deriving a fingerprint from a content item
monitored from the broadcast network; and interrogating the limited
list of content items with the fingerprint to identify the
monitored content item.
31. A method of authenticating video comprising at least a first
frame and a second frame, said method comprising: determining a
time stamp associated with the video; and providing a digital
signature of the video, wherein the digital signature comprises
data corresponding to at least a portion of the first frame and
data corresponding to at least a portion of the second frame, said
digital signature further comprising data corresponding to the time
stamp.
32. The method of claim 31, further comprising providing
geo-location information associated with the video, wherein the
digital signature further comprises data corresponding to the
geo-location information.
33. The method of claim 31, wherein the first frame and the second
frame are adjacent frames.
34. The method of claim 31, wherein the digital signature is
carried via a reversible digital watermark.
35. The method of claim 31, wherein the digital signature is
carried via a file header.
Description
RELATED APPLICATION DATA
[0001] This application claims the benefit of U.S. Provisional
Patent Application No. 60/396,893, filed Jul. 16, 2002, which is
herein incorporated by reference.
TECHNICAL FIELD
[0002] The invention relates to digital watermarking, digital
fingerprinting, and the use of such technology for copy protection,
digital asset management, access control, authentication, content
monitoring, and a variety of other applications.
BACKGROUND
[0003] Digital watermarking is a process for modifying physical or
electronic media content to embed a hidden machine-readable code
into the media. In digital watermarking, a media content signal,
such as an image or audio signal, is modified to embed a hidden,
digital auxiliary code signal such that the auxiliary signal is
imperceptible or nearly imperceptible to the user, yet may be
detected through an automated detection process. Most commonly,
digital watermarking is applied to media content signals such as
images, audio signals, and video signals. However, watermarking may
also be applied to other types of media objects, including
documents (e.g., through line, word or character shifting, through
background patterns or tints, etc.), software, multi-dimensional
graphics models, and surface textures of objects.
[0004] Digital watermarking systems typically have two primary
components: an encoder that embeds the watermark in a host media
signal, and a decoder that detects and reads the embedded watermark
from a signal suspected of containing a watermark (a suspect
signal). The encoder embeds a watermark by subtly altering the host
media signal. The reading component analyzes a suspect signal to
detect whether a watermark is present. In applications where the
watermark encodes information, the reader extracts this information
from the detected watermark.
[0005] Several particular watermarking techniques have been
developed. The reader is presumed to be familiar with the
literature in this field. Particular techniques for embedding and
detecting imperceptible watermarks in media signals are detailed,
e.g., in the assignee's co-pending U.S. patent application Ser. No.
09/503,881 and U.S. Pat. No. 6,122,403, which are each hereby
incorporated by reference.
[0006] Another technology referred to as fingerprinting, in
contrast to digital watermarking, does not embed auxiliary data in
a media signal, but rather, derives a unique content signal
identifier from the media signal itself. For some applications
where the signal undergoes a transformation in normal use, such as
compression, transmission, or digital to analog to digital
conversion, the fingerprint (or host signal characteristics used to
determined a fingerprint) preferably remains relatively unchanged,
allowing unique identification of the content signal. Fingerprints
for a wide selection of media signals may be stored in a database
and associated with information or actions to be taken upon
detection or calculation of a fingerprint.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1 shows a method of identifying songs with audio
fingerprints.
[0008] FIG. 2 shows an improved method of identifying songs with
fingerprints.
[0009] FIG. 3 shows a card creation process.
[0010] FIG. 4 shows a card usage process.
DETAILED DESCRIPTION
[0011] Efficient Embedding of Location Information in Cameras in
Combination with Cell Phones and GPS Units
[0012] Background and Summary of Efficient Embedding of Location
Information
[0013] As background, we believe that digital asset management
(DAM) systems will mature when the process of entering metadata
about content managed in the system is mostly automated--as the
searching capabilities are already pretty good and easy to use.
Location information associated with media content, e.g., a
geo-location of where an image was taken or locations depicted in
the image, is a good example of such metadata. Other metadata may
include, e.g., day, time, people depicted in an image, subject
matter or descriptive words ("1998 Disneyland Trip"), events,
authorship, copyright information, capture device particulars,
etc., etc.
[0014] We believe that most DAM systems fail, when they do so, due
to the time and effort needed to enter metadata into the system,
not because the systems are too difficult to use.
[0015] We can simplify this effort. For example, it would be ideal
if location information is automatically embedded in consumer
images, automatically used to categorize images in DAM systems, and
used when searching to find images from similar locations--thus,
increasing the usage of the consumer based image DAM systems. This
will allow consumers to search their images, as well as other's
images assuming the consumer is part of an image sharing or
subscription service, based upon location. (The file-sharing
techniques and systems described in assignee's U.S. patent
application Ser. No. 09/620,019, filed Jul. 20, 2000 and U.S.
Published Patent Application No. US 2002-0052885 A1 which are each
herein incorporated by reference, can be readily combined with
these techniques.) For example, I can search based upon the
geo-location of the Eiffel tower, and find all of my pictures and
my friends pictures that were taken near the Eiffel tower--or even
taken in France--by simply combining geographic maps of countries
(or, e.g., embedded geographic indicators) and location search
capabilities. (See, e.g., assignee's US Published Patent
Application Nos. US 2002-0122564 A1; US 2002-0154144 A1; and US
2002-0124024 A1, and pending patent application Ser. Nos.
10/423,489 and 10/423,834 for related methods and systems. Each of
these patent documents is herein incorporated by reference.).
[0016] To automatically embed location information from a camera
into the picture, the camera preferably knows where it is located.
A simplistic solution is to put a GPS detector in the camera, but
this is often too expensive.
[0017] A preferred inventive method and system combines the
function of cell phones and/or GPS units, which know where they
are, with a camera via a physical or wireless link, so that the
camera can inexpensively (e.g., without itself needing to include a
GPS receiver) embed location data. In fact, the system can use a
cell phone server so that the camera does not even require a GPS
receiver.
[0018] In one implementation, position data is determined by a cell
phone based on signals received from multiple cell sites (or
towers), e.g., based on received cell site transmission strength
and/or known cell site positions. (The location information is
relayed (e.g., wirelessly) from the cell phone to the camera. Or
the camera includes cell phone functionality, and communicates with
the cell towers itself.). Alternatively, position data is relayed
to the cell phone from a cell site. For example, multiple cell
sites can compare respective reception times or received signals
for a first transmission from the cell phone. A reception time
differential is determined for the first transmission and is then
used to determine a location for the cell phone based on known
locations of the cell sites. Or cell phone transmissions as
received by multiple cell sites, are compared to determine a
position of the cell phone, as in triangulation methods. This
position data is then communicated from a cell site (or tower) to
cell phone. The communicated position data is optionally encrypted
or otherwise scrambled to enhance security.
[0019] In another implementation, the camera communicates with a
GPS receiver to receive location information.
[0020] Detailed Description of Efficient Embedding of Location
Information
[0021] Embodiment 1--Location Based upon Cell Phone Tower(s)
[0022] Currently, many cell phones are including still and/or video
cameras. These cell phones can easily enable video conferencing
with video cameras and image sharing with MMS (multimedia messaging
services). A phone can also know where it is located within a cell
network by having its signal strength as received at a plurality of
cell towers calculated and/or compared (and/or a time delay of
signal reception, especially with CDMA, or Code-Division Multiple
Access as provided by Qualcomm Inc., where the system includes a
master clock). In alternative position determining technique, one
cell tower employs signal direction detection, such as a circular
array of sensors with strength varying in the circle, to determine
the location of a cell phone. This location information can be sent
to the cell phone for embedding into the image or video frame. (See
assigned U.S. Published Patent Application No. 2003-0040326 A1 for
related details on determining a cell phone location. This patent
document is herein incorporated by reference.). In other cases, a
cell phone determines its location, perhaps by comparing signals
received by multiple cell sites, or by using a timing scheme to
measure signals. Alternatively, location information can be
embedded by a network server into an image or other content, and
then sent from a cell site to a cell phone during, e.g., multimedia
messaging service (MMS) or video conferencing. In fact, an image
can be uploaded to the server from a cell phone, embedded with the
cell phone's location information, and then downloaded back to the
phone, even if not used for MMS or video conferencing. This means
that the cell phone/camera do not need to include a watermark
embedder.
[0023] The embedding process can include watermarking or out of
band methods, such as header tagging. The location of the cell
phone and/or camera combination during the capture of the image can
be redundantly embedded throughout the image, or more complex
calculations can be done to embed the location of the image (or
objects or areas depicted in an image) based upon the focal point
of the camera, which includes a calculation including the cell
phone/camera location, angle, focus length, and zoom, for example.
Or, perhaps, only the location of the focal point of the center of
the picture may be embedded, or the image can be broken into
smaller section with each section containing local location
information.
[0024] Embedding the complete location information in each section
can embed the local (or area specific) location information.
Alternatively, the local location information can be embedded by
embedding the central location and differential change of the focal
point (or cell phone camera combination location) into a group of
sections, and each section in that group contains an embedded
differential code. Thus, the location of that section can be
calculated by adding the differential code times the differential
change to the central location. This group may be part of a
complete picture. The group may include a predetermined
differential size, such that only the central location needs to be
embedded in the group, along with each section having a
differential code.
[0025] The cell phone and camera can be one device (as many cell
phones now have cameras onboard) or separate, but physically (or
wirelessly) connected devices. In either case, the connection,
whether internal or external, between the cell phone and camera
could be proprietary or based upon a standard such as USB or
BlueTooth.
[0026] Location information can be shared between a cell phone and
camera based upon using multimedia platforms, such as Qualcomm's
BREW, or requiring a cell phone modification.
[0027] If a network server does the embedding, the cell phone
preferably includes a control module (e.g., software or firmware)
to help facilitate the cell phone to upload and then receive each
embedded image. As such, this system requires minimal cost to
implement.
[0028] Embodiment 2--Linking Camera and GPS
[0029] In a consumer device world which is moving from physically
connected to wireless networking, such as via BlueTooth or IEEE
802.11b (a.k.a. WiFi), a camera and location device, such as cell
phone or GPS unit, do not have to be physically connected. The
camera and cell phone can act as described in embodiment 1 but via
a wireless connection. Alternatively, a GPS unit with wireless
capabilities can be used with the camera to provide location
information.
[0030] As such, the consumer whom has a camera, and cell phone or
GPS system, does not need to buy a combination. Thus, the consumer
will not end up with a multitude of devices, or several GPS
locators embedded within each device.
[0031] In some of the above embodiments, a server can determine a
location and create bits to be embedded for a predetermined image
size. For pseudo-random noise (PN) watermarking techniques, this
may include multiplying a watermark payload, including error
correction, by the PN sequence, and including an orientation and
synchronization signal, if applicable. The server can then download
this watermark signal to a cell phone, which can, in turn, add it
to the image or perform human visual modeling to reduce visibility
of the watermark, preferably using an efficient human visual model.
Of course, an image does not need to be uploaded, thus saving
bandwidth and consumer expense, while the cell phone requires less
CPU power to embed the watermark.
[0032] In addition, in both embodiments, a picture location can
have time and biometrics added to determine where, when and who
(e.g., who's depicted in the picture and who took the picture). The
camera or server can identify people in the picture and who took
the picture, via retinal or iris scanning in the eyepiece,
fingerprint recognition in the shutter button, or face recognition
using an image of the camera user. For a description of a capture
device that captures biometrics of the user and embeds the
biometric data in images, audio or video captured by the device,
see e.g., U.S. patent application Ser. No. 09/585,678, which is
herein incorporated by reference.
[0033] Watermarked Participant IDs Used for Proper Billing
[0034] Background and Summary for Participant IDs
[0035] In distribution of content, especially digital content, it
is sometimes difficult to determine which parties require royalty
payments. Audio royalties can be based upon the usage, such as
whether the content is played on the radio, or as background music
during a radio or TV show, etc. In addition, royalties include
parties involved in both the performing and recording process (and
can be extended to distribution and retail partners outside the
area of copyright, as described below). There are often at least
two copyrights associated with a particular piece of music, e.g.,
performance and recording. Video can be just as difficult,
especially for ads where actors have to be properly
compensated.
[0036] If a watermark in the content includes IDs for all parties
that should receive royalties, the system of determining who to pay
during distribution, especially digital file sharing, is made
easier. Biometric analysis, such as facial or voice recognition may
be used to identify people in images, video and audio to assist in
determining appropriate parties to which royalties are due.
[0037] A similar problem occurs for content bought from a retailer
(or distribution service provider acting as a retailer) that is
allowed to be super or re-distributed by the consumer, such as
content that is allowed to be shared on a file sharing system with
a billing methodology. The retailer should be compensated, as well
as copyright holders, whether or not, but optimally, if the file
sharing system receives revenue from the content. For example, this
can be audio shared on KazaA or VOD video shared with a SonicBlue
Replay 4000.TM. PVR (where the retailer is the VOD service
provider).
[0038] If the content includes a retailer (or distributor/VOD) ID,
then the file sharing system can identify the original retailer and
compensate them. The compensation can be per download, per
rendering (e.g., per listening or viewing), or a percentage of
royalty related to that song. More specifically, the retailer could
be compensated with a percentage of revenue of the P2P provider
related to the percentage of time that the retailer's song was
downloaded. Similar royalties can relate to copyright owners, such
as artists and record labels.
[0039] Detailed Description for Participant IDs
[0040] The digital watermarking system is a multi-step process
(with each step sometimes having multiple sub-steps):
[0041] 1. Each participant registers for a participant ID, and the
ID is linked to the participant in a database (e.g., via a web
based interface to a server or group of servers on the Internet or
other computer network).
[0042] 2. The content is embedded with the appropriate participant
IDs.
[0043] 3. The detection system reads the participant IDs, links to
the database to interpret them, and enables the correct response or
billing action.
[0044] Each participant preferably carries the first step out only
once. The database can be centrally located with all participant
IDs, centrally distributed, either mirrored or intelligently
distributed, or local and updated (see, e.g., assignee's U.S.
patent application Ser. No. 10/060,049 (published as US
2002-0162118 A1) and related PCT Application No. PCT/US02/02572
(published as WO 02/062009), each of which is herein incorporated
by reference).
[0045] The second step needs to be only done once for each piece of
content. The watermark can be chosen from a multitude of
watermarking methods depending upon the content format, desired
robustness, payload size, embedder and detector computational
complexity, and perceptibility.
[0046] The third step is preferably performed each time the content
goes through a transaction. For example, the third step may not
have to occur during a download, if the song has limited
functionality for evaluation, but should occur when the song is
purchased or enabled for longer evaluation.
[0047] The third step may involve several sub-steps. For example,
the participant ID is extracted from the content. Then, the
database is used to link the participant ID to the participant.
Next, an appropriate action is enabled.
[0048] The appropriate action may include a multitude of actions
and related sub-steps. The appropriate action may be crediting an
artist or retailer inside a P2P system or related billing system.
The participant is paid, either each time content is transacted
(e.g., a micro-payment) or as a larger transaction based upon the
time from the last payment (e.g., pay every month) or the amount
credited (e.g., pay when credit reaches $100).
[0049] The appropriate action may include logging the participant
ID so that it can be submitted to a collection agency, such as BMI,
ASCAP, and SESAC, and used by these agencies to determine how to
distribute royalties. A record label, for promotion and demography
research, may use the system. Or the system may be used to
inter-participant trading value, such as within bartering
systems.
[0050] Improved Broadcast Monitoring Based upon Reduced Fingerprint
Database Size
[0051] Background and Summary for Improved Broadcast Monitoring
[0052] Content fingerprints, which are methods to identify content
based upon the content itself, typically use a database to perform
some processing to match incoming fingerprint to fingerprints
stored in the database. The larger the database, the more
processing required by the database, and the less accurate the
system.
[0053] Content fingerprinting methods can be used to monitor
broadcasts, such as radio and TV. A dilemma is as follows. To
monitor more stations, the system may need a large database of
songs, which causes the system to be slower and less accurate.
However, when monitoring fewer stations, the system is less
economically attractive as a business because, once the
fingerprinting system is developed and implemented, it is cheaper
to monitor more stations as the only added cost is another radio
receiver.
[0054] The novel solution is, for each station that is being
monitored, the system only searches a database for content played
on that station, such as songs played on that radio station during
a predetermined period (e.g., a week, etc). The radio and TV
stations have play lists that can be provided to the monitoring
services.
[0055] Detailed Description for Improved Broadcast Monitoring
[0056] This system is applicable to radio and TV. The detailed
example is shown for audio, since these systems have currently
shown to be more practical. However, our inventive techniques can
be applied to other types of content, such as video as well. (When
considering video broadcast monitoring systems, or if audio
fingerprinting systems are used to monitor TV stations, a TV play
list will replaces a radio station song play list and the system
works as described for radio broadcast monitoring.)
[0057] FIG. 1 shows a conventional method of identifying songs with
fingerprints. Step 300 calculates the fingerprint for the song or
section of the song.
[0058] Step 310 compares the calculated fingerprint to the database
350 and finds the closest match. This step may involve some type of
distance calculation between the calculated fingerprints (sometimes
including sub-fingerprints) and the database entries, which is
computationally intensive and increases the likelihood of error as
the database becomes larger. Even in fingerprint methods
(potentially created in the future) that do not involve distance
measurements, a larger database increases the likelihood of error.
For example, if the fingerprint ID is erroneous due to background
noise or a voice over, it is more likely to match a database ID in
a larger database.
[0059] The third step involves identifying the song once the song's
fingerprint is matched in the database.
[0060] FIG. 2 shows an inventive method of identifying songs with
fingerprints, for systems such as radio monitoring, where a limited
play list can be used to limit the database search space. The first
step 300 is identical to the currently used method, where the
song's fingerprint is calculated.
[0061] Step 330 includes comparing the song's fingerprint to a
database (or data repository), which is limited to a play list of
the radio station (or equivalent audio service, such as an Internet
streaming system). The database used in step 330 is, e.g.,
preferably a selection of the songs from the radio station play
list 340, which is a subset of the complete database 350.
[0062] The database used in step 330 may be local to the
fingerprinting system, such as located on the same PC as
calculating the fingerprint. The local database should be updated
as the radio station play list changes. Alternatively, the database
may be accessible via a network, such as the internet in a central
database, or mirror or intelligently distributed database method.
The intelligently distributed database method is an initial step of
limiting number of database entries, where, for example, US songs
not played in Europe do not exist in the versions of the database
in Europe.
[0063] Wherever the database is located, the database can have only
entries for songs included in the play list, or songs in the play
list are highlighted as active. The latter means only the active
entries are changed when the play list is updated, whereas the
former means the database entries for newly added songs have to be
added and database entries for songs in the database not included
in the new play list have to be deleted.
[0064] The play list does not need to include times that the song
is played, just a total list of every song played for a period of
time, such as that week or month. The play list may have a
likelihood or number of times each song is played to help the
fingerprint system choose between two close matches.
[0065] As such, the song identification step 320 is now more
accurate, and most likely more efficient, since the database used
in step 330 is smaller than the database used in step 310.
[0066] Fair-Use via Watermark Continuity Measurements
[0067] Background and Summary for Fair-Use via Watermark Continuity
Measurements
[0068] United States Copyright law, as well as other copyright
laws, such as in Europe and Asia, provide fair use safe harbors.
Fair use usually enables certain people to use certain amounts of
content without requiring to pay the copyright owner. For example,
in the US, a teacher can use a short segment of a movie if certain
fair use criteria are met.
[0069] In our digital world it is easy to obtain content and hard
to determine fair use, so fair use is sometimes ignored. In
addition, there are content security systems that are being built
that violate fair use. As such, a technical method that enables
calculation or determination of fair use is desirable.
[0070] One prior technique to provide fair-use is disclosed in U.S.
Pat. No. 5,765,152 (Erickson), which is herein incorporated by
reference. In one embodiment of the Erickson patent, media content
is bundled in a container including minimum permissions (e.g., to
allow fair use of the content). The media content can be handled
only according to the minimum permissions.
[0071] My proposal, in contrast, uses a digital watermark embedded
in content that is used to measure contiguous time the content is
rendered or viewed. A playing or rendering system uses the measure
to determine fair use, and can limit contiguous access to protected
content within the system once the fair use limits have been
exhausted.
[0072] Detailed Description for Fair-Use via Watermark Continuity
Measurements
[0073] Before content distribution, a watermark is embedded in the
content with a payload including a static copy protection bit or
bits, a static content ID, and dynamic counter or time reference
indicator that can be used to determine the length of content. A
rendering system used in locations requiring fair use, such as
schools or research institutions, include a watermark detector to
measures contiguous time that protected content is played. If the
time is too long, or the content is used in a manner failing
outside of fair use's boundaries, the rendering system stops
working because fair use has been violated. The content ID is used
to determine whether the play is contiguous and is not from
different segments of protected content. (The protection bits can
also be used to signal that the content is protected content. In
some implementations, the copy protection bit(s) and the time
interval indicator comprise the same payload bits. Of course, in
other implementations, the copy protection bit(s) and the time
interval indicator are separate payload fields.).
[0074] Micro-Payment via Watermark Security Cards
[0075] Background and Summary for Micro-Payments
[0076] In a secure payment system "Pay by Touch" by Indivos Inc. of
Oakland, Calif., payments are made based upon a person's
fingerprint (from a person's finger, not to be confused with a
song's fingerprint as described above). The system involves a quick
sign-up process, which includes the calculation and storage of the
person's fingerprint. A checkout procedure involves a customer
placing her finger on a touch pad at the checkout and entering a
phone number or ZIP code. The reader compares the fingerprint scan
to the images on file and calls up the appropriate account number.
After the transaction authorization, the customer must also sign
the sales slip.
[0077] This type of security can alternatively, and preferably, be
provided via a secure driver's licenses or security card based upon
digital watermarks. The watermarks authenticate the card
originality as well as the picture as matching the card data. The
debt could be accumulated and submitted every week or when it went
above $100. By submitting larger amounts, the retailer can saves
money in terms of fixed fees for credit card transactions, as well
as being able to negotiate a lower rate due to an increased number
of larger transactions. However, the retailer is assuming debt, and
this amount can be based upon personal knowledge of the consumer
(if legal) and/or past shopping history.
[0078] Detailed Description for Micro-Payment Improvements
[0079] Two processes are discussed: one for the creation of a
customer card and one for the usage of the card at a point of
sale.
[0080] As shown in FIG. 3, the card creation process receives, in
step 400, personal information from a customer. For example, a
retailer can obtain the customer's name, address, phone number, and
credit card or bank information. The personal information can be
confirmed by a credit card company, as currently done with many
currently used systems, such as when ordering on the Internet.
[0081] Step 410 includes obtaining the person's picture. The
picture can be directly captured (e.g., via a digital camera) or
obtained from a photographic repository.
[0082] Step 420 includes creating and embedding a digital
watermark. A robust watermark to be embedded in the customer
picture is preferably based upon the personal information; that is,
a digital watermark payload is created and embedded in the picture
that somehow links the picture to the information on the card or to
information provided by the customer. The payload may include, for
example, a 20-bit MD5 hash of the name, address and phone number.
The likelihood that these 20 bits are not unique for people who
look similar is low enough to provide adequate security. In
addition, within the picture or background of the card, a fragile
watermark can be added that can identify whether the card is
original or a copy.
[0083] The card is printed in step 430. Most any printer can be
used as most printers usually have minimal effects on watermarks,
but the most secure system will include a list of recommended card
printers. This list will include printers that most accurately
represent the watermarks on the card.
[0084] As shown in FIG. 4, the card usage process is employed,
e.g., in step 440, where, at a checkout, a card reader
authenticates the card. The card reader may be part of a kiosk with
a window to place the card, may be a stand alone or tethered
hand-held reader, or a device that the card is slid into and pulled
out or automatically returned (i.e. a automatic scanner).
Typically, the reader will include an optical sensor to capture
optical scan data representing the card or a portion of the card
(e.g., the picture). A digital watermark decoder analyses the scan
data to decode the watermark and obtain the watermark payload.
[0085] If the reader is part of a kiosk, the kiosk system may also
be used to help link the user to previous buying habits and
pricing. For example, the user may be able to look up how much they
paid for milk a month ago, as well as how much milk they have
bought in the last month. In addition, they may be able to look up
warranty information for products bought at a store via the kiosk.
Similarly, if the store allows online browsing, the kiosk can link
to the online shopping. For example, the kiosk can link to a wish
created while shopping online. This linking is facilitated, at
least in part, by the decoded digital watermark. For example, the
payload includes an identifier--perhaps a hash of a customer's
personal information--which is used to link to information
regarding the customer. (See, e.g., assignee's U.S. patent
application Ser. No. 09/571,422, filed May 15, 2000, which is
herein incorporated by reference, for additional watermark-based
linking techniques.)
[0086] Since the ID created from the hash may not be entirely
unique and the kiosk system may sometimes need a unique ID for each
user, a PIN and/or person's name can be used to guarantee that the
ID is unique. Alternatively, a seperate unique ID can be added to
the image or background of the card based upon a central
registration process during the creation of the card to obtain the
unique ID.
[0087] In step 450, a store employee can optionally look at the
card and holder to make sure the picture matches the person.
Optionally, in addition to or instead of comparing the picture to
the person, a PIN (personal identification number) or biometric
sample can be entered. The ideal security system involves something
you have (i.e. ID card), something you are (e.g., biometric sample
like a fingerprint, voice print, retinal scan, etc.), and something
you know (e.g., PIN or sequence). The card is something you have.
Comparing the picture (or biometric) to the card user is something
you are, and requiring a PIN requires something you know. Thus, if
both optional steps are included, where each step involves minimal
cost, the system is more secure than just requiring a card or
fingerprint.
[0088] In step 460, the retailer adds the price of the purchase to
an account or data record associated with the customer, thus
accumulating the customer's debt. The retailer can accumulate debt
for a pre-determined amount of time and/or up to a pre-determined
amount. The pre-determined amount can be dependent, e.g., upon a
customer's past history of shopping at the store, her credit
history, or her personal relationship with the retailer (subject,
of course, to any applicable laws). Pre-determined amounts can
change over time, dependent upon the customer's usage patterns and
how quickly she pays the store or her account.
[0089] As such, the retailer is accumulating risk, balanced by
saving expenses by dealing with many small transactions with credit
card companies or banks. For example, a user may disappear and
cancel a credit card before the retailer bills the credit
card--which leaves the retailer out of money.
[0090] The pre-determined amounts can work similar to how credit
cards increase a credit limit over time. For example, for the first
6 months, the system may submit every week or when an account
reaches $50, which ever is first. Then, assuming the person
continues spending money and paying their debt, the values can be
increased to 10 days and $75, and so on.
[0091] At the correct time in step 470, based upon the previously
described rules, the debt is submitted to the credit card company
or other financial institution.
[0092] Alternatively, rather than using a credit card company, the
debt could be billed to the user directly, or highlighted for the
next time the user arrives for payment (especially if the retailer
is part of a user's club or co-op where the user visits
regularly).
[0093] This system saves the most money for retailers that have
repeat customers who spend minimal amounts each visit, such as for
a neighborhood market or convenience store, or hardware store.
[0094] This system can also be used with an Internet online
retailer, since a PC and camera can securely read a watermarked
card. In fact, the pre-determined amounts can be updated by ratings
of the Internet site from other user if the system involves
trading, such as for Web sites like eBay.com.
[0095] This system could also use a driver's license, when the
driver's license has digital watermarks authenticating the card,
instead of a proprietary store card. Similarly, instead of the
card, the fingerprint can be used with the described debt
accumulation system. The fingerprint can potentially along with the
PIN, signature and/or even a security card with a watermark or
magnetic strip (for something you have), could be used with the
method for accumulating micro-payments. In this system, the
fingerprint and signature (if included) are something you are, the
PIN (if included) is something you know, and security card (if
included) is something you have. Thus, if the optional parts are
required, the system has maximum security.
[0096] Digimarc MediaBridge Enabled Physical Internet Access Logon
Cards
[0097] Background and Summary for Access Logon Cards
[0098] Many Internet cafs and hotels with wireless networks and/or
central Internet-ready PCs use passwords to stop unauthorized
people from using an Internet link, as well as to track usage.
However, it is easy for users to share the passwords, as well as it
is expensive to maintain because the system requires a hotel to
change, remember, and provide the current password.
[0099] A logon card including a digital watermark embedded therein
that can be read by a PC optical camera, can enable Internet access
in computers with cameras. One type of watermarking scheme is
provided by Digimarc's (Tualatin, Oreg., USA) MediaBridge.TM.
technology. Our logon cards are more secure than passwords since
when people share them, a copy is not made. These logon cards are
more efficient than passwords because they don't need to be changed
since they cannot be duplicated. The logon card can be combined
with existing cards, such as with modern plastic hotel room keys
(that use a magnetic strip to open the door) or with personal
membership cards. This combination reduces cost because several
cards don't have to be created and monitored for loss.
[0100] Detailed Description for Access Logon Cards
[0101] Detailed examples involving a hotel and Internet caf are one
efficient way to explain this system.
[0102] For a hotel with a magnetically coded room keys, if every
room key is pre-watermarked with a unique ID, when the key is given
to the user and coded for their room, the watermark ID can be
linked to the room. Then, if the consumer wants to use a computer
in the business center of the hotel, for example, rather than
requiring a password, the user shows his/her watermarked room key
to the web camera on the PC in the business center, and the
computer is unlocked so the consumer can use it. Since the room key
is linked to the room, the computer usage can be automatically
billed to the user's room.
[0103] Similarly, our logon card system can be employed for
consumers to use a wireless network--which works as follows. A user
shows there watermarked room key to a wireless access terminal. The
access terminal captures an image of the room key, decodes the
watermark from captured image, check to see if the ID is valid
(e.g., ensuring the that user is a current quest at the hotel), and
then enables wireless access, if the ID iv valid. (The hotel or
affiliated provider can maintain a database of unique ID. IDs can
be flagged valid or invalid, or even removed from the database if
they have expired.) Thus, consumers with wireless network cards do
not need encryption codes that can easily be shared and hard to
find at the front desk to use the hotel's wireless network internet
connection, they only need a PC with a camera. In turn, the hotel
doesn't need to maintain the encryption system.
[0104] Once again, the system can automatically bill usage to the
user's room since the watermark is linked to the hotel room, even
though the consumer can work as they wonder throughout the hotel
(e.g., not locked in their room or a hotel business center).
[0105] This system can also work for in-room wired networks.
Although, the room number can be known in this case due to the
physical wire connected to the room, if the hotel is controlling
the PCs in its business center and wireless network with the room
key, it may be advantageous to have all the systems work
identically. In addition, with our logon card system, the rooms can
be wired with standard ethernet capabilities, where the physical
wires don't have to be linked to a room (and the PC doesn't have to
be identified so several people don't share a connection in one
room).
[0106] In an Internet caf example, a logon card given to the
consumer when they are ready to use a PC controls login. The logon
card is shown to a camera on the PC to log them on, as described
for the PCs in the hotel's business center. The usage can be timed
on the PC that the consumer is using (as identified by a watermark
ID on the logon card), or by a master PC, which determines when the
logon card was given to the consumer and when returned (by showing
it to the master PC's camera when given and received). With either
billing method, even if the consumer shares the logon card, they
are still billed since it can identify them.
[0107] In addition, if an Internet caf wants to have a membership
card, where a consumer gets 5 free hours with every 20 hours used,
for example, the membership card can be watermarked with a unique
ID that is used to log onto the computer and link to the member.
The card can also have security measures, if desired, such as
fragile watermarks to locate copies and watermarks that link the
picture to the personal data (via embedding a 20 bit hash of the
user's name, for example). The fragile watermark degrades when a
reproduction of the originally watermarked item is made, enabling a
watermark reader to differentiate reproductions from an original,
watermarked object.
[0108] The system level details include that after the card is
shown to the PC with the camera, the watermark ID is sent to a
database that links the ID to a room number and identifies that the
ID is active. If the ID is active, the consumer is allowed to
logon, and the appropriate billing action is started. If the ID is
not active, the logon is not allowed. They system should be secured
by authenticating the card reading software to the database, and
encrypting the watermark ID when sent within the PC or network.
Session keys should be changed to be resistant to replay attack, as
well known in the art of cryptography.
[0109] In the Internet caf system, which uses a master PC to
monitor the amount of time the card is checked-out, the PC that the
consumer uses only needs to verify that the watermark ID is active
or authentic to enable logon.
[0110] See, e.g., assignee's U.S. patent application Ser. Nos.
10/382,359 and 09/571,422, which are each herein incorporated by
reference, for related techniques and/or environments.
[0111] Surveillance Video Authentication
[0112] Background and Summary for Surveillance Video
Authentication
[0113] Authentication of surveillance video, such as that captured
by closed circuit TV (CCTV), is important to verify in court that
the surveillance video is authentic. The Digital Signature Standard
(DSS), or any digital signature (by definition, including the
private key encryption of a robust hash), can be used to
authenticate the accuracy of every bit in each frame of
surveillance video. The unique combination of private key usage,
frame splitting and date-time addition can improve the
authentication to guarantee that no frames were removed, that the
frames were recorded at the appropriate date and time, and that the
appropriate recorder performed the recording. Location information
can also be embedded in the video in the form of a digital
watermark to improve authentication.
[0114] Detailed Description for Surveillance Video
Authentication
[0115] The DSS, as described in Spec "Federal Information
Processing Standard (FIPS 186) Digital Signature Standard (DSS)"
available at <http://www.itl.nist.gov/fipspubs/fip186.htm>
and uses the secure hash algorithm (SHA-1) as described at
<http://www.itl.nist.gov/fipspu- bs/fip180-1.htm>, or any
other digital signature based upon a robust hash and public key
cryptography, is used to demonstrate that no bits in each frame
have been modified. The signature for each frame is saved as a
signature frame in a separate file or part of the video header,
with the whole signature saved in the video header or each
signature frame saved in each corresponding video frame's header.
The system also demonstrates that no frames have been added because
the correct signature cannot be calculated.
[0116] As an alternative and inventive option, the system can use
half of one frame and the other half of the next frame in the
signature for each frame. This system demonstrates that every bit
of each frame is authentic and that no frames have been added (as
before, as well as the fact that no frames have been removed). The
additional authentication that no frames have been removed is based
on the fact that each frame signature includes bits from itself and
another frame; thus, if any frame is removed the previous frame's
signature will not match. This system can be designed in many ways,
such as using thirds of a frame, the previous frame and the next
frame, as long as every bit of each frame is included somewhere in
the signature and one frame is not completely included in one
signature frame. The signature frames can be saved in a separate
file, as part of the video header, or as part of each frame
header.
[0117] Alternatively, the complete group of signature frames can be
encrypted with the private key to make sure no frames are removed.
Similarly, the system, for a fixed video, could consider all the
frames as one large message, and perform a DSS on the complete
message, as long as the message length does not exceed the limit
set by the digital signature.
[0118] Additionally, the date and time (and/or geo-location
associated with the video or area depicted in the video) can be
included as part of the frame signature to verify the date and time
(and/or location) of the video. The date time (and/or location) can
be added to the robust hash calculation (by converting the date
time stamp to bits and included as any additional frame bits) or
appending the date time stamp to the robust hash payload and then
encrypted with the private key. The date time stamp can have frame
or second accuracy (or maybe even minute accuracy).
[0119] Furthermore, each video recorder can share a system-level
private key or have its owner recorder private key. If separate
recorder private keys are used, the system can track the machine
from which the video was recorded based upon the one-to-one
relationship of the decryption public key and encryption private
key. More specifically, as well known, if the wrong public key is
used, the authentication process will fail; thus, the public key
identifies the private key used, which, in turn, identifies the
machine.
[0120] As discussed in the DSS or well known in the state of art
for cryptography and digital signatures, the authentication process
includes using the public key to decrypt the digital signature and
compare it to the robust hash calculation of the video frame data
(and possibly date and time, if included as part of the hash and
not a separate part of the payload). If they match, the video frame
data is authentic. If the date time stamp is included as an
appended part of the robust hash payload, the date time stamp can
be read from the digital signature to verify the date and time.
[0121] As such, if every additional option is used, every bit of
the video is shown to be authentic, meaning that no bits have not
been modified, removed or added. In addition, the date and time of
the video is known. Finally, the recorder can be identified based
upon the public decryption key.
[0122] Alternatively, the RSA algorithm, rather than DSA algorithm
can be chosen, as described at
<http://www.rsasecurity.com/rsalabs/faq/3-4-1.- html>, to
make the authentication (referred to as verification in the
referred web page) process faster than the signing process, but
this is probably not advantageous for this system since the signing
must be done in real time on the recorder and the authentication
can be done offline.
[0123] Furthermore, small segments (like 128.times.128 pixels) of
each frame can be used, with some segment overlapping between
frames. This has the advantage of demonstrating the location of any
manipulation, as well as linking frames together, so frames cannot
be removed. The digital signature for each segment within each
frame and for each segment started within the frame (and including
some of the next or other frame), can be stored in the header of
that frame or as a group of frames, and one needs to include a date
time stamp. In other words, the frame digital signature consists of
many smaller signatures, and includes a date time stamp.
[0124] The digital signatures could be included in reversible
watermarks within the frame content for each frame segment, rather
in header data. A reversible watermark is generally a watermark,
which can be removed from content without degrading (or without
significantly degrading) the content. In some cases, removing a
watermark implies restoring content to its unmarked state. Suitable
reversible watermarks are described, e.g., in assignee's pending
U.S. patent application Ser. No. 10/319,404, filed Dec. 12, 2002
and Ser. No. 10/435,517, filed May 8, 2003, which are each hereby
incorporated by reference.
CONCLUDING REMARKS
[0125] Having described and illustrated the principles of the
technology with reference to specific implementations, it will be
recognized that the technology can be implemented in many other,
different, forms.
[0126] To provide a comprehensive disclosure without unduly
lengthening the specification, applicants incorporate by reference
the patents and patent applications referenced above.
[0127] The methods, processes, and systems described above may be
implemented in hardware, software or a combination of hardware and
software. For example, the auxiliary data encoding processes may be
implemented in a programmable computer or a special purpose digital
circuit. Similarly, auxiliary data decoding and fingerprint
calculation may be implemented in software, firmware, hardware, or
combinations of software, firmware and hardware. The methods and
processes described above may be implemented in programs executed
from a system's memory (a computer readable medium, such as an
electronic, optical or magnetic storage device).
[0128] The particular combinations of elements and features in the
above-detailed embodiments are exemplary only; the interchanging
and substitution of these teachings with other teachings in this
and the incorporated-by-reference patents/applications are also
contemplated.
* * * * *
References