U.S. patent application number 14/598879 was filed with the patent office on 2015-07-23 for machine learning-based patent quality metric.
The applicant listed for this patent is Matthew BEERS, Elvir CAUSEVIC. Invention is credited to Matthew BEERS, Elvir CAUSEVIC.
Application Number | 20150206069 14/598879 |
Document ID | / |
Family ID | 53545096 |
Filed Date | 2015-07-23 |
United States Patent
Application |
20150206069 |
Kind Code |
A1 |
BEERS; Matthew ; et
al. |
July 23, 2015 |
MACHINE LEARNING-BASED PATENT QUALITY METRIC
Abstract
A machine-learning based artificial intelligence device for
finding an estimate of patent quality, such as patent lifetime or
term is disclosed. Such a device may receive a first set of patent
data and generate a list of binary classifiers. A candidate set of
binary classifiers may be selected and using a heuristic search,
for example an artificial neural network (ANN), a genetic
algorithm, a final set of binary classifiers is found by maximizing
iteratively a yield according to a cost function, such an area
under a curve (AUC) of a receiver operating characteristic (ROC).
The device may then receive patent information for a target patent
and report an estimate of patent quality according to the final set
of binary classifiers.
Inventors: |
BEERS; Matthew; (Ashland,
OR) ; CAUSEVIC; Elvir; (San Francisco, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
BEERS; Matthew
CAUSEVIC; Elvir |
Ashland
San Francisco |
OR
CA |
US
US |
|
|
Family ID: |
53545096 |
Appl. No.: |
14/598879 |
Filed: |
January 16, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61928806 |
Jan 17, 2014 |
|
|
|
Current U.S.
Class: |
706/13 ; 706/12;
706/25 |
Current CPC
Class: |
G06N 3/0454 20130101;
G06K 9/626 20130101; G06K 9/00442 20130101; G06K 9/6262 20130101;
G06N 3/086 20130101; G06K 9/6277 20130101; G06K 9/6227 20130101;
G06N 5/043 20130101; G06N 20/00 20190101 |
International
Class: |
G06N 99/00 20060101
G06N099/00; G06N 3/12 20060101 G06N003/12; G06N 3/08 20060101
G06N003/08; G06K 9/00 20060101 G06K009/00; G06K 9/62 20060101
G06K009/62 |
Claims
1. A machine-learning based artificial intelligence device for
finding an estimate of patent quality, the device comprising: a
patent data retriever configured to receive a first set of patent
data comprising at least one of patent application data and patent
data for a plurality of patents, and to generate a list of binary
classifiers based on the first set of patent data; a quantitative
data scalar configured to assign a standardized scaled score to
each binary classifier of the list of binary classifiers; and a
binary classifier optimizer configured to generate, using an
automated processor, a candidate set of binary classifiers from the
list of binary classifiers using a heuristic search and to
generate, using the automated processor, a final set of binary
classifiers by maximizing iteratively a yield according to a cost
function, wherein the device is configured to provide a signal
representing the final set of binary classifiers.
2. The device of claim 1, wherein the heuristic search comprises an
artificial neural network model.
3. The device of claim 2, wherein the maximizing iteratively
comprises changing a number of hidden layers of the artificial
neural network.
4. The device of claim 1, wherein the maximizing iteratively
comprises using a genetic algorithm.
5. The device of claim 1, wherein the maximizing iteratively
comprises using an artificial neural network model and a genetic
algorithm.
6. The device of claim 1, wherein the cost function is a receiver
operating characteristic and the yield is calculated according an
area under a curve.
7. The device of claim 1, wherein the estimate of patent quality
represents an estimate of a lifetime of the patent.
8. The device of claim 1, wherein the patent data retriever is
configured to receive a second set of patent data comprising at
least one of patent application data and patent data for a
plurality of patents, and wherein the device is configured to test
a validity of the final set of binary classifiers using the second
set of patent data.
9. The device of claim 1, further comprising a user information
manager configured to receive patent information for a target
patent and to report the estimate of patent quality according to
the final set of binary classifiers.
10. A system comprising the device of claim 1 and a second device
communicatively connected to the device over a network, the second
device comprising: a second automated processor: a user interface
receiving the patent information for the target patent; an estimate
requester requesting from the device the estimate of patent quality
for the target patent; and the user interface providing to a user a
signal representing the estimate of patent quality.
Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] The present non-provisional patent application claims the
benefit of priority from U.S. Provisional Patent Application No.
61/928,806, filed Jan. 17, 2014, the entire contents of which are
incorporated herein by reference.
BACKGROUND
[0002] 1. Field of the Invention
[0003] The present disclosure relates to a system comprising a CPU,
storage and database of patent grants or applications and other
relevant data for computation of an estimation of patent quality
utilizing machine learning algorithms for factor selection and
classification based on non-linear models.
[0004] 2. Related Art
[0005] Attempts have been made to assess or to estimate the value
or expected life of a patent or a patent application based on
historic data about patents. However, testable and reproducible
quantitative metrics are difficult to come by. Also, using a
combination of quantitative factors available from a universe of
patent information to arrive at a patent value or estimated patent
life or the like is difficult given the sheer number of
patent-related and patent application-related factors and given
that each patent represents a unique invention. Therefore, finding
the combination of factors that produces an optimal or maximized
patent quality/patent life profile has been a difficult task.
[0006] Existing methods of patent quality ratings depend on either
linear combinations of simple factors (e.g. the number of forward
citations combined with age of the patent) or traditional linear
and statistical mathematical tools based on an iterative human
driven factor selection process. Using a "brute force" approach to
finding the most relevant factors entails examining every factor
and every combination of factors. The solution space for a machine
learning problem should be considered as all possible combinations
of factors and coefficients. Therefore, the only way to find the
optimal solution using the brute force approach is consider every
element in the solution space iteratively; this process is known as
brute-force computation. As a simple example for a problem with two
factors, A and B, and no coefficients, the algorithm would need to
consider at least:
A
B
A+B
[0007] as the potential solutions to the problem. If a third
factor, C, was added, the brute-force approach would then need to
consider:
A
B
C
A+B
A+C
B+C
A+B+C
[0008] Generally, using a brute-force approach, each additional
factor, or combination of factors, increases the complexity and the
processing time exponentially.
SUMMARY OF THE DISCLOSURE
[0009] A machine-learning based artificial intelligence device for
finding an estimate of patent quality is disclosed. Such a device
may include:
a patent data retriever configured to receive a first set of patent
data comprising at least one of patent application data and patent
data for a plurality of patents, and to generate a list of binary
classifiers based on the first set of patent data; a quantitative
data scalar configured to assign a standardized scaled score to
each binary classifier of the list of binary classifiers; a binary
classifier optimizer configured to generate, using an automated
processor, a candidate set of binary classifiers from the list of
binary classifiers using a heuristic search and to generate, using
the automated processor, a final set of binary classifiers by
maximizing iteratively a yield according to a cost function,
wherein the device is configured to provide a signal representing
the final set of binary classifiers. The heuristic search may
include an artificial neural network model. The maximizing
iteratively may include changing a number of hidden layers of the
artificial neural network.
[0010] The maximizing iteratively may include using a genetic
algorithm or an artificial neural network model and a genetic
algorithm.
[0011] The cost function may be a receiver operating characteristic
and the yield may be calculated according an area under a
curve.
[0012] The estimate of patent quality may represent an estimate of
a lifetime of the patent.
[0013] The patent data retriever may be configured to receive a
second set of patent data comprising at least one of patent
application data and patent data for a plurality of patents,
and
[0014] wherein the device may be configured to test a validity of
the final set of binary classifiers using the second set of patent
data.
[0015] The device may also include a user information manager
configured to receive patent information for a target patent and to
report the estimate of patent quality according to the final set of
binary classifiers.
[0016] Also contemplated is a system that includes such a device in
combination with a second device communicatively connected to the
device over a network. Such a second device may include:
a second automated processor; a user interface receiving the patent
information for the target patent; an estimate requester requesting
from the device the estimate of patent quality for the target
patent; and the user interface providing to a user a signal
representing the estimate of patent quality.
[0017] Further aspects of the disclosure are explained in the
description below and in the accompanying Drawings.
BRIEF DESCRIPTION OF THE FIGURES
[0018] FIG. 1 is an example of ROC (Receiver Operating
Characteristic) curves generated by a heuristic such as NBC or ANN,
according to an aspect of the present disclosure.
[0019] FIG. 2 is an example of an overview of a machine learning
approach, according to an aspect of the present disclosure.
[0020] FIG. 3 is an example of a more detailed overview of a
machine learning approach, according to an aspect of the present
disclosure.
[0021] FIG. 4 is an example of a classifier selection process,
according to an aspect of the present disclosure.
[0022] FIG. 5 is an example of an artificial neural network model
iteration for finding binary classifiers, according to an aspect of
the present disclosure.
[0023] FIG. 6 is a schematic diagram showing an overview of a value
evaluation system connected over a network, according to an aspect
of the present disclosure.
[0024] FIG. 7 is an example of a patent value determination module
and some components, according to an aspect of the present
disclosure.
[0025] FIGS. 8A-8B contain a flowchart illustrating an example of
steps of a machine learning and patent life query responding
method, according to an aspect of the present disclosure.
DETAILED DESCRIPTION OF THE DISCLOSURE
[0026] A computer system, network platform including a server
computer, a processor-readable medium, a method, and means for
implementing the method according to the present disclosure employs
a set of algorithms based on training data receive from a database
of patent information, including granted patents and patents
applications in addition to other relevant patent data, including
aggregate data for patent examination, grant, opposition,
abandonment, annuity/maintenance fee payment, and the like. A
device or a system according to the present disclosure implements a
suite of binary classifiers to predict a measure of patent quality,
for example, whether a given issued patent will be maintained over
the lifetime of that patent. Other measures of quality may include
whether a patent will be licensed or upheld against legal
challenge, and the like. The system may also be adapted to predict
a measure of quality of other intangible assets.
[0027] Supervised machine learning algorithms are used to select an
optimal set of input factors from a number of raw and computed
inputs and then to find a set of binary classifier from a set of
classifiers, for example using Naive Bayes Classifier (NBC),
Artificial Neural Network (ANN) or Support Vector Machines (SVC).
The disclosed invention trains classifiers to represent a
prediction based on a non-linear computation of the input
factors.
[0028] The system selects features using a heuristic search
procedure such as a genetic algorithm or simulated annealing. The
algorithms accept as input a series of features identified from
information for a set of patents and patent applications. A random
or pseudo-random initial weight for each feature is assigned and
the search proceeds to iterate over the input set of data. At each
iteration the heuristic evaluates a cost function and determines
whether the current state of feature weights is more optimal than
the previous state. The final step is to mutate the feature weights
before starting the next iteration. The mutation computation varies
based on the algorithm; in a genetic algorithm, the weights are
mutated randomly or pseudo-randomly, while using simulated
annealing the weights are modified according to an energy
transition equation. The heuristic terminates after a given number
of iterations or when the changes to the feature weight drops below
a given threshold. The threshold may be a user-defined parameter
chosen based on experience with the system. The final selected
factors are used to train a binary classifier.
[0029] The cost function utilized by the search procedure heuristic
is used to optimize the area under a Receiver Operating
Characteristic (ROC) curve (FIG. 1). At each iteration, the current
set of factors under consideration--as identified by those with
non-zero factor weights--are used to train a binary classifier.
[0030] A larger sampling may yield a more accurate result for the
model. For example, 100,000 patent records may be used and divided
into the three sets to yield good sampling sets. The sets need not
be of equal size. However, it will be understood that more than
100,000 or fewer than 100,000 records may be used. The training,
validation and testing sets need not necessarily follow particular
size guidelines and may be dependent on the size of the total
population. For example, four million active U.S. patents versus
1.5 million active EPO patents may have different training set
sizes. The machine learning may be customized for country or
region, such that patent value/estimate of patent quality returned
for a patent queried may be based only on data obtained from patent
information for the country or region of the patent queried.
Similarly, the patent value/estimate of patent quality may be
customized for a given field of technology or scientific endeavor,
for example, mechanical arts, pharmaceuticals, chemical fields,
computer-related technology, and the like. In this way, the patent
value/estimate of patent quality returned for queried patent of
field of technology or scientific endeavor X may be based only on
data obtained based on patents/patent applications of field X.
[0031] The resultant binary classifier is evaluated against the
cross-validation set and an ROC curve is computed. FIG. 1 shows the
results of several iterations of factor selection and additionally
demonstrates that the search heuristic can select an optimal binary
classification algorithm in addition to the input factors.
[0032] Specificity is defined as the number of true negatives
divided by the total number of negatives. For example, in a data
set with ten total negatives and finding two of them, specificity
equals 2/10 or 0.2. The false positive rate is then 1-0.2 which
equals 0.8. "ANN" refers to the artificial neural network
classifier, and "NBC" refers to the Naive Bayes classifier.
[0033] The system maintains a database of raw patent factors that
are derived from the patent publication such as the number of
claims, number of citations, countries of issuance, patent
litigation and licensing, are also stored in the database.
Additional such factors may include:
TABLE-US-00001 TABLE 1 Raw Factors Issuing countries Priority
Application Date Foreign Priority Issuance Date Number of Assignees
Length of Pendency Number of Licensees Number of Inventors Number
of Attorneys Number of Figures Number of Office Actions Number of
Claims PCT Issuance Number of Independent Claims Patent Family Size
Number of Dependent Claims Number of words in Description Average
Number of Words in Claims Number of words in Title Total number of
words in claims Number of Backward Citations Number of different
words in claims Number of Forward Citations Claim Type Number of
Patent Classifications Number of words in Abstract Number of
Foreign Citations
[0034] From these raw factors, the input features to be evaluated
by the search heuristic are calculated through a number of methods
including: [0035] 1. Linear combination (e.g., adding factors
together) [0036] 2. Non-linear calculations (e.g., squaring a
factor or taking the square root) [0037] 3. Ratios of raw factors
(e.g., number of patents with 10 claims against all the total
number of all patients) These methods produce around 200 features,
with approximately 30 from raw factors, 50 ratio factors and the
rest being combinations. With a base set of features calculated,
the set of available features can be further expanded by computing
linear and non-linear combinations of all features to be evaluated
by the search heuristic. This expansion results in a combinatorial
increase in the number of available features. The binary
classifiers are trained using supervised machine learning with
three sets of data: training set, cross-validation set, and a
testing set. The input sets comprise a random or pseudo-random
sampling of issued patents from a given patent office. In a
preferred embodiment, the system then creates multiple binary
classifiers, each predicting the maintenance of patent for a given
maintenance period. The final output of each classifier is combined
into a final score.
[0038] A significant advantage of the use of machine learning when
identifying input factors and computing the classification model is
that the model can be continuously updated in response to changes
in the market, such as increased rates of abandonment or
litigation- or to the availability of additional raw factors. In
this way the rating can be constantly maximized for prediction
accuracy.
[0039] The system effectively works in two different phases. The
first phase, described in FIG. 2, is used in the regular
computation of the patent scores. It utilizes the classifier and
parameters that were calculated during the second phase (FIG. 3 and
FIG. 4).
[0040] For regular score computation, the system starts by
downloading the electronic record for each published and granted
patent from a given patent office (e.g., the European Patent
Office) and stores the downloaded information in a database. In a
preferred embodiment, the download process runs automatically in
response to external events; e.g., if the issued patents are
published on Tuesday morning, the system may automatically start to
download that week's issued patents on Tuesday afternoon.
[0041] Most patent offices publish the patent data in a
standardized XML format. The downloading process parses the data
from the XML and stores the information in the database. The system
then proceeds to compute the input features to the classifier using
the raw factors from the patent record. The final score is computed
using the trained classifier and then saved with the patent
record.
In addition to information in the electronic record stored,
additional raw factors may be calculated from the data in the
electronic record. A list of raw factors can be found in Table 1.
However, it will be understood that such a list is not exhaustive
and that many other such raw factors may be used in addition to, or
instead of those listed.
[0042] The system computes the model by first computing a set of
features from the electronic patent data stored in the database.
The features fall into two categories. The first category is the
raw factors on a patent basis from Table 1. The second are features
that are computed over multiple records of patent data (i.e., over
the entire set or over a subset). A list of the features considered
when training the model is listed in Table 2.
TABLE-US-00002 TABLE 2 Computed Features pcnt_abandoned Number of
patents abandoned by year of patent against total number patents
filed on a given date AB abandyear Number of years after issuance
the patent was abandoned, or 20 if still in-force Percent abandoned
grouped by assignee based on the patents in the period (now-4.5
years) and (now-8.5 years) pcnt_abandoned_assg Percentage of
patents abandoned by the assignee in the given date range A
pcnt_abandoned_assg_avg Average percentage of patents abandoned by
the assignee in the given date range C pcnt_abandoned_assg_stdev
Standard deviation of percentage of patents abandoned by the
assignee in the given date range E pcnt_abandoned_assg_median
Median percentage of patents abandoned by the assignee in the given
date range Percent abandoned grouped by assignee based on the
patents in the period (now-4.5 years) and (now-16.5 years)
pcnt_abandoned_assg_16 Percentage of patents abandoned by the
assignee in the given date range B pcnt_abandoned_assg_16_avg
Average percentage of patents abandoned by the assignee in the
given date range D pcnt_abandoned_assg_16_stdev Standard deviation
of percentage of patents abandoned by the assignee in the given
date range F pcnt_abandoned_assg_16_median Median percentage of
patents abandoned by the assignee in the given date range Percent
abandoned grouped by attorney based on the patents in the period
(now-4.5 years) and (now-8.5 years) pcnt_abandoned_atty Percentage
of patents abandoned by attorney in the given date range
pcnt_abandoned_atty_avg Average percentage of patents abandoned by
attorney in the given date range pcnt_abandoned_atty_stdev Standard
deviation of percentage of patents abandoned by attorney in the
given date range pcnt_abandoned_atty_median Median percentage of
patents abandoned by attorney in the given date range Percent
abandoned grouped by attorney based on the patents in the period
(now-4.5 years) and (now-16.5 years) pcnt_abandoned_atty_16
Percentage of patents abandoned by attorney in the given date range
pcnt_abandoned_atty_16_avg Average percentage of patents abandoned
by attorney in the given date range pcnt_abandoned_atty_16_stdev
Standard deviation of percentage of patents abandoned by attorney
in the given date range pcnt_abandoned_atty_16_median Median
percentage of patents abandoned by attorney in the given date range
Data calculated based on the legal status codes of the patents. G
impact_plus Count of positive legal status codes H impact_minus
Count of negative legal status codes I impact_neutral Count of
neutral legal status codes J assg_avg_ip Average of Impact Plus per
assignee K assg_avg_im Average of Impact Minus per assignee L
assg_avg_in Average of Impact Neutral per assignee M assg_stdev_ip
Standard deviation of Impact Plus per assignee N assg_stdev_im
Standard deviation of Impact Minus per assignee O assg_stdev_in
Standard deviation of Impact Neutral per assignee P assg_median_ip
Median of Plus per assignee Q assg_median_im Median of Minus per
assignee R assg_median_in Median of Neutral per assignee S
atty_avg_ip Average of Impact Plus per attorney T atty_avg_im
Average of Impact Minus per attorney U atty_avg_in Average of
Impact Neutral per attorney V atty_stdev_ip Standard deviation of
Impact Plus per attorney W atty_stdev_im Standard deviation of
Impact Minus per attorney X atty_stdev_in Standard deviation of
Impact Neutral per attorney Y atty_median_ip Median of Plus per
attorney Z atty_median_im Median of Minus per attorney AA
atty_median_in Median of Neutral per attorney Data calculated based
on the pendency, calculated as (filed - issued) pendancy_month
Number of months between filing and issuance per patent
pendancy_month_avg_by_week Average pendency grouped by week of
issuance pendancy_month_stdev_by_week Standard deviation of the
pendency grouped by week of issuance pendancy_month_median_by_week
Median pendency grouped by week of issuance Data calculated based
on number of independent claims iclaim_avg_by_week Average number
of independent claims group by issuance week AC
iclaim_stdev_by_week Standard deviation of independent claims group
by issuance week AD iclaim_median_by_week Median of independent
claims group by issuance week AE iclaim_avg_by_ipc Average number
of independent claims group by International Patent Classification
AF iclaim_stdev_by_ipc Standard deviation of independent claims
group by International Patent Classification AG
iclaim_median_by_ipc Median of independent claims group by
International Patent Classification
[0043] In Table 2, legal status code refers to events during the
lifetime of the patent. These include office actions, change of
ownership, abandonment, maintenance and expiration. "Week of
issuance" may refer to a week number of the year that patent was
granted. (e.g. January 1 is week 1, etc). "iClaim" means
independent claim, Claim type "A" refers to an apparatus claim,
claim type "S" to a system claim, claim type "C" to a claim for a
compound, and claim type "M" refers to a method claim. "Pendency"
may be the time between the application initial filing date or a
provisional initial filing date and the final action, such as a
Notice of Allowance or issuance of a patent.
[0044] The plus sign ("+") on the appended list of legal status
codes indicates those status codes that are positive, meaning
having received some positive treatment, the minus sign ("-")
indicates those status codes that are negative. Those with neither
sign mean neutral treatment. Individual specific legal status codes
need not be counted, but a total number of status codes that are
positive, negative and neutral may be counted.
[0045] In a preferred embodiment, training the model begins by
exporting three sets of randomly or pseudo-randomly selected issued
patent records. The first is the training set that is used to
evaluate a set of parameters in the model to determine how accurate
the prediction is. The second is a cross-validation set that is
used as second check for the accuracy of the prediction. By using a
different set to evaluate a set of parameters instead of the
training set, the model achieves a greater level of accuracy. The
training and cross-validation sets are both used to select
parameters in the model. The final set is a testing set that is
used to evaluate a complete model for accuracy. The sets are not
required to be of any particular size, nor are they required to be
the same size.
[0046] One further step to prepare for model training is to
normalize the input features, with each feature falling between 0
and 1. This step prevents any one set of features from unduly
influencing the model. The normalization step produces a scaling
weight for each feature that is applied to the features before said
feature is used in training or score calculation. For example the
range of values for number of backward citations is 0 to 141, so
the normalized backward citation for a patent with 40 citations
would be 0.28.
[0047] What follows is an example of a reduced set of inputs to
demonstrate the implementation in model training. Table 3 contains
a sample of issued patent records; Table 4 contains a sample
training set; Table 5 contains the sample training set with scaled
features. In each of Tables 3-5, the columns of each row are
continued on the second page of the table (for example, for Table
3, the first column of the second page shows the number of
inventors for the documents listed on the first page).
TABLE-US-00003 TABLE 3 bwd fwd doc number filed issued ctry_codes
assignee_name num_assg attorney_name num_atty cites cites
inventor_name 20040016127 Jul. 6, Oct. 31, AT BE BG CH KATHREIN- 1
Flach, Dieter 1 5 0 ZEHETNER, 2004 2007 CY CZ DE DK WERK Dipl.-Ing
HERMANN EE ES FI FR GB GR HU IE IT . . . 20050000163 Jan. 4, Sep.
9, 2009 AT BE BG CH XEROX 1 Gronecker, 1 5 0 FRAZIER, 2005 CY CZ DE
DK CORPORA Kinkeldey, ISAAC S. EE ES FI FR Stockmair & GB GR HU
IE Schwanh? IS .sctn.usse 19870810708 Dec. 1, Aug. 28, CH DE FR GB
LOOSER 1 Ritscher, 1 3 1 LOOSER, 1987 1991 LI GOTTLIE Thomas, Dr.
GOTTLIEB 20060300035 Jan. 17, Apr. 9, 2008 AT BE BG CH ALCATEL 1
Hervouet, 1 4 0 ROBISON, 2006 CY CZ DE DK LUCENT Sylvie ANDREW EE
ES FI FR GB GR HU IE IS . . . 20060002755 Feb. 7, May 14, AT BE BG
CH SONY 1 MUELLER & 1 5 0 KOIZUMI, 2006 2008 CY CZ DE DK
CORPORATI HOFFMANN YOSHIHIRO EE ES FI FR Patentanwolte GB GR HU IE
IS 20050734111 Mar. 22, Oct. 19, AT BE BG CH TRW 1 Sties, Jochen 1
HANSEMANN, 2005 2011 CY CZ DE DK AUTOMOTIV VOLKER EE ES FI FR GB GR
HU IE 20050077290 Dec. 17, Aug. 29, AT BE BG CH BIOSENSE 1 Mercer,
1 2 0 GOVARI, 2002 2012 CY CZ DE DK WEBST Christopher ASSAF EE ES
FI FR Paul GB GR IE IT LI . . . 20040819222 Nov. 23, Jan. 30, AT BE
BG CH Novartis AG 1 Leon, Susanna 1 BAESCHLIN, 2004 2013 CY CZ DE
DK Iris DANIEL EE ES FI FR KASPAR GB GR HU IE IS . . . num 1st 1st
claim 1st claim invt claim_type total_words diff_words title
abstract claim_one 1 A 112 64 LIGHTNING ARRESTER An antenna
installation lightning Lightning protection device for FOR ANTENNA
protection unit has a radio antenna systems, with a plurality
ARRANGEMENTS transparent protective housing of radiator elements
and radiator (5). . . arrangements (3) arranged offset . . . 4 A
181 79 IMPROVED A sheet feeder and separator A sheet feeder and
separator REPLACEMENT METHOD assembly for separating and assembly
(11) for separating and AND ASSEMBLY FOR sequentiallyfeeding
individual print sequentially feeding individual PAPER PICK ROLLERS
media sheets . . . print media sheets . . . 1 M 170 64 WINDING
METHOD AND A method of winding a A method of winding a APPARATUS
continuously moving web (10), continuously moving web (10) such as
a flexible polymer film . . . consisting of an essentially flexible
material . . . 1 M 122 45 METHOD FOR The invention provides a
system A method for controlling a CONTROLLING A and a method for
controlling a request for a resource from a PROCESS RESOURCE
request for a resource from a process (110) operating on a ACCESS
VIA A PARENT process . . . microprocessor-enabled machine PROCESS
(100) . . . 6 A 160 60 RECORDING APPARATUS, A recording apparatus
includes: A recording apparatus REPRODUCTION recording means having
a drive comprising: recording means (8) APPARATUS AND part
including rotational drive having a drive part (91.87) CONTROL
METHOD means for rotating an optical . . . including rotational
drive . . . 3 S 107 57 RUBBER BEARING, The invention relates to a
rubber A rubber bearing (10), in ESPECIALLY FOR A bearing (10),
especially for a motor particular for a motor pump unit MOTOR PUMP
UNIT OF A pump unit (12) of a power steering (12) of a power
steering POWER STEERING system . . . system . . . SYSTEM 1 M 199 92
IMPLANTABLE AND Apparatus for determining the Apparatus (20) for
determining INSERTABLE PASSIVE position of an object within a body
the position of an object (22) TAGS of a subject includes at least
one within a body of a subject, acoustic wave generator . . .
comprising: at least one acoustic wave generator (11, 13, 15) . . .
3 C 411 85 ORGANIC COMPOUNDS Disclosed are CE -amino- - A compound
having formula hydroxy- a-aryl-alkanoic acid (I)whereinR1 is
hydrogen, amide compounds of formula halogen, optionally
halogenated (I) and the salts thereof, alkyl, . . . having
renin-inhibiting properties . . .
TABLE-US-00004 TABLE 4 A B C D E F G document # pcnt_abandone
pcnt_abandone pcnt_abandone pcnt_abandone pcnt_abandone_assg.
pcnt_abandone impact_plus 20040016127 0.0195241 0.101213 0.0126767
0.07313 0.0161905 0.0926465 5 20050000163 0.0214966 0.121255
0.0150865 0.0899635 0.0163899 0.107652 5 19870810708 0 0 0.212828
0.212828 0.13399 0.13399 5 20060300035 0.0345951 0.118892 0.0100198
0.0738601 0.0345279 0.105121 6 20060002755 0.0188527 0.109995
0.0143231 0.0864105 0.0157964 0.0979499 6 20050734111 0.0265622
0.0524952 0.0153431 0.0328344 0.0302613 0.0476627 3 20050077290
0.0197951 0.0527218 0.0133056 0.0506772 0.0161905 0.0371031 4
20040819222 0.0352521 0.068873 0.00294144 0.0384033 0.0346951
0.0560363 4 19850309337 0.0238328 0.176999 0.0138066 0.0855378
0.0215054 0.178412 5 19810400286 0 0 0.212828 0.212828 0.13399
0.13399 3 H I J K L M N document # impact_minus impact_neutral
assg_avg_ip assg_avg_im assg_avg_in assg_stdev_ip assg_stdev_im
20040016127 23 7 4.65094 9.36792 6.59434 1.70736 7.39669
20050000163 0 2 5.09486 0.598155 1.8621 1.43756 1.28656 19870810708
5 3 3 10 11 0 0 20060300035 22 7 4.35834 6.16016 4.11809 1.49673
6.77812 20060002755 3 3 3.97511 0.785045 1.84866 1.63357 1.59374
20050734111 14 9 3.89091 1.89091 2.27273 1.27181 2.41655
20050077290 5 9 4.3399 4.59606 4.38424 1.50502 5.90556 20040819222
1 11 3.77778 5.33333 8.22222 1.71594 4.8734 19850309337 7 5 3.17966
4.89401 3.57559 1.26971 5.73916 19810400286 5 2 4.08772 2.4386
3.29825 1.55692 3.07193 O P Q R S T U document # assg_stdev_in
assg_median_ip assg_median_im assg_median_in atty_avg_ip
atty_avg_im atty_avg_in 20040016127 3.77152 5 11 8 5.13158 8.39474
6 20050000163 1.62292 5 0 2 3.9665 2.25741 2.93347 19870810708 0 3
10 11 0 0 0 20060300035 3.46319 4 1 3 5.09155 12.3169 5.76761
20060002755 2.12393 4 0 1 4.17358 1.09065 2.21382 20050734111
2.15557 4 1 2 3.66735 3.86448 4.77207 20050077290 4.01114 5 1 3
4.59333 5.99333 9.4 20040819222 5.35672 4 4 11 2.94545 3.4 6.21818
19850309337 4.12182 3 1 1 0 0 0 19810400286 2.45283 4 1 3 4 1 2 V W
X Y Z AA AB document # atty_stdev_ip atty_stdev_im atty_stdev_in
atty_median_ip atty_median_im atty_median_in abandyear 20040016127
1.50981 7.29139 3.77044 5 9 7 20 20050000163 1.64693 4.29542
3.12237 4 1 2 20 19870810708 0 0 0 0 0 0 9 20060300035 1.30414
7.16608 2.81502 5 15 7 20 20060002755 1.67789 2.31697 2.15989 4 0 2
6 20050734111 1.65807 5.67121 3.70177 4 1 4 20 20050077290 1.21012
4.6074 4.17085 5 7 9 20 20040819222 1.39335 5.46233 6.47128 3 1 3
20 19850309337 0 0 0 0 0 0 8 19810400286 0 0 0 4 1 2 13 AC AD AE AF
AG document # iclaim_stdev_by iclaim_median_ iclaim_avg_by_
iclaim_stdev_by iclaim_median_by_ipc 20040016127 4.87303 2 2.85074
3.15911 2 20050000163 4.01328 2 3.24691 3.58188 2 19870810708
2.47977 1 3.24691 3.58188 2 20060300035 4.52145 2 3.40922 3.65869 2
20060002755 4.03512 2 3.54489 3.6441 2 20050734111 4.52227 2
2.78329 3.07821 2 20050077290 4.56603 2 3.73529 4.29976 2
20040819222 4.04123 2 3.11477 3.58162 2 19850309337 2.34696 1
3.91159 4.03486 2 19810400286 2.68575 1 3.30404 3.5839 2
TABLE-US-00005 TABLE 5 A B C D E F G H document # pcnt_abandone
pcnt_abandone pcnt_abandone pcnt_abandone pcnt_abandone
pcnt_abandone impact_plus impact_minus 20040016127 0.0195241
0.101213 0.0126767 0.07313 0.0161905 0.0926465 0.666666667 1
20050000163 0.0214966 0.121255 0.0150865 0.0899635 0.0163899
0.107652 0.666666667 0 19870810708 0 0 0.212828 0.212828 0.13399
0.13399 0.666666667 0.217391304 20060300035 0.0345951 0.118892
0.0100198 0.0738601 0.0345279 0.105121 1 0.956521739 20060002755
0.0188527 0.109995 0.0143231 0.0864105 0.0157964 0.0979499 1
0.130434783 20050734111 0.0265622 0.0524952 0.0153431 0.0328344
0.0302613 0.0476627 0 0.608695652 20050077290 0.0197951 0.0527218
0.0133056 0.0506772 0.0161905 0.0371031 0.333333333 0.217391304
20040819222 0.0352521 0.068873 0.00294144 0.0384033 0.0346951
0.0560363 0.333333333 0.043478261 19850309337 0.0238328 0.176999
0.0138066 0.0855378 0.0215054 0.178412 0.666666667 0.304347826
19810400286 0 0 0.212828 0.212828 0.13399 0.13399 0 0.217391304 I J
K L M N O document # impact_neutral assg_avg_ip assg_avg_im
assg_avg_in assg_stdev_ip assg_stdev_im assg_stdev_in 20040016127
0.555555556 0.606096345 0.771550103 0.436428682 0.829166521
0.833333333 0.586727201 20050000163 0 0.725627869 0.010383305
0.032690867 0.698140184 0.15817562 0.252474151 19870810708
0.111111111 0.212013535 0.824128884 0.812303858 0 0 0 20060300035
0.555555556 0.675511116 0.817858725 0.314645335 0.46097416
0.802351215 0.373033589 20060002755 0.111111111 0.5372749
0.027202025 0.044080071 0.592127358 0.054846103 0.089827276
20050734111 0.777777778 0.505670097 0.077381064 0.056476005
0.245401395 0.093784103 0.091656022 20050077290 0.777777778
0.674200271 0.594540578 0.241508653 0.468919644 0.744852162
0.458792602 20040819222 1 0.522563281 0.760252149 0.772491776
0.671074222 0.545407915 0.760100318 19850309337 0.333333333
0.269287447 0.750428615 0.567148153 0.2978666 0.740863421
0.723731199 19810400286 0 0.5 0.5 0.5 0.5 0.5 0.5 P Q R S T U V
document # assg_median_ip assg_median_im assg_median_in atty_avg_ip
atty_avg_im atty_avg_in atty_stdev_ip 20040016127 0.722222222
0.833333333 0.580645161 0.833333333 0.567968942 0.531914894
0.749855473 20050000163 0.722222222 0 0.096774194 0.649196545
0.152731207 0.260059397 0.817956878 19870810708 0.166666667
0.833333333 0.822580645 0 0 0 0 20060300035 0.444444444 0.208333333
0.177419355 0.833333333 0.833333333 0.511312943 0.647708332
20060002755 0.444444444 0 0.016129032 0.757181246 0.151647748
0.196260638 0.833333333 20050734111 0.444444444 0.05 0.096774194
0.665339743 0.537330666 0.423055851 0.833333333 20050077290
0.722222222 0.05 0.177419355 0.833333333 0.833333333 0.833333333
0.723747324 20040819222 0.666666667 0.8 0.822580645 0.613635417
0.833333333 0.833333333 0.833333333 19850309337 0.25 0.5
0.071428571 0 0 0 0 19810400286 0.5 0.5 0.5 0.5 0.5 0.5 0 W X Y Z
AA AB AC document # atty_stdev_im atty_stdev_in atty_median_ip
atty_median_im atty_median_in abandyear iclaim_stdev_by 20040016127
0.833333333 0.485535062 0.833333333 0.5 0.648148148 0.791666667
0.733254076 20050000163 0.499508332 0.402080423 0.666666667
0.055555556 0.185185185 0.791666667 0.587172044 19870810708 0 0 0 0
0 0.21875 0.218377404 20060300035 0.833333333 0.3625017 0.833333333
0.833333333 0.648148148 0.791666667 0.683449185 20060002755
0.340457915 0.278137916 0.666666667 0 0.185185185 0.0625
0.631837212 20050734111 0.833333333 0.476692143 0.666666667
0.119047619 0.37037037 0.772727273 0.627210769 20050077290
0.702905171 0.537097504 0.833333333 0.833333333 0.833333333
0.772727273 0.721785122 20040819222 0.833333333 0.833333333 0.625
0.833333333 0.833333333 0.772727273 0.705118442 19850309337 0 0 0 0
0 0.173913043 0.514594639 19810400286 0 0 0.5 0.5 0.5 0.5 0.5 AD AE
AF AG document # iclaim_median_ iclaim_avg_by_ iclaim_stdev_by
iclaim_median_by_ipc 20040016127 0.75451151 0.25295427 0.258251691
0.258251691 20050000163 0.592978587 0.413524065 0.414998977
0.414998977 19870810708 0.167200864 0.413524065 0.414998977
0.414998977 20060300035 0.734071547 0.479309165 0.443477249
0.443477249 20060002755 0.599042444 0.534296933 0.438067823
0.438067823 20050734111 0.734299219 0.225616429 0.228257001
0.228257001 20050077290 0.746449145 0.564678497 0.625197872
0.625197872 20040819222 0.728038015 0.282891905 0.362413839
0.362413839 19850309337 0.348904211 0.618507263 0.591346991
0.591346991 19810400286 0.5 0.5 0.5 0.5
[0048] Examples of additional factors that may be used and
heuristically searched are provided in the following lists.
TABLE-US-00006 List Part 1 CCRE - BE: expiry of a complementary
protection certificate EN - FR: translation not filed EN3 - FR:
translation not filed ** decision concerning opposition EUG - SE:
european patent has lapsed FDY - File destroyed FITB - IT: spc for
herbicidal products: suspended FITG - IT: spc for herbicidal
products: definitive refusal FITM - IT: spc for herbicidal
products: withdrawal of spc application FITN - IT: spc for
herbicidal products: annulment of spc FITO - IT: spc for herbicidal
products: expiry FITP - IT: spc for herbicidal products:
renunciation of spc GBAW - GB: application withdrawn GBDW - GB: gb
designation withdrawn GBGD - GB: date of publication of the new
specification of the patent under article 103 (1977) ** grant date
withdrawn GBGR - GB: grant date revoked GBGW - GB: grant date
withdrawn GBPC - GB: european patent ceased through non-payment of
renewal fee GBPR - GB: patent revoked under art. 102 of the ep
convention designating the uk as contracting state GBV - GB: ep
patent (uk) treated as always having been void in accordance with
gb section 77(7)/1977 LTIE - LT: invalidation of european patent or
patent extension LTLA - LT: lapse of european patent or patent
extension R29U - Interruption of proceedings (correction) [after
grant] RVAA - Decision on revocation request is admissible (for
revocation filed after opposition period) RVDA - Decision on
revocation request is admissible (for revocation filed during
opposition period) RX1 - Cancellation of first publication RX2 -
Cancellation of second publication X - Document not published X1 -
No entry under this number 17A + Application maintained 17P +
Request for examination filed 17Q + First examination report 18RA +
Date of receipt of request for re-establishment of rights 18RR +
Re-established 19F + Date of resumption (after stay of proceedings)
[before grant] 19W + Date of resumption (after interruption of
proceedings)[before grant] 25N + Valid in all designated states 26D
+ Opposition deemed not to have been filed 26N + No opposition
filed 26U + Inadmissible opposition 27C + Termination of opposition
procedure 27O + Opposition rejected
TABLE-US-00007 List Part 2 28 + Re-established 29F + The resumption
of a previous incorrect announcement of a suspension of proceedings
(correction) [after grant] 29W + Date of resumption (after
interruption of proceedings) [after grant] 31R + Resumption 31W +
Resumption A4 + Supplementary search report A5 + Separate
publication of the ep or int. search report AK + Designated
contracting states: AKX + Payment of designation fees AX +
Extension or validation of the european patent to AXX + Payment of
extension fees BERR + BE: reestablished CCHV + BE: grant of a
complementary protection certificate for herbicides CCPV + BE:
grant of a complementary protection certificate D19F + Previously
announced "resumption after interruption of proceedings" was
erroneous D25 + Lapsed in a contracting state (deleted) DBV +
Designated contracting states (deleted) EAL + SE: european patent
in force in sweden EL + FR: translation of claims filed EL1 + FR:
translation or corrected translation of claims filed EM + FR:
revised translation of claims filed GBTC + GB: corrected
translation (of ep patent) filed (gb section 80(3)/1977) IECL + IE:
translation for ep claims filed INTG + Announcement of intention to
grant ITCL + IT: translation for ep claims filed ITF + IT:
translation for a ep patent filed MEDD + IT: spc for pharmaceutical
products: granted NLE + NL: notifications concerning applications
NLR3 + NL: receipt of modified translations in the netherlands
language after an opposition procedure NLR4 + NL: receipt of
corrected translation in the netherlands language at the initiative
of the proprietor of the patent PGFP + Postgrant: annual fees paid
to national office PGRI + Postgrant: patent reinstated in
contracting state R17C + Date of despatch of first examination
report R17P + Request for examination filed (correction) R18X +
Re-established (correction) R19F + The resumption of a previous
incorrect announcement of a stay of proceedings (correction)
[before grant] R19W + Resumption after interruption of proceedings
(correction) [before grant] R26D + Opposition deemed not to have
been filed (corr.) R26U + Inadmissible opposition (correction) R27A
+ Maintained as amended (correction)
TABLE-US-00008 List Part 3 EN4 + FR: notification of non filing
translation in an earlier bopi is erroneous EPTA + LU: last paid
annual fee ET + FR: translation filed ET1 + FR: translation filed
** revision of the translation of the patent or the claims ET2 +
FR: translation filed ** revision of the translation of the
modified patent after opposition ET3 + FR: translation filed **
decision concerning opposition ETR + FR: translation filed **
restoration of the right FITD + IT: spc for herbicidal products:
granted GBA + GB: translation amended (gb section 77(6)(a)/1977)
GBAT + GB: amendment of translation allowed (of ep patent) (gb
sect. 80 (3)/1977) GBC + GB: translation of claims filed (gb
section 78(7)/1977) GBC8 + GB: translation of claims filed (gb
section 80(3)/1977) GBCC + GB: corrected translation (of claims)
filed (gb section 80(3)/1977) GBDL + GB: delete "european patent
ceased" from journal GBRH + GB: ep (uk) patent reinstated (gb rule
100) GBRI + GB: ep (uk) patent reinstated (gb rule 110(3)a/1987)
GBT + GB: translation of ep patent filed (gb section 77(6)(a)/1977)
GBT8 + GB: translation filed (gb section 80(3)/1977) GBTA + GB:
translation of amended ep patent filed (gb section 77(6) (b)/1977)
R27O + Opposition rejected (correction) R28 + Re-established
(correction) R28E + Date of receipt of request for re-establishment
of rights (art 122) (corr.) R29W + Resumption after interruption of
proceedings (correction)[after grant] RA1 + Date and kind of first
publication (correction) RA4 + Date and kind of supplementary
search report (correction) RB1 + Date and kind of second
publication (correction) RB2 + Date and kind of third publication
(correction) RBV + Designated contracting states (correction): RJL1
+ Rejection of limitation - substantive refusal RJL2 + Rejection of
limitation - inadmissible for formal reasons RJL3 + Rejection of
limitation - no or late reply to subset report RJL4 + Rejection of
limitation - request allowed but requirements not fulfilled SC4A +
PT: translation is available T1 + DK: translation of the claims of
ep patent T3 + DK: translation of ep patent T4 + DK: translation of
amended ep patent T5 + DK: corrected translation of ep patent TCAT
+ AT: translation of patent claims filed TCNL + NL: translation of
patent claims filed TDAT + AT: translation of application
published
TABLE-US-00009 List Part 4 110E Request for conversion into a
national patent application 111L Licenses 111R Other rights "in
rem" 111Z Registering of licences or other rights 16A New documents
discovered after completion of the EP-search report 27A Maintained
as amended 33 Transfer of rights 34E Establishment of other rights
"in rem" 34G Grant of licenses 34L Legal means of execution 34TL
Transfer of licenses 34TR Transfer of other rights "in rem" 35
Correction 710B GB: proceeding under rule 110(4) patents act 1977
AC Divisional application (art. 76) of: AF Successive application
(art. 61) AKNL NL: corrections (part 1 heading g) BECA BE: change
of holder's address BECH BE: change of holder BECN BE: change of
holder's name CCPA BE: application for a complementary protection
certificate CND3 Copied from national register on demand of third
party DAX Extension of the european patent to (deleted) DB1 Date
and kind of second publication (deleted) DB2 Date of publication of
new second specification ** last entry deleted DET DE: translation
of patent claims DIN1 Inventor (deleted) DIN2 Inventor (deleted) DX
Miscellaneous: (deleted) FIT IT: spc for herbicidal products: no
action taken FITA IT: spc for herbicidal products: examined FITC
IT: spc for herbicidal products: partially granted FITE IT: spc for
herbicidal products: interlocutary refusal FITF IT: spc for
herbicidal products: office refusal FITH IT: spc for herbicidal
products: board of appeal FITI IT: spc for herbicidal products:
court of cassation FITL IT: spc for herbicidal products: court of
justice GBRJ GB: reinstated under rule 110(4) - alteration of time
limits GBTH GB: translations filed: amended european patents -
correction INTC Former communication of intention to grant
cancelled ITCP IT: complementary protection certificate ITPR IT:
changes in ownership of a european patent ITTA IT: last paid annual
fee K1C0 Previously announced correction of patent application
cancelled
TABLE-US-00010 List Part 5 CRD3 Copy of epo register on demand of
third party D11X Legal means of execution (deleted) D17D Search
report (deleted) D17P Request for examination filed (deleted) D17Q
First examination report (deleted) D18D EP-application deemed to be
withdrawn: (deleted) D18R Refused (deleted) D18W Withdrawal
(deleted) D18Z Request for re-establishment (deleted) D20
Corrections of a patent specification (deleted) D26 Opposition
filed (deleted) D26N No opposition filed (deleted) D27A Maintained
as amended (deleted) D27C Opposition finished (deleted) D27O
Opposition rejected (deleted) D27W Revoked (deleted) D8RA Date of
receipt of request for re-establishment of rights (art 122)
(deleted) DA1 Date and kind of first publication (deleted) DA4 Date
and kind of supplementary search report (deleted) DAC Divisional
application (art. 76) of: (deleted) DAF Successive application
(art. 61) (deleted) DAHF Divisional application (art 76) in:
(deleted)
TABLE-US-00011 List Part 6 DAF Successive application (art. 61)
(deleted) DAHF Divisional application (art 76) in: (deleted) DAX
Extension of the european patent to (deleted) DB1 Date and kind of
second publication (deleted) DB2 Date of publication of new second
specification ** last entry deleted DET DE: translation of patent
claims DIN1 Inventor (deleted) DIN2 Inventor (deleted) DX
Miscellaneous: (deleted) FIT IT: spc for herbicidal products: no
action taken FITA IT: spc for herbicidal products: examined FITC
IT: spc for herbicidal products: partially granted FITE IT: spc for
herbicidal products: interlocutary refusal FITF IT: spc for
herbicidal products: office refusal FITH IT: spc for herbicidal
products: board of appeal FITI IT: spc for herbicidal products:
court of cassation FITL IT: spc for herbicidal products: court of
justice GBRJ GB: reinstated under rule 110(4) - alteration of time
limits GBTH GB: translations filed: amended european patents -
correction INTC Former communication of intention to grant
cancelled ITCP IT: complementary protection certificate ITPR IT:
changes in ownership of a european patent ITTA IT: last paid annual
fee K1C0 Previously announced correction of patent application
cancelled K1C1 Correction of patent application (title page)
published K1C2 Correction of patent application (partial reprint)
published K1C3 Correction of patent application (complete reprint)
published K2 Correction of patent specification published K2C0
Announced rectification cancelled K2C1 Correction of patent
specification (title page) published K2C2 Correction of patent
specification (partial reprint) published K2C3 Correction of patent
specification (complete reprint) published KL Correction list LIM1
Limitation is admissible LIM2 Limitation is inadmissible LIM3
Limitation deemed not to be filed LIM4 Limitation deemed not to be
filed, opposition pending or filed LIM5 Limitation withdrawn MED
IT: spc for pharmaceutical products: no action taken MEDA IT: spc
for pharmaceutical products: examined MEDC IT: spc for
pharmaceutical products: partially granted MEDE IT: spc for
pharmaceutical products: interlocutary refusal MEDF IT: spc for
pharmaceutical products: office refusal MEDH IT: spc for
pharmaceutical products: board of appeal MEDI IT: spc for
pharmaceutical products: court of cassation MEDL IT: spc for
pharmaceutical products: court of justice NLR2 NL: decision of
opposition NLR5 NL: patents in respect of which a request to
provide a certificate of prior use has been filed NLR6 NL: patents
in respect of which a decision has been taken on a request
concerning prior use NLS NL: assignments of EP-patents NLT1 NL:
modifications of names registered in virtue of documents presented
to the patent office pursuant to art. 16 a, paragraph 1 NLT2 NL:
modifications (of names), taken from the european patent bulletin
NLUE NL: license registered with regard to european patents NLXE
NL: other communications concerning EP-patents (part 3 heading xe)
PRVG Petition for review by the enlarged board of appeal granted
PRVN Petition for review by the enlarged board of appeal not
granted R110 Filing of a request for conversion (correction) R11L
Granting of a license (correction) R11X Legal means of execution
(correction) R16A New documents discovered after completion of the
EP-search report (correction) R17D Search report (correction) R18Z
Request for re-establishment (correction) R19A Stay of proceedings
(correction)[before grant] R20 Corrections of a patent
specification R26N No opposition filed (correction) R27C Opposition
finished R80 Public notification if the address of the addressee
cannot be established RAC Divisional application (art. 76) of:
(correction) RAF Successive application (art. 61) (correction) RAG
Has successive application (art. 61) (correction) RAP1 Transfer of
rights of an ep application RAP2 Transfer of rights of an ep
publication RAP3 Correction of the address or name of applicant (a
document) RAP4 Correction of name or address of patent owner (b
document) RAX Extension of the european patent to (correction) REF
Corresponds to: REG Reference to a national code RHK1 Main
classification (correction) RHK2 Main classification (correction)
RIC1 Classification (correction) RIC2 Classification (correction)
RIN1 Inventor (correction) RIN2 Inventor (correction) RTI1 Title
(correction) RTI2 Title (correction) T2 DK: corrected translation
of the claims of ep patent XX Miscellaneous: ZE NL: corrections to
earlier entries in headings pe - xe
[0049] After the data is selected and the feature scaling factors
are computed, the training of the model starts by randomly or
pseudo-randomly choosing features as the input to classification
model trainer. Using a genetic algorithm search heuristic, a
population of sets of features is created and in each set the
features included are randomly or pseudo-randomly selected.
[0050] Each population is then used to train a binary classifier in
the next step. The output of the classification-training step is a
value that indicates how well that collection of features performs
on the training set of patent data.
[0051] The Artificial Neural Network (ANN) model will now be
described to illustrate an aspect of the system's search heuristic
to find an optimal classifier.
[0052] The size of the input layer to the ANN is defined as the
number of selected features provided by the feature selector. In an
Artificial Neural Network, the calculations "flow" from the input
nodes on the left, through the nodes of the hidden layers and
finally to the output node, as illustrated in FIG. 5. The size of
the output layer is set at 1. During this step, the system varies
iteratively the number of hidden layers and the number of nodes at
each layer. Each node represents a mathematical combination of its
inputs, and so the weights attached to the lines that represent the
connections between nodes adjust how much affect one node has on
another node. Given this, the hidden layers serve to increase the
complexity that the classifier is able to model. In an ANN with no
hidden layers, the maximum complexity is a linear system. Each
additional layer means that arbitrarily complex domains can be
represented and potentially give more accurate classifications.
[0053] The system initially considers an ANN with a single hidden
layer with size equal to the half the size of the input features.
The ANN is then trained using a feedforward cost function and
backpropagation algorithms to compute the gradient of errors.
Feedforward refers to the process of values propagating from the
input along the edges to the hidden nodes and then the computed
values from the hidden nodes propagating to the output node.
Backpropagation refers to the process of computing the difference
between the final output of the classifier against the test set and
then computing the error that each of the hidden nodes contributed
to that output. Backpropagation then computes the amount of error
that each of the input nodes contributed to the final calculation.
The result of backpropagation is the gradient of errors, which is a
measure of the amount of error at each node along each path through
the network. During training, the gradient of errors is used to
alter the weights in the neural network to reach the optimal
classifier. The errors are computed by evaluating the current ANN
on the cross-validation set. This avoids the problem of the
classifier being too specific (i.e., overfit) to the training
set.
[0054] Once an optimal solution, as defined by a minimization of
the difference in the output of the ANN and the testing set, is
found, the Area Under the Curve (AUC) of the ROC curve is
calculated by iterating over possible thresholds from 0 to 1 that
the ANN uses to determine the output of the classifier. For
example, if the threshold is 0.4, then any input to the output
layer that is greater than 0.4 will be considered a prediction that
the patent would be maintained. For each threshold, the system
computes the true and false positive and the true and false
negative rates. These data points are used to plot the ROC curve
and compute the AUC.
[0055] After training the first classifier using the previously
described method, the system alters the number of nodes in the
hidden layer. The system then repeats the training and AUC
computation for the new ANN. If the prediction is better, meaning
that a larger area under the ROC curve is yielded, the new
parameters are saved and the number of nodes in the input layer is
again altered. This proceeds until the maximal AUC is found.
[0056] The system then increases the number of hidden layers to two
and sets the size of the nodes in each layer to be half the inputs
of that hidden layer's inputs. In the current example, this means
the first hidden layer has 10 nodes and the second hidden layer has
5.
[0057] The system trains the ANN using the previously described
method. The number of nodes in the hidden layers is then altered
and the new ANN is trained. This proceeds until the maximal AUC is
found. The classifier-training step returns the parameters of the
binary classifier that had the maximal AUC.
[0058] The last step in the process is that the feature selection
search heuristic changes the set of features using a genetic
algorithm. The genetic algorithm selects the best sets of features
to use in the next iteration of the search by choosing those
features that performed best as measured by the maximum AUC that
set. The best sets of features are combined and mutated (slight,
random or pseudo-random changes) to create a new population of
candidate solutions.
[0059] The classification-training step is then executed again, and
the feature selection search heuristic collects all of the AUC
outputs, then selects the best feature sets and creates a new
population. This process continues until the selection process no
longer finds better solutions. The best solution from all
iterations of the feature selection search is the model that will
be used in calculation of the patent scores.
[0060] The output of a binary classifier during training may be
executed through a step function so that the actual prediction is a
binary decision. The raw patent score output from this system is
this value not executed through the step function; this raw
comparative score forms the basis for the computation of additional
scores.
[0061] By way of an example of an implementation, FIG. 6
illustrates patent value determination application 40 residing or
running on Value Evaluation System 20, which may be a server
connected to the internet for providing information about the value
of a patent to Patent Evaluation Requestor 31, a terminal connected
to the Internet. Such a server may include network interface 21 for
communicating with a network, operating system 22 for running the
device, and a processor 23 and memory 24.
[0062] Patent value determination application 40 may obtain
information from a database 33 or more than one such database. One
or more software applications providing the functionality herein
described may be provided by a server or server bank in the cloud
or on a proprietor's premises, or may be downloaded to a computer
or portable device of the user to make possible the delivery of
patent or patent application value to a requesting user. Patent
value determination application 40 illustrated in FIG. 6 may
include a number of components or software modules under control of
application controller 41. For example, judged patent information
receiver 43 may receive identifying and other detailed information
about a patent of interest or a target patent document to be
evaluated. Patent information extractor 44 can obtain relevant
information from database 33 to be used for generating the sets by
set generator 48 and to produce the factor inputs for the algorithm
by modules 46 and 47. Iteration controller 50 of patent value
determination application 40 can control the iterations of the
Genetic algorithm and/or the simulated annealing algorithm
performed by modules 51 and 52, respectively, and ROC generator and
AUC calculator 52 can obtain the best-fitting results using NBC,
ANN and/or a support vector machine implemented by modules 61, 62
and 63, respectively. Patent evaluator 69 provides a result to the
requestor based on the optimal factors obtained.
[0063] The present methods, functions, systems, computer-readable
medium product, or the like may be implemented using hardware,
software, firmware or a combination of the foregoing, and may be
implemented by one or more automated processors or computer chips
or cores, in one or more computer or other processing system, such
that no human operation may be necessary.
[0064] FIGS. 8A-B contain a flowchart showing steps of an example
of a machine learning, according to an aspect of the present
disclosure.
[0065] After system start, the system at S2 retrieves a set of
training patent data, for example, over a network, such as the
Internet. As illustrated in FIG. 6, a patent information database
33, such as the European Patent Office, WIPO, U.S. Patent Office
Database, a private database with patent information or a
combination of the foregoing may be accessed online. A proprietary
database located on site or off site may be used in addition to or
instead of the foregoing. At S3, a list of features of potential
interest is made and a weighted scale or standardized score is
assigned to each feature. At S4, a heuristic search method, such as
ANN, is used to generate a first set of binary classifiers.
Iteratively, the ANN model is modified, at S5, by changing a number
of hidden layers. This second set of binary classifiers is then
compared with the first set with reference to a cost function, such
as an area under a curve (AUC) of a ROC at S6. At S7, the ANN model
may be further iterated through by changing the number of hidden
layers and, at S8, the result is compared with the highest yielding
binary classifier set thus far. At S9, a genetic algorithm may be
used to improve upon the candidate set of binary classifiers. At
S10, iterations of the genetic algorithm are continued to maximize
the area under the curve of the ROC.
[0066] FIG. 8B contains S11. At S11 the iteration of the genetic
algorithm is continued until no improved set of candidates binary
classifiers is produced. At S12, the final set of binary
classifiers is reported or outputted. This set of binary
classifiers to be used or validated and tested may be reported
(S13).
[0067] At S14, a validation patent data set may be received.
However, it will be understood that the training patent data, the
validation patent data and the testing patent data may all be
received at the same time and randomly or pseudo randomly assigned
to one of the three groups. At S15, the validation patent data is
used to validate the final set of binary classifiers. At S16,
testing patent data are received, and at S17 the testing patent
data set is used to validate the final set of binary
classifiers.
[0068] At S18, a patent of interest is received by the system, and
at S19, an estimate of patent life or other patent quality estimate
is generated using the binary classifiers arrived at through the
machine learning algorithm. At S20, a report of the patent quality
estimate, such as the patent life for the patent of interest is
reported.
[0069] Thus described is a machine learning solution that may be
more efficient, more speedy and may improve the functioning of a
computer including an automated data processor or a set of
automated data processors carrying out the machine learning when
compared with, for example, a device implementing brute-force
solutions. Utilizing a machine-learning approach as described
herein according to the present disclosure, the solution space to
be searched may be reduced in every iteration because combinations
of factors and coefficients that do not lead to an optimal solution
can be excluded. Theoretically, while reducing the number of
solutions evaluated may exclude the optimal solution, the
randomness introduced by the mutation step may be sufficient to
reduce this risk. Further, a machine-learning approach as described
herein according to the present disclosure may significantly reduce
the system resources needed to calculate the solution by more
intelligently and efficiently selecting factors and coefficients
for evaluation.
[0070] For the reasons discussed above, such machine learning
approaches may be more expensive to implement, more time and
resource intensive, and may consume significantly more computer
processing resources. A machine learning solution as described
according to the present disclosure may consume less energy and
generate less heat when carried out on an automated data processor
or set of automated data processors.
[0071] A computer system for implementing the foregoing methods,
functions, systems and computer-readable storage medium may include
a memory, preferably a random access memory, and may include a
secondary memory. Examples of a memory or a computer-readable
storage medium product include a removable memory chip, such as an
erasable programmable read-only memory (EPROM), a programmable
read-only memory (PROM), removable storage unit or the like. The
methods and functions can be performed entirely automatically
through machine operations, but need not be entirely performed by
machines. Similarly, the systems and computer-readable media may be
implemented entirely automatically through machine operations but
need not be so. A computer system may include one or more
processors in one or more units for performing the system according
to the present disclosure and these computers or processors may be
located in a cloud or may be provided in a local enterprise setting
or off premises at a third party contractor, and may communicate
with a user requesting an evaluation or estimation of patent or
patent application quality on site via a wired or wireless
connection, such a through a LAN or WAN, or off site via internet
protocol-enabled communication, via a cellular telephone provider
or via other such means. Similarly, the information stored and/or
the patent database from which the sets of data are extracted, may
be stored in a cloud, in an official or third party patent
information database, or may be stored locally or remotely. The
computer system or systems that enable the user to interact with
content or features can include a GUI (Graphical User Interface),
or may include graphics, text and other types of information, and
may interface with the user via desktop, laptop computer or via
other types of processors, including handheld devices, telephones,
mobile telephones, smartphones or other types of electronic
communication devices and systems.
[0072] The communication interface of the Value Evaluation System
shown in FIG. 6 may include a wired or wireless interface
communicating over TCP/IP paradigm using HTTP or other types of
protocols, and may communicate via a wire, cable, fire optics, a
telephone line, a cellular link, a satellite link, a radio
frequency link, such as WI-FI or Bluetooth, a LAN, a WAN, VPN, the
world wide web or other such communication channels and networks,
or via a combination of the foregoing.
[0073] While the preferred embodiments of the invention have been
illustrated and described, modifications and adaptations, and other
combinations or arrangements of the structures and steps described
come within the spirit and scope of the application and the claim
scope.
* * * * *