U.S. patent number RE44,807 [Application Number 09/865,469] was granted by the patent office on 2014-03-18 for method and system for tracking multiple regional objects by multi-dimensional relaxation.
This patent grant is currently assigned to Colorado State University Research Foundation. The grantee listed for this patent is Aubrey B. Poore, Jr.. Invention is credited to Aubrey B. Poore, Jr..
United States Patent |
RE44,807 |
Poore, Jr. |
March 18, 2014 |
Method and system for tracking multiple regional objects by
multi-dimensional relaxation
Abstract
A method and system for real-time tracking of objects .[.is.].
.Iadd.are .Iaddend.disclosed. A region is repeatedly scanned
providing a plurality of images or data sets having points
corresponding to .[.ojbects.]. .Iadd.objects .Iaddend.in the region
to be tracked. Given a previously determined track for each object
in the region, an M-dimensional combinatorial optimization
assignment problem is formulated using the points from M-1 of the
images or data sets, wherein each point is preferably used in
extending at most one track. The M-dimensional problem is
subsequently solved for an optimal or near-optimal assignment of
the points to the tracks, extending the tracking of the objects so
that a response to each object can be initiated by the system in
real-time. Speed and accuracy is provided by an iterative
Lagrangian Relaxation technique wherein a plurality of constraint
dimensions are relaxed simultaneously to yield a reduced
dimensional optimization problem whose solution is used to
formulate an assignment problem of dimensionality less than M. The
iterative reducing of dimensions terminates when exact solutions
are determined for two-dimensional cases. A recovery procedure is
used for determining a higher dimensional assignment problem
solution from a problem having one less dimension. The procedure is
useful when the reduced dimensional optimizational problem has two
constraint dimensions.
Inventors: |
Poore, Jr.; Aubrey B. (Fort
Collins, CO) |
Applicant: |
Name |
City |
State |
Country |
Type |
Poore, Jr.; Aubrey B. |
Fort Collins |
CO |
US |
|
|
Assignee: |
Colorado State University Research
Foundation (Fort Collins, CO)
|
Family
ID: |
50240553 |
Appl.
No.: |
09/865,469 |
Filed: |
May 25, 2001 |
Related U.S. Patent Documents
|
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
Issue Date |
|
|
08404024 |
Mar 14, 1995 |
5537119 |
|
|
|
08171327 |
Dec 21, 1993 |
5406289 |
|
|
Reissue of: |
08682904 |
Jul 16, 1996 |
5959574 |
Sep 28, 1999 |
|
|
Current U.S.
Class: |
342/96 |
Current CPC
Class: |
G01S
13/726 (20130101) |
Current International
Class: |
G01S
13/00 (20060101) |
Field of
Search: |
;342/96 |
References Cited
[Referenced By]
U.S. Patent Documents
Other References
A V. Aho, J.E. Hopcroft, and J.D. Ullman. Design and Analysis of
Computer Algorithms. Addison-Wesley, MA, 1974. cited by applicant
.
Y. Bar-Shalom. Multitarget-Multisensor Tracking: Advanced
Applications. Artech House, Dedham, MA, 1990. cited by applicant
.
Y. Bar-Shalom and T.E. Fortmann. Tracking and Data Association.
Academic Press, Boston, MA, 1988. cited by applicant .
D. P. Bertsekas and D.A. Castanon. A forward/reverse auction
algorithm for asymmetric assignment problems. Computational
Optimization and Applications, 1: 277-298, 1992). cited by
applicant .
C.R. Reeves ed. Modern Heuristic Techniques for Combinatorial
Problems. Halstead Press, Wiley, New York, NY, 1993. cited by
applicant .
A.M. Frieze and J. Yadegar. An algorithm for solving 3-dimensional
assignment problems with application to scheduling a teaching
practice. Journal of the Operational Research Society, 32:989-995,
1981. cited by applicant .
M.R. Garey and D.S. Johnson. Computers and Inrtactability. W.H.
Freeman and Company, San Francisco, CA, 1979. cited by applicant
.
J. L. Goffin. On convergence rates of subgradient optimization
methods, mathematical programming. Mathematical Programming,
13:329-347, 1977. cited by applicant .
J.-B. Hiriart-Urruty and C. Lemarechal. Coonvex Andlysis and
Minimization Algorithms I&II. Springer-Verlag, Berlin, 1993.
cited by applicant .
K.C. Kiwiel. Methods of descent for nondifferentiable optimization.
In A. Dold and B. Eckmann, editors, Lecture Notes in Mathematics,
vol. 1133, Berlin, 1985. Springer-Verlag. cited by applicant .
K.C. Kiwiel. An aggregate subgradient method for non-smooth convex
minimization. Mathematical Programming, 27:320-341, 1983. cited by
applicant .
C. Lemarechal and R. Mifflin. Nonsmooth Optimization. Pergamon
Press, Oxford, UK, 1978. cited by applicant .
G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial
Optimization. Wiley, New York, NY, 1988. cited by applicant .
O.E. Drummond. Multiple target tracking with multiple frame,
probabilistic data association. In Signal and Data Processing of
Small Targets, SPIE Proceddings, vol. 1954, pp. 394-408, 1993.
cited by applicant .
O.E. Drummond. Feedback in track fusion without process noise. In
Signal and Data processing of Small Targets, SPIE Proceddings, vol.
2561, pp. 369-383, 1995. cited by applicant .
O.E. Drummond. Multiple sensor tracking with multiple frame,
probabilistic data association. In Signal and Data Processing of
Small Targets, SPIE Proceddings, vol. 2561, pp. 322-336, 1995.
cited by applicant .
C.H. Papadimitriou and K.Steiglitz. Combinatorial Optimization:
Algorithms and Complexity. Prentice-Hall, Inc., Englewood Cliffs,
NJ, 1982. cited by applicant .
J. Pearl. Heuristics: Intelligent Search Strategies for Computer
Problem Solving. Addison-Wesley, Reading, MA, 1984. cited by
applicant .
W. Pierskalla. The tri-substitution method for the
three-dimensional assignment problem. Journal du CORS, 5:71-81,
1967. cited by applicant .
B.T. Poljak. Subgradient method: A survey of Soviet research. In C.
Lemarechal and R. Mifflin, editors, Nonsmooth Optimization, pp.
5-29, N.Y., 1978. Pergamon Press. cited by applicant .
A.B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994. cited by applicant .
A.B. Poore. Multidimensional assignments and multitarget tracking:
Partitioning data sets. In P. Hansen, I.J. Cox, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, vol. 19, pp. 169-198, Providence, R.I., 1995.
American Mathematical Society. cited by applicant .
A.B. Poore and O.E. Drummo Track initiation and maintenance using
multidimensional assignment problems. In D.W. Hearn, W.W. Hager,
and P.M. Pardalos, editors, Network Optimization, vol. 450, pp.
407-422, Boston, 1996. Kluwer Academic Publishers B.V. cited by
applicant .
A.B. Poore and N. Rijavec. A lagrangian relaxation algorithm for
multidimensional assignment problems arising from multitarget
tracking. SIAM Journal of Optimization, 3, No. 3:544-563, 1993.
cited by applicant .
A.B. Poore and N. Rijavec. A numerical study of some data
association problems arising in multitarget tracking. Large Scale
Optimization: State of the Art, pp. 339-361, 1994. cited by
applicant .
A.B. Poore and N. Rijavec. Partitioning multiple data sets:
multidimentional assignments and lagrangian relaxation. In P.M.
Pardalos and H. Wolkowicz, editors, Quadratic Assignment and
Related Problems: DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, vol. 16, pp. 25-37, 1994. cited by
applicant .
A.J. Robertson III. A class of lagrangian relaxation algorithms for
the multidimensional assignment problem. Ph.D. Thesis, Colorado
State University, Ft. Collins, CO, 1995. cited by applicant .
K.R. Pattipati, S. Deb, and Y. Bar-Shalom. A s-dimensional
assignment algorithm for track initiation. In Proceedings of the
IEEE Systems Conference, Kobe, Japan, pp. 127-130, 1992. cited by
applicant .
K.R. Pattipati, S. Deb, and Y. Bar-Shalom. A
multisensor--multitarget data association algorithm for
heterogeneous sensors. In Proceedings of the IEEE Transaction on
Aerospace and Electronic Systems, vol. 29, No. 2, pp. 560-568, Apr.
1993. cited by applicant .
Y. Bar-Shalom, S. Deb, K.R. Pattipati, and H. Tsanakis. A new
algorithm for the generalized multidimensional assignment problem.
In Proceedings of the IEEE International Conference in Systems,
Math, and Cybernetics, Chicago, pp. 132-136, 1992. cited by
applicant .
H. Schramm and J. Zowe. A version of the bundle idea for minimizing
a nonsmooth function: Conceptual idea, convergence analysis,
numerical results. SIAM Journal on Optimization, 2, No. 1:121-152,
Feb. 1992. cited by applicant .
N. Z. Shor. Minimization Methods for Non-Differentiable Functions.
Springer-Verlag, New York, 1985. cited by applicant .
S.S. Blackman. Multiple Target Tracking with Radar Applications.
Artech House, Dedham, MA, 1986. cited by applicant .
T. Kurien. Issues in the designing of practical multitarget
tracking algorithms. In Multitarget-Multisensor Tracking: Advances
Applications by Y. Bar-Shalom. Artech House, MA, 1990. cited by
applicant .
P. Wolfe. A method of conjugate subgradients for minimizing
nondifferentiable functions. Mathematical Programming Study,
3:145-173, 1975. cited by applicant .
P. Wolfe. Finding the nearest point in a polytope. Mathematical
Programming Study, 11:128-149, 1976. cited by applicant.
|
Primary Examiner: Hellner; Mark
Attorney, Agent or Firm: Freund; Samuel M. Cochran; William
W. Cochran Freund & Young LLC
Government Interests
.Iadd.STATEMENT REGARDING FEDERAL RIGHTS .Iaddend.
.Iadd.This invention was made with government support under Grant
Nos. F49620-00-1-0108, F49620-93-1-0133, F49620-95-1-0136, and
F49620-97-1-0273 awarded by the U.S. Air Force Office of Scientific
Research to Colorado State University Research Foundation. The
government has certain rights in the invention. .Iaddend.
Parent Case Text
RELATED .[.APPLICATIONS.]. .Iadd.U.S. APPLICATION DATA
.Iaddend.
This application is a .Iadd.reissue of U.S. Pat. No. 5,959,574
which is a .Iaddend.continuation-in-part of U.S. patent application
Ser. No. 08/404,024, filed Mar. 14, 1995 and issued Jul. 16, 1996
as U.S. Pat. No. 5,537,119, which is a continuation-in-part of U.S.
patent application Ser. No. 08/171,327 filed Dec. 21, 1993, now
U.S. Pat. No. 5,406,289.
Claims
What is claimed is:
.[.1. A method for tracking a plurality of objects, comprising:
repeatedly scanning a region containing a set consisting of one or
more moving objects and generating N sequential images or data sets
of said region, a plurality of observations in said images or data
sets providing positional information for objects in said set;
determining a plurality of tracks, at least one track for each
object in said set; determining a plurality of costs, wherein each
cost is for assigning one of said observations to one of said
tracks; defining a linear programming problem: .times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..ltoreq-
..times..times..times..ltoreq..times..times..times..times..times..times..t-
imes. ##EQU00141## wherein each c.sub.i.sub.1 .sub.. . t.sub.N is
included in said plurality of costs, each M.sub.i, i=1, . . . ,N,
being one of: (a) a number of observations in an i.sup.th image or
data set of said N sequential images or data sets; (b) a sum of a
number of tracks in said plurality of tracks, and a number of said
observations in the i.sup.th image or data set not assigned to one
of said tracks; and (c) a number of tracks in said plurality of
tracks; solving said linear programming problem for values of
z.sub.i.sub.1 . . . i.sub.N for each i1 . . . iN; determining a
value z.sub.1.sub.1 .sub.. . . i.sub.N in {0,1} for each i1 . . .
iN corresponding to each z.sub.ij . . i.sub.N, wherein said values
zi1 . . . iN provide an optimal or near optimal solution to said
linear programming problem; taking one or more of the following
actions based on said optimal or near-optimal assignment of said
plurality of points to said plurality of tracks; sending a warning
to aircraft or a ground or sea facility, controlling air traffic,
controlling anti-aircraft or anti-missile equipment, taking evasive
action, working on one of said one or more objects, surveilling one
of said one or more objects..].
.Iadd.2. A method for tracking a plurality of objects, comprising:
using remote scanning apparatus to repeatedly scan a region
containing a set consisting of one or more moving objects and
generating N sequential images or data sets of said region, a
plurality of observations in said images or data sets providing
positional information for objects in said set; using a computer
system to determine a plurality of tracks, at least one track for
each object in said set; using a computer system to determine a
plurality of costs, wherein each cost is for assigning one of said
observations to one of said tracks; defining a linear programming
problem:
.times..times..times..times..times..times..times..times..times.
##EQU00142##
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times.
.times..ltoreq..times..times..ltoreq..times..times..times..times..times..-
times..times. ##EQU00142.2## wherein each c.sub.i1 . . . iN is
included in said plurality of costs, each M.sub.i, i=1, . . . ,N,
being one of: (a) a number of observations in an i.sup.th image or
data set of said N sequential images or data sets; (b) a sum of a
number of tracks in said plurality of tracks, and a number of said
observations in the i.sup.th image or data set not assigned to one
of said tracks; and (c) a number of tracks in said plurality of
tracks; using a computer system for solving said linear programming
problem for values of z.sub.i1 . . . iN for each i.sub.1 . . .
i.sub.N; using a computer system for determining a value z.sub.i1 .
. . .sub.iN in {0,1} for each i.sub.1 . . . i.sub.N corresponding
to each z.sub.i1 . . . .sub.iN, wherein said values z.sub.i1 . . .
.sub.iN provide an optimal or near optimal solution to said linear
programming problem; and using a computer system to determine which
one or more of the following actions will be taken based on said
optimal or near-optimal assignment of said plurality of points to
said plurality of tracks; sending a warning to aircraft or a ground
or sea facility, controlling air traffic, controlling anti-aircraft
or anti-missile equipment, taking evasive action, working on one of
said one or more objects, and surveilling one of said one or more
objects. .Iaddend.
.Iadd.3. The method of claim 2, wherein the remote scanning
apparatus for repeatedly scanning a region containing a set
consisting of one or more moving objects comprises radar apparatus.
.Iaddend.
Description
.[.FIELD OF THE INVENTION.].
.[.The invention relates generally to computerized techniques for
processing data obtained from radiation reflections used to track
multiple discrete object..].
BACKGROUND OF THE INVENTION
.Iadd.a. Field of the Invention .Iaddend.
The invention relates generally to computerized techniques for
processing data obtained from radar to track multiple discrete
objects.
.Iadd.b. Description of the Background .Iaddend.
There are many situations where the courses of multiple objects in
a region must be tracked. Typically, radar is used to scan the
region and generate discrete images or "snapshots" based on sets of
returns or observations. In some types of tracking systems, all the
returns from any one object are represented in an image as a single
point unrelated to the shape or size of the objects. "Tracking" is
the process of identifying a sequence of points from a respective
sequence of the images that represents the motion of an object. The
tracking problem is difficult when there are multiple closely
spaced objects because the objects can change .[.their.]. speed and
direction rapidly and move into and out of the line of sight for
other objects. The problem is exacerbated because each set of
returns may result from noise as well as echoes from the actual
objects. The returns resulting from the noise are also called false
positives. Likewise, the radar will not detect all echoes from the
actual objects and this phenomena is called a false negative or
"missed detect" error. For tracking airborne objects, a large
distance between the radar and the objects diminishes the signal to
noise ratio so the number of false positives and false negatives
can be high. For robotic applications, the power of the radar is
low and as a result, the signal to noise ratio can also be low and
the number of false positives and false negatives high.
In view of the proximity of the objects to one another, varied
motion of the objects and false positives and false negatives,
multiple sequential images should be analyzed collectively to
obtain enough information to properly assign the points to the
proper tracks. Naturally, the larger the number of images that are
analyzed, the greater the amount of information that must be
processed.
While identifying the track of an object, a kinematic model
.Iadd.may be generated .Iaddend.describing the .[.object's.].
location, velocity and acceleration .[.may be generated.]. .Iadd.of
the object.Iaddend.. Such a model provides the means by which the
.[.object's.]. future motion .Iadd.of the object .Iaddend.can be
predicted. Based upon such a prediction, appropriate action may be
initiated. For example, in a military application there is a need
to track multiple enemy aircraft or missiles in a region to predict
their objective, plan responses and intercept them. Alternatively,
in a commercial air traffic control application there is a need to
track multiple commercial aircraft around an airport to predict
their future courses and avoid collision. Further, .[.in.]. these
and other applications, such as robotic applications, may use
radar, sonar, infrared or other object detecting radiation
bandwidths for tracking objects. In particular, in robotic
applications reflected radiation can be used to track a single
object which moves relative to the robot (or vice versa) so the
robot can work on the object.
Consider the very simple example of two objects being tracked and
no false positives or false negatives. The radar, after scanning at
time t.sub.1, reports objects at two locations in a first
observation set. That is, it returns a set of two observations
{o.sub.11, o.sub.12}. At time t.sub.2 it returns a similar set of
two observations {o.sub.21, o.sub.22} from a second observation
set. Suppose from prior processing that track data for two tracks
T.sub.1 and T.sub.2 includes the locations at t.sub.0 of two
objects. Track T.sub.1 may be extended through the points in the
two sets of observations in any of four ways, as may track T.sub.2.
The possible extensions of T.sub.1 can be described as: {T.sub.1,
o.sub.11, o.sub.21}, {T.sub.1, o.sub.11, o.sub.22}, {T.sub.1,
o.sub.12, o.sub.21} and {T.sub.1, o.sub.12, o.sub.22}. Tracks can
likewise be extended from T.sub.2 in four possible ways.Iadd.,
.Iaddend.including.[.,.]. {T.sub.2, o.sub.12, o.sub.21}. FIG. 1
illustrates these five (out of eight) possible tracks (to simplify
the problem for purposes of explanation) . The five track
extensions are labeled h.sub.11, h.sub.12, h.sub.13, h.sub.14, and
h.sub.21 wherein h.sub.11 is derived from {T.sub.1, o.sub.11,
o.sub.21}, h.sub.12 is derived from {T.sub.1, o.sub.11, o,.sub.22},
h.sub.13 is derived from {T.sub.1, o.sub.12, o.sub.21}, h.sub.14 is
derived from {T.sub.1, o.sub.12, o.sub.22}, and h.sub.21 is derived
from {T.sub.2, o.sub.11, o.sub.21}. The problem of determining
which such track extensions are the most likely or optimal is
hereinafter known as the assignment problem.
It is known from prior art to determine a figure of merit or cost
for assigning each of the points in the images to a track. The
figure of merit or cost is based on the likelihood that the point
is actually part of the track. For example, the figure of merit or
cost may be based on the distance from the point to an
extrapolation of the track. FIG. 1 illustrates costs .delta..sub.21
.delta..sub.22 21 modeled target characteristics. The function to
calculate the cost will normally incorporate detailed
characteristics of the sensor, such as probability of measurement
error, and track characteristics, such as likelihood of track
maneuver.
FIG. 2 illustrates a two by two by two matrix, c, that contains the
costs for each point in relation to each possible track. The cost
matrix is indexed along one axis by the track number, along another
axis by the image number and along the third axis by a point
number. Thus, each position in the cost matrix lists the cost for a
unique combination of points and a track, one point from each
image. FIG. 2 also illustrates a {0, 1} assignment matrix, z, which
is defined with the same dimensions as the cost matrix. Setting a
position in the assignment matrix to "one" means that the
equivalent position in the cost matrix is selected into the
solution. The illustrated solution matrix selects the {h.sub.14,
h.sub.21 } solution previously described. Note that for the above
example of two tracks and two snapshots, the resulting cost and
assignment matrices are .[.three dimensional.].
.Iadd.three-dimensional.Iaddend.. As used in this patent
application, the term "dimension" means the number of axes in the
cost or assignment matrix while size refers to the number of
elements along a typical axis. The costs and assignments have been
grouped in matrices to facilitate computation.
A solution to the assignment problem satisfies two
constraints--first, the sum of the associated costs for assigning
points to a track extension is minimized and, second, if no false
positives or false negatives exist, then each point is assigned to
one and only one track.
When false positives exist, however, additional hypothetical track
extensions incorporating the false positives will be generated.
Further note that the random locations of false positives will, in
general, not fit well with true data and such additional
hypothetical track extensions will result in higher costs. Also
note that when false negative errors exist, then the size of the
cost matrix must grow to include hypothetical track extensions
formulated with "gaps" (i.e., data omissions where there should be
legitimate observation data) for the false negatives. Thus, the
second criteria must be weakened to reflect false positives not
being as signed and also to permit the gap filler to be multiply
assigned. With hypothetical cost calculated in this manner then the
foregoing criteria for minimization will tend to materialize the
false negatives and avoid the false positives.
For a .[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.problem,
as is illustrated in FIG. 1, but with N.sub.1 (initial) tracks,
N.sub.2 observations in scan 1, N.sub.3 observations in scan 2,
false positives and negatives assumed, the assignment problem can
be formulated as:
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..ltoreq..times..times..times-
..times..times..times..times..times..ltoreq..times..times.
##EQU00001## where "c" is the cost and "z" is a point or
observation assignment, as in FIG. 2.
The minimization equation or equivalently objective function
.[.[1.0] (a).]. .Iadd.(0.1 (a)) .Iaddend.specifies the sum of the
element by element product of the c and z matrices. The summation
includes hypothesis representations
Z.sub.i.sub.1.sub.i.sub.2.sub.i.sub.3 with observation number zero
being the gap filler observation. Equation .[.[1.01] (b).].
.Iadd.(0.1 (b)) .Iaddend.requires that each of the tracks T.sub.1,
. . . , T.sub.N.sub.1 be extended by one and only one hypothesis.
Equation .[.[1.0](c).]. .Iadd.(0.1 (c)) .Iaddend.relates to each
point or observation in the first observation set and requires that
each such observation, except the gap filler, can only associate
with one track but.Iadd., .Iaddend.because of the "less than"
condition.Iadd., .Iaddend.it might not associate with any track.
Equation .[.[1.01] (d).]. .Iadd.(0.1 (d)) .Iaddend.is like .[.[1.0]
(c).]. .Iadd.(0.1 (c)) .Iaddend.except that it is applicable to the
second observation set. Equation .[.[1.0] (e).]. .Iadd.(0.1 (e))
.Iaddend.requires that elements of the solution matrix z be limited
to the zero and one values.
The only known method to solve Problem Formulation .[.[1.0].].
.Iadd.(1.0) .Iaddend.exactly is a method called "Branch and Bound."
This method provides a systematic ordering of the potential
solutions so that solutions with a same partial solution are
accessible via a branch of a tree describing all possible solutions
whereby the cost of unexamined solutions on a branch are
incrementally developed as the cost for other solutions on the
branch are determined. When the developing cost grows to exceed the
previously known minimal cost (i.e., the bound) then enumeration of
the tree branch terminates. Evaluation continues with a new branch.
If evaluation of the cost of a particular branch completes, then
that branch has lower cost than the previous bound so the new cost
replaces the old bound. When all possible branches are evaluated or
eliminated then the branch that had resulted in the last used bound
is the solution. If we assume that N.sub.1=N.sub.2=N.sub.3=n and
that branches typically evaluate to half there full length, then
workload associated with "branch and bound" is proportional to
(n!|n/2)!).sup.2. This workload is unsuited to real time
evaluation.
The Branch and Bound algorithm has been used in past research on
the Traveling Salesman Problem. Messrs. Held and Karp showed that
if the set of constraints was relaxed by a method of Lagrangian
Multipliers (described in more detail below) then tight lower
bounds could be developed in advance of enumerating any branch of
the potential solution. By initiating the branch and bound
algorithm with such a tight lower bound, significant performance
improvements result in that branches will typically evaluate to
less than half their full length.
Messrs. Frieze and Yadagar in dealing with a problem related to
scheduling combinations of resources, as in job, worker and work
site, showed that Problem Formulation .[.[1.0].]. .Iadd.(1.0)
.Iaddend.applied. They further described a solution method based
upon an extension of the Lagrangian Relaxation previously
mentioned. The two critical extensions provided by Messrs. Frieze
and Yadagar were: (1) an iterative procedure that permitted the
lower bound on the solution to be improved (by "hill climbing"
described below) and (2) the recognition that when the lower bound
of the relaxed problem was maximized, then there existed a method
to recover the solution of the non-relaxed problem in most cases
using parameters determined at the maximum. The procedures
attributed to Messrs. Frieze and Yadagar are only applicable to the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.problem posed
by Problem Formulation .[.[1.0].]. .Iadd.(1.0) .Iaddend.and where
the cost matrix is fully populated. However, tracking multiple
airborne objects usually requires solution of a much higher
dimensional problem.
FIGS. 1 and 2 illustrate an example where "look ahead" data from
the second image improved the assignment accuracy for the first
image. Without the look ahead, and based only upon a simple nearest
neighbor approach, the assignments in the first set would have been
reversed. Problem Formulation .[.[1.0].]. .Iadd.(1.0) .Iaddend.and
the prior art only permit looking ahead one image. In the prior art
it was known that the accuracy of assignments will improve if the
process looks further ahead, however no practical method to
optimally incorporate look ahead data existed. Many real radar
tracking problems involve hundreds of tracks, thousands of
observations per observation set and matrices with dimensions in
the range of 3 to 25 including many images of look ahead.
It was also known that the data assignment problem may be
simplified (without reducing the dimension of the assignment
problem) by eliminating from consideration for each track those
points which, after considering estimated limits of speed and
turning ability of the objects, could not physically be part of the
track. One such technique, denoted hereinafter the "cone method,"
defines a cone as a continuation of each previously determined
track with the apex of the cone at the end of the previously
defined track. The length of the cone is based on the estimated
maximum speed of the object and the size of the arc of the cone is
based on the estimated maximum turning ability of the object. Thus,
the cone defines a region outside of which no point could
physically be part of the respective track. For any such points
outside of the cones, an infinite number could be put in the cost
matrix and a zero could be preassigned in the assignment matrix. It
was known for the tracking problem that these elements will be very
common in the cost and selection matrices (so these matrices are
"sparse").
It was also known in the prior art that one or more tracks which
are substantially separated geographically from the other tracks
can be separated also in the assignment problem. This is done by
examining the distances from each point to the various possible
tracks. If the distances from one set of points are reasonably
short only in relation to one track, then they are assigned to that
track and not further considered with the remainder of the points.
Similarly, if a larger group of points can only be assigned to a
few tracks, then the group is considered a different assignment
problem. Because the complexity of assignment problems increases
dramatically with the number of possible tracks and the total
number of points in each matrix, this partitioning of the group of
points into a separate assignment problem and removal of these
points from the matrices for the remaining points, substantially
reduces the complexity of the overall assignment problem.
A previously known Multiple Hypothesis Testing (MHT) algorithm (see
.Iadd.Chapter 10 of S.S. .Iaddend.Blackman.[.,.]..Iadd..
.Iaddend.Multiple-Target Tracking with Radar Applications.[.,
Chapter 10,.]..Iadd.. .Iaddend.Artech House.Iadd.,
.Iaddend.Norwood.Iadd., .Iaddend.MA, 1986.Iadd...Iaddend.) related
to formulation and scoring. The MHT procedure describes how to
formulate the sparse set of all reasonable extension hypothesis
(for FIG. 1 the set {h.sub.11 . . . h.sub.24}) and how to calculate
a cost of the hypothesis {T.sub.i, o.sub.1j, o.sub.2k} based upon
the previously calculated cost for hypothesis {T.sub.i, o.sub.1j}.
The experience with the MHT algorithm, known in the prior art, is
the basis for the assertion that look ahead through k sets of
observations results in improved assignment of observations from
the first set to the track.
In theory, the MHT procedure uses the extendable costing procedure
to defer assignment decision until the accumulated evidence
supporting the assignment becomes overwhelming. When it makes the
assignment decision.Iadd., .Iaddend.it then eliminates all
potential assignments invalidated by the decision in a process
called "pruning the tree." In practice, the MHT hypothesis tree is
limited to a fixed number of generations and the overwhelming
evidence rule is replaced by a most likely and feasible rule. This
rule considers each track independently of others and is therefore
a local decision rule.
A general object of the present invention is to provide an
efficient and accurate process for assigning each point object in a
region from multiple images to a proper track and then taking an
action based upon the assignments.
A more specific object of the present invention is to provide a
technique of the foregoing type which determines the solution of a
k-dimensional assignment problem where "k" is greater than or equal
to three.
SUMMARY OF THE INVENTION
The present invention relates to a method and apparatus for
tracking objects. In particular, the present invention tracks
movement or trajectories of objects by analyzing radiation
reflected from the objects, the invention being especially useful
for real-time tracking in noisy environments.
In providing such a tracking capability, a region containing the
objects is repeatedly scanned to generate a multiplicity of
sequential images or data observation sets of the region. One or
more points (or equivalently observations), in each of the images
or observation sets are detected wherein each such observation
either corresponds to an actual location of an object or is an
erroneous data point due to noise. Subsequently, for each
observation detected, figures of merit or costs are determined for
assigning the observation to each of a plurality of previously
determined tracks. *Afterwards, a first optimization problem is
specified which includes: (a) a first objective function for
relating the above mentioned costs to potential track extensions
through the detected observations (or simply observations); and (b)
a first collection of constraint sets wherein each constraint set
includes constraints to be satisfied by the observations in a
particular scan to which the constraint set is related. In general,
there is a constraint for each observation of the scan, wherein the
constraint indicates the number of track extensions to which the
observation may belong.
In particular, the first optimization problem is formulated,
generated or defined as an M-dimensional assignment problem wherein
there are M constraint sets in the first collection of constraint
sets (i.e., there are M scans being examined) and the first
objective function minimizes a total cost for assigning
observations to various track extensions wherein terms are included
in the cost, such that the terms have the figures of merit or costs
for hypothesized combinations of assignments of the observations to
the tracks. Subsequently, the formulated M-dimensional assignment
problem is solved by reducing the complexity of the problem by
generating one or more optimization problems each having a lower
dimension and then solving each lower dimension optimization
problem. That is, the M-dimensional assignment problem is solved by
solving a plurality of optimization problems each having a lower
number of constraint sets.
The reduction of the M-dimensional assignment problem to a lower
dimensioned problems is accomplished by relaxing the constraints on
the points of one or more scans thereby permitting these points to
be assigned to more than one track extension. In relaxing the
constraints, terms having penalty factors are added into the
objective function thereby increasing the total cost of an
assignment when one or more points are assigned to more than one
track. Thus, the reduction in complexity by this relaxation process
is iteratively repeated until a sufficiently low dimension is
attained such that the lower dimensional problem may be solved
directly by known techniques.
In one embodiment of the invention, each k-dimensional assignment
problem .[.2<k.ltoreq.M,.]. .Iadd.(2<k.ltoreq.M) .Iaddend.is
iteratively reduced to a .[.k-1 dimensional.].
.Iadd.(k-1)-dimensional .Iaddend.problem until a
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem is
specified or formulated. Subsequently, the .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem formulated is solved
directly and a "recovery" technique is used to iteratively recover
an optimal or near-optimal solution to each k-dimensional problem
from a derived .[.(k-1) dimensional.]. .Iadd.(k-1)-dimensional
.Iaddend.problem k=2, 3, 4, . . . M.
In performing each recovery step (of obtaining a solution to a
k-dimensional problem using a solution to a (k-1)-dimensional
problem).Iadd., .Iaddend.an auxiliary function.[.,.]. is utilized.
In particular, to recover an optimal or near-optimal solution to a
k-dimensional problem, an auxiliary function, .PSI..sub.k-1.[.,.].
k=4, 5, . . . , M.Iadd.).Iaddend., is specified and a region or
domain is determined wherein this function is maximized, whereby
values of the region determine the penalty factors of the
(k-1)-dimensional problem such that another .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem can be formulated which
determines a solution to the k-dimensional problem using the
penalty factors of the (k-1)-dimensional problem.
Each Auxiliary function .PSI..sub.k.Iadd., k=3, . . . , M-1, is a
function of .Iaddend.of both lower dimensional .[.problem.].
.Iadd.problems, .Iaddend.penalty factors.Iadd., .Iaddend.and a
solution at the dimension k at which the penalized cost function is
solved directly (typically a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem). Further, in determining,
for auxiliary function .PSI..sub.k, a peak region, a gradient of
the auxiliary function is determined, and utilized to identify the
peak region. Thus, gradients are used for each of the approximation
functions .PSI..sub.3, .PSI..sub.4,.Iadd.. . . ,
.PSI..Iaddend..sub.M-1 determining penalty factors .[.penalty
factors.]. until .[.tm-1.]. .Iadd.M-1 .Iaddend.is used in
determining the penalty factors for the .[.M-1 dimensional.].
.Iadd.(M-1)-dimensional .Iaddend.problem. Subsequently, once the
M-dimensional problem is solved (using a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem to go from an .[.(M-1)
dimensional.]. .Iadd.(M-1)-dimensional .Iaddend.solution to an
M-dimensional solution), one or more of the following actions are
taken based on the track assignments: sending a warning to aircraft
or a ground or sea facility, controlling air traffic, controlling
anti-aircraft or anti-missile equipment, taking evasive action,
working on one of the objects.
According to one feature of this first embodiment of the present
invention, the following steps are also performed before the step
of defining the auxiliary function. A preliminary auxiliary
function is defined for each of the lower dimensional problems
having a dimension equal or one greater than the dimension at which
the penalized cost function is solved directly. The preliminary
auxiliary function is a function of lower order penalty values and
a solution at the dimension at which the penalized cost function
was solved directly. In determining a gradient of the preliminary
auxiliary function, step in the direction of the gradient to
identify a peak region of the preliminary auxiliary function and
determine penalty factors at the peak region. Iteratively repeat
the defining, gradient determining, stepping and peak determining
steps to define auxiliary functions at successively higher
dimensions until the auxiliary function .[.at 6-18 (k-1).].
dimension .Iadd.(k-1) .Iaddend.is determined. In an alternative
second embodiment of the present invention, instead of reducing the
dimentiality of the M-dimensional assignment problem by a single
dimension at a time, a plurality of dimensions are relaxed
simultaneously. This new strategy has the advantage that when the
M-dimensional problem is relaxed directly to a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem, then all
computations may be performed precisely without utilizing an
auxiliary function such as .PSI..sub.k as in the first embodiment.
More particularly, the second embodiment solves the first
optimization problem (i.e., the M-dimensional assignment problem)
by specifying (i.e., creating, generating, formulating and/or
defining) a second optimization problem. The second optimization
problem includes a second objective function and a second
collection of constraint sets wherein: a) the second objective
function is a combination of the first objective function and
penalty factors or terms determined for incorporating .[.M-m
constraint.]. .Iadd.(M-m)-constraint .Iaddend.sets of the first
optimization problem into the second objective function; b) the
constraint sets of the second collection include only m of the
constraint sets of the first collection of constraints, wherein
2.ltoreq.m.ltoreq.M-1. Note that, once the second optimization
problem has been specified or formulated, an optimal or
near-optimal solution is determined and that solution is used in
specifying (i.e., creating, generating, formulating and/or
defining) a third optimization problem of .[.M-m.]. .Iadd.(M-m)
.Iaddend.dimensions (or equivalently constraint sets). The third
optimization problem is subsequently solved by decomposing it using
the same procedure of this second embodiment as was used to
decompose the first optimization problem above. Thus, a plurality
of instantiations of the third optimization problem are specified,
each successive instantiation having a lower number of dimensions,
until an instance of the third optimization problem is a .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment problem
which can be solved directly. Subsequently, whenever an instance of
the third optimization problem is solved, the solution is used to
recover a solution to the instance of the first optimization
problem from which this instance of the third optimization was
derived. Thus, an optimal or near-optimal solution to the original
first optimization problem may be recovered through iteration of
the above steps.
As mentioned above, the second embodiment of the present invention
is especially advantageous when m=2, since in this case all
computations may be performed precisely and without utilizing
auxiliary functions.
Other features and benefits of the present invention will become
apparent from the detailed description with the accompanying
drawings contained hereinafter.
BRIEF DESCRIPTION OF THE FIGURES
FIG. 1 is a graph of images or data sets generated by a scan of a
region and possible tracks within the images or data sets according
to the prior art.
FIG. 2 illustrates cost and assignment matrices for the data sets
of FIG. 1 according to the prior art.
FIG. 3 is a block diagram of the present invention.
FIG. 4 is a flow chart of a process according to the prior art for
solving a .[.3-dimensional.]. .Iadd.three-dimensional
.Iaddend.assignment problem.
FIG. 5 .[.Is.]. .Iadd.is .Iaddend.a flow chart of a process
according to the present invention for solving a k-dimensional
assignment problem where "k" is greater than or equal to 3.
FIG. 6 is a graph of various functions used to explain the present
invention.
FIG. 7 is another block diagram of the present invention for
solving the k-dimensional assignment problem where "k" is greater
than or equal to .[.3.]. .Iadd.three (3).Iaddend..
FIG. 8 is a flowchart describing the procedure for solving .[.a.].
.Iadd.an .Iaddend.n-dimensional assignment problem according to the
second embodiment of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Referring now to the other figures in detail wherein like reference
numerals indicate like elements throughout the several views, FIG.
3 illustrates a system generally designated 100 for implementing
the present invention. System 100 comprises, for example, a radar
station 102 (note sonar, microwave, infrared and other radiation
bandwidths are also contemplated) for scanning a region which may
be, for example, an aerial region (in aerial surveillance
applications) or a work region (in robotic applications) and
generating signals indicating locations of objects within the
region. The signals are input to a converter 104 which converts the
signals to data points or observations in which each object (or
false positive) is represented by a single point. The output of the
converter is input to and readable by a computer 106. As described
in more detail below, the computer 106 assigns the points to
respective tracks, and then displays the tracks and extrapolations
of the tracks on a monitor 110. Also, the computer 106 determines
an appropriate action to take based on the tracks and track
extensions. For example, in a commercial application at an airport,
the computer can determine if two aircraft being tracked are on a
collision course and if so, signal a transmitter 112 to warn each
aircraft, or if a scheduled take-off will pose the risk of
collision, delay the take-off. For a military application on a ship
or base, the computer can determine subsequent coordinates of enemy
aircraft and send the coordinates to an antiaircraft gun or missile
120 via a communication channel 122. In a robotic application, the
computer controls the robot to work on the proper object or portion
of the object.
The invention generates k-dimensional matrices where k is the
number of images or sets of observation data in the look ahead
window plus one. Then, the invention formulates a k-dimensional
assignment problem as in .[.[1.0].]. .Iadd.(0.1)
.Iaddend.above.
The k-dimensional assignment problem is subsequently relaxed to a
(k-1)-dimensional problem by incorporating one set of constraints
into the objective function using a Lagrangian relaxation of this
set. Given a solution of the (k-1)-dimensional problem, a feasible
solution of the k-dimensional problem is then reconstructed. The
(k-1)-dimensional problem is solved in a similar manner, and the
process is repeated until it reaches the two-dimensional problem
that can be solved exactly. The ideas behind the Lagrangian
relaxation scheme are outlined next.
Consider the integer programming problem
.times. .times..function..times..times..times..times..times.
.times..ltoreq..times..ltoreq..times..times..times..times..times..times..-
times..times..times..times..times..times..di-elect
cons..times..times..times. ##EQU00002## where the partitioning of
the constraints is natural in some sense. Given a multiplier vector
u.gtoreq.0, the Lagrangian relaxation of .[.[1.0.1].]. .Iadd.(0.2)
.Iaddend.relative to the constraints Bz.ltoreq.d is defined to be
.[..PHI..PHI.TT.].
.PHI..times..times..times.
.times..times..times..times..times..times.
.times..times..ltoreq..times..times..times..ltoreq..times..times..times..-
times..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times. ##EQU00003## If the constraint set
Bz.ltoreq.d is replaced by Bz=d, the .[.nonnegativity.].
.Iadd.non-negativity .Iaddend.constraint on u is removed.
.[.L=C.sup.Tz+u.sup.T(Bz-d).]. .Iadd.L=c.sup.Tz+u.sup.T(Bz-d)
.Iaddend.is the Lagrangian relative to the constraints Bz.ltoreq.d,
and hence the name Lagrangian relaxation. The following fact gives
the relationship between the objective functions of the original
and relaxed problems.
FACT A.1. If z is an optimal solution to .[.[1.0.1].].
.Iadd.(0.2).Iaddend., then .[..PHI..ltoreq.v( z).].
.Iadd..PHI.(u).ltoreq.v( z) .Iaddend.for all u.gtoreq.0. If an
optimal solution {circumflex over (z)} of .[.[1.0.2].]. .Iadd.(0.3)
.Iaddend.is feasible for .[.[1.0.2].]. .Iadd.(0.3).Iaddend., then
{circumflex over (z)} is an optimal solution for .[.[1.0.1].].
.Iadd.(0.2) .Iaddend.and .[..PHI. Algor ithm.].
.Iadd..PHI.(u)=v({circumflex over (z)}). .Iaddend.
.[.k.sub.k.sup..infin.=0.].
.Iadd.This leads to the following algorithm: .Iaddend.
.Iadd.Algorithm. Construct a sequence of multipliers
{u.sub.k}.sup..infin..sub.k=0 .Iaddend.converging to the solution
of Maximize .[.{.PHI..gtoreq.0}.].
.Iadd.{.PHI.(u):u.gtoreq.0}.Iaddend. and a corresponding sequence
of feasible solutions {Z.sub.k}.sub.k=0of .[.[1.0.1].]. .Iadd.(0.2)
.Iaddend.as follows: 1. Generate .[.in.]. .Iadd.an .Iaddend.initial
approximation u.sub.0. 2. Given u.sub.k, choose a search direction
s.sub.k and a search distance .alpha..sub.k .[..PSI..sub.k
.alpha..sub.k k .PSI..sub.k.]. .Iadd.so that
.PHI.(u.sub.k+.alpha..sub.ks.sub.k)>.PHI.(u.sub.k). Update the
multiplier .Iaddend.estimate u.sub.k by
.[.u.sub.k+1=u.sub.k+.alpha..sub.k k.].
.Iadd.u.sub.k+1=u.sub.k+.alpha..sub.ks.sub.k. .Iaddend. 3. Given
u.sub.k+1 and a feasible solution .[.{circumflex over (Z)}.sub.k+1
(u.sub.k+1).]. .Iadd.{circumflex over (z)}.sub.k+1 (u.sub.k+1)
.Iaddend.of .[.[1.0.2].]. .Iadd.(0.3).Iaddend., recover a feasible
solution z.sub.k+1 (u.sub.k+1) of the integer programming problem
.[.[1.0.1].]. .Iadd.(0.2).Iaddend.. 4. Check the termination
criteria. If the algorithm is not finished, set k=k+1 and return to
Step 2. Otherwise, terminate the algorithm.
If z is an optimal solution of .[.[1.0.1].]. .Iadd.(0.2).Iaddend.,
then .[..PSI..sub.k.ltoreq..PSI..ltoreq.V ( z).ltoreq.(z.sub.k).].
Since the optimal solution {circumflex over (z)}={circumflex over
(z)}(u) of .[.[1.0.2].]. .Iadd.(0.3) .Iaddend.is usually not a
feasible solution of .[.[1.0.1].]. .Iadd.(0.2) .Iaddend.for any
choice of the multipliers,
.PHI..PHI..alpha..alpha..PHI..alpha..alpha. ##EQU00004##
.Iadd..PHI.( ) is usually strictly less than v( z). The difference
v( z)-.PHI.( ) is called the "duality gap," which can be estimated
by comparing the best relaxed and recovered solutions computed in
the course of maximizing .PHI.(u). .Iaddend.
.[..sub.p.ltoreq.p.ltoreq.k.].
.Iadd.Thus, to eliminate the "<" in the constraint formulations
(e.g., (0.1)(a)-(0.1)(d)) that resulted from false positives, the
constraint sets have been modified, as indicated above, to
incorporate a gap filler in each observation set to account for
false positives. Thus, a zero for an index, i.sub.p,
l.ltoreq.p.ltoreq.k .Iaddend.in z.sup.k.sub.i.sub.1.sub.. . .
i.sub.k in problem .[.[1.1].]. .Iadd.(0.4) .Iaddend.below indicates
that the track extension represented by z.sup.k.sub.i.sub.1 .sub..
. . i.sub.k includes a false positive in the p.sup.th observation
set. Note that this implies that a hypothesis be formed
incorporating an observation with k-1 gap fillers, e.g., z.sub.0 .
. . 0i.sub.p .sub.. . . 0, i.sub.p.noteq.0. Thus, the resulting
generalization of Problem Formulation .[.[1.0].]. .Iadd.(0.1)
.Iaddend.without the "less than" complication within the
constraints is the following k-Dimensional Assignment Problems in
which k.ltoreq.3:
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..di-elect
cons..A-inverted..times..times..times..times. ##EQU00005## where c
and z are similarly dimensioned matrices representing costs and
hypothetical assignments. Note, in general, for tracking problems
these matrices are sparse.
After formulating the k-dimensional assignment problem as in
.[.[1.1].]. .Iadd.(0.4).Iaddend., the present invention solves the
resulting problem so as to generate the outputs required by devices
110, 112, 122 and 130. For each observation set o.sub.i received
from converter 104 at time t.sub.i where i=1 . . . , N.sub.1, the
computer processes O.sub.i in a batch together with the other
observation sets O.sub.i-k+1, . . . .Iadd., .Iaddend.O.sub.i and
the track T.sub.i-k, i.e., T.sub.j is the set of all tracks that
have been defined up to but not including O.sub.j. (Note, bold type
designations refer to the vector of elements for the indicated
time, i.e., the set of all observations in the scan or tracks
existing at the time, etc.) The result of this processing is the
new set of tracks T.sub.i-k+1 and a set of cost weighted possible
solutions indicating how the tracks might extend to the current
time t.sub.i. At time t.sub.i+1 the batch process is repeated using
the newest observation set and deleting the oldest. Thus, there is
a moving window of observation sets which is shifted forward to
always incorporate the most recent observation set. The effect is
that input observation sets are reused for .[.k-1.]. .Iadd.(k-1)
.Iaddend.cycles and then on the observation set's .[.k-th.].
.Iadd.k.sup.th .Iaddend.reuse each observation within the
observation set is integrated into a track.
FIG. 7 illustrates various processes implemented upon receipt of
each observation set. Except for the addition of the k-dimensional
assignment solving process 300 and the modification to scoring
process 154 to build data structures suitable for process 300, all
processes in FIG. 7 are based upon prior art. The following
processes 150 and 152 extend previously defined tracks
h.sub.i-1.[...]. based on new observations. Gate formulation and
output process 156 determines, for each of the previously defined
tracks, a zone wherein the track may potentially extend based on
limits of velocity, maneuverability and radar precision. One such
technique to accomplish this is the cone method described
previously. The definition of the zone is passed to gating process
150. When a new observation set O.sub.i is received, the gating
process 150 will match each member observation with the zone for
each member of the hypothetical set h.sub.i-1. After all input
observations from O.sub.i are processed.Iadd., .Iaddend.the new
hypothesis set h.sub.i is generated by extending each track of the
prior set of hypothetical tracks h.sub.i-1 either with missed
detect gap fillers or with all new observation elements satisfying
the track's zone. This is a .[.many to many.]. .Iadd.many-to-many
.Iaddend.matching in that each hypothesis member can be extended to
many new observations and each new observation can be used to
extend many hypotheses. It, however, is not a full matching in that
any hypothesis will neither be matched to all observations nor vice
versa. It is this matching characteristic that leads to the sparse
matrices involved in the tracking process. Subsequently, gating 150
forwards the new hypothesis set h.sub.i to filtering process 152.
Filtering process 152 determines a smooth curve for each member of
h.sub.i. Such a smooth curve is more likely than a sharp turn from
each point straight to the next point. Further, the filtering
process 152 removes small errors that may occur in generating
observations. Note that in performing these tasks, the filtering
process 152 preferably utilizes a minimization of a least squares
test of the points in a track hypothesis or a Kalman
.[.Filtering.]. .Iadd.filtering .Iaddend.approach.
As noted above, the foregoing track extension process requires
knowledge of a previous track. For the initial observations, the
following gating process 158 and filtering process 160 determine
the "previous track" based on initial observations. In determining
the initial tracks, the points from the first observation set form
the beginning points of all possible tracks. After observation data
from the next observation set is received, sets of simple .[.two
point.]. .Iadd.two-point .Iaddend.straight line tracks are defined.
Then, promotion, gate formulation, and output step 162 determines a
zone in which future extensions are possible. Note that filtering
step 160 uses curve fitting techniques to smooth the track
extensions depending upon the number of prior observations that
have accumulated in each hypothesis. Further note that promotion,
gate formulation and output process 162 also determines when
sufficient observations have accumulated to form a basis for
promoting the track to processes 150 and 152 as described
above.
The output of each of the filtering processes 152 and 160 is a set
of hypothetical track extensions. Each such extension contains the
hypothetical initial conditions (from the previous track), the list
of observations incorporated in the extension, and distance between
each observation and the filtered track curve. Scoring process 154
determines the figure of merit or cost of an observation being an
extension of a track. In one embodiment, the cost is based on the
above-mentioned distance although the particular formula for
determining the cost is not critical to the present invention. A
preferred formula for determining the cost utilizes a negative log
likelihood function in which the cost is the negative of the sum
of: (a) the logs of the distances normalized by sensor standard
deviation parameters, and (b) the log likelihoods for events
related to: track initiation, track termination, track maneuver,
false negatives and false positives. Note that track maneuvers are
detected by comparing the previous track curve with the current
extension. Further note that some of the other events related to,
for example, false negatives and false positives are detected by
analyzing the relative relationship of gap fillers in the
hypothesis. Thus, after determining that one of these events
occurred, a cost for it can be determined based upon suitable
statistics tables and system input parameters. The negative log
likelihood function is desirable because it permits effective
integration of the useful components. Copies of the set of
hypothetical track extensions which are scored are subsequently
passed directly to one of the gate formulation and output steps 156
and 162. Note that the scoring process 154 also arranges the actual
scores in a sparse matrix based upon observation identifiers, and
passes them to k-dimensional assignment problem solving process
300.
The assignment solving process 300 is described below. Its output
is simply the list of assignments which constitute the most likely
solution of the problem described by Equation .[.[1.1].].
.Iadd.(0.4).Iaddend.. Note that both gate formulation and output
processes 156 and 162 use (at different times) the list of
assignments to generate the updated track history T.sub.i to
eliminate or prune alternative previous hypotheses that are
prohibited by the actual assignments in the list, and.Iadd.,
.Iaddend.subsequently.Iadd., .Iaddend.to output any required data.
Also note that when one of the gate formulation and output
processes 156 and 162 accomplish these tasks, the process will
subsequently generate and forward the new set of gates for each
remaining hypothesis and the processes will then be prepared to
receive the next set of observations. In one embodiment, the loop
described here will generate zones for a delayed set of
observations rather than the subsequent set. This permits processes
156 and 162 to operate on even observation sets while the scoring
step 154 and k-dimensional solving process operate on odd sets of
observations, or vice versa.
The assignment solving process 300 permits the present invention to
operate with .Iadd.a .Iaddend.window size of .Iadd.dimension
.Iaddend.k-1 for .Iadd.some .Iaddend.k.gtoreq.3. The upper limit on
k depends only upon the computational power of the computer 106 and
the response time constraints of system 100. The k-1 observation
sets within the processing window plus the prior track history
result in a k-dimensional Assignment Problem as described by
Problem Formulation .[.[1.1].]. .Iadd.(0.4).Iaddend.. The present
invention solves this generalized problem including the processes
required to consider false positives and negatives, and also the
processes required to consider sparse matrix problem
formulations.
I. A FIRST EMBODIMENT OF THE k-DIMENSIONAL ASSIGNMENT SOLVER
300
In describing a first embodiment of the k-dimensional assignment
solver 300, it is worthwhile to also discuss the process of FIG. 4
which is used by the solver 300. FIG. 4 illustrates use of the
Frieze and Yadagar process as shown in prior art for transforming a
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.assignment
problem into a .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem and then use a hill climbing algorithm
to solve the .[.3-dimensional.]. .Iadd.three-dimensional
.Iaddend.assignment problem. The solver 300 uses a Lagrangian
Relaxation technique (well known in the art) to reduce the
dimension of an original k-dimensional assignment problem (k>3)
down to a .[.3-dimensional.]. .Iadd.three-dimensional
.Iaddend.problem and then use the process of FIG. 4 to solve the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.problem.
Further note that the Lagrangian Relaxation technique is also
utilized by the process of FIG. 4 and that in using this technique
the requirement that each point is assigned to one and only one
track is relaxed. Instead, an additional cost, which is equal to a
respective Lagrangian Coefficient .[."u".]. .Iadd.u.Iaddend., is
added to the cost or objective function .[.[1.0].]. .Iadd.(0.1)
.Iaddend.(a) whenever a point is assigned to more than one track.
This additional cost can be picked to weight the significance of
each constraint violation differently, so this additional cost is
represented as a vector of coefficients u which are correlated with
respective observation points. Hill climbing will then develop a
sequence of Lagrangian Coefficients sets designated (u.sub.0, . . .
, u.sub.j, u.sub.j+1, . . . , u.sub.p). That correspond to an
optimum solution of the .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem. The assignments at this optimum
solution are then used to "recover" the assignment solution of the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.assignment
problem.
In step 200 of FIG. 4, initial values are selected for the u.sub.0
coefficients. Because the Lagrangian Relaxation process is
iterative, the initial values are not critical and are all
initially selected as zero. In step 202, these additional costs are
applied to the objective function .[.[1.0].]. .Iadd.(0.1
.Iaddend.(a). With the addition of the costs .[."u".].
.Iadd.u.Iaddend., the goal is still to assign the points which
minimize the total cost. This transforms Equation .[.[1.0].].
.Iadd.(0.1 .Iaddend.(a), written for k=3 and altered to exclude
mechanisms related to false positives and negatives, into objective
function .[.[2.1].]. .Iadd.(1.1) .Iaddend.(a).Iadd.. .Iaddend.In
the first iteration it is not necessary to consider the .[."u".].
.Iadd.u .Iaddend.matrix because all .[."u".]. .Iadd.u
.Iaddend.values are set to zero. To relax the requirement that each
point be assigned to one and only one track, the constraint
Equation .[.[1.0].]. .Iadd.(0.1 .Iaddend.(d) is deleted, thereby
permitting points from the last image to be assigned to more than
one track. Note that while any axis can be chosen for relaxation,
observation constraints are preferably relaxed. The effect of this
relaxation is to create a new problem which must have the same
solution in the first two axes but which can have a .[.differing.].
.Iadd.different .Iaddend.solution in the third axis. The result is
constraints .[.[2.1].]. .Iadd.(1.1) .Iaddend.(b-d).
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..di-elect
cons..times..times..A-inverted..times..times..times. ##EQU00006##
Step 204 then generates from the .[.3-dimensional.].
.Iadd.three-dimensional .Iaddend.problem described by Problem
Formulation .[.[1.0].]. .Iadd.(0.1) .Iaddend.a new
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem
formulation which will have the same solution for the first two
indices. As Problem Formulation .[.[2.1].]. .Iadd.(1.1)
.Iaddend.has no constraints on the 3.sub.rd axis, any value within
a particular 3.sub.rd axis can be used in a solution, but using
anything other than the minimum value from any .[.3-rd.].
.Iadd.3.sup.rd .Iaddend.axis has the effect of increasing solution
cost. Conceptually, the effect of step 204 is to change the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.arrays in
Problem Formulation .[.[2.1].]. .Iadd.(1.1) .Iaddend.into
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.arrays as shown
in Problem Formulation .[.[2.2].]. .Iadd.(1.2) .Iaddend.and to
generate the new .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.matrix m.sub.i.sub.1.sub.i.sub.2 defined as shown in
Equation .[.[2.3].]. .Iadd.(1.3).Iaddend..
.times..times..times..times..times..times..times..times..A-inverted..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..ltoreq..times..times..times..di-elect
cons..times..times..A-inverted..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es. ##EQU00007## The cost or objective function for the reduced
problem as defined by .[.[2.2].]. .Iadd.(1.2) .Iaddend.(a), if
evaluated at all possible values of u is a surface over the domain
of .[.U.sup.3.]. .Iadd.u.Iaddend.. This surface is referred to as
.[..PHI.u.]. .Iadd..PHI.(u) and is non-smooth but provably convex
(i.e., it has a single peak and several other critical
characteristics which form terraces). Due to the convex
characteristics of .PHI.(u), the results from solving Problem
Formulation (1.2) at any particular u.sub.j can be used to generate
a new set of .Iaddend.
.PHI. ##EQU00008## coefficients u.sub.j+1, whose corresponding
.[.Problem Formulation [2.2] problem solution is a.]. cost value
.Iadd.is .Iaddend.closer to the peak of .[..PHI. u
U.sub.Pdimensional.]. .Iadd..PHI.(u). The particular set of
Lagrangian Coefficients that will generate the two-dimensional
problem resulting in the maximum cost is designated u.sub.p. To
recover the solution to the three-dimensional .Iaddend.assignment
problem requires solving the Equation .[.2.2.]. .Iadd.(1.2)
.Iaddend.problem corresponding to u.sub.p.
In step 206, the .[.two dimensional.]. .Iadd.(two-dimensional)
.Iaddend.problem is solved directly using a technique known to
those skilled in the art such as Reverse Auction for the
corresponding cost and solution values. This is the evaluation of
one point on the surface or for the first iteration
.[..PHI.U.sub.0.]. .Iadd..PHI.(u.sub.0). .Iaddend.
Thus, after this first iteration, the points have been assigned
based on all "u" values being arbitrarily set to zero. Because the
"u" values have been arbitrarily assigned, it is unlikely that
these assignments are correct and it is likely that further
iterations are required to properly assign the points. Step 208
determines whether the points have been properly assigned after the
first iteration by determining if for this set of assignments
whether a different set of "u" values could result in a higher
total cost. Thus, step 208 is implemented by determining the
gradient of objective function .[.[2.2].]. .Iadd.(1.2 .Iaddend.(a)
with respect to u.sub.j. If the gradient is substantially non-zero
(greater than a predetermined limit) then the assignments are not
at or near enough to the peak of the .[..PHI.u.]. .[.U.sub.j+1.].
.Iadd.surface .PHI.(u) (decision 210), and the new set of
Lagrangian Coefficients u.sub.j+1 is determined. .Iaddend.
Hill climbing Step 212 determines the u.sub.j+1 values based upon
the u.sub.j values, the direction resulting from protecting the
previous gradient into the U.sup.3 domain, and a step size. The
solution value of the .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.problem is the set of coefficients that minimize the
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem and the
actual cost at the minimum. Those coefficients augmented by the
coefficients stored in m.sub.i.sub.1.sub.i.sub.2 permit the
evaluation (but not the minimization) of the cost term in Problem
Formulation .[.[2.1].]. .Iadd.(1.1).Iaddend.. These two cost terms
are lower and upper bounds on the actual minimized cost of the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.problem, and
the difference between them in combination with the gradient is
used to compute the step size.
With this new set of .[."u".]. .Iadd.u .Iaddend.values, steps
202-210 are repeated as a second iteration. Steps 212 and 202-210
are repeated until the gradient as a function of u determined in
step 208 is less than the predetermined limit. This indicates that
the .[.up.]. .Iadd.u.sub.p .Iaddend.values which locate the peak
area of the .[..PHI. u.]. .Iadd..PHI.(u) .Iaddend.surface are
determined and that the corresponding Problem Formulation
.[.[2.2].]. .Iadd.(1.2) .Iaddend.has been solved. Step 214 will
attempt to use the assignments that resulted from this particular
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem to recover the solution of the .[.3-dimensional.].
.Iadd.three-dimensional .Iaddend.assignment problem as described
below. If the limit was chosen properly so that the .[."u".].
.Iadd.u .Iaddend.values are close enough to the peak, this recovery
will yield the proper set of assignments that rigidly satisfies the
constraint that each point be assigned to one and only one track.
However, if the .[."u".]. .Iadd.u .Iaddend.values are not close
enough to the peak, then the limit value for decision 210 is
reduced and the repetition of steps 212 and 202-210 is
continued.
Step 214 recovers the .[.3-dimensional.]. .Iadd.three-dimensional
.Iaddend.assignment solution by using the assignment values
determined on the last iteration through step 208. Consider the
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.z assignment
matrix to have 1's in the locations specified by the list
L.sub.1=(a.sub.i, b.sub.i).sup.N.sub.i=1. If the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.z matrix is
specified by placing 1's at the location indicated by the list
.[.L.sub.2=(a.sub.i, b.sub.i, m.sub.a.sub.i.sub.b.sub.i).sub.N=1.].
.Iadd.L.sub.2=(a.sub.i, b.sub.i,
m.sub.a.sub.i.sub.b.sub.i).sup.N.sub.i-1 .Iaddend.then the result
is a solution of Problem Formulation .[.[2.1].].
.Iadd.(1.1).Iaddend.. Let
.[.L.sub.3=(m.sub.a.sub.i.sub.b.sub.i).sub.N=1.].
.Iadd.L.sub.3=(m.sub.a.sub.i.sub.b.sub.i).sup.N.sub.i=1 .Iaddend.be
the list formed by the third index. If each member of L.sub.3 is
unique then the L.sub.2 solution satisfies the third constraint so
it is a solution to Problem Formulation .[.[1.0].].
.Iadd.(0.1).Iaddend.. When this is not the case, recovery
determines the minimal substitutions required within list L.sub.3
so that it plus L.sub.1 will be a feasible solution, i.e., a
solution which satisfies the constraints of a problem formulation,
but which may not optimize the objective function of the problem
formulation. This stage of the recovery process is formulated as a
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem.[.:.]. .Iadd.as follows. .Iaddend.Form a new cost matrix
[c.sub.i,j].sup.N.sub.i,j=1 where
c.sub.i,j=c.sub.a.sub.i.sub.b.sub.i.sub.,j for j=1 . . . N.sub.i
and the N.sub.i term is the total number of cost elements in the
selected row of the .[.3-dimensional.]. .Iadd.three-dimensional
.Iaddend.cost matrix. Attempt to solve this .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem for the minimum using two
constraints sets. If a feasible solution is found then the result
will have the same form as list L.sub.1. Replace the first set of
.[.indexes.]. .Iadd.indices .Iaddend.by the indicated (a.sub.i,
b.sub.i) pairs taken from list L.sub.1 and the result will be a
feasible solution of Problem Formulation .[.[1.1].].
.Iadd.(0.4).Iaddend.. If no feasible solution to the new
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem exists
then further effort to locate the peak of .[..PHI. u.].
.Iadd..PHI.(u) is required. .Iaddend.
.times..PHI..PHI..PHI..PHI..PHI..times. .times. ##EQU00009##
.Iadd.I.1. Generalization to a Multi-Dimensional Assignment Solving
Process .Iaddend.
.Iadd.Let M be a fixed integer and assume that M is the dimension
of the initial assignment problem to be solved. Thus, initially,
the result of the scoring step 154 is a M-dimensional Cost Matrix
which is structured as a sparse matrix (i.e., only a small
percentage of the entries in the cost and assignment matrices are
filled or non-zero). Individual cost elements represent the
likelihood that a track T.sub.i as extended .Iaddend.by the set of
observations {o.sub. |i=1, . . . , M-1}, is not valid. Because the
matrix is sparse the list of cost elements is stored as a packed
list, and then for each dimension of the matrix, a vector of a
variable length list of pointers to the cost elements is generated
and stored. This organization means that for a particular
observation .[.o.sub. y.]. .Iadd.O.sub.ij for .Iaddend.the j.sup.th
list in the i.sup.th vector will be a list of pointers to all
hypotheses in which .[.o.sub. y.]. .Iadd.O.sub.ij
.Iaddend.participates. This structure is further explained in the
following section dealing with problem partitioning.
The objective of the assignment solving process is to select from
the set of all possible combinations of track extensions a subset
that satisfies two criteria. First, each point in the subset of
combinations should be assigned to one and only one track and
therefore, included in one and only one combination of the subset,
and second, the total of the scoring sums for the combinations of
the subset should be minimized. This yields the following
M-dimensional equations where .[.k=M.]..Iadd.k=M.Iaddend.:
.times..times..times..times..times..function..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s. ##EQU00010## and where c.sup.k is the cost matrix
[c.sup.k.sub.i.sub.1 . . . , .sub.i.sub.k] which is a function of
the distance between the observed point z.sup.k and the smoothed
track determined by the filtering step, and v.sub.k is the cost
function. This set of equations is similar to the set presented in
Problem Formulation .[.[1.1].]. .Iadd.(0.4) .Iaddend.except that it
includes the subscript and superscript k notation. Thus, in solving
the M-dimensional Assignment Problem the invention reduces this
problem to an .[.M-1 dimensional.]. .Iadd.(M-1)-dimensional
.Iaddend.Assignment Problem and then to an .[.N-2 dimensional.].
.Iadd.(N-2)-dimensional .Iaddend.Assignment Problem, etc. Further,
the symbol k.epsilon.{3, . . . ,M} customizes Problem Formulation
.[.[3.1].]. .Iadd.(0.4) .Iaddend.to a particular relaxation level.
That is, the notation is used to reference data from levels
relatively removed as in c.sup.k+1 are the cost coefficients which
existed prior to this level of relaxed coefficients c.sup.k. Note
that actual observations are numbered from 1 to N.sub.i.Iadd.,
.Iaddend.where N.sub.i is the number of observations in observation
set i. Further note that the added .[.0.]. .Iadd.O
.Iaddend.observation in each set of observations is the
unconstrained "gap filler." This element serves as a filler in
substituting for missed detects, and a sequence of these elements
including only one true observation represents the possibility that
the observation is a false positive. Also note that by being
unconstrained a gap filler may be used in as many hypotheses as
required.
While direct solution to .[.[3.1].]. .Iadd.(0.4) .Iaddend.would
give the precise assignment, the solution of k-dimensional
equations directly for large k is too complex and time consuming
for practice. Thus, the present invention solves this problem
indirectly.
The following is a short description of many aspects of the present
invention and includes some steps according to the prior art. The
first step in solving the problem indirectly is to reduce the
complexity of the problem by the previously known and discussed
Lagrangian Relaxation technique. According to the Lagrangian
Relaxation technique, the absolute requirement that each point is
assigned to one and only one track is relaxed such that for some
one image, points can be assigned to more than one track. However,
a penalty based on a respective Lagrangian Coefficient u.sup.k is
added to the cost function when a point in the image is assigned to
more than one track. The Lagrangian Relaxation technique reduces
the complexity or "dimension" of the formulation of the assignment
problem because constraints on one observation set are relaxed.
Thus, the Lagrangian Relaxation is performed iteratively to
repeatedly reduce the dimension until a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.penalized cost function problem
results as in Problem Formulation .[.[2.1].]. .Iadd.(1.1).Iaddend..
This .[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem is
solved then directly by a previously known technique such as
Reverse Auction. The penalized cost function for the
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.problem defines
a valley or convex shaped surface which is a function of various
sets of {u.sup.k|k=3, . . . ,M} penalty values and one set of
assignments for the points in two dimensions. That is, for each
particular u.sup.3 there is a corresponding .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.penalized cost function problem and
its solution. Note that the solution of the .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.penalized cost function problem
identifies the set of assignments for the particular u.sup.3 values
that minimize the penalized cost function. However, these
assignments are not likely to be optimum for any higher dimensional
problem because they were based on an initial arbitrary set of
u.sub.k values. Therefore, the next step is to determine the
optimum assignments for the related .[.3-dimensional.].
.Iadd.three-dimensional .Iaddend.penalized cost function problem.
There exists a .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.hill shaped function
.PHI.> ##EQU00011## .Iadd..PHI. which is a graph of the minimums
of all penalized cost functions at various sets of assignments in
two dimensions. For the three-dimensional problem, the function
.PHI. can be defined based on the solution to the foregoing
two-dimensional penalized cost function. By using the current
u.sup.3 values and the {u.sup.k|k>3} .Iaddend.values originally
assigned.[.. Then.]., the gradient of the hill-shaped
.[..PHI..].
.[.u.sup.3.].
.Iadd.function .PHI. is determined, which points toward the peak of
the hill. By usin the gradient and u.sup.3 values
.Iaddend.previously selected for the one point on the hill
(corresponding to the minimum of the penalized cost .[..PHI..].
.[.U.sup.3.]. .Iadd.function .PHI.) as a starting point, the
u.sup.3 values can be found for .Iaddend.which the corresponding
problem will result in the peak of the
.PHI..alpha..alpha. ##EQU00012## .Iadd.function .PHI.. The solution
of the corresponding two-dimensional problem is the proper values
for two of the three sets of indices in the three-dimensional
problem. These solution indices can select a subsection of the cost
array which maps to a two-dimensional array. The set of indices
which minimize the two-dimensional assignment problems based on
that array corresponds to the proper assignment of points in the
third dimension. The foregoing "recovery" process was known in the
prior art, but it is modified here to adjust for the sparse matrix
.Iaddend.characteristic. The next task is to recover the solution
of the proper u.sup.4 values for the .[.4-dimensional.].
.Iadd.four-dimensional .Iaddend.problem. The foregoing hill
climbing process will not work again because the foregoing hill
climbing process when required to locate the .[.4-dimensional.].
.Iadd.four-dimensional .Iaddend.u.sup.4 values for the peak of
.PHI..sup.3 exact definition of the .[..PHI..sup.3 case of
.PHI..sup.2.]. .PHI..sup.3 .Iadd.requires the exact definition of
the function .PHI..sup.3 (as was available in the case of
.PHI..sup.2) or an always less than approximation of .PHI..sup.3,
whereas .Iaddend.the iteration can result in a greater than
approximation of .[..PHI..sup.3.].
.times..times..psi..alpha..alpha..PHI..times..times..psi.
##EQU00013## .Iadd..PHI..sup.3. According to the present invention,
another three-dimensional function .PSI. is defined which is a
"less than approximation" the three-dimensional function .PHI. and
which can be defined based on the solution to the two-dimensional
penalized cost function and the previously assigned and determined
u.sup.k values. Next, the gradient of the function .PSI. is found
and hill climbing technique used to determine the u.sup.4 values at
the peak. Each selected u.sup.4 set .Iaddend.results in a new
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.problem and
requires the .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.hill climbing based upon new .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problems. At the peak of the
.[.3-dimensional .PSI..]. .Iadd.three-dimensional function .PSI.,
the solution values are a subset of the values required for the
four-dimensional solution. Recovery processing extends the subset
.Iaddend.to a complete solution. This process is repeated
iteratively until the u.sup.k values that result in a corresponding
solution at the peak of the highest order .[..PSI. .PHI..].
.[..PHI..]..Iadd.functions .PSI. and .PHI. are found. The final
recovery process then results in the solution of k-dimensional
problem. FIG. 5 illustrates process 300 in more detail.
.Iaddend.
.Iadd.I.2. Problem Formulation .Iaddend.
.Iadd.In problem formulation step 310, all data structures for the
subsequent steps are allocated and linked by pointers as required
for execution efficiency. The incoming problem is partitioned as
described in the subsequent section. This partitioning has the
effect of dividing the incoming problem into a set of independent
problems and thus reducing the total workload. The partitioning
process depends only on the actual cost matrix so the partitioning
can and is performed for all levels of the relaxation process.
.Iaddend.
.Iadd.I.2.1. Relaxation and Recovery .Iaddend.
.Iadd.Step 320 begins the Lagrangian Relaxation process for
reducing the M-dimensional problem by selecting all Lagrangian
.Iaddend.Coefficient u.sup.M penalty values initially equal to
zero. The Lagrangian Coefficients associated with the M.sup.th
constraint set are .[.a N.sub.M+1 element.]. .Iadd.an
(N.sub.M+1)-element .Iaddend.vector. The reduction of this
M-dimensional problem in step 322 to a .[.M-1 dimensional.].
.Iadd.(M-1)-dimensional .Iaddend.problem uses the two step process
described above. First, a penalty based on the value of the
respective u.sup.M coefficient is added to the cost function when a
point is assigned to more than one track and then the resultant
cost function is minimized. However, during the first iteration,
the penalty is zero because all u.sup.M values are initially set to
zero. Second, the requirement that no point from any image can be
assigned to more than one track is relaxed for one of the images.
In the extreme this would allow a point from the relaxed image to
.Iadd.be .Iaddend.associate with every track. However, the effect
of the previous penalty would probably mean that such an
association would not minimize the cost. The effect of the two
steps in combination is to remove a hard constraint while adding
the penalty to the cost function so that it operates like a soft
constraint. For step 322 this .[.two step.]. .Iadd.two-step
.Iaddend.process results in the following penalized cost function
problem with k=M:
.times.
.times..PHI..function..ident..times..times..times..PHI..functio-
n..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..function..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00014## Because the constraint on single assignment of
elements from the last image has been eliminated, .[.a M-1
dimensional.]. .Iadd.an (M-1)-dimensional .Iaddend.problem can be
developed by eliminating some of the possible assignments. As shown
in Equations .[.[3.3].]. .Iadd.(1.6).Iaddend., this is done by
selecting the smallest cost element from each of the M.sup.th axis
vectors of the cost matrix. Reduction in this manner yields a new,
.[.lower order.]. .Iadd.lower-order .Iaddend.penalized cost
function defined by Equations .[.[3.3].]. .Iadd.(1.6)
.Iaddend.which has the same minimum cost as does the objective
function defined by .[.[3.2].]. .Iadd.(1.5) .Iaddend.(a) above.
The cost vectors are selected as follows. Define .[.an index array
m.sup.k.sub.i.sub.1 . . . .sub.i.sub.k-.sub.1 and.]. .Iadd.a
.Iaddend.new cost array c.sup.k-1.sub.i.sub.1 .sub.. . . ,i.sub.k-1
by:
.times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times. .times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..noteq..times..times..times..times..times.-
.times..times..times..times..times. ##EQU00015## The resulting
.[.M-1 dimensional.]. .Iadd.(M-1)-dimensional .Iaddend.problem is
(where k=M):
.PHI..function..times..times..times..function..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00016## Assignment Problem .[.[3.1].]. .Iadd.(1.4)
.Iaddend.and Problem Formulation .[.[3.4].]. .Iadd.(1.7)
.Iaddend.differ only in the dimension M vs. M-1, respectively. An
optimal solution to .[.[3.4].]. .Iadd.(1.7) .Iaddend.is also an
optimal solution to equation .[.[3.2].]. .Iadd.(1.5).Iaddend.. This
relationship is the basis for an iterative sequence of reductions
indicated by steps 320-332 through 330-332 and 200-204 in which the
penalized cost function problem is reduced to a .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.problem. As these
formula will be used to describe the processing at all levels, the
lowercase k is used except where specific reference to the top
level is needed. In step 206, the .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.penalized cost function is solved
directly by the prior art Reverse Auction technique. Each execution
of 206 produces two results, the set of z.sup.2 values that
minimize the problem and the cost that results v.sup.2 when these z
values are substituted into the objective function .[.[3.4].].
.Iadd.(1.7) .Iaddend.(a).
In step 208, according to the prior art, solution z.sup.2 values
are substituted into the .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.derivative of the .Iadd.surface .Iaddend..PHI..sub.2. The
result indicates how the value of u.sup.3 should be adjusted so as
to perform the hill climbing function. As was previously described
the objective is to produce a sequence of u.sup.3.sub.i values
which ends when the .[.U.sup.3 .sub.p.]. .Iadd.u.sup.3.sub.p
.Iaddend.value .Iadd.is .Iaddend.in the domain of the peak of the
.Iadd.surface .Iaddend..PHI..Iadd.. The section "Determining
Effective Gradient" describes how new values are computed and how
it is determined that the current u.sub.i.sup.k points
.Iaddend.
.alpha..alpha. ##EQU00017## to the peak of .PHI..sub.2.Iadd.. When
no further adjustment is required the .Iaddend.flow moves to step
214 which will attempt to recover the .[.3-dimensional.].
.Iadd.three-dimensional .Iaddend.solution as previously described.
When further adjustment is required then the flow progresses to
step 212 and the new values of U.sup.k are computed. At the
.[.2-dimensional.]. .Iadd.three-dimensional .Iaddend.level the
method of the prior art could be used for the hill climbing
procedure. However, it is not practical to use this prior art hill
climbing technique to determine the updated Lagrangian Coefficients
u.sup.k or the Max on the next (or any) higher order .Iadd.surface
.Iaddend..PHI..Iadd.because the next (or any) higher dimensional
function .PHI. cannot be defined based on known information.
.Iaddend.
.PHI..times..times..times..times..times..times. ##EQU00018##
.Iadd.Instead, the present invention defines a new function based
on known information which is useful for hill climbing from the
third to the fourth and above dimensions, i.e., u.sup.k
.Iaddend.values which result in z values that are closer to the
proper z values for the highest k-dimension. This hill climbing
process (which is different than that used in the prior art of FIG.
4 for recovering only the .[.three dimensional.].
.Iadd.three-dimensional .Iaddend.solution) is used iteratively at
all lower dimensions k of the M-dimensional problem (including the
.[.3-dimensional.]. .Iadd.three-dimensional .Iaddend.level where it
replaces prior art) even when k is much larger than three. FIG. 6
helps to explain this new hill climbing technique and illustrates
the original k-dimensional cost function v.sub.k of Problem
Formulation .[.[3.1].]. .Iadd.(1.4).Iaddend.. However, the actual
k-dimensional cost surface v.sub.k defined by .[.[3.1].].
.Iadd.(1.4) .Iaddend.(a) comprises scalar values at each point
described by k-dimensional vectors and as such can not be drawn.
Nevertheless, for illustration purposes only, FIG. 6 ignores the
reality of ordering vectors and illustrates a concave function
v.sub.k(z.sup.k) to represent Equation .[.3.1..].
.Iadd.(1.4).Iaddend.. The .[.v.sub.k.]. surface .Iadd.v.sub.k
.Iaddend.is illustrated as being smooth to simplify the explanation
although actually it can be imagined to be terraced. The goal of
the assignment problem is to find the values of z.sup.k; these
values minimize the k-dimensional cost function v.sub.k.
For purposes of explanation, assume that in FIG. 6, k=4 (the
procedure is used for all k.ltoreq.3). This problem is reduced by
two iterations of Lagrangian Relaxation to a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.penalized cost function
.sub..PHI..sub.j.sup.(k-1).sup.i. This cost function, and all other
cost functions described below.Iadd., .Iaddend.are also non-smooth
and continuous but are illustrated in FIG. 6 as smooth for
explanation purposes. Solving the .PHI..sub.j.sup.(k-1).sup.i
problem results in one set of z.sup.2 assignments and the value of
.PHI..sub.(k-1).sub.i at the point u.sup.2.sub.ij. A series of
functions .PHI..sub.j.sup.(k-1).sup.i, . . . ,
.PHI..sub.1.sup.(k-1.sup.i.sup.).sup.i each generated from a
different u.sup.3 are shown. The particular series illustrates the
process of locating the peak of .PHI..sub.(k-1).sub.i. The
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.penalized cost
functions .PHI..sub.j.sup.(k-1).sup.i, . . . ,
.PHI..sub.1.sup.(k-1).sup.i can be solved directly. Each such
solution provides the information required to calculate the next
u.sup.3 value. Each iteration of the hill climbing improves the
selection of u.sup.3 values, i.e., yields .PHI..sup.2 solution is
closer to those at the solution of the .PHI..sup.3 The result of
solving .PHI..sub.1.sup.(k-1).sub.i is values that are on both
.PHI..sub.(k-1).sub.i and .PHI..sub.k .[..PHI..sub.k.]..Iadd.. FIG.
6 illustrates the surface .PHI..sub.k which comprises the minimums
of all k-dimensional penalized cost function surfaces
.Iaddend..PHI..sub.k.Iadd., i.e., if the Problem Formulations (1.5)
.Iaddend.and .[.[3.4].]. .Iadd.(1.7) .Iaddend.were solved at all
possible values of u.sup.k the function .PHI..sub.k .[.k
.PHI..sub.k.]. .Iadd.(u.sub.k) would result. The surface
.PHI..sub.k is always less .Iaddend.than the .[.v.sub.k.]. surface
.Iadd.v.sub.k .Iaddend.except at the peak as described in Equation
.[.3.5.]. .Iadd.(1.8) .Iaddend.and its maximum occurs where the
.[.v.sub.k.]. surface .Iadd.v.sub.k .Iaddend.is minimum. Because
the .[..PHI.k the .PHI..sub.k .PHI..sub.k v.sub.k surface. The.].
.PHI..sub.k .Iadd.represents the minimum of the surface
.PHI..sub.k, any point on the surface .PHI..sub.k can mapped to the
surface v.sub.k. The function .PHI..sub.k provides a lower bound on
the .Iaddend.minimization problem described by .[.[3.1].].
.Iadd.(1.4) .Iaddend.(a). Let z.sup.k be the unknown solution to
Problem Formulation .[.[3.1].]. .Iadd.(1.4) .Iaddend.and note that:
.PHI..sup.k(u.sup.k).ltoreq.v.sub.k(
z.sub.i.sup.k).ltoreq.v.sub.k-1(z.sup.k) .[.[3.5].]. .Iadd.(1.8)
.Iaddend. Consequently, the .[.z.sup.k.]. values .Iadd.z.sup.k
.Iaddend.at the peak of .PHI..sub.k .Iadd.(i.e., the
.Iaddend.greatest lower bound on the cost of the relaxed problem),
can be substituted into the k-dimensional penalized cost function
to determine the proper assignments. Consequently, the present
invention attempts to find the maximum on the .[..PHI..sub.k.].
surface .Iadd..PHI..sub.k.Iaddend.. However, it is not possible to
use the prior art hill climbing to hill climb to the peak of
.PHI..sub.k definition of .[..PHI..sub.k.]. functions
.Iadd..PHI..Iaddend.. As the solution of .PHI..sup.k.sub.i in that
higher order .[.u.]. values .Iadd.of u .Iaddend.are not yet
optimized, its value can be larger than the true peak of .PHI.k
lower bound on v.sub.k and it can not be used to recover the
required solution.
Instead, the present invention defines all auxiliary .[.function.].
.Iadd.functions .Iaddend..PSI..sub.k dimensional penalized cost
function problem, lower order z.sup.k values and u.sup.k values
determined previously by the reduction process. The .PSI..sub.k
.PHI..sub.k and its gradient is used for hill climbing to its peak.
The .[.z.sup.k.]. values .Iadd.z.sup.k .Iaddend.at the peak of the
.[..PSI..sub.k.]. into Problem Formulation .[.[3.2].]..Iadd.(1.5)
.Iaddend.to determine the proper assignments. To define the
.[..PSI..sub.k.]. explicitly makes the function .PHI..sub.k U.sup.k
order sets of Lagrangian Coefficients with the expanded notation:
.[..PHI..sub.k U.sup.k U.sup.k+1 U.sup.k .PSI..].
.Iadd..PHI..sub.k(u.sup.k; u.sup.k+1, . . . ,u.sup.k). Then, a new
set of functions .PSI., .Iaddend.is defined recursively, using the
.[..PHI..sub.k.]. .Iadd..PHI..sub.k's domain: .Iaddend.
.PSI..function..times..times..PHI..function..times..times..times..times..-
times. ##EQU00019## where v.sub.2 is the solution value for the
most recent .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.penalized cost function problem. For k>3
.PSI..function..times..times..PHI..function..times..times..times..PSI..fu-
nction..times..times..times..times..times. ##EQU00020## From the
.[.definition.]. .Iadd.definitions .Iaddend.of .[..PHI..sub.k k.].
.Iadd..PHI..sub.k and v.sub.k (Problem Formulation (1.7)
.Iaddend.compared with Problem Formulation .[.[3.2].].
.Iadd.(1.5).Iaddend.):
.PHI..function..times..function..times..times. ##EQU00021## it
follows that:
.PSI..function..times..times..PHI..function..times..ltoreq..function.
##EQU00022## and with that Equation .[.3.5.]. .Iadd.(1.8)
.Iaddend.is extended to: .PSI..sub.k(u.sup.3, . . . ,
u.sup.k-1;u.sup.k).ltoreq..PHI..sub.k(u.sup.k;u.sup.k+1, . . . ,
u.sup.M.ltoreq.v.sub.k( .sup.k).ltoreq.v.sub.k(u.sup.k) .[.[3.7].].
.Iadd.(1.11).Iaddend. This relationship means that either
.PHI..sub.k .Iadd.or .Iaddend..PSI..sub.k .Iadd.may be used in
.Iaddend.hill climbing to update the Lagrangian Coefficients u.
.PHI..sub.k .Iadd.is .Iaddend.the preferred choice, however it is
only available when the solution to Problem Formulation .[.[3.2].].
.Iadd.(1.5) .Iaddend.is a feasible solution to Problem Formulation
.[.[3.1].]. .Iadd.(1.4) .Iaddend.(as in hill climbing from the
second to third dimension which is why prior art worked). For
simplicity in implementation, the function .PSI..sub.k .Iadd.is
defined so .Iaddend.that it equals .PHI..sub.k .Iadd.when either
function could be used. It is .Iaddend.therefore always used, even
for hill climbing from the second dimension. The use of the
.Iadd.function .Iaddend..PSI. .Iadd.which is based on previously
assigned or determined u values and not higher order u values which
are not yet determined, is an important feature of the present
invention. .Iaddend.
.times..times..times..times..times..times. .times..times.
.times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..PSI..times..times.
##EQU00023##
.Iadd.I.2.2. Determining Effective Gradient .Iaddend.
.Iadd.After the function .PSI..sub.k is defined, the next steps of
hill .Iaddend.climbing/peak detection are to determine the gradient
of the .[..PSI..sub.k.]. .Iadd.function .PSI..sub.k, determine an
increasing portion of the gradient .Iaddend.and then move up the
.[..PSI..sub.k.]. .Iadd.surface .PSI..sub.k in the direction of
this .Iaddend.increasing portion of the gradient. As shown in FIG.
6, any upward step on the .[..PSI..sub.k the .PHI..sub.kj U.sup.k
.alpha. .alpha..]. .Iadd.surface .PSI..sub.k, for example, to the
minimum of the .PHI..sup.k.sub.j will yield a new set of values
u.sup.k (to the "left") that .Iaddend.is closer to the ideal set of
.Iadd.values .Iaddend.u.sup.k .[.values.]. which correspond to the
minimum of the .[.v.sub.k.]. function .Iadd.v.sub.k.Iaddend.. While
it is possible to iteratively step up this .[..PSI..sub.k.].
.Iadd.surface .PSI..sub.k with steps of fixed size .Iaddend.and
then determine if the peak has been reached, the present invention
optimizes this process by determining the single step size from the
starting point at the minimum of .[..PHI..sub.k.sub.i.].
.Iadd..PHI..sup.k.sub.i .Iaddend.that will jump to the peak and
then .[.calculating.]. .Iadd.calculate .Iaddend.the .Iadd.values
.Iaddend.u.sup.k .[.values.]. at the peak. Once the .[.f. "u"s.].
.Iadd.values .Iaddend.u at the peak .[.at.]. .Iadd.of
.Iaddend..PSI..sub.k .Iadd.are .Iaddend.determined, then the
.Iadd.values .Iaddend.u.sup.k .[.values.]. can be substituted into
Problem Formulation .[.[3.2].]. .Iadd.(1.5) .Iaddend.to determine
the proper assignment. (However, in a more realistic example, where
k is much greater than three, then the .Iadd.values
.Iaddend.u.sup.k .[.values.]. at the peak of the .Iadd.function
.Iaddend..PSI. along with the .[.lower order.]. .Iadd.lower-order
values .Iaddend.u.sup.k .[.values.]. and those assigned and yielded
by the reduction steps are used to define the next higher level
.Iadd.function .PSI.. This definition of a higher order function
.PSI. and hill climbing process are repeated iteratively until
.PSI..sub.k, the peak of .PSI..sub.k, and the values u.sup.k at the
peak of .PSI..sub.k are identified.) The following is a description
of how to determine the gradient of each surface .PSI. and how to
determine the single step size to jump to the peak from the
starting point on each surface .PSI.. .Iaddend.
.times..PSI..times..times..PSI..times..times. .times..times.
.times..times.
.times..times..times..PSI..times..times..PSI..times..times..times..times.-
.times..PSI..times. ##EQU00024## .[.determine the gradient of each
.PSI..].
.times..PSI..times..times..times. .times..times..times..PSI.
##EQU00025##
.Iadd.As noted above, each surface .PSI. is non-smooth. Therefore,
if a gradient was taken at a single point, the gradient may not
point toward the peak. Therefore, several gradients (a
.Iaddend."bundle") are determined at several points on the
.Iadd.surface .Iaddend..PSI..sub.k in the region of the starting
point (i.e., minimum of .PHI..sub.k.sub.i) and then averaged.
Statistically, the averaged result should point toward the peak of
the .Iadd.surface .Iaddend..PSI..sub.k.Iadd.. Wolfe's Conjugate
.Iaddend.Subgradient Algorithm (.[.Wolfe75, Wolfe79.]. .Iadd.P.
Wolfe. A method of conjugate subgradients for minimizing
non-differentialbe functions. Mathematical Programming Study,
3:145-173, 1975; P. Wolfe. Finding the nearest point in a polytope.
Mathematical Programming Study, 11:128-149, 1976.Iaddend.) for
minimization was previously known in another environment to
determine a gradient of a non-smooth surface using multiple
subgradients and can be used with modification in the present
invention. .[.Wolfe's algorithm is further described in "A Method
of Conjugate Subgradients for Minimizing Nondifferentiable
Functions" page 147-173 published by Mathematical Programming Study
3 in 1975 and "Finding the Nearest Point in a Polytope" page
128-149 published by Mathematical Programming Study 11 in 1976..].
The modification to Wolfe's algorithm uses the information
generated for .PSI..sub.k U.sup.k3 U.sup.kk-2 U.sup.kk-1 for
calculating the subgradients. The definition of a subgradient v of
.PSI..sub.k U.sup.k3 U.sup.kk subdifferential set defined as:
.delta..PSI..sub.k(u)=}v.epsilon.R.sup.A.sup.-k+1|(.PSI..sub.k(U.sup.k3,
. . . , U.sup.kk-1;U')-.PSI..sub.k(U.sup.k3, . . . ,
U.sup.j-1;U.sup.k)).gtoreq.v.sup.T(U'-U.sup.k)
.A-inverted.U'.epsilon.R.sup.N.sup.k+1} .[.(where v.sup.T is the
transpose of v).]. .Iadd.where v.sup.T is the transpose of v.
.Iaddend.
Next, a subgradient vector is determined from this function. If
z.sup.h is the solution of Problem Formulation .[.[3.2].].
.Iadd.(1.5).Iaddend., then differentiating .PSI..sub.k U.sup.3
U.sup.k-1 U.sup.k U.sup.k.sub.ik evaluating the result with respect
to the current selection matrix z.sup.k yields a subgradient
vector:
.times..times..times..times..times..times. ##EQU00026## The
iterative nature of the solution process at each dimension yields a
set of such subgradients. Except for a situation described below
where the resultant averaged gradient does not point toward the
peak, the most recent set of such subgradients are saved and used
as the "bundle" for the peak finding process for this dimension.
For example, at the .[.k.]. level .Iadd.k .Iaddend.there is a
bundle of subgradients of the .Iadd.surface .Iaddend..PSI..sub.k
.Iadd.near .Iaddend.the minimum of the .[..PHI..sub.k.sub.i.].
surface .Iadd..PHI..sup.k.sub.i .Iaddend.determined as a result of
solving Problem Formulation .[.[3.1].]. .Iadd.(1.4) .Iaddend.at all
lower levels. This bundle can be averaged to approximate the
gradient. Alternately, the previous bundle can be discarded so as
to use the new value to initiate a new bundle. This choice provides
a way to adjust the process to differing classes of problems, i.e.,
when data is being derived from two sensors and the relaxation
proceeds from data derived from one sensor to the other then the
prior relaxation data for the first sensor could be detrimental to
performance on the second sensors data.
I.2.3. Step Size and Termination Criterion
After the average gradient of the .Iadd.surface
.Iaddend..PSI..sub.k .Iadd.is .Iaddend.determined, the next step is
to determine a single step that will jump to the peak of the
.Iadd.surface .Iaddend..PSI..sub.k.Iadd.. The basic strategy
.Iaddend.is .[.to.]. first .Iadd.to .Iaddend.specify an arbitrary
step size, and then calculate the value of .PSI..sub.k .Iadd.at
this step size in the direction of the .Iaddend.gradient. If the
.PSI..sub.k .Iadd.value of .PSI..sub.k is larger than the previous
one, .Iaddend.this probably means that the step has not yet reached
the peak. Consequently, the step size is doubled and a new
.[..PSI..sub.k.]. value .Iadd.of .PSI..sub.k is .Iaddend.determined
and compared to the previous one. This process is repeated until
the new .Iadd.value of .Iaddend..PSI..sub.k .Iadd.is less than the
.Iaddend.previous one. At that time, the step has gone too far and
crossed over the peak. Consequently, the last doubling is
rolled-back and that step size is used for the iteration. If the
initial estimated .PSI..sub.k .Iadd.is less than previous value
then the .Iaddend.step size is decreased by 50%. If this still
results in a smaller .Iadd.value of .Iaddend..PSI..sub.k, then the
last step is rolled back and the previous step size is decreased by
25%. The following is a more detailed description of this
process.
With a suitable bundle of subgradients determined as just
described.[.,.]. .Iadd.. .Iaddend.Wolfe's algorithm can be used to
determine the effective subgradient .[.(d).]. .Iadd.d .Iaddend.and
the .[.Upgraded.]. .Iadd.upgraded .Iaddend.value U.sup.k.sub.j+1.
From the previous iteration, or from an initial condition, there
exists a step length value .[.(t).]. .Iadd.t.Iaddend.. The value,
u.sub.+=u.sup.k.sub.j+td is calculated as an estimate of
u.sup.k.sub.j+1. To determine if the current step size is valid the
we evaluate .PSI..sub.k u.sup.3 u.sup.k-2u.sub.+ If the result
represents an improvement then we double the step size. Otherwise
we halve the step size. In either case a new u.sub.+ is calculated.
The doubling or halving continues until the step becomes too large
to improve the result, or until it becomes small enough to not
degrade the result. The resulting suitable step size is saved with
d as part of the subgradient bundle. The last acceptable u.sub.+ is
assigned to u.sup.k.sub.j+1.
Three distinct .[.criterion.]. .Iadd.criteria .Iaddend.are used to
determine when u.sup.k.sub.j is close enough to .sup.k: 1. The
Wolfe's algorithm criterion of d=0 given that the test has been
repeated with the bundle containing only the most recent
subgradient. 2. The difference between the lower bound .PHI..sub.k
u.sup.k upper bound v.sub.k (z.sup.k, u.sup.kk) being less than a
preset relative threshold. (Six percent was found to be an
effective threshold for radar tracking problems.) 3. An iteration
count being exceeded.
The use of limits on the iteration are particularly effective for
iterations at the level 3 through .Iadd.(.Iaddend.n-1
.[.levels,.]..Iadd.) .Iaddend.as these iterations will be repeated
so as to resolve higher order coefficient sets. With limited
iterations the process is in general robust enough to improve the
estimate of upper order Lagrangian Coefficients. By limiting the
iteration counts then the total processing time for the algorithm
becomes deterministic. That characteristic means the process can be
effective for real time problems such as radar, where the results
of the last scan of the environment must be processed prior to the
next scan being received.
I.2.4. Solution Recovery
The following process determines if the u.sup.k values at what is
believed to be the peak of the .Iadd.function .Iaddend..PSI..sub.k
.Iadd.adequately .Iaddend.approximate the u.sup.k values at the
peak of the corresponding .[..PHI..sub.k.]. function
.Iadd..PHI..sub.k.Iaddend.. This is done by determining if the
corresponding penalized cost function is minimized. Thus, the
u.sup.k values at the peak of the .Iadd.function
.Iaddend..PSI..sub.k .Iadd.are first substituted into Problem
.Iaddend.Formulation .[.[3.2].]. .Iadd.(1.5) .Iaddend.to determine
a set of z assignments for k-1 points. During the foregoing
reduction process, Problem Formulation .[.[3.4].]. .Iadd.(1.7)
.Iaddend.yielded a tentative list of k selected z points that can
be described by their indices as: {(i.sup.j.sub.i. . .
i.sup.j.sub.k-1)}.sup.N.sup.0 where N.sub.0 is the number of cost
elements selected into the solution. One possible solution of
Problem Formulation .[.[3.1].]. .Iadd.(1.4) .Iaddend.is the
solution of Problem Formulation .[.[3.2].]. .Iadd.(1.5)
.Iaddend.which is described as {(i.sup.j.sub.i. . .
i.sup.j.sub.k-1m.sup.k.sub.i.sub.1. . . i.sub.k-1)}.sup.N.sub.j=1
.[.[3.3].]. .Iadd.(1.6).Iaddend.. If this solution satisfies the
.[.k-th.]. .Iadd.k.sup.th .Iaddend.constraint set.Iadd.,
.Iaddend.then it is the optimal solution.
However, if the solution for Problem Formulation .[.[3.2].].
.Iadd.(1.5) .Iaddend.is not feasible (decision 355), then the
following adjustment process determines if a solution exists which
satisfies the .[.k-th.]. .Iadd.k.sup.th .Iaddend.constraint while
retaining the assignments made in solving Problem Formulation
.[.[3.4].]. .Iadd.(1.7).Iaddend.. To do this, a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.cost matrix is defined based upon
all observations from the .[.k-th.]. .Iadd.k.sup.th .Iaddend.set
which could be used to extend the relaxed solution.[...]..Iadd.:
.Iaddend.
.times..times..times..times..times..times..times..times..times.
##EQU00027## If the resulting .[.2-dimensional.].
.Iadd.two-dimesional .Iaddend.assignment problem,
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.di-elect cons..times..times..times..times..times. ##EQU00028## has
a feasible solution.Iadd., .Iaddend.then the indices of that
solution map to the solution of Problem Formulation .[.[3.1].].
.Iadd.(1.4) .Iaddend.for the k-dimensional problem. The first index
in each resultant solution entry is the pointer back to an element
of the {(i.sup.j.sub.i, . . . i.sup.j.sub.k-im.sup.k.sub.i.sub.1 .
. . i.sub.k-1)}.sup.N.sup.0.sub.j=1 list. That element supplies the
first k-1 indices of the solution. The second index of the solution
to the recovery problem is the k.sup.th index of the solution.
Together these indexes specify the values of z.sup.k that solve
Problem Formulation .[.[3.1].]. .Iadd.(1.4) .Iaddend.at the
k.sup.th level.
If Problem Formulation .[.[3.9].]. .Iadd.(1.13) .Iaddend.does not
have a feasible solution.Iadd., .Iaddend.then the value of uki
which was thought to represent u.sup.k.sub.p is not representative
of the actual peak and further iteration at the k.sup.th level is
required. This decision represent the branch path from step 214 and
equivalent steps.
I.3. Partitioning
A partitioning process is used to divide the cost matrix that
results from the Scoring step 154 into as many independent problems
as possible prior to beginning the relaxation solution. The
partitioning process is included with the Problem Formulation Step
310. The result of partitioning is a set of problems to be solved,
i.e., there will be p.sub.1 problems that consist of a single
hypothesis, p.sub.2 problems that consist of two hypothesis, etc.
Each such problem is a group in that one or more observations or
tracks are shared between members of the group.
In partitioning to groups no consideration is given to the actual
cost values. The analysis depends strictly on the basis of two or
more cost elements sharing the same specific axis of the cost
matrix. In a .[.2-dimensional.]. .Iadd.two-dimensional
.Iaddend.case two cost elements must be in the same group if they
share a row or a column. If the two elements are in the same row,
then each other cost element that is also in the same row, as well
as any cost elements that are in columns occupied by members of the
row must be included in the group. The process continues
recursively. In literature it is referred to as "Constructing a
Spanning Forest." The k-dimensional case is analogous to the
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.case. The
specific method we have incorporated is a depth first search,
presented by Aho, Hopecraft, and Ullman .Iadd.(A. V. Aho.Iaddend.,
.[.in "Design.]. .Iadd.J. E. Hopcroft, and J. D. Ullman. Design
.Iaddend.and Analysis of Computer Algorithms.[.." section 5.2,
published by Addison-Westley.]. .Iadd.Addison-Wesley.Iaddend.,
Mass..Iadd., 1974).Iaddend..
The result of partitioning at level M is the set of problems
described as {P.sub.ij|i=1, . . . , p.sub.j and j=1, . . . N},
where N is the total number of hypothesis. The problems labeled
.[.{P.sub.ij|i=1, . . . , p.sub.j and j=1, . . . , N}.].
.Iadd.{P.sub.i1|=1, . . . , p.sub.1}.Iaddend.are the cases where
.[.their.]. .Iadd.there .Iaddend.is only one choice for the next
observation at each scan and that observation could be used for no
other track, i.e., it is a single isolated track. The hypothesis
must be in included in the solution set and no further processing
is required.
As .[.hypothesis.]. .Iadd.hypotheses .Iaddend.are constructed the
first element is used to refer to the track .[.id.].
.Iadd.ID.Iaddend.. Any problem in the partitioned set which does
not have shared costs in the first scan represent a case where a
track could be extended in several ways but none of the ways share
an observation with any other track. The required solution
hypothesis for this case is the particular hypothesis with the
maximum likelihood. For this case all likelihoods are determined as
was described in Scoring and the maximum is selected.
In addition to partitioning at the .[.M.]. level .Iadd.M.Iaddend.,
partitioning is applied to each subsequent level M-1, . . . , 2.
For each problem that was not eliminated by .[.either by.]. the
prior selection.Iadd., .Iaddend.partitioning is repeated, ignoring
observations that are shared in the last set of observations.
Partitioning recognizes that relaxation will eliminate the last
constraint set and thus partitioning is feasible for the lower
level problems that will result from relaxation. This process is
repeated for all levels down to k=3. The full set of partitionings
can be performed in the Problem Formulation Step 310, prior to
initiating the actual relaxation steps. The actual
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.solver used in
step 206 includes an equivalent process so no advantage would be
gained by partitioning at the k=2 level.
There are two possible solution methods for the remaining problems.
"Branch and Bound" as was previously described, or the relaxation
method that this invention describes. If any of the partitions have
5-10 possible tracks and less than 50 to 20 hypotheses, then the
prior art "Branch and Bound" algorithm generally executes faster
than does the relaxation due to its reduced level of startup
overhead. The "Branch and Bound" algorithm is executed against all
remaining M level problems that satisfy the size constraint. For
the remainder the Relaxation algorithm is used. The scheduling done
in Problem Formulation allows each Problem Formulation .[.[3.2].].
.Iadd.(1.5) .Iaddend.cost matrix resulting from the first step of
relaxation to be partitioned. The resulting partitions can be
solved by any of the four methods.[.,.]..Iadd.: .Iaddend.isolated
track direct inclusion, isolated track tree evaluation, small group
"Branch and Bound" or an additional stage of relaxation as has been
fully described.
The partitioning after each Level of Lagrangian Relaxation is
effective because when a problem is relaxed, the constraint that
requires each point to be assigned to only one track is eliminated
(for one image at a time) . Therefore, two tracks previously linked
by contending for one point will be unlinked by the relaxation
which permits the point to be assigned to both tracks. The
fruitfulness of partitioning increases for lower and lower levels
of relaxation.
The following is a more detailed description of the partitioning
method. Its application at all but the .[.M.]. level .Iadd.M
.Iaddend.depends upon the relaxation process described in this
invention. The recursive partitioning is therefore a distinct part
of this invention. The advantage of this method is greatly enhanced
by the sparse cost matrix resulting from tracking problems. However
the sparse nature of the problem requires special storage and
search techniques.
A hypothesis is the combination of the cost element c.sub.n the
selection variable z.sub.n and all observations that made up the
potential track extension. It can be written as,
.[.h.sub.n={c.sub.n, z.sub.n, {o.sub.n.sub.k=oki|k=1, , . . . ,
Mi.epsilon.{1, . . . , Nk}}}.]. .Iadd.h.sub.n={c.sub.n, z.sub.n,
{o.sub.n.sub.k=o.sub.k.sub.i|k=1, . . . , M.sub.i.epsilon.{1, . . .
, N.sub.k}}}, .Iaddend.i.e., cost, selection variable and
observations in the hypothesis. .[.n.epsilon.{1, . . . , N}.].
.Iadd.Here n.epsilon.{1, . . . , N}, .Iaddend.where N is the total
number of hypotheses in the problem. While the cost and assignment
matrices were previously referenced. these matrices are not
effective storage mechanisms for tracking applications. Instead the
list of all hypothesis and sets of lists for each dimension that
reference the hypothesis set are stored. The hypothetical set in
list form is: {h.sub.n}.sup.n+N.sub.n=1
For each dimension k=1M there exists a set of lists, with each list
element being a pointer to a particular hypothesis:
L.sub.k.sub.i={p.sub.k.sub.j|k.sub.j=1, . . . ,
N.sub.k.sub.i}.sup.i=N.sup.k.sup.,k=M.sub.i=1,k=1 where
N.sub.k.sub.i is .Iadd.a .Iaddend.number of hypothesis containing
the .[.i-th.]. .Iadd.i.sup.th .Iaddend.observation from scan k.
This structure is a multiply linked list in that any observation is
associated with a set of pointer to all hypothesis it participates
in, and any hypothesis has a set of pointers to all observations
that formed it. (These pointers can be implemented as true pointers
or indices depending upon the particular programming language
utilized.)
Given this description of the matrix storage technique then the
partitioning technique is as follows: Mark the first list
L.sub.k.sub.i and follow out all pointers in that list to the
indicated hypothesis h.sub.pk.sub.i for i=1, . . . , .[.N.sub.ki.].
.Iadd.N.sub.k.sub.i.Iaddend.. Mark all located hypothesis, and for
each follow pointers back the particular .[.L.sub.ok.].
.Iadd.L.sub.k .Iaddend.for k=1, . . . .Iadd., .Iaddend.M. Those L's
if not previously marked get marked and also followed out to
hypothesis elements and back to L's. When all such L's or h's being
located are marked, then an isolated sub-problem has been
identified. All marked elements can be removed from the lists and
stored as a unique problem to be solved. The partitioning problem
then continues by again starting at the first residual .[.L.]. set
.Iadd.L.Iaddend.. When none remain, the original problem is
completely partitioned.
Isolated problems can result from one source track having multiple
possible extensions or from a set of source tracks contending for
some observations. Because one of the indices of k (in our
implementation it is k=1) .[.indicate.]. .Iadd.indicates
.Iaddend.the source track then it is possible to categorize the two
problem types by observing if the isolated problem includes more
than one L-list from the index level associated with tracks.
I.4. Subsequent Track Prediction
As noted above, after the points are assigned to the respective
tracks, some action is taken such as controlling take-offs and
landings to avoid collision, advising airborne aircraft to change
course, warning airborne aircraft of an adjacent aircraft in a
commercial environment, or aiming and firing an anti-aircraft (or
anti-missile) missile, rocket or projectile, or taking evasive
action in a military environment. Also, the tracking can be used to
position a robot to work on an object. For some or all of these
applications, usually the tracks which have just been identified
are extended or extrapolated to predict a subsequent position of
the aircraft or other object. The extrapolation can be done in a
variety of prior art ways: for example, straight line extensions of
the most likely track extension hypothesis, parametric quadratic
extension of the x versus time, y versus time, etc. functions that
are the result of the filtering process described earlier, least
square path fitting or Kalman filtering of just the selected
hypothesis.
The process of gating, filtering, and gate generation as they were
previously described require that a curve be fit through
observations such that fit of observations to the likely path can
be scored and further so that the hypothetical target future
location can be calculated for the next stage of gating. In the
implementation existing, quadratic functions have been fit through
the measurements that occur within the window. A set of quadratics
is used, one per sensor measurement. To calculate intercept
locations these quadratics can be converted directly into path
functions. Intercept times are then calculated by prior methods
based upon simultaneous solution of path equations.
The use of the fitted quadratics is not as precise as more
conventional filters like the Kalman Filter. They are however much
faster and sufficiently accurate for the relative scores required
for the Assignment problem. When better location prediction is
required then the assignment problem is executed to select the
solution hypothesis and based upon the observations in those
hypothesis the more extensive Kalman filter is executed. The result
is tremendous computation savings when compared with the Kalman
Filter being run on all hypothetical tracks.
Based on the foregoing, apparatus and methods have been disclosed
for tracking objects. However, numerous modifications and
substitutions can be made without deviating from the scope of the
present invention. For example, the foregoing .Iadd.functions
.Iaddend..PSI. .Iadd.can also be defined as recursive approximation
problem in which several values of higher order .Iaddend.u.sup.k
values are used to eliminate the higher than approximation
characteristic of the .[..PHI..sub.k.].
.PSI..PHI. ##EQU00029## .Iadd.function .PHI..sub.k. The hill
climbining of the function .PSI. can be implemented by a high order
hill climbing using the enhanced function .PHI.. Although the
result would not .Iaddend.be as efficient it seems likely that the
method would converge. Therefore, the invention has been disclosed
by way of example and not limitation, and reference should be made
to the following claims to determine the scope of the present
invention.
II. AN ALTERNATIVE EMBODIMENT OF THE MULTI-DIMENSIONAL ASSIGNMENT
SOLVING PROCESS
The following description provides an alternative second embodiment
of the multi-dimensional assignment solving process of section I.1.
However, in order to clearly describe the present alternative
embodiment, further discussion is first presented regarding the
data assignment problem of partitioning measurements into tracks
and false alarms and, in particular, the representation of
multi-dimensional assignment problems.
II.1. Formulation of the Assignment Problem
The goal of this section is to explain the formulation of the data
association problems, and more particularly multi-dimensional
assignment problems, that govern large classes of problems in
centralized or hybrid centralized-sensor level
multisensor/multitarget tracking. The presentation is brief;
technical details are presented for both track initiation and
maintenance in .Iadd.(.Iaddend.A. B. Poore.[.,
Multi-dimensional.]..Iadd.. Multidimensional .Iaddend.assignment
formulation of data association problems arising from multitarget
tracking and multisensor data fusion.[.,.]..Iadd..
.Iaddend.Computational Optimization and Applications, 3 .[.(1994),
pp..]..Iadd.: .Iaddend.27-57, for .[.nonmaneuvering.].
.Iadd.non-maneuvering .Iaddend.targets and in .[.Ingmar J. Cox,
Pierre Hansen, and Beta Julesz, eds., Partitioning Data Sets,.].
.Iadd.(A. B. Poore. Multidimensional assignments and multitarget
tracking: Partitioning data sets. In P. Hansen, I. J. Cox, and B.
Julesz, editors, .Iaddend.DIMACS Series in Discrete Mathematics and
Theoretical Computer Science.Iadd., volume 19, pages 169-198,
Providence, R.I., 1995.Iaddend.. American Mathematical Society.[.,
Providence, R.I., v. 19, Feb. 1995, pp. 169-198.]..Iadd.)
.Iaddend.for maneuvering targets. Note that the present formulation
can also be modified to include target features (e.g., size and
type) into the scoring process 154.
The data assignment problems for multisensor and multitarget
tracking considered here are generally posed as that of maximizing
the posterior probability of the surveillance region (given the
data) according to
.times..function..GAMMA..gamma..function..GAMMA..gamma..gamma..times..di--
elect cons..GAMMA..times..times. ##EQU00030## where Z.sup.M
represents M data sets, .gamma. .Iadd.is a partition of indices
.Iaddend.of the data (and thus induces a partition of the data),
.GAMMA.* .Iadd.is .Iaddend.the finite collection of all such
partitions, .GAMMA. .[.*.gamma..sup.0.]. .Iadd..GAMMA. is a
discrete random element defined on .GAMMA.*, .gamma..sup.0 is a
reference partition, and .Iaddend.P(.GAMMA.=.gamma.|Z.sup.M) is the
posterior probability of a partition .gamma. true given the data
Z.sup.M. The term partition is defined below.
Consider M observation sets Z(k) (k=1, . . . , N) each of N.sub.k
observations {z.sup.k.sub.i.sub.k}.sup.N.sup.k.sub.i.sub.k.sub.-1,
and let Z.sup.M denote the cumulative data set defined by
z(k)={z.sup.k.sub.i.sub.k}.sup.N.sup.k.sub.i.sub.k.sub.-1 and
Z.sup.M={Z(1), . . . , Z(M)}, [5.2] respectively. In multisensor
data fusion and multitarget tracking the data sets Z(k) may
represent different classes of objects, and each data set can arise
from different sensors. For track initiation the objects are
measurements that must be partitioned into tracks and false alarms.
In the formulation of track extensions a moving window over time of
observations sets is used. The observation sets will be
measurements which are: (a) assigned to existing tracks, (b)
designated as false measurements, or (c) used for initiating
tracks. However, note that alternative data objects instead of
observation sets may also be fused such as in sensor level tracking
wherein each sensor forms tracks from its own measurements and then
the tracks from the sensors are fused in a central location. Note
that, as one skilled in the art will appreciate, both embodiments
of the present invention may be used for this type of data
fusion.
The data assignment problem considered presently is represented as
a case of set partitioning defined in the following way. First, for
notational convenience in representing tracks, a zero index is
added to each of the index sets in .[.[5.2].].
.Iadd.(2.2).Iaddend., a gap filler Z.sup.k.sub.o is added to each
of the data sets Z(k) in .[.[5.2].]. .Iadd.(2.2).Iaddend., and a
"track of data" is defined as (z.sup.1.sub.i.sub.1, . . . ,
z.sup.N.sub.i.sub.N).Iadd., .Iaddend.where i.sub.k .[.and
z.sup.k.sub.i.sub.k.]. can now assume the values of 0 .[.and
z.sup.k.sub.o, respectively.].. A partition of the data will refer
to a collection of tracks of data or track extensions wherein each
observation occurs exactly once in one of the tracks of data and
such that all data is used up. Note that the occurrence of the gap
filler is unrestricted. The gap filler z.sup.k.sub.o serves several
purposes in the representation of missing data, false observations,
initiating tracks, and terminating tracks. Note that a "reference
partition" may be defined which is a partition in which all
observations are declared to be false.
Next under appropriate independence assumptions it can be shown
that:
.function..GAMMA..gamma..function..GAMMA..gamma..ident..gamma..ident..tim-
es..times..times..di-elect
cons..gamma..times..times..times..times..times..times..times.
##EQU00031## where L.sup.k.sub.i.sub.1.sub.. . . i.sub.M is a
likelihood ratio containing probabilities for detection, maneuvers,
and termination as well as probability density functions for
measurement errors, track initiation and termination. Then if
c.sup.k.sub.i.sub.1 . . . .sup.i.sub.N=-lnL.sup.k.sub.i1 . . .
iM,
.function..GAMMA..gamma..function..GAMMA..gamma..times..di-elect
cons..gamma..times..times..times..times..times..times..times.
##EQU00032## .[.Using [5.3] and the zero-one variable
z.sup.k.sub.i.sub.1 . . . .sub.i.sub.N=1 if (i.sub.1, . . . ,
i.sub.M).].
.di-elect
cons..times..gamma..times..PHI..times..gamma..PHI..times..gamma-
..times..times..PHI..times..times..times..times..times..times..times..time-
s..times..times..times..times..gamma..times..times..times..times..times..t-
imes..times..times.
.times..PHI..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..di-elect
cons..times..times..times..times..times..times..times.
##EQU00033##
.Iadd.Using (2.3) and the zero-one variable z.sub.i.sub.1.sub.. . .
i.sub.N.sup.k=1 if (i.sub.1, . . . , i.sub.M).epsilon..gamma. and 0
otherwise, one can then write the problem (2.1) as the following
M-dimensional assignment problem (as also presented in section
Problem Formulaion (1.4) with k=M): .Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00034##
.Iadd.where c.sub.o . . . o is arbitrarily defined to be zero.
Here, each group of sums in the constraints .Iaddend..[.[5.5]
(b)-[5.5] (e).]. .Iadd.(2.5) (b)-(2.5) (e) represents the fact that
each non-gap filler observation occurs exactly once in a "track of
data." One can modify this formulation to include multiassignments
of one, some, or all the actual observations. The assignment
problem .Iaddend..[.[5.5].]. .Iadd.(2.5) is changed accordingly.
For example, if z.sup.k.sub.i.sub.k is to be assigned no more than,
exactly, or no less than n.sup.k.sub.i.sub.k times, then the "=1"
in the appropriate one of the constraints [5.5(b)-[5.5] (d) is
changed to ".ltoreq., =, .gtoreq. n.sup.k.sub.i.sub.k,"
respectively.Iaddend..
Expressions for the likelihood ratios L.sup.k.sub.i.sub.1.sub.. . .
i.sub.M .[.such as [5.5].]. .Iadd.appearing in the costs
c.sub.i.sub.1.sub.. . . i.sub.M=-1n(L.sub.i.sub.1.sub.. . .
i.sub.M) .Iaddend.are .[.well known.]. .Iadd.well-known.Iaddend..
In particular, discussions of these ratios may be found in.[.:.].
.Iadd.(.Iaddend.A.B. Poore.[., Multi-dimensional.]..Iadd..
Multi-dimensinal .Iaddend.assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion, Computational Optimization and
Applications.[., 3 (1994), pp. 27-57; and Ingemar J. Cox, Pierre
Hansen, and Beta Julesz, eds., Partitioning Data Sets,.]..Iadd.,
3:27-57, 1994; A. B. Poore. Multidimensional assignments and
multitarget tracking: Partitioning data sets. In P. Hansen, I. J.
Cox, B. Julesz, editors, .Iaddend.DIMACS Series in Discrete
Mathematics and Theoretical Computer Science.Iadd., volume 19,
pages 169-198, Providence, R.I., 1995.Iaddend.. American
Mathematical Society.[., Providence, R.I. v. 19, 1995. pp.
169-198..]..Iadd.) .Iaddend.Furthermore, the likelihood ratios are
easily modified to include target features and to account for
different sensor types. Also note that in track initiation, the M
observation sets provide observations from M sensors, possibly all
the same. Additionally note that for track maintenance, a sliding
window of M observation sets and one data set containing
established tracks may be used. In this latter case, the
formulation is the same as above except that the dimension of the
assignment problem is now M+1.
II.2. Overview of the New Lagrangian Relaxation Algorithms
Having formulated an M-dimensional assignment problem .[.[5.5].].
.Iadd.(2.5).Iaddend., we now turn to a description of the
Lagrangian relaxation algorithm. Subsection II.2.1 below presents
many of the relaxation properties associated with the relaxation of
an n-dimensional assignment problem to an m-dimensional one via a
Lagrangian relaxation of n-m sets of constraints wherein
.[.m<n<M.]. .Iadd.m<n.ltoreq.M .Iaddend.and preferably in
the present invention embodiment n-m>1. Although any n-m
constraint sets can be relaxed, the description here is based on
relaxing the last n-m sets of constraints and keeping the first m
sets. Given either an optimal or suboptimal solution of the relaxed
problem, a technique for recovering a feasible solution of the
n-dimensional problem is presented in subsection II.2.2 below and
an overview of the Lagrangian relaxation algorithm is given in
subsection II.2.3.
The following notation will be used throughout the remainder of the
work. Let M be an integer such that M.gtoreq.3 and let
n.epsilon.{3, . . . ,M}. The n-dimensional assignment problem
is
.times..times..times..times. ##EQU00035##
.times..times..times..function..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..d-
i-elect cons..times..times..times..times..times..times..times.
##EQU00035.2## To ensure that a feasible solution of .[.[6.1].].
.Iadd.(2.6) .Iaddend.always exists, all variables with exactly one
nonzero index (i.e., variables of the form z.sup.n.sub.o . . .
oi.sub.k.sub.o . . . o for i.sub.k.noteq.0) are assumed free to be
assigned and the corresponding cost coefficients are
well-defined.
II 2.1. The Lagrangian Relaxed Assignment Problem
The n-dimensional assignment problem .[.[6.1].]. .Iadd.(2.6)
.Iaddend.has n sets of constraints. A (N.sub.k+1)-dimensional
multiplier vector associated with the .[.k-th.]. .Iadd.k.sup.th
.Iaddend.constraint set will be denoted by u.sup.k=(u.sup.k.sub.0,
u.sup.k.sub.1, . . . , u.sup.kM.sub.k).sup.T with u.sup.k.sub.0=0
and k=1, . . . , n. The n-dimensional assignment problem
.[.[6.1].]. .Iadd.(2.6) .Iaddend.is relaxed to an m-dimensional
assignment problem by incorporating n-m of the n sets of
constraints into the objective function .[.[6.1].]. .Iadd.(2.6)
.Iaddend.(a). Although any constraint set can be relaxed, the
description of the relaxation procedure for .[.[6.1].]. .Iadd.(2.6)
.Iaddend.will be based on the relaxation of the last n-m sets of
constraints. The relaxed problem is
.PHI..function..times..ident..times..times..PHI..function..times..ident..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..function..times..times..times..times..times..times-
..times..times..times..ident..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00036## wherein we have added the multiplier
u.sup.k.sub.0.sub.k=0 for notational convenience. Thus, the
multiplier R with u.sup.k.sub.0=0 is fixed.
An optimal (or suboptimal) solution of .[.[6.2].]. .Iadd.(2.7)
.Iaddend.can be constructed from that of an m-dimensional
assignment problem. To show this, def ine f or each (i.sub.1, . . .
, i.sub.m) an index (j.sub.m+1, . . . , j.sub.n)=(j.sub.m+1(i.sub.1
, . . . , im), . . . , j.sub.n(i.sub.1, . . . , i.sub.m)) and a new
cost function c.sup.m.sub.i.sub.1. . . .sub.i.sub.m by
.times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..noteq..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times. .times. .times. ##EQU00037##
(If (j.sub.m+1, . . . , j.sub.nn) is not unique, choose the
smallest such j.sub.m+1, amongst those (n-m)-tuples with the same
j.sub.m+1 choose the smallest j.sub.m+2, etc., so that (j.sub.m+1,
. . . , j.sub.n) is uniquely defined.) Using the cost coefficients
defined in this way, the following m-dimensional assignment problem
is .[.obtaIned.]. .Iadd.obtained.Iaddend.:
.PHI..function..times..times..times..PHI..function..times..ident..functio-
n..ident..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00038## As an aside, observe that any feasible solution
z.sup.n of .[.[6.1].]. .Iadd.(2.6) .Iaddend.yields a feasible
solution z.sup.m of .[.[6.4].]. .Iadd.(2.9) .Iaddend.via the
construction
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times. ##EQU00039## Thus, the
mn-dimensional assignment problem .[.6.4].]. .Iadd.(2.9)
.Iaddend.has at least as many feasible solutions of the constraints
as the original problem .[.[6.1].]. .Iadd.(2.6).Iaddend..
The following Fact A.2 has been shown to be true. It states that an
optimal solution of Problem Formulation .[.[6.2].]. .Iadd.(2.7)
.Iaddend.can be computed from that of Problem Formulation
.[.[6.4].]. .Iadd.(2.9).Iaddend.. Furthermore, the converse to this
fact is provided in Fact A.3. Moreover, if the solution of either
of these two problems .[.[6.2].]. .Iadd.(2.9) .Iaddend.or
.[.[6.4].]. .Iadd.(2.0) .Iaddend.is .epsilon.-optimal (i.e., the
objective function associated with the suboptimal solution is
within ".epsilon.", of the objective function associated with the
optimal solution), then so is the other.
Fact A.2. Let w.sup.m be a feasible solution to problem .[.[6.4].].
.Iadd.(2.9) .Iaddend.and define w.sup.n by
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..noteq..times..times..times..times..times..times..times.-
.times..noteq..times..times..times..times..times..noteq..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..ltoreq..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times.>.times..times. ##EQU00040## Then w.sup.n is a
feasible solution of the Lagrangian relaxed problem .[.[6.2].].
.Iadd.(2.7) .Iaddend.and .PHI..sub.mn.gamma..sup.n m+1 n
{circumflex over (.PHI.)}.sub.minw.sup.m;u.sup.m+1, . . . ,
u.sup.n)-.SIGMA..sup.n.sub.k=m+1.SIGMA..sup.Mk.sub.i.sub.k=ou.sup.k.sub.i-
.sub.k If, in addition, w.sup.m is optimal for .[.[6.4].].
.Iadd.(2.9).Iaddend., then w.sup.n is an optimal solution of
.[.[6.2].]. .Iadd.(2.7) .Iaddend.and .PHI..sub.mn.gamma..sup.m+1 n
{circumflex over (.PHI.)}.sub.mn(u.sup.m+1, . . . ,
u.sup.n)-.SIGMA..sup.n.sub.k=m+1.SIGMA..sup.Mk.sub.i.sub.k-ou.sup.k.sub.i-
.sub.k
With the exception of one equality being converted to an
inequality, the following Fact is a converse of Fact A.2 and has
also been shown to be true.
Fact A.3. Let w.sup.n be a feasible solution to problem .[.[6.2].].
.Iadd.(2.7) .Iaddend.and define W.sup.m by
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..noteq..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times.>.times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..ltoreq..times..times..times..times..-
times..times..times..times. ##EQU00041## Then w.sup.m is a feasible
solution of the problem .[.[6.4].]. .Iadd.(2.9) .Iaddend.and
.PHI..function..times..gtoreq..PHI..function..times..times..times.
##EQU00042## If, in addition, w.sup.n is optimal for .[.[6.2].].
.Iadd.(2.7).Iaddend., then w.sup.m is an optimal solution of
.[.[6.4].]. .Iadd.(2.9).Iaddend., .PHI..sub.mn.gamma..sup.n m+1
{circumflex over (.PHI.)}.sub.mn(w.sup.m;u.sup.m+1, . . . ,
u.sup.n)-.SIGMA..sup.n.sub.k=m+1.SIGMA..sup.Mk.sub.i.sub.k-ou.sup.k.sub.i-
.sub.k, and .PHI..sub.mn.gamma..sup.n+1 n {circumflex over
(.PHI.)}.sub.mn(u.sup.m+1, . . . ,
u.sup.n)-.SIGMA..sup.n.sub.k=m+1.SIGMA..sup.M.sup.k.sub.i.sub.k-ou.sup.k.-
sub.i.sub.k.
II.2.2. The Recovery Procedure
The next objective is to explain a recovery procedure or method for
recovering a solution to the n-dimensional problem of .[.[6.1].].
.Iadd.(2.6) .Iaddend.from a relaxed problem having potentially
substantially fewer dimensions than .[.[6.1].].
.Iadd.(2.6).Iaddend.. Note that this aspect of the alternative
embodiment of the present invention is substantially different from
the method disclosed in the first embodiment of the
multi-dimensional assignment solving process of section I.1 of this
specification. Further, this alternative embodiment provides
substantial benefits in terms of computational efficiency and
accuracy over the first embodiment, as will be discussed. Thus,
given a feasible (optimal or suboptimal) solution w.sup.m of
.[.[6.4].]. .Iadd.(2.9) .Iaddend.(or w.sup.n if Problem Formulation
.[.[6.2].]. .Iadd.(2.7) .Iaddend.is constructed via Fact A.2), an
explanation is provided here regarding how to generate a feasible
solution z.sup.n of .[.[6.1].]. .Iadd.(2.6) .Iaddend.which is close
to w.sup.m in a sense to be specified. We first assume that no
variables in .[.[6.1].]. .Iadd.(2.6) .Iaddend.are preassigned to
zero (this assumption will be removed shortly). The difficulty with
the solution w.sup.n is that it need not satisfy the last n-m sets
of constraints in .[.[6.1].]. .Iadd.(2.6).Iaddend.. Note, however,
that if w.sup.m is an optimal solution for .[.[6.4].]. .Iadd.(2.9)
.Iaddend.and w.sup.n (constructed as in Fact A.2) satisfies the
relaxed constraints, then w.sup.n is optimal for .[.[6.1].].
.Iadd.(2.6).Iaddend.. The recovery procedure described here is
designed to preserve the 0-1 character of the solution w.sup.m of
.[.[6.4].]. .Iadd.(2.9) .Iaddend.as far as possible. That is, if
w.sup.m.sub.i.sub.1. . . i.sub.m=1 and i1w.noteq.0 for at least one
1=1, . . . , m, then the corresponding feasible solution z.sup.n of
.[.[6.1].]. .Iadd.(2.6) .Iaddend.is constructed so that
z.sup.n.sub.i.sub.1. . . i.sub.n=1 for some (i.sub.m+1, . . .
,i.sub.n). Note that by this reasoning, variables of the form
z.sup.n.sub.0 . . . 0t.sub.m+1.sub.. . . i.sub.n can be assigned to
a value of 1 in the recovery problem only if w.sub.m.sub.0 . . .
0=1. However, variables z.sup.n.sub.0 . . . 0i.sub.m+1.sub.. . .
i.sub.n will be treated differently in the recovery procedure in
that they can be assigned 0 or 1 independent of the value
w.sup.m.sub.0 . . . 0. This increases the feasible set of the
recovery problem, leading to a potentially better solution.
Let {(i.sup.j.sub.1, . . . ,i.sup.j.sub.m)}.sup.N.sup.0.sub.j=1 be
an enumeration of indices of w.sup.m (or the first m indices of w1
constructed as in Fact A.2) such that w.sup.m.sup.j.sub.i.sub.1. .
. j.sup.j=1.sub.m and (i.sup.j.sub.1, . . . ,
i.sup.j.sub.m).noteq.(0, . . . , 0). Set (i.sup.0.sub.1, . . .
i.sup.0.sub.m)=(0, . . . , 0) for j=0 and define
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times. ##EQU00043## Let Y denote the solution
of the (n-m+1)-dimensional assignment problem:
.times.
.times..times..times..times..times..times..times..times..times.-
.times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times. .times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..function..times.
##EQU00044## The recovered feasible solution z.sup.n of .[.[6.1].].
.Iadd.(2.6) .Iaddend.corresponding to the multiplier set {u.sup.m,
. . . , u.sup.n} is then defined by
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times. ##EQU00045##
This recovery procedure is valid as long as all cost coefficients
c.sup.n are defined and all zero-one variables in z.sup.n are free
to be assigned. Note that modifications are necessary for sparse
problems. If the cost coefficient .[.c.sup.nj.sub.i.sub.1. .
.i.sup.j.sub.mi.sub.m+1.sub.. . . i.sub.n.].
.Iadd.c.sup.n.sub.i.sub.1.sub.j.sub.. . .
i.sub.m.sub.j.sub.i.sub.m+1.sub.. . . i.sub.n .Iaddend.is well
defined and the zero-one variable .[.z.sup.nj.sub.i.sub.1. . .
i.sup.j.sub.mi.sub.m+1. . . i.sub.n.].
.Iadd.c.sup.n.sub.i.sub.j.sub.1.sub.. . .
i.sub.j.sub.m.sub.i.sub.m+1.sub.. . . i.sub.n/n .Iaddend.is free to
be assigned to zero or one, then define c.sup.n-m+1.sub.ji.sub.m+1.
. . .sub.i.sub.n=cn.sub.ji.sub.1. . . ij.sub.mi.sub.m+1. . .
i.sub.n as in .[.[6.7].]. .Iadd.(2.12) .Iaddend.with
z.sup.n-m+1.sub.ji.sub.m+1 . . . .sub.i.sub.n being free to be
assigned. Otherwise, z.sup.n-m+1.sub.ji.sub.m+1 . . . .sub.i.sub.n
is preassigned to zero. To ensure that a feasible solution exists,
we now only need ensure that the variables z.sup.n-m+1.sub.j0...0
are free for j=0, 1, . . . , N.sub.0. (Recall that all variables of
the form z.sup.n.sub.0...i.sub.k.sub....0 are free (to be assigned)
and all coefficients of the form c.sup.n.sub.0...i.sub.k.sub....0
are well defined for k=1, . . . , n.) This is accomplished as
follows. If the cost coefficient
c.sup.n.sub.i.sub.1j.sub.i.sub.2j.sub....i.sub.mj.sub.0...0 is well
defined and z.sup.n.sub.i.sub.1j.sub.i.sub.2j.sub.. . .
i.sub.mj.sub.0...0 is free, then define
c.sup.n-m+1.sub.j0...0=c.sup.n.sub.i.sub.1ij.sub.2...i.sup.j.sub.m0
. . . 0 with z.sup.n-m+1.sub.j0...0 being free. Otherwise, since
all variables of the form z.sup.n.sub.0...i.sub.k . . . 0 are known
feasible and have well-defined costs, put
.times..times..times..times..noteq..times..times..times..times..times..-
times. .times.
.times..times..times..times..noteq..times..times..times..times..times..ti-
mes. ##EQU00046##
II.3. The Multi-Dimensional Lagrangian Relaxation Algorithm for the
n-Dimensional Assignment Problem
Starting with the M-dimensional assignment problem .[.[6.1].].
.Iadd.(2.6).Iaddend., i.e., n=M, the algorithm described below is
recursive in that the M-dimensional assignment problem is relaxed
to an m-dimensional problem by incorporating (n-m) sets of
constrains into the objective function using Lagrangian relaxation
of this set. This problem is maximized with respect to the Lagrange
multipliers, and a good suboptimal solution to the original problem
is recovered using an (n-m+1) -dimensional assignment problem. Each
of these two (the m-dimensional and the (n-m+1) -dimensional
assignment problems) can be solved in a similar manner.
More precisely, reference is made to FIG. 8 which presents a
flowchart of a procedure embodying the multi-dimensional relaxation
algorithm, referred to immediately above. This procedure, denoted
MULTI.sub.13 DIM_RELAX in FIG. 8, has three formal parameters, n,
PROB_FORMULATION and ASSIGNMT_SOLUTION, which are used to transfer
information between recursive instantiations of the procedure. In
particular, the parameter, n, is an input parameter supplying the
dimension for the multi-dimensional assignment problem (as in
.[.[6.1].]. .Iadd.(2.6).Iaddend.) which is to be solved (i.e., to
obtain an optimal or near-optimal solution). The parameter.
PROB_FORMULATION, is also an input parameter supplying the data
structure(s) used to represent the n-dimensional assignment problem
to be solved. Note that PROB_FORMULATION at least provides access
to the cost matrix, [cnu.sup.n], and the observation sets whose
observations are to be assigned. Additionally, the parameter,
ASSIGNMT_SOLUTION, is used as an output parameter for returning a
solution of a lower dimensioned assignment problem to an
instantiation of MULTI_DIM_RELAX which is processing a higher
dimensioned assignment problem.
A description of FIG. 8 follows. Assuming MULTI_Dim_RELAX is
initially invoked with M as the value for the parameter n and the
PROB_FORMULATION having a data structure(s) representing an
M-dimensional assignment problem as in .[.[5.51].].
.Iadd.(2.5).Iaddend., in step 500 an integer m, 2.ltoreq.m<n is
chosen such that the constraint sets .[.or.]. .Iadd.corresponding
to .Iaddend.dimensions m+1, . . . , n are to be relaxed so that an
m-dimensional problem formulation as in .[.[6.21].]. .Iadd.(2.7)
.Iaddend.results. In step 504 an initial approximation is provided
for {u.sup.m+1.sub.0, . . . , u.sup.n.sub.056 . Subsequently, in
step 508 the above initial values for {u.sup.m+1.sub.0, . . .
,u.sup.n.sub.0} are used in an iterative process in determining (
.sup.m+1, . . . , .sup.n) which maximizes
{.PHI..sub.mn.gamma..sup.m+1 n k.epsilon.R.sup.M.sup.k+i and k=m+1,
. . . ,n} for a feasible solution w.sup.n subject to the
constraints of Problem Formulation .[.[6.2].].
.Iadd.(2.7).Iaddend.. Note that by maximizing
.PHI..sub.mn.gamma..sup.m+1 n constraints that were relaxed are
being forced to be satisfied and in so doing information is built
into a solution of this function for solving the input assignment
problem in "PROB_FORMULATION." Further note that a non-smooth
optimization technique is used here and that a preferred method of
determining a maximum in step 508 is the bundle-trust method of
Schramm and Zowe .[.as described in.]. .Iadd.(.Iaddend.H. Schramm
and J. Zowe.[.,.]..Iadd.. .Iaddend.A version of the bundle idea for
minimizing a non-smooth function: Conceptual idea, convergence
analysis, numerical results.[.,.]..Iadd.. .Iaddend.SIAM Journal on
Optimization, 2 .[.(1992).]., .[.pp. 121-152.]. .Iadd.No.
1:121-152, February, 1992).Iaddend.. This method, along with
various other methods for determining the maximum in step 508, are
discussed below.
Also note that for m>2, a solution to the optimization problem
of step 508 is NP-hard and therefore cannot be solved optimally.
That is, there is no known computationally tractable method for
guarantying an optimal solution. Thus, there are two possibilities:
either (a) allow m to be greater than 2 and use auxiliary functions
similar to those disclosed in the first embodiment of the
k-dimensional assignment solver 300 in section I to compute a
near-optimal solution, or (b) make m=2 wherein algorithms such as
the forward/reverse auction algorithm of D. P. Bertsekas and D. A.
Castanon .[.in their paper.]. .Iadd.(D. P. Bertsekas and D. A.
Castanon.Iaddend.. A forward/reverse auction algorithm for
asymmetric assignment problems.[.,.]..Iadd.. .Iaddend.Computational
Optimization and Applications, 1 .[.(1992), pp..]..Iadd.:
.Iaddend.277-298, .Iadd.1992) .Iaddend.provides an optimal
solution.
If option (a) immediately above is chosen, then the auxiliary
functions are used as approximation functions for obtaining at
least a near-optimal solution to the optimization problem of step
508. Note that the auxiliary functions used depend on the value of
m. Thus, auxiliary functions used when m=3 will likely be different
from those for m=4. But in any case, the optimization procedure is
guided by using the merit function, .PHI..sub.2n(u.sup.3, . . . ,
u.sup.n), which can be computed exactly via a .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem for guiding the
maximization process.
Alternatively, if option (b) above is chosen, then two important
advantages result.[.:.]..Iadd., namely, .Iaddend.the optimization
problem of step 508 can be always solved optimally and without
using auxiliary approximation functions. Thus, better solutions to
the original M-dimensional assignment problem are likely since
there is no guarantee that when the non-smooth optimization
techniques are applied to the auxiliary functions the techniques
will yield an optimal solution to step 508. Furthermore, it is
important to note that without auxiliary functions, the processing
in step 508 is both conceptually easier to follow and more
efficient.
Subsequently, in step 512 of FIG. 8, the solution ( .sup.m+1, . . .
, ).sup.n .[.and w.sup.n.]. is used in determining an optimal
solution.[.,.]. w.sup.m to Problem Formulation .[.[6.4].].
.Iadd.(2.9) .Iaddend.as generated according to .[.[6.3].].
.Iadd.(2.8) .Iaddend.and Fact A.3.
In step 516, the solution w.sup.n is used in defining the cost
matrix C.sup.n-m+1 as in .[.[6.7].]. .Iadd.(2.12).Iaddend..
Subsequently, if n-m+1=2, then the assignment problem .[.[6.8].].
.Iadd.(2.13) .Iaddend.may be solved straightforwardly using known
techniques, such as forward/reverse auction algorithms. Following
this, in step 528, the solution to the .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem is assigned to
the variable ASSIGNMT_SOLUTION and in step 532 ASSIGNMT_SOLUTION is
returned to a dimension three level recursion of MULTI_DIM_RELAX
for solving a .[.3-dimensional.]. .Iadd.three-dimensioanl
.Iaddend.assignment problem.
Alternatively, if in step 520, n-m+1>2, then in step 536 the
data structure(s) representing a problem formulation as in
.[.[6.8].]. .Iadd.(2.13) .Iaddend.is generated and assigned to the
parameter, PROB_FORMULATION. Subsequently, in step 540 a recursive
copy of MULTI_DIM_RELAX is invoked to solve the lower dimensional
assignment problem represented by PROB_FORMULATION. Upon the
completion of step 540, the parameter, ASSIGNMT_SOLUTION, contains
the solution to the .[.n-m+1 dimensional.].
.Iadd.(n-m+1)-dimensional .Iaddend.assignment problem. Thus, in
step 544, the .[.n-m+1.]. .Iadd.(n-m+1).sup.th .Iaddend.solution is
used to solve the n-dimensional assignment problem as discussed
regarding equations .[.[6.9].]. .Iadd.(2.14).Iaddend.. Finally, in
steps 548 and 552 the solution to the n-dimensional assignment
problem is returned to the calling program so that, for example, it
may be used in taking one or more actions such as (a) sending a
warning to aircraft or sea facility; (b) controlling air traffic;
(c) controlling anti-aircraft or anti-missile equipment; (d) taking
evasive action; or (e) surveilling an object.
II.3.1. Comments on the Various Procedures Provided by FIG. 8
There are many procedures described by FIG. 8. One such procedure
is the first embodiment of the multi-dimensional assignment solving
process of section I.1. That is, by specifying m=n-1 in step 500, a
single set of constraints is relaxed in step 508. Thus, one set of
constraints .Iadd.is .Iaddend.incorporated .[.is.]. into the
objective function via the Lagrangian problem formulation,
resulting in an .[.m=n-1 dimensional.]. .Iadd.(m=n-1)-dimensional
.Iaddend.problem. The relaxed problem is subsequently maximized in
step 512 with respect to the corresponding Lagrange multipliers and
then a feasible solution is reconstructed for the n-dimensional
problem using a two-dimensional assignment problem. The second
procedure provided by FIG. 8 is a novel approach which is not
suggested by the first embodiment of section I.1. In fact, the
second procedure is somewhat of a mirror image of the first
embodiment in that n-2 sets of constraints are simultaneously
relaxed, yielding immediately an .[.m=2 dimensional.].
.Iadd.(m=2)-dimensional .Iaddend.problem in step 512. Thus, a
feasible solution to the n-dimensional problem is then recovered
using a recursively obtained solution to an .[.n-1 dimensional.].
.Iadd.(n-1)-dimensional .Iaddend.problem via step 540. In this
case, the function values and subgradients of
.PHI..sub.2n.gamma..sup.3 n optimally via a .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem. The significant
advantage here is, that there is no need for the merit or auxiliary
functions, .PSI..sub.k embodiment of the multi-dimensional
assignment solving process of section I.1 above and also there is
no need for the more general merit or auxiliary functions
.PSI..sub.mn subsection II.4.2 below. Further, note that all
function values and subgradients used in the .[.non-smooth.].
.Iadd.nonsmooth .Iaddend.maximization process are computed exactly
(i.e., optimally) in this second procedure. Moreover, problem
decomposition is now carried out for the n-dimensional problem;
however, decomposition of the .[.n-1 dimensional.].
.Iadd.(n-1)-dimensional .Iaddend.recovery problem (and all lower
recovery problems) is performed only after the problem is
formulated.
Between these two procedures are a host of different relaxation
schemes based on relaxing n-m sets of constraints to an
m-dimensional problem (2<m<n), but these all have the same
difficulties as the procedure for the first embodiment of section
I.1 in that the relaxed problem is an NP-hard problem. To resolve
this difficulty, we use an auxiliary or merit function .PSI..sub.nm
.gamma.notation .PSI..sub.k.PSI..sub.n-1,nm<n-1, the recovery
procedure is still based on an NP-hard .[.n-m+1 dimensional.].
.Iadd.n-m+1)-dimensional .Iaddend.assignment problem. Note that the
partitioning techniques similar to those discussed in Section I.3
may be used to identify the assignment problem with a layered graph
and then to find the disjoint components of this graph. In general,
all relaxed problems can be decomposed prior to any
.[.non-smooth.]. .Iadd.nonsmooth .Iaddend.computations because
their structure stays fixed throughout the-algorithm of FIG. 8.
However, all recovery problems cannot be decomposed until they are
formulated, as their structure changes as the solutions to the
relaxed problems change.
II.4. Details, and Refinements Relating to the Flowchart of FIG.
8
Further explanation is provided here on how various steps of FIG. 8
are solved. Note that the refinements presented here can
significantly increase the speed of the relaxation procedure of
step 508.
II.4.1 Maximization of the Non-smooth Function
One of the key steps in the Lagrangian relaxation algorithm in
section II.3 is the solution of the problem Maximize
{.PHI..sub.mn.gamma..sup.n+1 n kR.sup.M.sup.k+1; k=m+1, . . . , n }
[7.1] where u.sub.o.sup.k=0 for all k=m+1, . . . ,n. The evaluation
of .PHI..sup.nm.gamma..sup.m+1 n corresponding minimization problem
.[.[6.2]Zn varies for each instance of (u.sup.m+1, . . .
,u.sup.n).]. .Iadd.(2.7).Iaddend.. The following discussion
provides some properties of these functions.
Fact A.4. Let u.sup.m+1, . . . ,u.sup.n be multiplier vectors
associated with the (m+1).sup.st through the n.sup.th set of
constraints .[.[6.1].]. .Iadd.(2.6).Iaddend., let .PHI..sub.mn
.gamma.n.sup.n objective function value of the n-dimensional
assignment problem in equation .[.[6.1].]. .Iadd.(2.6).Iaddend.,
let z.sup.n be any feasible solution of .[.[6.1].].
.Iadd.(2.6).Iaddend., and let z.sup.n be an optimal solution of
.[.[6.1].]. .Iadd.(2.6).Iaddend.. Then, .PHI..sub.mn.gamma..sup.m+1
n {u.sup.m+1, . . . , u.sup.n}and .PHI..sub.mn.gamma..sup.m+1
n.gtoreq.V.sub.n( z.sup.n).gtoreq.V.sub.n(z.sup.n) .[.[7.2a].].
.Iadd.(2.16) .Iaddend. Furthermore, .PHI..sub.m-1n.gamma..sup.m m+1
n.gtoreq..PHI..sub.mn.gamma..sup.n+1 n .[.[7.2a].]. .Iadd.(2.17)
.Iaddend. for all u.sup.kR.sup.M.sup.k+1 with u.sub.o.sup.k=0 and
k=m, . . . n.
Most of the procedures for non-smooth optimization are based on
generalized gradients called subgradients, given by the following
definition.
.Iadd.Definition.Iaddend.. At u=(u.sup.m+1, . . . ,u.sup.n) the set
.delta..PHI..sub.mn.gamma. subdifferential of .PHI..sub.mn
.Iadd.and defined by .Iaddend.
.delta..PHI..sub.mn.gamma..epsilon.R.sup.M.sup.m+11x. . .
xR.sup.M.sup.n+1|.PHI..sub.mn.gamma..PHI..sub.mn.gamma..gtoreq.q.sup.T(w0-
u).A-inverted.w.epsilon.R.sup.M.sup.n+1+1x. . . xR.sup.M.sup.n+1}
[7.3] .[.A vector q.epsilon..delta..PHI..sub.mn.gamma..]. .Iadd.A
vector q.epsilon..delta..PHI..sub.mn(u) is called a
subgradient.Iaddend.. If z.sup.n is an optimal solution of
.[.[6.2].]. .Iadd.(2.7) .Iaddend.computed during evaluation of
.[..PHI..sub.mn.gamma..PHI..sub.mn
i.sub.n.sup.n.]..Iadd..PHI..sub.mn(u), differentiating .PHI..sub.mn
with respect to u.sup.n.sub.i.sub.n .Iaddend.yields the following
.[.i.sub.n-th.]. .Iadd.i.sub.n.sup.th .Iaddend.component of a
subgradient g of for .[..PHI..sub.mn.gamma..].
.Iadd..PHI..sub.mn(u) .Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times. ##EQU00047## If z.sup.n is the unique optimal solution of
.[.[6.2].]. .Iadd.(2.7).Iaddend., .[..delta..PHI..sub.mn.gamma.
.PHI..sub.mn.]. .Iadd..differential..PHI..sub.mn(u)={g}and
.PHI..sub.mn .Iaddend.unique, then there are finitely many such
solutions, say z.sup.n(1), . . . ,z.sup.n(K). Given the
corresponding subgradients, g.sup.1, . . . , g.sup.K, the
subdifferential .[..delta..PHI..gamma..]. .Iadd..delta..PHI.(u) is
the convex hull of .Iaddend.{g.sup.1, . . . , g.sup.K}.
II.4.2 Mathematical Description of a Merit or Auxiliary
Function
For real-time needs, one must address the fact that the non-smooth
optimization problem of step 508 (FIG. 8) requires the solution of
an NP-hard problem for m>2. One approach to this problem is to
use the following merit or auxiliary function to decide whether a
function value has increased or decreased sufficiently in the line
search or trust region methods:
.PSI..function..times..times..PHI..function..times..times..times..times..-
times..times..times..times..times..times..times..times..PHI..times..functi-
on..times..times. ##EQU00048## where the multipliers .sup.3, . . .
, .sup.m that appear in lower order relaxations used to construct
(suboptimal) solutions of the m-dimensional relaxed problem
.[.(6.2).]. .Iadd.(2.7) .Iaddend.have been explicitly included.
Note that .[..PSI..sub.mn.gamma..]. .Iadd..PSI..sub.mn is
well-defined since (2.7) can always .Iaddend.be solved optimally if
m=2. For sufficiently small problems .[.(6.2).]. .Iadd.(2.7)
.Iaddend.or .[.(6.4).]. .Iadd.(2.9).Iaddend., one can more
efficiently solve the NP-hard problem by branch and bound. This is
the reason for the inclusion of the first case; otherwise, the
relaxed function .PHI..sub.2n .Iadd.is used to guide the nonsmooth
optimization phase. That .Iaddend.the merit function provides a
lower bound for the optimal solution follows directly from Fact A.4
and the following fact.
Fact A.5. Given the definition of .[..PSI..sub.mn.gamma..].
.Iadd..PSI..sub.mn in (2.20) above, .Iaddend.
.PSI..sub.mn.gamma..sup.3 m m+1
n.ltoreq..PHI..sub.mn.gamma..sup.m+1 n .sup.3, . . . , .sup.m,
u.sup.m+1, . . . ,u.sup.n .[.[7.6].]. .Iadd.(2.21) .Iaddend. The
actual function value used in the optimization phase is
.[..PSI..sub.mn.gamma..]. .Iadd..PSI..sub.mn; however, the
subgradients are computed as in (2.19), but .Iaddend.with the
solution z.sub.i.sub.1.sup.n . . . i.sub.n being a suboptimal
solution constructed from a relaxation procedure applied to the
m-dimensional problem. Again, the use of these lower order relaxed
problems is the reason for the inclusion of the multipliers .sup.3
. . . , .sup.m.
.[.To explain how the merit function is used, suppose there is a
current multiplier set (u.sub.old.sup.m+1, . . . , u.sub.old.sup.n)
and it is desirable to update to a new multiplier set
(u.sub.new.sup.m+1, . . . ,
.DELTA..sup.n.PSI..sub.mn.gamma..sub.old.sup.3.sub.old.sup.m.sub.old.sup.-
m+1.sub.old.sup.n.DELTA..gamma..sub.old.sup.3.sub.old.sup.m.DELTA.
obtained during the relaxation process used to compute a high
quality solution to the relaxed m-dimensional assignment problem
(6.2) at (u.sup.m+1, . . . ,u.sup.n)=(u.sub.old.sup.m+1, . . .
,u.sub.old.sup.n). The decision to accept (u.sub.n.sup.m+1ew. . .
,u.sup.n.sub.new) is then based on
.PSI..gamma..sub.mn.sup.3old.sup.m.sub.oldu.sub.old.sup.m+1, . . .
,
u.sub.old.sup.n)>.PSI..sub.mn.gamma..sub.new.sup.3.sub.new.sup.m.s-
ub.new.sup.m+1.sub.new.sup.n.DELTA. stopping criteria commonly used
in line searches. Again, ( .sub.new.sup.3, . . . , .sub.new.sup.m)
represents the multiplier set used in the lower level relaxation
procedure to construct a high quality feasible solution to the
m-dimensional relaxed problem (6.2) at (u.sup.m+1, . . . ,
u.sup.n)=(u.sub.new.sup.m+1, . . . , u.sub.new.sup.n). The point is
that each time one changes (u.sup.m+1, . . . , u.sup.n) and uses
the merit function .PSI..sub.mn.gamma..sup.3.sub.m m+1 n.DELTA.
generally change the lower level multipliers ( .times..sup.3, . . .
, .sup.m)..]. .Iadd.To explain how the merit function is used,
suppose there is a current multiplier set (u.sub.old.sup.m+1, . . .
,u.sub.old.sup.n) and it is desirable to update to a new multiplier
set (u.sub.new.sup.m+1, . . . ,u.sub.new.sup.n) via
(u.sub.new.sup.m+1, . . . ,u.sub.new.sup.n)=(u.sub.old.sup.m+1, . .
. ,u.sub.old.sup.n)+(.DELTA.u.sup.m+1, . . . ,.DELTA.u.sup.n). Then
.PSI..sub.mn ( .sub.old.sup.3, . . . , .sub.old.sup.m;
u.sub.old.sup.m+1, . . . ,u.sub.old.sup.n) is computed where (
.sub.old.sup.3, . . . , .sub.old.sup.m) is obtained during the
relaxation process used to compute a high quality solution to the
relaxed m-dimensional assignment problem (2.7) at (u.sup.m+1, . . .
,u.sup.n)=(u.sub.old.sup.m+1, . . . ,u.sub.old.sup.n). The decision
to accept (u.sub.new.sup.m+1, . . . ,u.sub.new.sup.n) is then based
on .PSI..sub.mn( .sub.old.sup.3, . . . , .sub.old.sup.m;
u.sub.old.sup.m+1, . . . ,u.sub.old.sup.n)<.PSI..sub.mn (
.sub.new.sup.3, . . . , .sub.new.sup.m; u.sub.new.sup.m+1, . . .
,u.sub.new.sup.n) or some other stopping criteria commonly used in
line searches. Again, ( .sub.new.sup.3, . . . , .sub.new.sup.m)
represents the multiplier set used in the lower level relaxation
procedure to construct a high quality feasible solution to the
m-dimensional relaxed problem (2.7) at (u.sup.m+1, . . .
,u.sup.n)=(u.sub.new.sup.m+1, . . . ,u.sub.new.sup.n). The point is
that each time one changes (u.sup.m+1, . . . ,u.sup.n) and uses the
merit function .PSI..sub.mn ( .sup.3, . . . , .sup.m; u.sup.m+1, .
. . ,u.sup.n) for comparison purposes, one must generally change
the lower level multipliers ( .sup.3, . . ., .sup.m). .Iaddend.
An illustration of this merit function for m=n-1 is given in
.[."PARTITIONING MULTIPLE DATA SETS. MULTIDIMENSIONAL ASSIGNMENTS
AND LAGRANGIAN RELAXATION", by Aubrey B. Poore and Nenad Rijavec,
and in "QUADRATIC ASSIGNMENT AND RELATED PROBLEMS, DIMACS SERIES IN
DISCRETE MATHEMATICS AND THEORETICAL COMPUTER SCIENCE", in American
Mathematical Society, Providence, R.I., Vol. 16, 1994, pp. 25-37
edited by Panos M. Pardalos and Henry Wolkowicz.]. .Iadd.(A. B.
Poore and N. Rijavec. Partitioning Multiple Data Sets,
Multidimensional Assignments and Lagrangian Relaxation. In P. M.
Pardalos and H. Wolkowicz, editors, Quadratic Assignment and
Related Problems: DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, volume 16, pages 25-37,
1994).Iaddend..
II.4.3 Non-smooth Optimization Methods
By Fact A.4 the function .[..PHI..sub.mn.gamma..DELTA..].
.Iadd..PHI..sub.mn(u) is a continuous, piecewise .Iaddend.affine,
and concave, so that the negative of
.[..PHI..sub.mn.gamma..DELTA..]. .Iadd..PHI..sub.mn(u) is convex.
.Iaddend.Thus the problem of maximizing
.[..PHI..sub.mn.gamma..DELTA..]. .Iadd..PHI..sub.mn(u) is one of
nonsmooth .Iaddend.optimization. Since there is a large literature
on such problems, only a brief discussion of the primary classes of
methods for solving such problems is provided. A first class of
methods, known as subgradient methods, are reviewed and analyzed in
.[.the book by.]. .Iadd.(.Iaddend.N. Z. Shor.[.,.]. .Iadd..
.Iaddend.Minimization Methods for Non-Differentiable
Functions.[.,.]. Springer-Verlag, New York, 1985.Iadd.).Iaddend..
Despite their relative simplicity, subgradient methods have some
drawbacks that make them inappropriate for the tracking problem.
They do not guarantee a descent direction at each iteration, they
lack a clear stopping criterion, and exhibit poor convergence (less
than linear).
A more recent and sophisticated class of methods are the bundle
methods; excellent developments are presented in .[.the books of
Hiriart-Urruty and Lemarechal:.]. .Iadd.(.Iaddend.J.-B.
Hiriart-Urruty and C. Lemarechal.[.,.]..Iadd.. .Iaddend.Convex
Analysis and Minimization Algorithms I and II.[.,.]..Iadd..
.Iaddend.Springer-Verlag, Berlin, 1993; .[.and the book of.]. K. C.
Kiwiel.[.:.]. .Iadd.. .Iaddend.Methods of Descent for
Non-Differentiable Optimization.Iadd.. In A. Dold and B.
Eckman.Iaddend., .Iadd.editors, .Iaddend.Lecture Notes in
Mathematics, .[.Vol..]. .Iadd.volumen .Iaddend.1133, .Iadd.Berlin,
1985. .Iaddend.Springer-Verlag.[., Berlin, 1985.]..Iadd.).Iaddend..
Bundle methods retain a set (or bundle) of previously computed
subgradients to determine the best possible descent direction at
the current iteration. Because of the .[.non-smoothness.].
.Iadd.nonsmoothness .Iaddend.of .PHI..sub.mn.Iadd., the resulting
direction .Iaddend.may not provide a "sufficient" decrease in
.PHI..sub.mn.Iadd.. In this case, .Iaddend.bundle algorithms take a
"null" step, wherein the bundle is enriched by a subgradient close
to u.sub.k. As a result, bundle methods are non-ascent methods
which satisfy the relaxed descent condition
.PHI..sub.mn.gamma..sub.k+1.DELTA..PHI..sub.mn.gamma..sub.k.DELTA..sub.k+-
1.noteq.u.sub.k. These methods have been shown to have good
convergence properties. In particular, bundle method variants have
been proven to converge in a finite number of steps for piecewise
affine convex functionals in .Iadd.(.Iaddend.K..
Kiwiel.[.,.]..Iadd.. .Iaddend.An .[.Aggregate Subgradient Method
for Non-smooth Convex Minimization,.]. .Iadd.aggregate subgradient
method for non-smooth convex minimization. .Iaddend.Mathematical
Programming.Iadd., .Iaddend.27.[., (1983) pp..]. .Iadd.:
.Iaddend.320-341, .[.and.]. .Iadd.1983; .Iaddend.H. Schramm and J.
Zowe.[.,.]..Iadd.. .Iaddend.A version of the bundle idea for
minimizing a non-smooth function: Conceptual idea, convergence
analysis, numerical results.[.,.]..Iadd.. .Iaddend.SIAM Journal on
Optimization, 2 .[.(1992).]., .[.pp..]. .Iadd.No.
1:.Iaddend.121-152.Iadd., February, 1992).Iaddend..
All of the above non-smooth optimization methods require the
computation
.[..PHI..sub.mn.gamma..DELTA..epsilon..delta..PHI..sub.mn.gamma..DELTA..]-
. .Iadd.of .PHI..sub.mn(u) and a g .epsilon..delta..PHI..sub.mn(u).
These in turn require .Iaddend.the computation of the relaxed cost
coefficients .[.[6.3].]. .Iadd.(2.8).Iaddend.. In both the first
and second procedures discussed in section II.3.1, maximization of
.[..PHI..sub.2n.gamma..DELTA..]. .Iadd..PHI..sub.2n(u) must be
repeatedly evaluated. .Iaddend.In the most efficient
implementations presently known of these two procedures, it was
found that at least 95% of the computational effort of the entire
procedure is spent in the evaluation of the relaxed cost
coefficients .[.[6.3].]. .Iadd.(2.8) .Iaddend.as part of computing
.[..PHI..sub.2n.gamma..DELTA..]. .Iadd..PHI..sub.2n(u). Thus,
generally a method should be chosen .Iaddend.that makes as
efficient use of the subgradients and function values as .[.is.].
possible, even at the cost sophisticated manipulation of the
subgradients. In evaluating three different bundle procedures: (a)
the conjugate subgradient method of Wolfe used in section I.1 of
the first embodiment of the present invention; (b) the aggregate
subgradient method of Kiwiel (.[.described in.]. K. C.
Kiwiel.[.,.]..Iadd.. .Iaddend.An .[.Aggregate Subgradient Method
for Non-smooth Convex Minimization,.]. .Iadd.aggregate subgradient
method for non-smooth convex minimization. .Iaddend.Mathematical
Programming.Iadd., .Iaddend.27.[., (1983) pp..]..Iadd.:
.Iaddend.320-341.Iadd., 1083.Iaddend.); and (c) the bundle trust
method of Schramm and Zowe (.[.described in.]. H. Schramm and J.
Zowe.[.,.]..Iadd.. .Iaddend.A version of the bundle idea for
minimizing a non-smooth function: Conceptual idea, convergence
analysis, numerical results.[.,.]..Iadd.. .Iaddend.SIAM Journal on
Optimization, 2 .[.(1992).]., .[.pp..]. .Iadd.No.
1:.Iaddend.121-152.Iadd., February, 1992.Iaddend.), it was
determined that for a fixed number of non-smooth iterations, say,
ten, the bundle-trust method provides good quality solutions with
the fewest number of function and subgradient evaluations of all
the methods, and is therefore the preferred method.
II.4.4 The Two Dimensional Assignment Problem
The forward/reverse auction algorithm of Bertsekas and Castanon
(.[.as described in.]. D. P. Bertsekas and D. A.
Castafion.[.,.]..Iadd.. .Iaddend.A forward/reverse auction
algorithm for asymmetric assignment problems.[.,.]..Iadd..
.Iaddend.Computational Optimization and Applications, 1 .[.(1992),
pp..]..Iadd.: .Iaddend.277-298.Iadd., 1992.Iaddend.) is used to
solve the many relaxed .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.problems that occur in the course of execution.
II.4.5. Initial Multipliers and Hot Starts
The effective use of "hot starts" is fundamental for real-time
applications. A good initial set of multipliers can significantly
reduce the number of non-smooth iterations (and hence the number of
.[..PHI..sub.mn.DELTA..]. .Iadd..PHI..sub.mn evaluations) required
for a high .Iaddend.quality recovered solution. Further, there are
several ways that multipliers can be reused. First, if .[.a.].
.Iadd.an .Iaddend.n-dimensional problem is relaxed to .[.a.].
.Iadd.an .Iaddend.m-dimensional problem, relaxation provides the
multiplier set {u.sup.m+1, u.sup.m+2, . . . ,u.sup.n}. These can be
used as the initial multipliers for the .[.n-m+1 dimensional.].
.Iadd.(n-m+1)-dimensional .Iaddend.recovery problem for n-m+1>2.
This approach has also worked well to reduce the number of
non-smooth iterations during recovery.
Further, for track maintenance, initial feasible solutions are
generated as follows. When a new scan of information (a new
observation set) arrives from a sensor, one can obtain an initial
primal feasible solution by matching new reports to existing tracks
via a .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem. This is known as the track-while-scan
(TWS) approach. Thus, an initial primal solution exists and then we
use the above relaxation procedure is to make improvements to this
TWS solution. Also for track maintenance, multipliers are available
from a previously solved and closely related
.[.multi-dimensional.]. .Iadd.multidimensional .Iaddend.assignment
problems for all but the new observation set.
II.4.6. Local Search Methods
Given a feasible solution of the .[.multi-dimensional.].
.Iadd.multidimensional .Iaddend.assignment problem, one can
consider local search procedures to improve this result, as
described in .Iadd.(.Iaddend.C. H. Papadimitriou and K.
Steiglitz.[.,.]..Iadd.. .Iaddend.Combinatorial Optimization:
Algorithms and Complexity.[.,.]..Iadd.. .Iaddend.Prentice-Hall,
.Iadd.Inc., .Iaddend.Englewood Cliffs, N.J., 1982.[., and.]..Iadd.;
.Iaddend.J. Pearl.[.,.]..Iadd.. .Iaddend.Heuristics: .[.intelligent
search strategies for computer problem solving,.].
.Iadd.Intelligent Search Strategies for Computer Problem Solving.
.Iaddend.Addison-Wesley, .Iadd.Reading, Mass.,
.Iaddend.1984.Iadd.).Iaddend.. One method is the idea of
.[.k-interchanges.]. .Iadd.k interchanges.Iaddend.. Since for
sparse problems only those active arcs that participate in the
solution are stored, it is difficult to efficiently identify
eligible variable exchange sets with some data structures for
solving the assignment problem. However, a local search that may be
more promising is to use the .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment solver in the following
way. Given a feasible solution to the .[.multi-dimensional.].
.Iadd.multidimensional .Iaddend.assignment problem, the indices
that correspond to active arcs in the solution are enumerated.
Subsequently, one coordinate is freed to formulate a .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment problem
with one index corresponding to the enumeration and the other to
the freed coordinate, and then solve a .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem to improve the
freed index position.
II.4.7. Problem Decomposition
The procedures described thus far are all based on relaxation. Due
to the sparsity of assignment problems, however, frequently a
decomposition of the problem into a collection of disjoint
components can be done wherein each of the components can be solved
independently. Due to the setup costs of Lagrangian relaxation, a
branch and bound procedure is generally more efficient for small
components, say ten to twenty feasible 0-1 variables (i.e.,
z.sub.i.sub.1 . . . i.sub.n). Otherwise, the relaxation procedures
described above are used. Perhaps the easiest way to view the
decomposition method is to view the reports or measurements as a
layered graph. A vertex is associated with each observation point,
and an edge is allowed to connect two vertices only if the two
observations belong to at least one feasible track of observations.
Given this graph, the decomposition problem can then be posed as
that of identifying the connected subcomponents of a graph which
can be accomplished by constructing a spanning forest via a depth
first search algorithm, as described in .Iadd.(.Iaddend.A. V. Aho,
J. E. Hopcroft, and J. D. Ullman.[.,.]..Iadd.. he Design and
Analysis of Computer Algorithms.Iaddend..[.,.]..Iadd..
.Iaddend.Addison-Wesley .[.Publishing Company, Reading.]., Mass.,
1974.Iadd.).Iaddend.. Additionally, decomposition of a different
type might be based on the identification of bi- and tri-connected
components of a graph and enumerating on the connections. Here is a
technical explanation. Let Z-={z.sub.i.sub.1.sub.,.sub.2 .sub.. . .
i.sub.n|zi1i2 . . . in is not preassigned to zero} denote the set
of assignable variables. Define an undirected graph G(N,A) where
the set of nodes is N={z.sub.i.sub.n.sup.min=1, . . . ,n; in=1, . .
. , Nm} and arcs, .[.A={(z.sub.j.sub.n.sup.n,zj1.sup.1)|n.noteq.1,
j.sub.n.noteq.0, j.sub.1.noteq.0 and there exists zi1i2 . . .
in.epsilon.Z such that j.sub.n=i.sub.n and j.sub.1-i.sub.1}..].
.Iadd.A={(z.sub.j.sub.n.sup.n,z.sub.j.sub.1.sup.1)[|n.noteq.1]|n.noteq.1,
j.sub.n.noteq.0, j.sub.1.noteq.0 and there exists
z.sub.i.sub.1.sub.i.sub.2.sub.. . . i.sub.n.epsilon.Z-.
.Iaddend.
Note that the nodes corresponding to zero index have not been
Included in the above defined graph, since two variables that have
only the zero index in common can be assigned independently.
Connected components of the graph are then easily found by
constructing a spanning forest via a depth first search. (A
detailed algorithm can be found in the book by Aho, Hopcroft and
Ullman cited above). Furthermore, this procedure is used at each
level in the relaxation. i.e., is applied to each assignment
problem .[.[3.1].]. .Iadd.(1.4) .Iaddend.for n=3, . . . ,M.
The original relaxation problem is decomposed first. All relaxed
assignment problems can be decomposed a priori and all recovery
problems can be decomposed only after they are formulated. Hence,
in the n-to-(n-1) case, we have n-2 relaxed problems that can all
be decomposed initially, and the recovery problems that are not
decomposed (since they are all .[.two dimensional.].
.Iadd.two-dimensional.Iaddend.). In the n-to-2 case, we have only
one relaxed problem that can be decomposed initially. This case
yields n-3 recovery problems, which can be decomposed only after
they are formulated.
.Iadd.III. New Approaches to Track Initiation and Maintenance Using
Multidimensional Assignment Problems .Iaddend.
The ever-increasing demand for sensor surveillance systems is to
accurately track and identify objects in real-time, even for dense
target scenarios and in regions of high track contention. The use
of multiple sensors, through more varied information, has the
potential to greatly improve target state estimation and
identification. However, to take full advantage of the available
data, a multiple frame data association approach is needed. The
most popular such approach is an enumerative technique called
multiple hypothesis tracking (MHT). As an enumerative technique,
MHT can be too computationally intensive for real-time needs
because this problem is NP-hard. A promising approach is to utilize
the recently developed Lagrangian relaxation algorithms .Iadd.(K.
P.Pattipati, S. Deb, and Y.Bar-Shalom. A multisensor-multitarget
data association algorithm for heterogeneous sensors. IEEE
Transactions on Aerospace and Electronic Systems, 29, No.
2:560-568, April 1993; A. B. Poore and N.Rijavec. Partitioning
multiple data sets: multidimensional assignments and lagrangian
relaxation, in quadratic assignment and related problems, P. M.
Pardalos and H.Wolkowicz, editors, DIMACS series in Discrete
Mathematics and Theoretical Computer Science, 16:25-37, 1994; A. J.
Robertson III. A class of lagrangian relaxation algorithms for the
multidimensional problem. Ph.D. Thesis, Colorado State University,
Ft. Collins, Colo., 1995) .Iaddend.for the multidimensional
assignment problem; however, there are many other potentially good
approaches to these assignment problems such as LP relaxation
combined with an interior point method, GRASP, and
parallelization.
These data association problems are fundamentally important in
tracking. Thus, our first objective is to present an overview of
the tracking problem and the context within which these data
association problems fit. Following a brief review of the
probabilistic framework for data association, the second objective
is to formulate two models for track initiation and maintenance as
multidimensional assignment problems. This formulation is based on
a window moving over the frames of data. The first and simpler
model uses the same window length for track initiation and
maintenance, while the second model uses different lengths. As the
window moves over the frames of data, one creates a sequence of
these multidimensional assignment problems and there is an overlap
region of frames common to adjacent windows. Thus, given the
solution of one problem, one can develop a warm start for the
solution to the next problem in the sequence, as shown hereinafter.
Such information is critically important to the design of real-time
algorithms.
.Iadd.III.2. .Iaddend.Overview of the Tracking Problem
Tracking and data fusion are complex subjects of specialization
that can only be briefly summarized as they are related to the
subject of this paper. The processing of track multiple targets
using data from one or more sensors is typically partitioned into
two stages or major functional blocks, namely, signal processing
and data .[.process.]. .Iadd.processing (Y.Bar-Shalom,
Multitarget-Multisensor Tracking: Advanced Applications. Artech
House, Dedham, Mass., 1990; Y.Bar-Shalom and T. E. Fortmann,
Tracking and Data Association, Academic Press, Boston, Mass., 1988;
S. S. Blackman. Multiple Target Tracking with Radar Applications.
Artech House, Dedham, Mass., 1986).Iaddend.. The first stage is the
signal processing that takes raw sensor data and outputs reports.
Reports are sometimes called observations, threshold exceedances,
plots, hits, or returns, depending on the type of sensor. The true
source of each report is usually unknown and can be due to a target
of interest, a false signal, or persistent background objects that
can be moving in the field of view of the sensor.
Two principal functions of data processing are tracking and
discrimination or target identification. The discrimination or
target identification function distinguishes between tracks that
are for targets of interest and other tracks such as those due to
background. Also, if there is enough information, each track is
classified as to the type of target it appears to represent. Target
identification and discrimination will not be discussed further in
this paper except to comment here on the use of attributes
(including identification information) in the data association.
There are two types of attributes or features, namely, discrete
valued variables and continuous valued variables. Available
attributes of either or both types should be used in computing the
log likelihoods or scores for data association. In discussing
tracking in this paper, the attributes will not be mentioned
explicitly but it should be understood that some reports and some
tracks may include attributes information that should be and can be
used in the track processing (both data association and filtering)
if it is useful.
.Iadd.III.2.1 .Iaddend.Tracking Functions
The tracking function accepts reports for each frame of data and
constructs or maintains a target state estimate, the
variance-covariance matrix of the state estimation error, and
refined estimates (or probabilities) of the target attributes. The
state estimate typically includes estimates of target position and
velocity in three dimensions and possibly also acceleration and
other variables of interest.
The tracking function is typically viewed in two stages, namely,
data association and filtering. Also, the process of constructing
new tracks, called track initiation, is different from the process
of updating existing tracks, called track maintenance.
In track maintenance, the data association function decides how to
relate the reports from the current frame of data to the previously
computed tracks. In one approach, at most one report is assigned to
each track, and in other approachesm, weights are assigned to the
pairings of reports to a track. After the data association, the
filter function updates each target state estimate using the one or
more (with weights) reports that were determined by the data
association function. A filter commonly used for multiple target
tracking and data fusion is the well know Kalman filter (or the
extended Kalman filter in the case of nonlinear problems) or a
simplified version of it .Iadd.(Y.Bar-Shalom,
Multitarget-Multisensor Tracking: Advanced Applications. Artech
House, Mass., 1990; Y.Bar-Shalom and T. E. Fortmann, Tracking and
Data Association, Academic Press, Boston, Mass., 1988; S. S.
Blackman. Multiple Target Tracking with Radar Applications. Artech
House, Dedham, Mass., 1986).Iaddend..
In track initiation, typically a sequence of reports is selected
(one from each of a few frames of data) to construct a new track.
In track initiation, the filtering function constructs a target
state estimate and related information based on the selected
sequence of reports. The new track is later updated by the track
maintenance processing of the subsequent frames of reports.
In some trackers, there is also a tentative tracking function for
processing recently established tracks until there is enough
confidence to include them in track maintenance. While for
simplicity of presentation, tentative tracking will not be included
in the processing discussed in this paper, the techniques discussed
could readily include one or more tentative tracking functions.
There have been numerous approaches developed to perform the data
association function. Since optimal data association is far too
complex to implement, good but practical sub-optimal approaches are
pursued. Data association approaches can be classified in a number
of ways. .[.On.]. .Iadd.One .Iaddend.way to classify data
association approaches is based on the number of data frames used
in the association process .Iadd.(O. E. Drummond. Multiple sensor
tracking with multiple frame, probabilistic data association. In
Signal and Data Processing of Small Targets, SPIE Proceedings,
volume 2561, pages 322-336, 1995).Iaddend.. In single frame data
association for track maintenance, "hard decisions" are made
.[.each frame as to how the.]. .Iadd.on the assignment of
.Iaddend.reports .[.are to be related.]. to the tracks. Some single
frame approaches include: individual nearest neighbor, FDA, JPDA,
and global nearest neighbor (sequential most probable hypothesis
tracking), which uses a .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment algorithm .Iadd.(Y. Bar-Shalom.
Multitarget-Multisensor Tracking: Advanced Applications. Artech
House, Mass., 1990; Y. Bar-Shalom and T. E. Fortmann. Tracking and
Data Association. Academic Press, Boston, Mass., 1988; S. S.
Blackman. Multiple Target Tracking with Radar Applications. Artech
House, Dedham, Mass., 1986).Iaddend.. In most single frame data
association approaches, only one track per object is carried
forward to be used in processing the next frame of reports.
Multiple frame data association is more complex and frequently
involves purposely .[.carry.]. .Iadd.carrying .Iaddend.forward more
than one track per target to be used in processing the next frame
of reports. By retaining more than one track per target, the
tracking performance is improved at the cost of increased
processing load. The best known multiple frame data association
approach is an enumerative technique called multiple hypothesis
tracking (MHT) which enumerates all the global hypothesis with
various pruning rules.
As shown hereinafter, the log likelihood of the probability of a
global hypothesis can be decomposed into the sum of scores of each
track that is contained in the hypothesis. As a consequence, the
most probable hypothesis can be identified with the aid of a
non-enumerative algorithm for the assignment so formulated.
Interface Between Track Initiation And Track Maintenance
There are a number of ways that track initiation and track
maintenance functions can interact. Two ways that are especially
pertinent to the methods of this paper will be discussed in this
section.
One approach is to treat these two functions sequentially, by first
assigning reports to tracks and then using the information that
remains to form new tracks. A better approach is to integrate both
processes where initiating tracks compete for the new frame of
reports on an equal basis, i.e., at the same time.
In the integrated approach, the data association for track
initiation and track maintenance processing are combined and
conducted simultaneously. One assignment array is created that
includes the track scores for potential tracks for both track
initiation and track maintenance. The first dimension of the
assignment problem includes only all the tracks either created or
updated in the processing of the prior frames of reports. Each of
the remaining dimensions accommodates one frame of reports.
After the assignment algorithm finds a solution, each of the
previously established tracks are updated with the report assigned
to it from the second dimension of the assignment array. The
remaining reports in the second dimension that were in the
assignment solution are firmly established as track initiators. The
unassigned reports in the second dimension of the assignment array
are discarded as false signals. The processing of the next frame of
reports repeats this process using these updated tracks and the
newly identified initiators in the first dimension of the cost
array for processing the new frame of reports. Details of this
approach are discussed hereinafter.
The integrated approach just discussed, uses the same number of
frames of reports for both track initiation and track maintenance.
The goal in using the multi-dimensional assignment algorithm is to
provide improved performance while minimizing the amount of
processing required. Typically, track initiation will benefit from
more frames of reports than will track maintenance. Thus.Iadd.,
.Iaddend.a second approach that integrates track initiation and
track maintenance is discussed hereinafter, wherein the number of
frames of reports is not the same for these two functions. This is
a novel approach that is introduced in this paper.
Multiple Sensor Processing
There are many advantages and many ways .[.to.]. .Iadd.of
.Iaddend.combining data from multiple sensors. There are also many
ways of categorizing the different algorithmic architectures for
processing data from multiple sensors. One approach outlines four
generic algorithmic architectures .Iadd.(O. E. Drummond. Multiple
sensor tracking with multiple frame, probabilistic data
association. In Signal and Data Processing of Small Targets, SPIE
Proceedings, volume 2561, pages 322-336, 1995).Iaddend.. Two of
these generic architectures are especially pertinent to this paper
and are summarized briefly.
In the Centralized Fusion algorithmic architecture, reports are
combined from the various sensors to form global tracks. This
algorithmic architecture is also called Central Level Tracking,
Centralized Algorithmic Architecture, or simply the Type IV
algorithmic architecture. In track maintenance.Iadd., .Iaddend.for
example, if a single frame data association is used, then for data
association a frame of reports from one sensor is processed with
the latest global tracks; then the global tracks are updated by the
filter function. After the processing of this frame of reports is
completed, a frame of the reports from another (or the same) sensor
is processed with these updated global tracks. This process is
continued as new frames of data become available to the system as a
whole.
In Centralized Fusion, using the multi-dimensional assignment
algorithm for the data association with multiple sensors is similar
to the processing of data from a single sensor. Instead of using
multiple frames of reports from a single sensor, the multiple
frames of reports come from multiple sensors. The frames of reports
from all the sensors are ordered based on the nominal time each
frame of reports is obtained and independent of which sensor
provided the reports. In this way the approaches discussed in this
paper can be extended to process reports from multiple sensors
using the Centralized Fusion algorithmic architecture.
The second pertinent algorithmic architecture is Track Fusion. This
approach is also called the Hierarchical or Federated algorithmic
architecture, sensor level tracking or simply the Type II
algorithmic architecture. In track maintenance.Iadd., .Iaddend.for
example, a processor for each sensor computes single sensor tracks;
these tracks are then forwarded to the global tracker to compute
global tracks based on data from all sensors.
After the first time a sensor processor forwards tracks to the
global tracker, then subsequent tracks for the same targets are
cross-correlated with the existing global tracks. This
track-to-track cross-correlation is due to the common history of
the current sensor tracks and the tracks from the same sensor that
were forwarded earlier to the global tracker. The processing must
take this cross-correlation into account and there are a number of
ways of compensating for this-cross-correlations. One method for
dealing with this cross-correlation is to decorrelate the sensor
tracks that are sent to the global tracker. There are a variety of
ways to achieve this decorrelation and some are summarized in
.[.Drummond, 1996, Signal Data Proc..]..Iadd.recent paper (O. E.
Drummond, Feedback in track fusion without process noise. In Signal
and Data Processing .Iaddend.of Small Targets .[.1995.]..Iadd.,
SPIE Proceedings.Iaddend., .[.-2561:369-383.]. .Iadd.volume 2561,
pages 369-383, 1995.Iaddend..
Once the sensor tracks are decorrelated they can be processed by
the global tracker in almost the same way as reports are processed.
In the case of track fusion the association process is referred to
as track (or track-to-track) association rather than data (or
report-to-track) association. If the sensor tracks are
decorrelated, the global tracker can process the tracks from the
various sensor processors in much the same way that the global
tracker of Centralized Fusion processes reports. Accordingly the
methods described in this paper can be readily extended to
processing data from multiple sensors using either the Centralized
Fusion or Track Fusion or even a hybrid combination of both
algorithmic architectures.
.Iadd.III.3 .Iaddend.Formulation of the Data Association
Problem
The goal of this section is to briefly outline the probabilistic
framework for the data association problems presented in this work.
The technical details are presented elsewhere .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57,
1994).Iaddend.. The data association problems for multisensor and
multitarget tracking considered in this work are generally posed
.Iadd.(A. B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994) .Iaddend.as that of maximizing the
posterior probability of the surveillance region (given the data)
according to Maximize
{P(.GAMMA.=.gamma.|Z.sup.N+1)|.gamma..epsilon..GAMMA.*} .[.(3.1).].
where Z.sup.N+1 represents N+1 data sets, .gamma. .Iadd.is a
partition .Iaddend.indices of the data (and thus induces a
partition of the data) .Iadd.T* is the finite collection of all
such Partitions, F is a discrete random element defines on F*,
.Iaddend..[.partition,.].and P(.GAMMA..gamma.|Z.sup.N+1) is the
posterior probability of a partition .gamma. .[.N+1 is defined
below; .]..Iadd.being true given the data Z.sup.N+1. The term
partition is defined below; .Iaddend.however, this framework is
currently sufficiently general to cover set packings and
coverings.
Consider N+1 data sets Z(k)(k=1, . . . , N+1), each consisting of
M.sub.k actual reports and a dummy report z.sub.o.sup.k, and let
denote the cumulative data set defined by
Z(k)={z.sub.i.sub.k.sup.k}.sub.i.sub.k.sup.M.sup.k=0 and
Z.sup.N+1={Z(1), . . . , Z(N+1)}, (3.2) respectively. (The dummy
report z.sub.o.sup.k serves several purposes in the representation
of missing data, false reports, initiating tracks, and terminating
tracks .Iadd.(A. B. Poore. Multidimensional assignment formulation
of data association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994).Iaddend..) In multisensor data fusion
and multitarget tracking the data sets Z(k) may represent different
classes of objects, and each data set can arise from different
sensors. For track initiation the objects are reports that must be
partitioned into tracks and false alarms. In our formulation of
track maintenance, which uses a moving window, one data set will be
tracks and remaining data sets will be reports which are assigned
to existing tracks, as false reports, or to initiating tracks.
We specialize the problem to the case of set partitioning .Iadd.(A.
B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994) .Iaddend.defined in the following way.
Define a "track of data" as {z.sub.i.sub.1.sup.1, . . . ,
z.sub.i.sub.N+1.sup.N+1} where each i.sub.k and z.sub.i.sub.k.sup.k
can assume .[.the.]. .Iadd.zero or nonzero .Iaddend.values .[.of 0
and z.sub.o.sup.k, respectively.].. A partition of the data will
refer to a collection of tracks of data wherein each report occurs
exactly once in one of the tracks of data and such that all data is
used up; the occurrence of dummy report is unrestricted. The
reference partition .Iadd..gamma..sup.o .Iaddend.is that in which
all reports are declared to be false.
Next, under appropriate independence assumptions one can show
.function..GAMMA..gamma..function..GAMMA..gamma..ident..times..times..gam-
ma..ident..pi..times..times..times..di-elect
cons..gamma..times..times..times..times. ##EQU00049## L.sub.i.sub.1
.sub.. . . iN+1 is a likelihood ratio containing probabilities for
detection, maneuvers, and termination as well as probability
density functions for report errors, track initiation and
termination. Define
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times. ##EQU00050##
Then, recognizing that
.function..GAMMA..gamma..times..times..GAMMA..gamma..function..gamma..tim-
es..GAMMA..gamma. .function..times.
.times..times..times..times..times..di-elect
cons..gamma..times..times..times..times..times. ##EQU00051## the
problem (3.1) can be expressed as the .[.N+1-dimensional.].
.Iadd.(N+1)-dimensional .Iaddend.assignment problem
.times.
.times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..di-elec-
t cons..times..times..times..times..times..times..times.
##EQU00052## where c.sub.O . . . O is arbitrarily defined to be
zero. Here, each group of stuns in the constraints represents the
fact that each non-dummy report occurs exactly once in a "track of
data." One can modify this formulation to include
.[.multiassignments.]. .Iadd.multi-assignments .Iaddend.of one,
some, or all the actual reports. The assignment problem (3.5) is
changed accordingly. For example, if z.sub.i.sub.k.sup.k is to be
assigned no more than, exactly, or no less than n.sup.i.sub.k.sup.k
times, then the "=1" in the constraint (3.5) is changed to
".ltoreq., =, .gtoreq.n.sub.i.sub.k.sup.i," respectively. In making
these changes, one must pay careful attention to the independence
assumptions, which need not be valid in many applications.
Expressions for the likelihood ratios can be found in the work of
.[.Poore, 1994, Computational Optimization & Appl., 3:27-57,.].
.Iadd.(A. B. Poore. Multidimensional assignments and multitarget
tracking, partitioning data sets, I. J. Cox, P. Hansen, and B.
Julesz, editors. DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, American Mathematical Society,
Providence, R.I., 19:169-198, 1995) .Iaddend.and the references
therein.
.Iadd.III.4. .Iaddend.Track Initiation and Maintenance
In this section we explain two multiframe assignment formulations
to the track initiation and maintenance problem. The continued use
of all prior information is computationally intensive for tracking,
so that a window sliding over the frames of reports is used as the
framework for track maintenance and track initiation within the
window. The objectives are to describe an improved version of a
simple method and then to put this into a more general framework in
which track initiation and maintenance have different length moving
windows.
.Iadd.III.4.1 .Iaddend.The First Approach to Track Maintenance and
Initiation
The first method as explained in this section is an improved
version of our first track maintenance scheme .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57, 1994)
.Iaddend.and uses the same window length for track initiation and
maintenance after the initialization step. The process is to start
with a window of length N+1 anchored at frame one. In the first
step .[.one.]. there is only .Iadd.one .Iaddend.track initiation in
that we assume no prior existing tracks. In the second and all
subsequent frames, there is a window of length N anchored at frame
k plus a collection of tracks up to frame k. This window is denoted
by {k.[.;.]..Iadd., .Iaddend.k+1, . . . , k+N}. The following
explanation of the steps is much like mathematical induction in
that we explain the first step and then step k to step k+1.
Track Maintenance and Initiation: Step 1. Let
{i.sub.1(l.sub.2),i.sub.2(l.sub.2) . . . ,
i.sub.N(l.sub.2),i.sub.N+1(l.sub.2)}.sub.l.sub.2.sup.L.sup.2=1
.[.(4.1).]. .Iadd.(3.6) .Iaddend. be an enumeration of all those
zero-one variables in the solution of the assignment problem (3.5)
(i.e., z.sub.i.sub.1.sub.i.sub.2 .sub.. . . i.sub.N+1.sub.=1)
excluding all the false reports in the solution (i.e., all those
zero-one variables with exactly one nonzero index) and zero-one
variables in the solution for which (i.sub.1,i.sub.2)=(0,0). (The
latter can correspond to tracks that initiate on frames three and
higher.) These denote our initial tracks.
Consider only the first two index sets in this enumeration
.[.(4.1).]. .Iadd.(3.6) .Iaddend.and add the zero index l.sub.2=0
with the corresponding values of i.sub.1 and i.sub.2 being zero.
Thus, the enumeration is now
{i.sub.1(l.sub.2),i.sub.2(l.sub.2)}.sub.l.sub.2.sup.L.sup.2=0. The
notation
T.sub.2(l.sub.2).ident.(z.sub.i.sub.1.sub.(l.sub.2.sub.),Z.sub.l-
.sub.2.sub.(l.sub.2.sub.)) will be used for this pairing. Suppose
now that the next data set, i.e., the (N+2).sup.th set, is added to
the problem.
To explain the costs for the new problem, one starts with the
hypothesis that a partition .gamma..epsilon..GAMMA.* being true is
now conditioned on the truth of the pairings on the first two
frames being correct. Corresponding to the sequence {T.sub.2,
(l.sub.2),z.sub.i.sub.3, . . . ,z.sub.i.sub.n+2}, the likelihood
function is then given by
.gamma..ident..pi..function..di-elect
cons..gamma..times..times..times..times..times..times..times..times..time-
s..times..times..ident..function..times..function..times..times..times..ti-
mes. ##EQU00053## Next, define the cost and the corresponding
zero-one variable
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..function..times..times.
##EQU00054## respectively. Then the track maintenance problem
Maximize {L.gamma.|.gamma..epsilon..GAMMA.* assignment
.times.
.times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times.
.times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times.
##EQU00055##
Track .[.Maintence.]. .Iadd.Maintenance .Iaddend.and Initiation:
Step k. At the beginning of the k.sup.th step, we solve the
following (N+1)-dimensional assignment problem.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times. ##EQU00056## where for the first stepl.sub.i and L.sub.1
are replaced by i.sub.1 and M.sub.1, respectively. The first index
l.sub.k in the subscripts corresponds to the sequence of tracks
{T.sub.k(l.sub.k)}.sub.l.sub.k.sup.L.sup.k=0, where
T.sub.k(l.sub.k)={z.sub.i.sub.1.sup.1l(l.sub.k), . . . ,
z.sub.i.sub.k.sup.k(l.sub.k)} is a track of data from the solution
of the problem prior to the formulation of .[.(4.4) or if.].
.Iadd.(3.10). If .Iaddend.k=1.Iadd., .Iaddend.then it is just the
first data set in frame one.
.Iadd.Basic Assumption. .Iaddend.Suppose problem .[.(4.4).].
.Iadd.(3.10) .Iaddend.has been solved and let the solution, i.e.,
those zero-one variables equal to one, be enumerated by
{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1), . . .
,i.sub.k+N(l.sub.k+1))}.sub.l.sub.k+1.sup.L.sup.k+1=1 .[.(4.5).].
.Iadd.(3.11) .Iaddend. with the following excpetions.
a. All .[.zero one.]. .Iadd.zero-one .Iaddend.variables for which
(l.sub.k,i.sub.k+1)=(0,0) are excluded. Thus, tracks that initiate
on frames after the (k+1).sup.th are not included in the list.
b. All zero-one variables whose subscripts have the form l.sub.k=0
and exactly one nonzero index in the remaining indices {i.sub.k+1,
. . . ,i.sub.k+N} are excluded These correspond to false reports on
frames p=k+1, . . . ,k+N.
c. All variables .[.(z.sub.i.sub.k.sub.j.sub.k+1 .sub.. . .
l.sub.k+N).]. .Iadd.z.sub.l.sub.k.sub.j.sub.k+1.sub.. . . i.sub.k+N
.Iaddend.for which (i.sub.k+1, . . . , i.sub.k+N)=(0,0, . . . ,0)
and l.sub.k(l.sub.k)=0 .[.in (4.5).]. .Iadd.are excluded from
(1.11).Iaddend.. In other words, the reports on the last N+1 frames
in the {T.sub.k(l.sub.k),.sub.i.sub.k+N.sup.z.sup.k+1} are all
dummy. Any solution with this feature corresponds to a terminated
track.
Given the enumeration .[.(4.5).]. .Iadd.(3.11).Iaddend., one now
fixes the assignments on the first two index sets in the list
.[.(4.5).]. .Iadd.(3.11).Iaddend.. The zero index l.sub.k+1=0 is
added to the enumeration to specify (l.sub.k(0),i.sub.k+1(0))=(0,0)
that is used to represent false reports and tracks that initiate on
frame k+2 or later, so that the enumeration .[.(4.5).].
.Iadd.(3.11) .Iaddend.is now
{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1))}.sub.l.sub.k+1.sup.L.sup.k+1=0
.[.(4.6).].
Then, for l.sub.k+1=1, . . . ,L.sub.k+1, the 1.sub.l.sub.k+1.sup.th
such track is denoted by
{T.sub.k+1(l.sub.k+1)={T.sub.k(l.sub.k(l.sub.k+1)),zk+1.sub.i.sub.k+1(l.s-
ub.k+1)} and the (N+1)-tuple
{T.sub.k+1(1.sub.k+1),z.sub.i.sub.k+2.sup.k+2, . . .
,z.sub.l.sub.k+1+N.sup.k+1+N} will denote a track
T.sub.k+1(1.sub.k+1) plus a set of reports
{z.sub.l.sub.k+2.sup.k+2, . . . ,z.sub.i.sub.k+1+N.sup.k+1+N},
actual or dummy, that are feasible with the track
T.sub.k+1(1.sub.k+1). The (N+1)-tuple
{T.sub.k+1(0),z.sub.i.sub.k+2.sup.k+2, . . .
,z.sub.i.sub.k+1+N.sup.k+1+N} will denote a track that initiates in
the sliding window, i.e., on subsequent frames. A false report in
the sliding window is one with all but one non-zero index i.sub.p
for some p=k+2, . . . , k+1+N in the (N+1)-tuple
{T.sub.k+1(0),z.sub.i.sub.k+2.sup.k+2, . . .
,z.sub.i.sub.k+1+N.sup.k+1+N}.
The corresponding hypothesis about a partition
.gamma..epsilon..GAMMA.* being true is now conditioned on the truth
of the L.sub.k+1 tracks existing at the beginning of the N-frame
window. (Thus the assignments prior to this sliding window are
fixed.) The likelihood function is given by
.times..times..times..times..times..function..times..times..function..tim-
es..times..times..times..times. ##EQU00057## Next, define the cost
and the zero-one variable by
.[.said at least one inert gas has an amount, on a molar basis,
that is greater than the amount of said oxygen in said pressurized
medium;.].
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..ident..times..times..function..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..function..times..times.
##EQU00058## respectively, so that the track extension problem,
which was originally formulated as Maximize
{L.gamma.|.gamma..epsilon..GAMMA.*}, can be expressed as exactly
the same multi-dimensional assignment in (3.4) .[.but.]. with k
replaced by k+1. Thus, we do not repeat it here.
.Iadd.III.4.2. .Iaddend.The Second Approach to Track Maintenance
and Initiation
In the first approach the window lengths were the same for both
track maintenance and initiation. This can be inefficient in that
one can usually use a shorter window for track maintenance than for
track initiation. This section addresses such a formulation.
The General Step k. To formulate a problem for track initiation and
maintenance we consider a moving window centered as frame k of
length I+J+1 denoted by .[.[k-I, . . . ,k, . . . ,k+J.].. In this
representation, the window length for track maintenance is J and
that for initiation, I+J+1. The objective will be to explain the
situation at this center and then the move to the same length
window at center k+1 denoted by .[.[k+1-I.]. .Iadd.k+1-I.Iaddend.,
. . . ,k+1, . . . ,.[.k+1+J].]., i.e., by moving the .[.frame.].
.Iadd.window to the right .Iaddend.one frame at time .[.to the
right.].. The explanation from the first step follows
hereinafter.
The notation for a track of data is
T.sub.k(l.sub.k)={z.sub.i.sub.1.sup.1(l.sub.k), . . .
,z.sub.i.sub.p.sup.p(l.sub.k), . . . ,z.sub.i.sub.k.sup.k(l.sub.k)}
.[.(4.8).]. .Iadd.(3.14) .Iaddend. where the index l.sub.k is used
for an enumeration of those reports paired together. We also use
the notation T.sub.p,k(l.sub.k) to denote the sequence of reports
belonging to track T.sub.k(l.sub.k) but restricted to frames prior
to and including p. Thus, T.sub.k(l.sub.k)=T.sub.k,k(l.sub.k)
T.sub.k(l.sub.k)={z.sub.i.sub.1.sup.1(l.sub.k), . . .
,z.sub.i.sub.p.sup.p.sub.(l.sub.k.sub.)}
T.sub.k,k(l.sub.k)=T.sub.p,k(l.sub.k).orgate.{z.sub.i.sub.p+1.sup.p+1,
. . .
,z.sub.i.sub.k-1.sup.k-1.sub.(l.sub.k.sub.),z.sub.i.sub.k.sup.k.sub.(-
l.sub.k.sub.)} for p.ltoreq.k-1. .[.(4.9).]. .Iadd.(3.15) .Iaddend.
Given this notation for the tracks and partition of the data in the
frames {k-I, . . . ,k, . . . ,k+J}.
L.sub.T.sub.p,k.sub.)l.sub.k.sub.) will denote the accumulated
likelihood ratio up to and including frame p(p.ltoreq.k) for a
track that is declared as existing on frame k as a solution of the
assignment problem. In this notation, the likelihood for
T.sub.p,k(l.sub.k) and that of the association of
{z.sub.l.sub.k+1.sup.k+1, . . . , z.sub.l.sub.k+N.sup.k+N} with
track T.sub.k(l.sub.k) is given by
L.sub.t.sub.k.sub.(l.sub.k.sub.)=L.sub.T.sub.p,k.sub.(l.sub.k.sub.)L.sub.-
i.sub.p+1.sub.(l.sub.k.sub.) . i.sub.k.sub.(i.sub.k.sub.) for any
p.ltoreq.k-1, L.sub.T.sub.k.sub.(l.sub.k.sub.).sub.k+1
.sub.i.sub.k+N=L.sub.T.sub.pk.sub.(l.sub.k.sub.)L.sub.i.sub.p+1.sub.(l.su-
b.k.sub.) i.sub.k.sub.(i.sub.k.sub.).sub.i+N for any p.ltoreq.k-1,
.[.(4.10).]. .Iadd.(3.16) .Iaddend. respectively.
The cost for the assignment of {k.sub.l.sub.k+1.sup.k+1, . . .
,z.sub.l.sub.k+N.sup.k+N} to track T.sub.k(l.sub.k) and the
corresponding .[.zero one.]. .Iadd.zero-one .Iaddend.variable are
given by
.times..times..times..times..times..function..times..times..times..times.-
.times..times..function..times..function..times..times..times..times..func-
tion..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..function..times.
##EQU00059## respectively. Likewise, for costs associated with the
false reports on the frames k-I to k and as associated with
{z.sub.i.sub.k+1.sup.k+1, . . . ,z.sub.l.sub.k+J.sup.k+J} and the
corresponding zero-one .[.variable.]. .Iadd.variables .Iaddend.are
given by:
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times.
##EQU00060##
For the window of frames in the range .[.[k-I, . . . ,k].].
.Iadd.{k-I, . . . ,k}.Iaddend., the easiest explanation of the
partition of the reports is based on the definitions
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..function..times..times..times..times..times..times..-
times..times..times..times..times..times. ##EQU00061## so that all
of those reports not used in the tracks T.sub.k(l.sub.k) on frame p
are put into the set of available reports F.sub.p,k for the
formation of tracks over the entire window. Thus, we formulate the
assignment problem as
.times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times.
.times..noteq..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..di-e-
lect
cons..times..times..noteq..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..d-
i-elect
cons..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..noteq..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..gtoreq..times.
##EQU00062##
.Iadd.Basic Assumptions. .Iaddend.Suppose problem .[.(4-14).].
.Iadd.(1.20) .Iaddend.has been solved and let the solution,
i.e..[.,.]. those zero-one variables equal to one, be
.function..times..function..times..times..times..function..function..time-
s..times..times..function..times..function..times..times..times..function.-
.times..times. ##EQU00063## with the following exceptions.
a. All .[.zero one.]. .Iadd.zero-one .Iaddend.variables in the
second list for which (i.sub.k-1, . . . ,i.sub.k,i.sub.k+1)=(0, . .
. , 0,0) are excluded. Thus, tracks that initiate on frames after
the (k+1).sup.th are not included in the list.
b. All false reports .[.axe.]. .Iadd.are .Iaddend.excluded, i.e.,
all zero-one variables in the second list whose subscripts have
exactly one nonzero index.
c. All variables z.sub.l.sub.k.sub.l.sub.k+1.sub.. . .
i.sub.k+N.sup.T for which (i.sub.k+1, . . . ,i.sub.K+N)=(0,0, . . .
,0) and z(p).andgate.T.sub.k(l.sub.k)=0 for p=k-K, . . . ,k where
K.gtoreq.0 is user specified. Thus the track T.sub.k(l.sub.k) is
terminated if it is not observed over K+J+1 frames.
Given the enumeration .[.(4.15).]. .Iadd.(3.20).Iaddend., one now
fixes the assignments on the all index sets up to and including the
(k+1).sup.th index sets.
.function..function..function..function..times..times..times..function..t-
imes..function..function..times..times..times..times.
##EQU00064##
Thus one can now formulate the assignment problem for the next
problem exactly as in .[.(4-14).]. .Iadd.(3.20) .Iaddend.but with k
replaced by k+1. Thus, we do not repeat it here.
The Initial Step. Here is one explanation for the initial step.
First, assume that N=I+J. In this case, we start the track
initiation with a solution of (3.5). Let
{i.sub.1(l.sub.2),l.sub.2(l.sub.2) . . .
i.sub.N(l.sub.2),i.sub.N+1(l.sub.2)}.sub.l.sub.2.sup.L.sup.2=0 be
an enumeration of the solution set of (3.5), i.e., those zero-one
variables
z.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.) . . .
i.sub.N+1.sub.(l.sub.2.sub.)=1, including z.sub.OO . . . O=1
corresponding to l.sub.2=0, but excluding all those zero-one
variables that are assigned to one and correspond to false reports
(i.e.. there is exactly one nonzero index in the subscript of
z.sub.i.sub.1.sub.i.sub.2 .sub.. . . i.sub.N+1), all those zero-one
variables that are assigned to one and correspond to tracks that
initiate on frames higher than I+2. Then we fix the data
association decisions corresponding to the reports in our list of
tracks prior to and including frame k+1=I+2. This defines the k for
problem (4.4) and one can then continue the development by adding a
frame to the window as in the general case.
If I+J>N, then one possibility is to start the process with N+1
frames, and assuming J.ltoreq.N, proceed as before replacing I by
N-J for the moment, and continue to add frames without lopping off
the first frame in the window until reaches a window of length
I+J+1. Then we proceed as in the previous paragraph.
If I+J<N, then one can solve the track initiation problem (3.5),
formulate the problem with the center of the window at k+1=N+1-J,
enumerate the solutions as above, and lop off the first N-J-I
frames. Then, we proceed just as in the case I+J=N.
A primary objective in this work has been to demonstrate how
multidimensional assignment problems arise in the tracking
environment. The problem of track initiation and maintenance has
been formulated within the framework of a moving window over the
frames of data. The solution of these NP-hard, noisy, large scale,
and sparse problems to the noise level in the problem is
fundamental to superior track estimation and identification. Thus,
one must utilize the special structure in the problems as well take
advantage of special information that is available. Since these
moving windows are overlapping, there are some algorithm
efficiencies that can be identified and that take advantages of the
overlap in the windows from one frame of reports to the next. Here
is an example of the use of a primal solution of one problem to
warm start the solution of the next problem in the sequence.
.Iadd.III.5. CONCLUDING COMMENTS .Iaddend.
Suppose we have solved problem .[.(4.4).]. .Iadd.(3.10)
.Iaddend.and have enumerated all those zero-one variables in the
solution of .[.(4.4).]. .Iadd.(3.10) .Iaddend.as in .[.(4.5).].
.Iadd.(3.11).Iaddend.. Add the zero index l.sub.k+1=0, so that the
enumeration is {(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1), . . .
,i.sub.k+N(l.sub.k+1))} .[.(5.1).]. .Iadd.(3.23) .Iaddend. With
this enumeration one can define the cost by
c.sub.l.sub.k+1.sub.J.sub.k+1+N=c.sub.i.sub.k.sub.(i.sub.k+1.sub.),j.sub.-
k+1.sub.(l.sub.k+1.sub.), . . . ,
i.sub.N+1.sub.(j.sub.k+1.sub.)i.sub.k+1+N .[.(5.2).]. .Iadd.(3.24)
.Iaddend. and the .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem
.PHI..ident..times.
.times..times..times..times..times..times..times..times..ident..function.-
.times..times..times.
.times..times..times..times..times..times..times..times..times..times..di-
-elect cons..times..times..times..times..times..times..times.
##EQU00065## Let w be an optimal or feasible solution of two
two-dimensional assignment problem and define
.times..times..times..times..times..times..times..times..function..times.-
.function..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times. ##EQU00066## This need
not satisfy the constraints in that there are usually many objects
left unassigned. Thus. one can complete the assignment by using the
zero-one variables in .[.(4.4).]. .Iadd.(3.10) .Iaddend.with k
replaced by k+1 with exactly one nonzero index corresponding to any
unassigned object or data report.
For the dual solutions, the multipliers arising from the solution
of the .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem .[.(5.3).]. .Iadd.(3.25)
.Iaddend.corresponding to the second variable, i.e.,
{u.sub.l.sub.k+1+N.sup.k+1+N}.sub.l.sub.k+1+N.sup.M.sup.k+1+N=0.
These are good initial values .[.for.]. .Iadd.to .Iaddend.use in a
relaxation scheme .[.[11.12].]. .Iadd.(A. B. Poore and N. Rijavec.
Partitioning multiple data sets: multidimensional assignments and
Lagrangian relaxation, in quadratic assignment and related
problems; P. M. Pardalos and H. Wolkowicz, editors, DIMACS series
in Discrete Mathematics and Theoretical Computer Science, 16:25-37,
1994; A. J. Robertson III. A class of lagrangian relaxation
algorithms for the multidimensional assignment problem. Ph.D.
Thesis, Colorado State University, Ft. Collins, Colo.,
1995).Iaddend.. Finally, note that one can also develop a warm
start for problem .[.(4.14).]. .Iadd.(3.20) .Iaddend.in a similar
fashion.
.Iadd.IV. REVIEW OF NEW RELAXATION SCHEMES .Iaddend.
.Iadd.IV.1. Introduction .Iaddend.
Multidimensional assignment problems govern the central problem of
data association in multisensor and multitarget tracking. i.e., the
problem of partitioning observations from multiple scans of the
surveillance region and from either single or multiple sensors into
tracks and false alarms. This fundamental problem can be stated as
.Iadd.(A. B. Poore, Multidimensional assignments and multitarget
tracking, partitioning data sets, I. J. Cox, P. Hansen, and B.
Julesz, editors, DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, American Mathematical Society,
Providence, R.I., 19:169-198, 1995; A. B. Poore. Multidimensional
assignment formulation of data association problems arising from
multitarget tracking and multisensor data fusion. Computational
Optimization and Applications, 3:27-57, 1994) .Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
di-elect
cons..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..di-elect
cons..times..times..times..times..times..times..times..times.
##EQU00067##
where c.sub.o . . . o is arbitrarily defined to be zero and is
included for notational convenience. One can modify this
formulation to include .[.multiassignments.].
.Iadd.multi-assignments .Iaddend.of one, some, or all of the actual
reports. The zero index is used in representing missing data, false
alarms, initiating and terminating tracks .Iadd.as described in (A.
B. Poore, Multidimensional assignments and multitarget tracking,
partitioning data sets, I. J. Cox, P. Hansen, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, American Mathematical Society, Providence, R.I.,
19:169-198, 1995; A. B. Poore. Multidimensional assignment
formulation of data association problems arising from multitarget
tracking and multisensor data fusion. Computational Optimization
and Applications, 3:27-57, 1994).Iaddend.. In these problems, we
assume that all zero-one variables z.sub.i.sub.1 .sub.. . . i.sub.N
with precisely one nonzero index are free to be assigned and that
the corresponding cost coefficients are well-defined. (This is a
valid assumption in the tracking environment .Iadd.(A. B. Poore,
Multidimensional assignments and multitarget tracking, partitioning
data sets, I. J. Cox, P. Hansen, and B. Julesz, editors, DIMACS
Series in Discrete Mathematics and Theoretical Computer Science,
American Mathematical Society, Providence, R.I., 19:169-198, 1995;
A. B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994).Iaddend..) Although not required,
these cost coefficients with exactly one nonzero index can be
translated to zero by cost shifting .Iadd.(A. B. Poore and N.
Rijavec. A lagrangian relaxation algorithm for multidimensional
assignment problems arising from multitarget tracking. SIAM Journal
of Optimization, 3, No. 3:544-563, 1993) .Iaddend.without changing
the optimal assignment. Finally, this formulation is of sufficient
generality to include the symmetric problem and the asymmetric
inequality problem .Iadd.(A. J. Robertson III. A class of
lagrangian algorithms for the multidimensional assignment problem.
Ph.D. Thesis, Colorado State University, Ft. Collins, Colo.
1995).Iaddend..
The data association problems in tracking that are formulated as
.[.(1.1).]. .Iadd.Equation (4.1) .Iaddend.have several
characteristics. They are normally sparse, the cost coefficients
are noisy and the problem is NP-hard .Iadd.(M. R. Galey and D. S.
Johnson. Computers and Intractability. W. H. Freeman and Company,
San Francisco, Calif., 1979).Iaddend., but it must be solved in
real-time. The only known methods for solving this NP-hard problem
optimally are enumerative in nature, with branch-and-bound being
the most efficient; however, such methods are much too slow for
real-time applications. Thus one must resort to suboptimal
approaches. Ideally, any such algorithm should solve the problem to
within the noise level, assuming, of course, that one can measure
this noise level in the physical problem and .Iadd.that
.Iaddend.the mathematical method provides a way to decide if the
criterion has been met.
There are many algorithms that can be used to construct suboptimal
solutions to NP-hard combinatorial optimization problems. These
include greedy (and its many variants), relaxation, simulated
annealing, tabu search, genetic algorithms, and neural .[.netork.].
algorithms .Iadd.(C. H. Papadimitriou and K. Steiglitz.
Combinatorial Optimization: Algorithm and Complexity.
Prentice-Hall, Inc., Englewood Cliffs, N.J., 1982; J. Pearl.
Heuristics: Intelligent Search Strategies for Computer Problem
Solving. Addison-Wesley, Reading, Mass., 1984; C. R. Reeves ed.
Modern Heuristic Techniques for Combinatorial Problems. Halstead
Press, Wiley, N.Y., 1993).Iaddend.. For the .[.three dimensional.].
.Iadd.three-dimensional .Iaddend.assignment problem. Pierskalla
.Iadd.(W. Pierskalla. The tri-substitution for three-dimensional
assignment problem. Journal du CORS, 5:71-81, 1967)
.Iaddend.developed the tri-substitution method. which is a variant
of the simplex method. Frieze and Yadegar .Iadd.(A. M. Frieze and
J. Yadegar, An algorithm for solving 3-dimensional assignment
problems with application to scheduling a teaching practice.
Journal of the Operational Research Society, 39:989-955,
1981).Iaddend.introduced a method based on Lagrangian relaxation in
which a feasible solution is recovered using information provided
by the relaxed solution. Our choice of approaches is strongly
influenced by the need to balance real-time performance and
solution quality. Lagrangian relaxation based methods have been
used successfully in prior tracking applications .Iadd.(K. R.
Pattipati, S. Deb, and Y. Bar-Shalom. A s-dimensional assignment
algorithm for track initiation. In Proceedings of the IEEE Systems
Conference, Kobe, Japan, pages 127-130, 1992; Y. Bar-Shalom, S.
Deb, K. R. Pattipati, and H. Tsanakis. A new algorithm for the
generalized multidimensional assignment problem. In Proceedings of
the IEEE International Conference on Systems, Math, and
Cybernetics, Chicago, pages 132-136, 1992; A. B. Poore and N.
Rijavec. A numerical study of some data association problems
arising in multitarget tracking. Large Scale Optimization: State of
the Art, W. W. Hager, D. W. Hearn and P. M. Pardalos, editors.
Kluwer Academic Publishers B.V., Boston, pages 339-361, 1994; A. B.
Poore and N.Rijavec. Partitioning multiple data sets:
multidimensional assignments and lagrangian relaxation. In P. M.
Pardalos and H. Wolkowicz, editors, Quadratic assignment and
related problems: DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, volume 16, pages 25-37, 1994; A. B.
Poore and N. Rijavec. A lagrangian relaxation algorithm for
multidimensional assignment problems from multitarget tracking.
SIAM Journal of Optimization,3, No. 3:544-563, 1993).Iaddend.. An
advantage of these methods is that they provide both an upper and
lower bound on the optimal solution, which can then be used to
measure the solution quality. These works extend the method of
Frieze and Yadegar .Iadd.(A. M. Frieze and J. Yadegar. An algorithm
for solving 3-dimensional assignment problems with application to
scheduling a teaching practice. Journal of the Operational Research
Society, 32:989-995, 1981) .Iaddend.to the multidimensional
case.
.Iadd.IV.2. .Iaddend.Probabilistic Framework for Data Association.
.[.(ABP).].
The goal of this section is to explain the formulation of the data
association problems that governs large classes of data association
problems in centralized or hybrid centralized-sensor level
multisensor/multitarget tracking. The presentation is brief;
technical details are presented for both track initiation and
maintenance in the work of .[.Poore.]. .Iadd.(A. B. Poore.
Multidimensional assignments and multitarget tracking, partitioning
data sets, I. J. Cox, P. Hansen, and B. Julesz, editors, DIMACS
Series in Discrete Mathematics and Theoretical Computer Science,
American Mathematical Society, Providence, R.I., 19:169-198, 1995;
A. B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994).Iaddend.. The formulation is of
sufficient generality to cover the MHT work of Reid, Blackman and
Stein .Iadd.(S. S. Blackman. Multiple Target Tracking with Radar
Applications. Artech. House, Norwood, Mass., 1986) .Iaddend.and
modifications by Kurien .Iadd.(T.Kurien. Issues in the designing of
practical multitarget tracking algorithms. In
Multitarget-Multisensor Tracking: Advanced Applications by Y.
Bar-Shalom. Artech House, Mass., 1990) .Iaddend.to include
maneuvering targets. As suggested by Blackman .Iadd.(S. S.
Blackman, Multiple Target Tracking with Radar Applications, Artech
House, Norwood, Mass., 1986).Iaddend., this formulation can also be
modified to include target features (e.g., size and type) into the
scoring function. The recent work .[.of Poore and Drummond.].
.Iadd.(A. B. Poore and O. E. Drummond. Track initiation and
maintenance using multidimensional assignment problems. In D. W.
Hearn, W. W. Hager, and P. M. Pardalos, editors, Network
Optimization, volume 450, pages 407-422, Boston, 1996. Kluwer
Academic Publishers B.V.) .Iaddend.significantly extends the work
of this section to new approaches for multiple sensor centralized
tracking. Future work will involve extensions to track-to-track
correlation.
The data association problems for multisensor and multitarget
tracking considered in this work are generally posed as that of
maximizing the posterior probability of the surveillance region
(given the data) according to Maximize
.DELTA.P(.GAMMA.=.gamma.|Z.sup.N)|.gamma..GAMMA.*} .[.(2.1).].
.Iadd.(4.2) .Iaddend. where Z.sup.N represents N data sets. .gamma.
.Iadd.is a partition of indices .Iaddend.of the data (and thus
induces a partition of the data). .[..GAMMA..delta..].
.GAMMA..GAMMA..times..times..delta..UPSILON. ##EQU00068##
.[.P(.GAMMA. .gamma.|Z.sup.N).]. .Iadd..GAMMA.* is the finite
collection of all such partitions, .GAMMA. is a discrete random
element defined on .GAMMA.*, and P(.GAMMA.=.gamma.|Z.sup.N)
.Iaddend.is the posterior probability of a partition .gamma.
.Iadd.being .Iaddend.true given the data Z.sup.N. The term
partition is defined below; however, this framework is currently
sufficiently general to cover set packings and coverings .Iadd.(A.
B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994; A. B. Poore. Multidimensional
assignments and multitarget tracking, partitioning data sets, I. J.
cox, P. Hansen, and B. Julesz, editors, DIMACS Series in Discrete
Mathematics and Theoretical Computer Science, American Mathematical
Society, Providence, R.I., 19:169-198, 1995).Iaddend..
Consider N data sets Z(k) (k=I, . . . ,N) each of M.sub.k reports
{z.sub.i.sub.k.sup.k}.sub.i.sub.k.sup.M.sub.k=1, and let Z.sup.N
denote the cumulative data set defined by
Z(k)={z.sub.i.sub.k.sup.k}.sub.i.sub.k.sup.Mk=1 and Z.sup.N={Z(1),
. . . ,Z(N)} .[.(2.2).]. .Iadd.(4.3) .Iaddend. respectively. In
multisensor data fusion and multitarget tracking the data sets Z(k)
may represent different classes of objects, and each data set can
arise from different sensors. For track initiation the objects are
measurements that must be partitioned into tracks and false alarms.
In our formulation of track maintenance .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57, 1994;
A. B. Poore. Multidimensional assignments and multitarget tracking,
partitioning data sets, I. J. cox, P. Hansen, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, American Mathematical Society, Providence, R.I.,
19:169-198, 1995).Iaddend., which uses a moving window over time,
one data set will be tracks and remaining data sets will be
measurements which are assigned to existing tracks, as false
measurements, or to initiating tracks. In sensor level tracking,
the objects to be fused are tracks .Iadd.(S. S. Blackman. Multiple
Target Tracking with Radar Applications. Artech House, Norwood,
Mass., 1986).Iaddend.. In centralized fusion .Iadd.(S. S. Blackman.
Multiple Target Tracking with Radar Applications. Artec House,
Norwood, Mass., 1986).Iaddend., the objects may all be measurements
that represent targets or false reports, and the problem is to
determine which measurements emanate from a common source.
We specialize the problem to the case of set partitioning .Iadd.(A.
B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994; A. B. Poore. Multidimensional
assignments and multitarget tracking, partitioning data sets, I. J.
Cox, P. Hansen, and B. Julesz, editors, DIMACS Series in Discrete
Mathematics and Theoretical Computer Science, American Mathematical
Society, Providence, R.I., 19:169-198, 1995) .Iaddend.defined in
the following way. First, for notational convenience in
representing tracks, we add a zero index to each of the index sets
in .[.(2.2).]. .Iadd.Equation (4.3).Iaddend., a dummy report
z.sub.o.sup.k to each of the data sets Z(k) in .[.(2.2).].
.Iadd.Equation (4.3).Iaddend., and define a "track of data" as
(z.sub.i.sub.j.sup.1,z.sub.i.sub.N.sup.N) where i.sub.k and
z.sub.i.sub.i.sup.k can now assume the .[.values.]. .Iadd.value
.Iaddend.of 0 and z.sub.O.sup.k .[.respectively.].. A partition of
the data will refer to a collection of tracks of data wherein each
report occurs exactly once in one of the tracks of data and such
that all data is used up; the occurrence of dummy report is
unrestricted. The dummy report z.sub.o.sup.k serves several
purposes in the representation of missing data, false reports,
initiating tracks, and terminating tracks .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57, 1994;
A. B. Poore. Multidimensional assignments and multitarget tracking,
partitioning data sets, I. J. cox, P. Hansen, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, American Mathematical Society, Providence, R.I.,
19:169-198, 1995).Iaddend.. The reference partition is that in
which all reports are declared to be false.
Next under appropriate independence assumptions one can show
.Iadd.(A. B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994; A. B. Poore. Multidimensional
assignments and multitarget tracking, partitioning data sets, I. J.
cox, P. Hansen, and B. Julesz, editors, DIMACS Series in Discrete
Mathematics and Theoretical Computer Science, American Mathematical
Society, Providence, R.I., 19:169-198, 1995).Iaddend..
.function..GAMMA..UPSILON..function..GAMMA..UPSILON..ident..UPSILON..iden-
t..times..times..times..di-elect
cons..times..UPSILON..times..times..times..times..times..times.
##EQU00069## .Iadd.where .gamma..sup.0 is a reference partition,
.Iaddend.L.sub.i.sub.1 .sub.. . . i.sub.N is a likelihood ratio
containing probabilities for detection, maneuvers, and termination
as well as probability density functions for measurement errors,
track initiation and termination. Then if c.sub.i.sub.1 .sub.. . .
i.sub.N=-1nL.sub.i.sub.1 .sub.. . . i.sub.N,
.times..function..GAMMA..UPSILON..function..GAMMA..UPSILON..times..times.-
.times..di-elect
cons..times..UPSILON..times..times..times..times..times..times..times..ti-
mes. ##EQU00070## Using .[.(2.3).]. .Iadd.Equation (4.4)
.Iaddend.and the zero-one variable z.sub.i.sub.1 .sub.. . .
i.sub.N=1 (if i.sub.1, . . . , i.sub.N).epsilon..gamma. and 0
otherwise, one can then write the problem .[.(2.1).]. .Iadd.(4.4)
.Iaddend.as the following N-dimensional assignment problem:
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00071##
where c.sub.o . . . o is arbitrarily defined to be zero. Here, each
group of sums in the constraints represents the fact that
each-non-dummy report occurs exactly once in a "track of data." One
can modify this formulation to include .[.multiassignments.].
.Iadd.multi-assignments .Iaddend.of one, some, or all the actual
reports. The assignment problem .[.(2.5).]. .Iadd.Equation (4.6)
.Iaddend.is changed accordingly. For example, if
z.sub.i.sub.k.sup.k is to be assigned no more than, exactly, or no
less than n.sub.i.sub.k .sup.k times, then the "=1" in the
constraint .[.(2.5).]. .Iadd.(4.6) .Iaddend.is changed to .ltoreq.,
=, .gtoreq.n.sub.i.sub.k.sup.k," respectively. Modifications for
group tracking and multiresolution features of the surveillance
region will be addressed in future work. In making these changes,
one must pay careful attention to the independence assumptions that
need not be valid in many applications.
Expressions for the likelihood ratios L.sub.i.sub.1.sub., . . .
,l.sub.N, can be found in the work of Poore .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57, 1994;
A. B. Poore. Multidimensional assignments and multitarget tracking,
partitioning data sets, I. J. cox, P. Hansen, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, American Mathematical Society, Providence, R.I.,
19:169-198, 1995).Iaddend.. These expressions include the
developments of Reid, Kurien .Iadd.(T. Kurien, Issues in the
designing of practical multitarget tracking algorithms. In
Multitarget-Multisensor Tracking: Advanced Applications by Y.
Bar-Shalom. Artech House, Mass., 1990).Iaddend., and Stein and
Blackman .Iadd.(S. S. Blackman. Multiple Target Tracking with Radar
Applications. Artech House, Norwood, Mass., 1986).Iaddend.. What's
more, they are easily modified to include target features and to
account for different sensor types. In track initiation, the N data
sets all represent reports from N sensors, possibly all the same.
For track maintenance, we use a sliding window of N data sets and
one data set containing established tracks. .Iadd.(A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57, 1994;
A. B. Poore. Multidimensional assignments and multitarget tracking,
partitioning data sets, I. J. cox, P. Hansen, and B. Julesz,
editors, DIMACS Series in Discrete Mathematics and Theoretical
Computer Science, American Mathematical Society, Providence, R.I.,
19:169-198, 1995).Iaddend.. The formulation is the same as above
except that the dimension of the assignment problem is now N+1.
.[.@@@.].
.[.To explain the costs for the new problem, one starts with the
hypothesis that a partition .gamma..epsilon..GAMMA.* conditioned on
the truth of the pairings on the first two frames being correct.
Corresponding to the sequence {T.sub.2(l.sub.2), z.sub.i.sub.3, . .
. z.sub.j.sub.N+2}, the likelihood function is then given by.].
.times..GAMMA..times..times..gamma..ident..function..times..di-elect
cons..gamma..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..ident..function..times..function..ti-
mes..times..times..times..times..times. ##EQU00072##
.[.Next, define the coset and the corresponding zero-one
variable.].
.times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..function..times..times..times.
##EQU00073## .[.respectively. Then the track maintenance problem
Maximize {L.gamma.|.gamma..epsilon..GAMMA.* assignment.].
.times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times.
##EQU00074##
.[.Track Maintence and Initiation: Step k. At the beginning of the
k.sup.th step. we solve the following (N+1)-dimensional assignment
problem..].
.times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times.
.times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times.
##EQU00075## .[.where for the first step l.sub.1 and L.sub.1 are
replaced by i.sub.1 and M.sub.1, respectively. The first index
l.sub.k in the subscripts corresponds to the sequence of tracks
{T.sub.k(l.sub.k)}.sub.l.sub.k.sup.L.sup.k=0 where
T.sub.k(l.sub.k)={z.sub.i.sub.1.sup.1(l.sub.k), . . .
,z.sub.i.sub.k.sup.k(l.sub.k)} is a track of data from the solution
of the problem prior to the formulation of (4.4) or if k=1 then it
is just the first data set in frame one..].
.[.Suppose problem (4.4) has been solved and let the solution,
i.e., those zero-one variables equal to one, be enumerated by.].
.[.{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1), . . .
,i.sub.k+N(l.sub.k+1))}.sub.l.sub.k+1.sup.Lk+1=1 (4.5).]. .[.with
the following exceptions..].
.[.a. All zero one variables for which (l.sub.k,l.sub.k+1)=(0,0)
are excluded. Thus, tracks that initiate on frames after the
(k+1).sup.th are not included in the list..].
.[.b. All zero-one variables whose subscripts have the form
l.sub.k=3 and exactly one nonzero index in the remaining indices
{i.sub.k+1, . . . ,i.sub.k+N} are excluded. These correspond to
false reports on frames p=k+1, . . . ,k+N..].
.[.c. All variables Z.sub.l.sub.k.sub.i.sub.k.sub.+l . . . ik+N)
for which (ik+1, . . . , ik+N)=(0,0, . . . ,0) and
i.sub.k(I.sub.k)=0 in (4.5). In other words, the reports on the
last N+1 frames in the {T.sub.k(l.sub.k), z.sub.i.sub.k+N.sup.k+1}
are all dummy. Any solution with this feature corresponds to a
terminated track..].
.[.Given the enumeration (4.5), one now fixes the assignments on
the first two index sets in the list (4.5). The zero index
l.sub.k+1=0 is added to the enumeration to specify
(l.sub.k(0),i.sub.k+1(0))=(0,0) that is used to represent false
reports and tracks that initiate on frame k+2 or later, so that the
enumeration (4.5) is now.].
.[.{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1))}.sub.l.sub.k+1.sup.Lk+1=0
(4.6).].
.[.Then, for l.sub.k+1=1, . . . ,L.sub.k+1, the l.sub.k+1.sup.th
such track is denoted by
T.sub.k+1(l.sub.k+1={T.sub.k(l.sub.k(l.sub.k+1)),z.sub.i.sub.k+1
k.sup.k+1}, and the (N+1)-tuple
{T.sub.k+1(l.sub.k+1),z.sub.i.sub.k+2.sup.k+1, . . .
,z.sub.l.sub.k+1+N.sup.k+1+N} will denote a track
T.sub.k+1(I.sub.k+1) plus a set of reports {.sub.l.sub.k+2.sup.k+2,
. . . ,z.sub.i.sub.k+1+N.sup.k+1+N}, actual or dummy. that are
feasible with the track T.sub.k+1(l.sub.k+1). The (N+1)-tuple
{T.sub.k+1(0), z.sub.l.sub.k+2.sup.k+2, . . .
,z.sub.i.sub.k+1+N.sup.k+1+N}, will denote a track that initiates
in .sup.2the sliding window, i.e., on subsequent frames. A false
report in the sliding window is one with all but one non-zero index
i.sub.P for some p=k+2, . . . , k+1+N in the (N+1)-tuple
{T.sub.k+1(0), z.sub.l.sub.k+2.sup.k+2, . . .
,z.sub.l.sub.k+1+N.sup.k+i+N}..].
.[.The corresponding hypothesis about a partition
.gamma..epsilon..GAMMA.* being true is now conditioned on the truth
of the L.sub.k+1 tracks existing at the beginning of the N-frame
window. (Thus the assignments prior to this sliding window are
fixed.) The likelihood function is given by.].
.times..times..times..gamma..function..times..di-elect
cons..gamma..times..times..times..times..times..times..times..times..time-
s..function..times..times..times..function..times..times. .times.
##EQU00076## .[.Next, define the cost and the zero-one variable
by.].
.times..times..times..times..times..times..times. .times.
##EQU00077## .[.respectively, so that the track extension problem,
which was originally formulated as Maximize
{L.sub..gamma.|.gamma..epsilon..GAMMA..delta. as exactly the same
multi-dimensional assignment in (3.4) but with k replaced by k+1.
Thus, we do not repeat it here..].
.[.The Second Approach to Track Maintenance and Initiation.].
.[.In the first approach the window lengths were the same for both
track maintenance and initiation. This can be inefficient in that
one can usually use a shorter window for track maintenance than for
track initiation. This section addresses such a formulation..].
.[.The General Step k. To formulate a problem for track initiation
and maintenance we consider a moving window centered as frame k of
length I+J+1 denoted by [k-I, . . . , k, . . . ,k+J]. In this
representation, the window length for track maintenance is J and
that for initiation, I+J+1. The objective will be to explain the
situation at this center and then the move to the same length
window at center k+1 denoted by [k+1-I, . . . ,k+1, . . . .
,k+1+J], i.e., by moving the frame one frame at time to the right.
The explanation from the first step follows hereinafter..].
.[.The notation for a track of data is.].
.[.T.sub.k(l.sub.k)={z.sub.i.sub.1.sup.1(l.sub.k), . . .
,z.sub.i.sub.p.sup.p(l.sub.k), . . . ,
z.sub.i.sub.k.sup.k(l.sub.k)} (4.8).]. .[.where the index l.sub.k
is used for an enumeration of those reports paired together. We
also use the notation T.sub.p,k(l.sub.k) to denote the sequence of
reports belonging to track T.sub.k(l.sub.k) but restricted to
frames prior to and including p. Thus,.].
.[.T.sub.k(l.sub.k)=T.sub.k,k(l.sub.k).].
.[.T.sub.k(l.sub.i)={z.sub.i.sub.j.sup.1(l.sub.k), . . .
,z.sub.i.sub.p.sup.p(l.sub.k)}.].
.[.T.sub.k,k(l.sub.k)=T.sub.p,k(l.sub.k).orgate.{z.sub.i.sub.p+1.sub.(l.s-
ub.k.sub.).sup.p+1, . . .
,z.sub.i.sub.k-1.sub.(l.sub.k.sub.).sup.k-1,z.sub.i.sub.k.sub.(l.sub.k.su-
b.).sup.5} for p.ltoreq.k-1.(4.9).]. .[.Given this notation for the
tracks and partition of the data in the frames {k-I, . . . ,k, . .
. , k+J}. .sup.LT.sub.p,k(I.sub.k) will denote the accumulated
likelihood ratio up to and including frame p(p.ltoreq.k) for a
track that is declared as existing on frame k as a solution of the
assignment problem. In this notation, the likelihood for T
.sub.p,k(l.sub.k) and that of the association of
{z.sub.i.sub.k+1.sup.k+1, . . . ,z.sub.i.sub.k+n.sup.k+N} with with
track T.sub.k(l.sub.k) is given by.].
.times..times..times..times..times..function..function..times..times-
..times..times..times..times..function..function..times..function..times..-
times..times..times..function..times..times..times..times..times..times..t-
imes..times..times..times..times..times. ##EQU00078##
.[.respectively..].
.[.The cost for the assignment of {z.sub.i.sub.k+1.sup.k+1, . . .
,z.sub.i.sub.k+N.sup.k+N} to track T.sub.k(l.sub.k) and the
corresponding zero one variable are given by.].
.times..times..times..times..times..function..function..times..times..t-
imes..times..times..times..function..function..times..function..times..tim-
es..times..function..times..times..times..times..times..times..times..time-
s..times..times..times..times..times. ##EQU00079##
.[.is assigned to track T.sub..lamda.(l.sub.k), respectively.
Likewise, for costs associated with the false reports on the frames
k-I to k and as associated with {z.sub.l.sub.k+1.sup.k+1, . . .
,z.sub.i.sub.k+j.sup.k+J} and the corresponding zero-one variable
are given by:.].
.times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times. ##EQU00080##
.[.For the window of frames in the range [k-I, . . . ,k], the
easiest explanation of the partition of the reports is based on the
definitions.].
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..function..times..times..times..-
times..times..times..times..times..times..times. ##EQU00081## .[.so
that all of those reports not used in the tracks T.sub.k(I.sub.k)
on frame p are put into the set of available reports F.sub.p,k for
the formation of tracks over the entire window. Thus, we formulate
the assignment problem as.].
.times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..no-
teq..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..di-elect
cons..times..times..noteq..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..gtoreq..times..times.
.times..times. ##EQU00082##
.[.Suppose problem (4.14) has been solved and let the solution,
i.e., those zero-one variables equal to one, be enumerated
by.].
.times..function..times..function..times..times..times..times..functi-
on..times..times..times..function..times..function..times..times..times..t-
imes..function. ##EQU00083## .[.with the following
exceptions..].
.[.a. All zero one variables in the second list for which
(i.sub.k-1, . . . , k.sub.k,i.sub.k+1)=(0, . . . ,0,0) are
excluded. Thus, tracks that initiate on frames after the
(k+1).sup.th are not included in the list..].
.[.b. All false reports are excluded, i.e., all zero-one variables
in the second list whose subscripts have exactly one nonzero
index..].
.[.c. All variables z.sub.i.sub.k.sub.1.sub.k+1 . . . l.sub.k+N for
which (i.sub.k+1, . . . ,i.sub.k+N=(0, 0, . . . ,0) and
Z(p).andgate.T.sub.k(l.sub.k)={0} for p=k-K, . . . ,k where
K.gtoreq.0 is user specified. Thus the track T.sub.k(I.sub.k) is
terminated if it is not observed over K+J+1 frames..].
.[.Given the enumeration (4.15), one now fixes the assignments on
the all index sets up to and including the (k+1).sup.th index
sets..].
.times..function..function..times..times..times..times..function..times-
..function..function..times..times..times..times. ##EQU00084##
.[.Thus one can now formulate the assignment problem for the next
problem exactly as in (4.14) but with k replaced by k+1. Thus, we
do not repeat it here..].
.[.The Initial Step. Here is one explanation for the initial step.
First, assume that N=I+J. In this case, we start the track
initiation with a solution of (3.5). Let.].
.[.{i.sub.2(l.sub.2),i.sub.2(l.sub.2) . . .
,i.sub.N(l.sub.2),i.sub.N+1(l.sub.2)}.sub.l.sub.2.sup.L.sup.2=0 .].
.[.be an enumeration of the solution set of (3.5), i.e., those
zero-one variables
z.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.) . . .
i.sub.N+1.sub.(l.sub.2.sub.)=1, including z.sub.OO . . . O=1
corresponding to l.sub.2=0, but excluding all those zero-one
variables that are assigned to one and correspond to false reports
(i.e., there is exactly one nonzero index in the subscript of
z.sub.i.sub.1.sub.i.sub.2 . . . i.sub.N+1), all those zero-one
variables that are assigned to one and correspond to tracks that
initiate on frames higher than I+2. Then we fix the data
association decisions corresponding to the reports in our list of
tracks prior to and including frame k+1=I+2. This defines the k for
problem (4.4) and one can then continue the development by adding a
frame to the window as in the general case..].
.[.If I+J>N, then one possibility is to start the process with
N+1 frames, and assuming J.ltoreq.N, proceed as before replacing I
by N-J for the moment, and continue to add frames without lopping
off the first frame in the window until reaches a window of length
I+J+1. Then we proceed as in the previous paragraph..].
.[.If I+J<N, then one can solve the track initiation problem
(3.5), formulate the problem with the center of the window at
k+1=N+1-J, enumerate the solutions as above, and lop off the first
N-J-I frames. Then, we proceed just as in the case I+J=N..].
.[.A primary objective in this work has been to demonstrate how
multidimensional assignment problems arise in the tracking
environment. The problem of track initiation and maintenance has
been formulated within the framework of a moving window over the
frames of data. The solution of these NP-hard, noisy, large scale,
and sparse problems to the noise level in the problem is
fundamental to superior track estimation and identification. Thus,
one must utilize the special structure in the problems as well take
advantage of special information that is available. Since these
moving windows are overlapping, there are some algorithm
efficiencies that can been identified and that take advantages of
the overlap in the windows from one frame of reports to the next.
Here is an example of the use of a primal solution of one problem
to warm start the solution of the next problem in the
sequence..].
.[.Suppose we have solved problem (4.4) and have enumerated all
those zero-one variables in the solution of (4.4) as in (4.5). Add
the zero index l.sub.k+1=0, so that the enumeration is.].
{.[.(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1), . . .
,i.sub.k+N(l.sub.k+1))}.sub.i.sub.k+1.sup.Lk+1=0 (5.1).]. .[.With
this enumeration one can define the cost by.].
.[.c.sub.i.sub.k+1.sub.i.sub.k+1+N=c.sub.i.sub.k.sub.(l.sub.k+1.sub.).sub-
.k+1.sub.(l.sub.k+1.sub.), . . . ,
i.sub.N+1.sub.(l.sub.k+i.sub.)i.sub.k+1+N (5.2).]. .[.and the two
dimensional assignment problem.].
.times..PHI..ident..times.
.times..times..times..times..times..times..times..ident..function..times.-
.times..times..times.
.times..times..times..times..times..times..times..times..times..times..di-
-elect cons..times..times..times..times..times..times. ##EQU00085##
.[.Let w be an optimal or feasible solution to this two-dimensional
assignment problem and define.].
.times..times..times..times..times..times..times..times..function..time-
s..function..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times. ##EQU00086##
.[.This need not satisfy the constraints in that there are usually
many objects left unassigned. Thus, one can complete the assignment
by using the zero-one variables in (4.4) with k replaced by k+1
with exactly one nonzero index corresponding to any unassigned
object or data report..].
.[.For the dual solutions, the multipliers arising from the
solution of the two dimensional assignment problem (5.3)
corresponding to the second variable. i.e.,
{u.sub.i.sub.k+1+N.sup.k+1+N}.sub.i.sub.k+1+N.sup.M.sup.k+1+N=0.
These are good initial values for use in a relaxation scheme
[11.12]. Finally, note that one can also develop a warm start for
problem (4.14) in a similar fashion..].
.[.Multidimensional assignment problems govern the central problem
of data association in multisensor and multitarget tracking, i.e.,
the problem of partitioning observations from multiple scans of the
surveillance region and from either single or multiple sensors into
tracks and false alarms. This fundamental problem can be stated
as.].
.times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..d-
i-elect
cons..times..times..times..times..times..times..times..times..time-
s. ##EQU00087## .[.where c.sub.O . . . 0 is arbitrarily defined to
be zero and is included for notational convenience. One can modify
this formulation to include multiassignments of one, some, or all
of the actual reports. The zero index is used in representing
missing data, false alarms, initiating and terminating tracks. In
these problems, we assume that all zero-one variables z.sub.ij . .
. i.sub.N with precisely one nonzero index are free to be assigned
and that the corresponding cost coefficients are well-defined.
(This is a valid assumption in the tracking environment.) Although
not required, these cost coefficients with exactly one nonzero
index can be translated to zero by cost shifting without changing
the optimal assignment. Finally, this formulation is of sufficient
generality to include the symmetric problem and the asymmetric
inequality problem..].
.[.The data association problems in tracking that are formulated as
(1.1) have several characteristics. They are normally sparse, the
cost coefficients are noisy and the problem is NP-hard, but it must
be solved in real-time. The only known methods for solving this
NP-hard problem optimally are enumerative in nature, with
branch-and-bound being the most efficient; however, such methods
are much too slow for real-time applications. Thus one must resort
to suboptimal approaches. Ideally, any such algorithm should solve
the problem to within the noise level, assuming, of course, that
one can measure this noise level in the physical problem and the
mathematical method provides a way to decide if the criterion has
been met..].
.[.There are many algorithms that can be used to construct
suboptimal solutions to NP-hard combinatorial optimization
problems. These include greedy (and its many variants), relaxation,
simulated annealing. tabu search, genetic algorithms, and neural
netork algorithms. For the three dimensional assignment problem.
Pierskalla developed the tri-substitution method, which is a
variant of the simplex method. Frieze and Yadegar introduced a
method based on Lagrangian relaxation in which a feasible solution
is recovered using information provided by the relaxed solution.
Our choice of approaches is strongly influenced by the need to
balance real-time performance and solution quality. Lagrangian
relaxation based methods have been used successfully in prior
tracking applications. An advantage of these methods is that they
provide both an upper and lower bound on the optimal solution,
which can then be used to measure the solution quality. These works
extend the method of Frieze and Yadegar to the multidimensional
case..].
.[.Probabilistic Framework for Data Association. (ABP).].
.[.The goal of this section is to explain the formulation of the
data association problems that governs large classes of data
association problems in centralized or hybrid centralized-sensor
level multisensor/multitarget tracking. The presentation is brief;
technical details are presented for both track initiation and
maintenance in the work of Poore. The formulation is of sufficient
generality to cover the MHT work of Reid, Blackman and Stein and
modifications by Kurien to include maneuvering targets. As
suggested by Blackman, this formulation can also be modified to
include target features (e.g., size and type) into the scoring
function. The recent work of Poore and Drummond significantly
extends the work of this section to new approaches for multiple
sensor centralized tracking. Future work will involve extensions to
track-to-track correlation..].
.[.The data association problems for multisensor and multitarget
tracking considered in this work are generally posed as that of
maximizing the posterior probability of the surveillance region
(given the data) according to.]. .[.Maximize
{P(.GAMMA.=.gamma.|Z.sup.N)|.gamma..epsilon..GAMMA.*} (2.1).].
.[.where Z.sup.N represents N data sets, .gamma. of the data (and
thus induces a partition of the data), .GAMMA..gamma..].
.times..GAMMA..times..times..times..GAMMA..times..times..delta..gamma..ti-
mes. ##EQU00088## .[.P(.GAMMA. .gamma.|Z.sup.N) is the posterior
probability of a partition .gamma. true given the data Z.sup.N. The
term partition is defined below; however, this framework is
currently sufficiently general to cover set packings and
coverings..].
.[.Consider N data sets Z(k) (k=I, . . . ,N) each of M.sub.k
reports {z.sub.i.sub.k.sup.k}.sub.i.sub.k.sup.M.sup.k=1, and let
Z.sup.N denote the cumulative data set defined by.].
.[.Z(k)={z.sub.i.sub.k.sup.k}.sub.i.sub.k.sup.Mk=1 and
Z.sup.N={(Z(I), . . . ,Z(N)} (2.2).]. .[.respectively. In
multisensor data fusion and multitarget tracking the data sets Z(k)
may represent different classes of objects, and each data set can
arise from different sensors. For track initiation the objects are
measurements that must be partitioned into tracks and false alarms.
In our formulation of track maintenance, which uses a moving window
over time, one data set will be tracks and remaining data sets will
be measurements which are assigned to existing tracks, as false
measurements, or to initiating tracks. In sensor level tracking,
the objects to be fused are tracks. In centralized fusion, the
objects may all be measurements that represent targets or false
reports, and the problem is to determine which measurements emanate
from a common source..].
.[.We specialize the problem to the case of set partitioning
defined in the following way. First, for notational convenience in
representing tracks, we add a zero index to each of the index sets
in (2.2), a dummy report z.sub.o.sup.k to each of the data sets
Z(k) in (2.2), and define a "track of data" as
(z.sub.i.sub.1.sup.i,z.sub.i.sub.N.sup.N) where @@i.sub.k and
z.sub.i.sub.k.sup.k can now assume the values of 0 and
z.sub.O.sup.k, respectively. A partition of the data will refer to
a collection of tracks of data wherein each report occurs exactly
once in one of the tracks of data and such that all data is used
up; the occurrence of dummy report is unrestricted. The dummy
report z.sub.O.sup.k serves several purposes in the representation
of missing data, false reports, initiating tracks, and terminating
tracks. The reference partition is that in which all reports are
declared to be false..].
.[.Next under appropriate independence assumptions one can
show.].
.times..function..GAMMA..gamma..function..GAMMA..gamma..ident..times..t-
imes..gamma..ident..times..times..times..di-elect
cons..gamma..times..times..times..times..times..times. ##EQU00089##
.[.L.sub.i.sub.1 . . . i.sub.N is a likelihood ratio containing
probabilities for detection, maneuvers, and termination as well as
probability density functions for measurement errors, track
initiation and termination. Then if c.sub.i.sub.1 . . .
i.sub.N=--InL.sub.i.sub.1 . . . i.sub.N,.].
.times..function..gamma..function..gamma..times..times..times..di-elec-
t cons..gamma..times..times..times..times..times. ##EQU00090##
.[.Using (2.3) and the zero-one variable z.sub.i.sub.1 . . . N==1
i.sub.1, . . . N).].
.times..di-elect
cons..gamma..gamma..times..DELTA..times..PHI..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00091##
.[.where c.sub.O . . . O is arbitrarily defined to be zero. Here,
each group of sums in the constraints represents the fact that
each-non-dummy report occurs exactly once in a "track of data." One
can modify this formulation to include multiassignments of one,
some, or all the actual reports. The assignment problem (2.5) is
changed accordingly. For example, if z.sub.l.sub.k.sup.k is to be
assigned no more than, exactly, or no less than n.sub.i.sub.k.sup.k
times, then the "=1" in the constraint (2.5) is changed to
.ltoreq., =, .gtoreq.n.sub.i.sub.k.sup.k," respectively.
Modifications for group tracking and multiresolution features of
the surveillance region will be addressed in future work. In making
these changes, one must pay careful attention to the independence
assumptions that need not be valid in many applications..].
.[.Expressions for the likelihood ratios L.sub.l.sub.1. ,i.sub.N,
can be found in the work of Poore. These expressions include the
developments of Reid, Kurien, and Stein and Blackman. What's more,
they are easily modified to include target features and to account
for different sensor types. In track initiation, the N data sets
all represent reports from N sensors, possibly all the same. For
track maintenance, we use a sliding window of N data sets and one
data set containing established tracks. The formulation is the same
as above except that the dimension of the assignment problem is now
N+1..].
Overview of the Lagrangian Relaxation Algorithm. .[.(ABP).].
Having discussed the N-dimensional assignment problem .[.(1.1).].
.Iadd.(3.1).Iaddend., we now turn to a description of the
Lagrangian relaxation algorithm. The algorithm will proceed
iteratively for a loop k=1, . . . , N-2. At the completion, there
remains one two-dimensional assignment problem that provides the
last step which yields an optimal (sometimes) or near-optimal
solution to the original N-dimensional assignment problem. In step
k of this loop (summarized in Section .[.4.]. .Iadd.IV.3.Iaddend.)
one starts with the following (N-k+1)-dimensional assignment
problem with one change in notation. If k=1, then the index
notation l.sub.1 and L.sub.1 are to be replaced by i.sub.1 and
M.sub.1, respectively.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..di-elec-
t
cons..times..times..times..times..times..times..times..times..times..tim-
es. ##EQU00092## To ensure that a feasible solution of .[.(3.1).].
.Iadd.Equation (4.7) .Iaddend.always exists for is a sparse
problem, all variables with exactly one nonzero index (i.e.,
variables of the form z.sub.l.sub.k.sub.O . . . O .sup.N-k+1 o for
.[.l.sub.k, . . . ,L.sub.k.]..Iadd.l.sub.k=1, . . . ,L.sub.k
.Iaddend.and z.sub.l.sub.0 .sub.. . . l.sub.O . . . O.sup.N-k+1 for
i.sub.p=1, . . . , M.sub.l.sub.p and p=k+12, . . . ,N) assumed free
to be assigned with the corresponding cost coefficients being
well-defined. This assumption is valid in the tracking environment
.Iadd.(A. B. Poore. Multidimensional assignments and multitarget
tracking, partitioning data sets, I. J. cox, P. Hansen, and B.
Julesz, editors, DIMACS Series in Discrete Mathematics and
Theoretical Computer Science, American Mathematical Society,
Providence, R.I., 19:169-198, 1995; A. B. Poore. Multidimensional
assignment formulation of data association problems arising from
multitarget tracking and multisensor data fusion. Computational
Optimization and Applications, 3:27-57, 1994).Iaddend...
.[.Subsection 3.1.]. .Iadd.Section IV.3.1 .Iaddend.presents some of
the properties associated with the Lagrangian relaxation of
.[.(3.1).]. .Iadd.(4.7) .Iaddend.based on relaxing the last
(N-k)-sets of constraints to a two-dimensional one. .[.A.].
.Iadd.Section IV.3.2 describes a .Iaddend.new approach to the
problem of recovering a high quality feasible solution of the
original (N-k+1)-dimensional problem given a feasible solution
(optimal or suboptimal) of the relaxed two-dimensional problem is
described hereinafter. A summary of the relaxation algorithm is
given .[.hereinafter.]. .Iadd.in Section IV.3.3.Iaddend.,
.[.following which.]. .Iadd.and in Section IV.3.4, .Iaddend.we
establish the maximization of the Lagrangian dual (an important
aspect of the relaxation 3procedure) to be an unconstrained
nonsmooth optimization problem and then present a method for
computing the subgradients.
.Iadd.IV.3.1 .Iaddend.The Lagrangian Relaxed Assignment
Problem.[...].
The (N-k+1)-dimensional problem .[.(3.1).]. .Iadd.(4.7)
.Iaddend.has (N-k+1) sets of constraints. A (M.sub.p+1)-dimensional
multiplier vector (i.e., .nu..sup.p .epsilon. R.sup.M.sup.p.sup.+1)
associated with the .[.p-th.]. .Iadd.p.sup.th .Iaddend.constraint
set will be denoted by u.sup.p=(u.sub.0.sup.p,u.sub.1.sup.p, . . .
,u.sub.M.sub.p.sup.p).sup.T with u.sub.O.sup.p=0 being fixed for
each p=k+2, . . . , N and included for notational convenience only.
Now, the .[.(N-k+1)dimensional.]. .Iadd.(N-k+1)-dimensional
.Iaddend.assignment problem .[.(3.1).]. .Iadd.(4.7) .Iaddend.is
relaxed to a two-dimensional assignment problem by incorporating
(N-k-1) sets of constraints into the objective function via the
Lagrangian. Although any (N-k-1) constraint sets can be relaxed, we
choose the last (N-k-1) sets of constraints for convenience. The
.Iadd.relaxed.Iaddend. problem is
.PHI..function..times..ident..times..times..PHI..function..times..ident..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..ident..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times. ##EQU00093## One of the major steps
in the algorithm is the maximization of .PHI..sub.N-k+1(u.sup.k+2,
. . . ,u.sup.N) with respect to the multipliers (u.sup.k+2, . . .
,u.sup.N). It turns out that .PHI..sub.N-k+1 is a concave,
continuous, and piecewise affine function of the multipliers
(u.sup.k+2, . . . ,u.sup.N), so that the maximization of
.PHI..sub.N-k+1 is a problem of nonsmooth optimization. Since many
of these algorithms .[.[???].]. require a function value and a
subgradient of .PHI..sub.N-k+1 at any required multiplier value
(u.sup.k+2, . . . ,u.sup.N), we address this problem in the next
subsection. We note, however, that there are other ways to maximize
.PHI..sub.N-k+1 and the next subsection just addresses one such
method.
.Iadd.IV.3.2 .Iaddend.Properties .Iadd.of the .Iaddend.Lagrangian
Relaxed Assignment Problem
For a function evaluation of .[..PHI..sub.N-k+1.].
.Iadd..PHI..sub.N-k+1.Iaddend., we show that an optimal (or
suboptimal) solution of this relaxed problem .[.(3.2).].
.Iadd.(4.8) .Iaddend.can be constructed from that of a
two-dimensional assignment problem. Then, the nonsmooth
characteristics of .PHI..sub.N-k+1 are addressed, followed by a
method for computing the function value and a subgradient.
Evaluation of .PHI..sub.N-k+1. Define for each (l.sub.k,i.sub.k+1)
an index (j.sub.k+2, . . . ,
j.sub.N)=(j.sub.k+2(l.sub.k,i.sub.k+1), . . .
,j.sub.N(l.sub.k,i.sub.k+1)) and a new cost function
c.sub.l.sub.k.sub.i.sub.k+1.sup.2 by
.function..times..function..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
function..times..times..times..function..times..function..times..times..ti-
mes..times..noteq..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times. ##EQU00094## Given an
index pair (I.sub.k,i.sub.k+1),(j.sub.k+2, . . . , j.sub.N) need
not be unique, resulting in the potential generation of several
subgradients .[.(3.11).]. .Iadd.(4.17).Iaddend.. Then,
.PHI..function..times..times..times..PHI..function..times..ident..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00095##
As an aside, two observations are in order. The first is that the
search procedure needed for the computation of the relaxed cost
coefficients in .[.(3.3).]. .Iadd.(4.9) .Iaddend.is the most
computationally intensive part of the entire relaxation algorithm.
The second is that a feasible solution z.sup.N-k+1 of a sparse
problem .[.(3.1).]. .Iadd.(4.7) .Iaddend.yields a feasible solution
z.sup.2 of .[.(3.4).]. .Iadd.(4.10) .Iaddend.via the
construction
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times. ##EQU00096## thus, there are
generally solutions other than the one nonzero index solution.
The following Theorem .[.3.1.]. .Iadd.4.1 .Iaddend.gives a method
for evaluating .PHI..sub.N-k+1 and states that an optimal solution
of .[.(3.2).]. .Iadd.(4.10) .Iaddend.can be computed from that of
.[.(3.4).]. .Iadd.(4.10).Iaddend.. Furthermore, if the solution of
either of these two problems is .epsilon.-optimal, then so is the
other. The converse is contained in Theorem .[.3.2.].
.Iadd.(4.2).Iaddend..
.[.Theorem 3.1. Let .omega..sup.2 dimensional assignment problem
(3.4) and define .omega..sup.N-k+1.].
.times.
.times..times..times..times..times..omega..times..times..times.-
.times..times..times..times..times..times..noteq..times..times..times..tim-
es..times..times..times..times..times..noteq..times..times..times..noteq..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..ltoreq..times..times..times..times..times..times..times..-
times..times..times..times..times..times.>.times. .times.
##EQU00097## .[.Then .omega..sup.N-k+1 .nu. problem and
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1.nu..sup.k+2 . . . ,
.nu..sup.N {circumflex over (.PHI.)}.sub.N-k-1.gamma..omega..sub.2
.nu..sup.k+2 .nu..sup.N.DELTA. addition,
.omega..sub.2.omega..sup.N=k+1 .PHI..sub.N-k+1.gamma..nu..sup.k+2
.nu..sup.N.DELTA. {circumflex over
(.PHI.)}.sub.N-k+1.gamma..nu..sup.k+2 .nu..sup.N.DELTA..].
.[.Proof. Let .omega..sup.N-k+1 .omega..sup.2
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1 .nu..sup.k+2
.nu..sup.N.DELTA. {circumflex over (.PHI.)}
.gamma..sub.N-k+1.sup.2.nu..sup.k+2 .nu..sup.N) objective function
values of (3.2) and (3.4), respectively. Direct verification shows
that .omega..sup.n-k+1 in (3.2) and
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1
.nu..sup.k+2.nu..sup.N{circumflex over
(.PHI.)}.sub.N-k+1.gamma..omega..sup.2.nu..sup.k+2, . . .
,.nu..sup.N remainder of the proof, assume that
.omega..sup.-.gamma..DELTA..]. .[.Let .chi..sup.N-k+1
.gamma..DELTA..].
.times..chi..times..times..times..times..times..times..chi..times..time-
s..times..times..times..times..times..times..times..noteq..times..times..c-
hi..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..ltoreq..times..times..times..times..times..tim-
es..times..times..times..chi..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times.>.times..times..ti-
mes..times..times..times..times. .times. ##EQU00098## .[.Note that
.chi..sup.2.gamma..DELTA..PHI..sub.N-k+1.gamma..chi..sup.N-k+1.nu..sup.k+-
2.nu..sup.N.DELTA..gtoreq.{circumflex over
(.PHI.)}.sub.N-k+1(.chi..sup.2.nu..sup.k+2
.nu..sup.N.DELTA..gtoreq. {circumflex over (.PHI.)}.sub.N-k+1
(.omega..sup.2 .nu..sup.k+2 .nu..sup.N.DELTA.
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1 .nu..sup.k+2
.nu..sup.N.DELTA. .chi..sup.N-k+1 .gamma. .DELTA. .omega.
.omega..sup.N-k+1 optimal solution of (3.2). Next,
.PHI..sub.N-k+1.gamma..nu..sup.k+2 .nu..sup.N.DELTA. {circumflex
over (.PHI.)}.sub.N-k+1 (.nu..sup.k+2 .nu..sup.N.DELTA.
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1 .nu..sup.k+2
.nu..sup.N.DELTA. .PHI..sub.N-k+1(.omega..sup.2 .nu..sup.k+2
.nu..sup.N.DELTA. .omega..sup.2 .gamma..DELTA..].
.[.With the exception of one equality being converted to an
inequality, the following theorem is a converse of Theorem
3.1..].
.[.Theorem 3.2. Let .omega..sup.N-k+1.gamma..DELTA. and define
.omega..sup.2.].
.times..omega..times..times..times..times..times..times..omega..times..-
times..times..times..times..times..times..times..times..noteq..times..time-
s..omega..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..ltoreq..times..times..times..times..time-
s..times..times..times..times..omega..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times.>.times..t-
imes..times..times..times..times..times..times..times. .times.
.times. ##EQU00099## .[.Then .omega..sup.2 .gamma. .DELTA.
.PHI..sub.N-k+1 .gamma..omega..sup.N-k+1,
.nu..sup.N.DELTA..gtoreq.{circumflex over
(.PHI.)}.sub.N-k+1(.omega..sup.2 .nu..sup.k+2 .nu..sup.N.DELTA.
.omega..sup.N-k+1 .gamma. .DELTA. .omega..sup.2 (3.4),
.PHI..sub.N-k+1.gamma..omega..sup.N-k+1 .nu..sup.k+2
.nu..sup.N.DELTA.={circumflex over (.PHI.)}.sub.N-k+1(.omega..sup.2
.nu..sup.k+2 .nu..sup.N.DELTA. and .PHI..sub.N-k+1 (.nu..sup.k+2
.nu..sup.N.DELTA. {circumflex over
(.PHI.)}.sub.N-k+1.gamma..nu..sup.k+2 .nu..sup.N.DELTA..].
.[.Proof: Let .omega..sup.N-k+1 .omega..sup.2
.PHI..sub.N-k+1(.omega..sup.N-k+1 .nu..sup.k+2 .nu..sup.N.DELTA.
and {circumflex over (.PHI.)}.sub.N-k+1(.omega..sup.2 .nu..sup.k+2
.nu..sup.N.DELTA. objective function values of (3.2) and (3.4),
respectively. Direct verification shows that .omega..sup.2 (3.4)
and .PHI..sub.N-k+1.gamma..omega..sup.N-k+1
.nu..sup.k+2.nu..sup.N.DELTA..gtoreq.{circumflex over
(.PHI.)}.sub.N-k+1(.omega..sup.2.nu..sup.k+2.nu..sup.N remainder of
the proof, assume that .omega..sup.N-k+1 .gamma..DELTA. and
construct .omega..sup.2 .omega. .omega..sup.N-k+1 by replacing
.omega..sup.N-k+1 .gamma. .DELTA. .omega..sup.N-k+1 .omega.
.PHI..sub.N-k+1.gamma..sup. .omega.N-k+1.nu..sup.k+2
.nu..sup.N.DELTA.={circumflex over (.PHI.)}.sub.N-k+1(.omega..sup.2
.nu..sup.k+2
.nu..sup.N.DELTA..gtoreq..PHI..sub.N=k+1.gamma..omega..sup.N-k+1;
.nu..sup.k+2 .nu..sup.N.DELTA. .omega..sup.N=k+1 inequality is in
face an equality. This provides the theorem..].
.Iadd.Theorem 4.1. Let w.sup.2 be a feasible solution to the
two-dimensional assignment problem (4.10) and define w.sup.N-k+1 by
.Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..noteq..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..noteq..tim-
es..times..times..times..times..noteq..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..ltoreq..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times.> ##EQU00100##
.Iadd.Then w.sup.N-k+1 is a feasible solution of the Lagrangian
relaxed problem and .Iaddend.
.Iadd..PHI..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, . . . ,u.sup.N)={acute
over (.PHI.)}.sub.-k-1(w.sub.2; u.sup.k+2, . . . ,u.sup.N).
.Iaddend. .Iadd.If, in addition, w.sup.2 is optimal for the
two-dimensional problem, then w.sup.N-k+1 is an optimal solution of
the relaxed problem and .PHI..sub.N-k+1(u.sup.k+2, . . .
,u.sup.N)={acute over (.PHI.)}.sub.-k+1(u.sup.k+2, . . . ,u.sup.N).
.Iaddend. .Iadd.Proof. Let w.sup.N-k+1 and w.sup.2 be as in the
hypotheses, and let .Iaddend.
.Iadd..PHI..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, . . . ,u.sup.N)
.Iaddend. .Iadd.and .Iaddend. .Iadd.{acute over
(.PHI.)}.sub.n-k+1(z.sup.2;u.sup.k+2, . . . ,u.sup.N) .Iaddend.
.Iadd.denote the objective function values of (4.8) and (4.10),
respectively. Direct verification shows that w.sup.N-k+1 satisfies
the constraints in (4.8) and .Iaddend.
.Iadd..phi..sub.N-k+1(w.sup.N-k+1;u.sup.k+2; . . . ,u.sup.N)={acute
over (.phi.)}.sub.-k+1(w.sup.2;u.sup.k+2, . . . ,u.sup.N).
.Iaddend. .Iadd.For the remainder of the proof, assume that w.sup.2
is optimal for (4.10). Let X.sup.N-k+1 satisfy the constraints in
(4.8) and define .Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..noteq..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..ltoreq..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times.>.times..times..times.-
.times..times..times..times. ##EQU00101## .Iadd.Note that x.sup.2
satisfies the constraints in (4.10) and .Iaddend.
.PHI..function..times..gtoreq..times..PHI.'.function..times..gtoreq..time-
s..PHI.'.function..times..times..PHI..function..times. ##EQU00102##
for any feasible solution x.sup.N-k+1 of the constraints (4.8).
This implies w.sup.N-k+1 is an optimal solution of (4.8). Next,
.Iadd..PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N)={acute over
(.PHI.)}.sub.N-k+1(u.sup.k+2, . . . ,u.sup.N) .Iaddend.
.Iadd.follows immediately from .Iaddend.
.Iadd..PHI..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, . . .
,u.sup.N)=.PHI..sub.N-k+1(w.sup.2;u.sup.k+2, . . . ,u.sup.N)
.Iaddend. .Iadd.for an optimal solution w.sup.2 of (4.10). With the
exception of one equality being converted to an inequality, the
following theorem is a converse of Theorem 4.1. .Iaddend.
.Iadd.Theorem 4.2. Let w.sup.N-k+1 be a feasible solution to
problem (4.8) and define w.sup.2 by .Iaddend.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..noteq..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..ltoreq..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times.>-
.times..times..times..times..times..times..times..times.
##EQU00103## .Iadd.Then w.sup.2 a feasible solution of the problem
(4.10) and .Iaddend. .Iadd..phi..sub.N-k+1(w.sup.N-k+1;w.sup.k+2, .
. . ,u.sup.N).gtoreq.{acute over
(.phi.)}.sub.N-k+1(w.sup.2;u.sup.k+2, . . . ,u.sup.N).
.Iaddend.
.Iadd.If, in addition, whu N-k+1 is optimal for (4.8), then w.sup.2
is an optimal solution of (4.10), .Iaddend.
.Iadd..phi..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, . . . ,u.sup.N)={acute
over (.phi.)}.sub.N-k+1(w.sup.2;u.sup.k+2, . . . ,u.sup.N)
.Iaddend. .Iadd.and .Iaddend. .Iadd..PHI..sub.N-k+1(u.sup.k+2, . .
. ,u.sup.N)={acute over (.PHI.)}.sub.N-k+1(u.sup.k+2, . . .
,u.sup.N). .Iaddend. .Iadd.Proof: Let w.sup.N-k+1 and w.sup.2 be as
in the hypotheses, and let .phi..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, .
. . ,u.sup.N) and {acute over (.phi.)}.sub.N-k+1(w.sup.2;u.sup.k+2,
. . . ,u.sup.N) denote the objective function values of (4.8) and
(4.10), respectively. Direct verification shows that w.sup.2
satisfies the constraints in (4.10) and .Iaddend.
.Iadd..phi..sub.N-k+1(w.sup.N-k+1;u.sup.k+2, . . .
,u.sup.N).gtoreq.{acute over (.phi.)}.sub.N-k+1(w.sup.2;u.sup.k+2,
. . . ,u.sup.N). .Iaddend.
.Iadd.For the remainder of the proof, assume that w.sup.N-k+1 is
optimal for (4.8) and construct w.sup.2 as above. Using Theorem
4.1, construct w.sup.N-k+1 by replacing w.sup.N-k+1 in (4.11) with
w.sup.N-k+1. Then, from that theorem .Iaddend.and
.PHI..function..times..times..PHI.'.function..times..ltoreq..times..PHI..-
function..times. ##EQU00104## .Iadd.Optimality of w.sup.N-k+1 then
implies the last inequality is in fact an equality. This proves the
theorem. .Iaddend.
The Nonsmooth Optimization Problem. Next, we address the nonsmooth
properties of the function .[..PHI..sub.N-k+1.gamma..nu..sup.k+2
.nu..sup.N.DELTA..]. .Iadd..PHI..sub.N-k+1(u.sup.k+2, . . .
,u.sup.N) as .Iaddend.explained in the following theorem.
Theorem .[.3.4.]. .Iadd.4.3. (G. L. Nemhauser and L. A. Wolsey.
Interger and Combinatorial Optimization, Section II.3.6. Wiley, New
York, N.Y., 1988). .Iaddend.Let .[..PHI..sub.N-k+1
.gamma..DELTA..PHI..sub.N-k+1.gamma..sup.N-k+1).].
.Iadd..PHI..sub.N-k+1 be as defined in (4.7), let
V.sub.N-k+1(z.sup.N-k+1) .Iaddend.be the .[.object.].
.Iadd.objective .Iaddend.function value of the (N-k+1)-dimensional
assignment problem in .[.equation (3.1).]. .Iadd.(4.7)
.Iaddend.corresponding to a feasible solution Z.sup.N-k+1 of the
constraints in .[.(3.1).]. .Iadd.(4.7).Iaddend., and let
z.sup.N-k+1 be an optimal solution of .[.(3.1).].
.Iadd.(4.7).Iaddend.. Then,
.[..PHI..sub.N-k+1.gamma..nu..sup.k+2.nu..sup.N.DELTA..].
.Iadd..PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N) .Iaddend.is
piecewise affine, concave and continuous in .[.{.nu..sup.k+2
.nu..sup.N.]. .Iadd.{u.sup.k+2, . . . ,u.sup.N}.Iaddend.and
.PHI..sub.N-k+1(u.sup.k+2, . . .
,u.sup.N).ltoreq.V.sub.N-k+I(.sup.N-k+1).ltoreq.V.sub.N-k+1(.sup.N--
k+1). .Iadd.(4.14) .Iaddend. Most of the algorithms for nonsmooth
optimization are based on generalized gradients, called
subgradients, given by the following definition for a concave
function. Definition .[.3.5..]. .Iadd.4.1. .Iaddend.At
u=(u.sup.k+2, . . . ,u.sup.N) the set .delta..PHI..sub.N-k+1(u) is
called a subdifferential of .PHI..sub.N-k+1 and is defined for the
concave function .PHI..sub.N-k+1(u) by
.differential..PHI..sub.N-k+1(u)={g.epsilon.R.sup.M.sub.k+2.sub.+1
x . . .
xR.sup.M.sub.N+1|.PHI..sub.N-k+1(.omega.)-.PHI..sub.N-k+1(u).ltoreq.g.s-
up.T(.omega.-u) for all .OMEGA..epsilon.R.sup.M.sub.k+2.sub.1 x . .
. xR.sup.M.sub.N+1). where g.sub.o=.omega..sub.o.sup.p=0 are all
permanently fixed. (Recall that these were used for notational
conveience only.) A vector ge.epsilon..differential..PHI..sub.N-k+1
is called a subgradient.
There is a large literature on such problems, .Iadd.e.g., (J.-B.
Hiriart-Urruty and C. Lemarechal. Concex Analysis and Minimization
Algorithms I& II. Springer-Verlag, Berlin, 1993; K.C. Kiwiel.
Methods of descent for nondifferentiable optimization. In Lecture
Notes in Mathematics 1133, A. Dold and B. Eckmann, eds.
Springer-Verlag, Berlin, 1985; C. Lemarechal and R. Mifflin.
Nonsmooth Optimization. Pergamon Press, Oxford, UK, 1978; B. T.
Polyak. Subgradient method: A survey of Soviet research. In C.
Lemarechal and R. Mifflin, editors, Nonsmooth Optimization, pages
5-29, N.Y., 1978. Pergamon Press.; H. Schramm and J. Zowe. A
version of the bundle idea for minimizing a nonsmooth function:
Conceptual idea, convergence analysis, numerical results. SIAM
Journal on Optimization, 2, No.1:121-152, February, 1992; N. Z.
Shor. Minimization Methods for Non-Differentiable Functions.
Springer-Verlag, New York, 1985; P. Wolfe. A method of conjugate
subgradients for minimizing nondifferentiable functions.
Mathematical Programming Study, 3:145-173, 1975), .Iaddend.and we
have tried a variety of methods including subgradient methods
.Iadd.(N. Z. Shor. Minimization Methods for Non-Differentiable
Functions. Springer-Verlag, New York, 1985) .Iaddend.and bundle
methods .Iadd.(J.-B. Hiriart-Urruty and C. Lemarechal. Convex
Analysis and Minimization Algorithms I& II. Springer-Verlag,
Berlin, 1993; K. C. Kiwiel. Methods of descent for
nondifferentiable optimization. In Lecture Notes in Mathematics
1133, A. Dold and B. Eckmann, eds, Springer-Verlag, Berlin,
1985).Iaddend.. Of these, we have determined that for a fixed
number of nonsmooth iterations (e.g., twenty), the bundle trust
method .Iadd.of .Iaddend.Schramm and Zowe .Iadd.(H. Schramm and J.
Zowe. A version of the bundle idea for minimizing a nonsmooth
function: Conceptual idea, convergence analysis, numerical results.
SIAM Journal on Optimization, 2, No. 1:121-152, February, 1992)
.Iaddend.provides excellent quality solutions with the fewest
number of function and subgradient evaluations, and is therefore
our currently recommended method.
An Algorithm for Computing .PHI..sub.N-k+1 and a Subgradient. Most
current software for maximizing the concave function
.PHI..sub.N-k+1 requires the value of the function and a
subgradient at a point (u.sup.k+2, . . . ,u.sup.N). Based on the
previous two sections, this can be summarized as follows.
1. Starting with problem .[.(3.1).]. .Iadd.(4.7).Iaddend., form the
relaxed problem .[.(3.2).]. .Iadd.(4.8).Iaddend.;
2. To solve .[.(3.2).]. .Iadd.(4.8) .Iaddend.optimally, defined the
.[.two dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem .[.(3.4).]. .Iadd.(4.10) .Iaddend.via the transformation
.[.(3.3).]. .Iadd.(4.0).Iaddend.;
3. Solve the two-dimensional problem .[.(3.4).]. .Iadd.(4.10)
.Iaddend.optimally;
4. Reconstruct the optimal solution, say w.sup.N-k+1, of
.[.(3.2).]. .Iadd.(4.8) .Iaddend.via equation .[.(3.5).].
.Iadd.(4.9) .Iaddend.as in Theorem .[.3.1.].
.Iadd.4.1.Iaddend.;
5. Compute the function
.PHI..function..times..ident..PHI..function..times..ident..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times.
##EQU00105##
6. A subgradient is given by
.differential..PHI..function..times..differential..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times. ##EQU00106## Several
remarks are in order. First, the optimal solution of the
.[.minimiation.]. .Iadd.minimization .Iaddend.problem .[.(3.2).].
.Iadd.(4.8) .Iaddend.is required before one can remove the minimum
sign, replace z.sup.N-k+1 by the minimizer w.sup.N-k+1 and
differentiate with respect to u.sub.i.sub.p.sup.p to obtain a
subgradient as in .[.(3.11).]. .Iadd.(4.17).Iaddend.. If
w.sup.N-k+1 is an approximate solution of .[.(3.2).].
.Iadd.(4.8).Iaddend., then the subgradient and function values are
only approximate with .[.accruacy.]. .Iadd.accuracy
.Iaddend.depending on that of w.sup.N-k+1. Although one can
evaluate the sums in .[.(3.10).]. .Iadd.(4.16) .Iaddend.and
.[.(3.11).]. .Iadd.(4.17) .Iaddend.in a .[.straightforward.].
.Iadd.straight forward .Iaddend.manner, another method is based on
the following observation. Given the multiplier vector u.sub.k+2, .
. . ,u.sub.N), let
{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1)}.sub.l.sub.k+1.sup.L.sup.k+1=0,
be an enumeration of indices {l.sub.k,i.sub.k+1} of w.sup.2 (or the
first 2 indices of w.sup.N-k+1 constructed in equation .[.3.5.].
.Iadd.4.9.Iaddend.)) such that))
w.sub.i.sub.k.sub.(l.sub.k+1.sub.),i.sub.k+1.sub.(l.sub.k+1.sub.).sup.2=1
for (l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1)).noteq.(0.0) and
(l.sub.k(l.sub.k+1), i.sub.k+1(l.sub.k+1))=(0,0) for l.sub.k+1=0
regardless of whether w.sub.oo.sup.2=1 or not. (The latter can only
improve the recovered feasible solution.) The evaluation of the
bracketed quantity in .[.(3.11).]. .Iadd.(4.17) .Iaddend.for a
specific i.sub.p>=1 and p=k+2, . . . ,N is one minus the number
of times the index value i.sub.p appears .Iadd.in .Iaddend.the
(p-k+1).sup.th position of the (N-k+1)tuple in the list
({l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1), j.sub.k+2, . . .
,j.sub.N)}.sub.l.sub.k+1.sup.L.sup.k+1=0.
Finally, we have presented a method for computing one subgradient.
If w.sup.N-k+1 is the unique optimal solution of .[.(3.2).].
.Iadd.(4.8).Iaddend., then .PHI..sub.N-k+1(u) is differentiable at
u, g is just the gradient at u, and the subdifferential
.delta..PHI..sub.N-k+1(u)={g} is a singleton. If, corresponding to
u,w.sup.N-k+1 is not unique, then there are finitely many such
solutions, say {w.sup.N-k+1(1), . . . ,w.sup.N-k+1(m)} with
respective subgradients {g(1), . . . ,g(m)}, the subdifferential
.delta..PHI.(u)is the convex hull of {g(1) g(m)} .Iadd.(J. L.
Goffin. On convergence rates of subgradient optimization methods,
mathematical programming. Mathematical Programming, 13:329-347,
1977).Iaddend.. These nonunique solutions arise in two ways. First,
if the optimal solution of the .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment problem is not unique,
then one can .[.general.]. .Iadd.generate .Iaddend.all optimal
solutions of the .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem .[.(3.4).]. .Iadd.(4.10).Iaddend..
Corresponding to each of these solutions and to each index pair
(l.sub.k,i.sub.k+1)in each solution, compute the indices
(j.sub.k+2, . . . ,j.sub.N) in .[.(3.3).]. .Iadd.(4.9).Iaddend.. If
these j's are not unique, then we can enumerate all the possible
optimal solutions w.sup.N-k+1 of .[.(3.2).]. .Iadd.(4.8).Iaddend..
Given these solutions, one can then compute the corresponding
subgradients from .[.(3.11).]. .Iadd.(4.17).Iaddend..
In most nonsmooth optimization algorithms, one uses an
.[.epsilon-subdifferential.]. .Iadd..epsilon.-subdifferential.
.Iaddend.
Definition .[.3.x.]. .Iadd.4.2.Iaddend.. At u=(u.sup.k+2, . . .
,u.sup.N) the set .delta..sub..epsilon..PHI..sub.N-k+1(u) is called
an .[.epsilon subdifferential.]. .Iadd..epsilon.-subdifferential
.Iaddend.of .PHI..sub.N-k+1 and is defined for the concave funtion
.PHI..sub.N-k+1(u) by
.differential..PHI..sub.N-k+1(u)={g.epsilon.R.sup.M.sub.k+2.sub.+1x
. . . xR.sup.M.sub.N+1|1
.sub.N-k+1(.omega.)-.PHI..sub.N-k+i(u).ltoreq.g.sup.T(.omega.-u)+.epsilon-
. for all .omega.R.sup.M.sub.k+1}. (3.9) where
g.sub.o.sup.p=.omega..sub.o.sup.p=u.sub.o.sup.p=0 are all
permanently fixed. (Recall that these were used for notational
convenicence only.) A vector
g.epsilon..differential..PHI..sub.N-k+1(u) is called .[.a
subgradients.]. .Iadd.an .epsilon.-subgradient.Iaddend..
HOW DO YOU COMPUTE g.epsilon..differential..PHI..sub.N-k+i(u)
.Iadd.IV.3.3 .Iaddend.The Recovery Procedure.
The next objective is to explain a recovery procedure, i.e., given
a feasible (optimal or suboptimal) solution
.[..omega..sup.2.gamma..DELTA..gamma..omega..sup.N-k+1.gamma..DELTA..].
.Iadd.w.sup.2 of (4.10) (or w.sup.N-k+1 of (4.8) constructed
.Iaddend.via Theorem .[.3.1.]. .Iadd.4.1.Iaddend.), generate a
feasible solution z.sup.N-k+1 of equation (3.1) which is close to
.[..omega..sup.2.]. .Iadd.w.sup.2 in a sense to be
.Iaddend.specified. We first assume that no variables in
.[.(3.1).]. .Iadd.(4.7) .Iaddend.are .[.presassigned.].
.Iadd.preassigned .Iaddend.to zero; this assumption will .Iadd.be
.Iaddend.removed shortly. The difficulty with the solution
.[..omega..sup.N-k+1 .omega..]. .Iadd.w.sup.N-k+1 .Iaddend.it need
not satisfy the last (N-k-1) sets of constraints in .[.(3.1).].
.Iadd.(4.7).Iaddend.. (Note, however, that if .[..omega..sup.2
(3.4).]. .Iadd.w.sup.2 is an optimal solution (4.10) .Iaddend.and
.[..omega..sup.N-k+1.]. .Iadd.w.sup.N-k+1, as constructed in
Theorem 4.1, satisfies the .Iaddend.to relaxed constraints, then
.[..omega..sup.N-k+1.gamma..DELTA..DELTA..omega..].
.Iadd.w.sup.N-k+1 is optimal for (4.7.) The .Iaddend.recovery
proceudre described here is designed to preserve the 0-1 character
of the solution .[..omega..sup.2.gamma..DELTA..]. .Iadd.w.sup.2 of
(4.10) as far as possible: .Iaddend.If
.[..omega..sub.i.sub.k.sub.i.sub.i+1.sup.2=1 and l.sub.k.noteq.0 or
i.sub.k+1.noteq.0.]., .Iadd.w.sup.2.sub.l.sub.k.sub.i.sub.k+1-1 and
l.sub.k.noteq.0 or i.sub.k+1.noteq.0 .Iaddend.the corresponding
feasible solution z.sup.N-k+1 of .[.(3.1).]. .Iadd.(4.7)
.Iaddend.is construed so that z.sub.i.sub.0. . . i.sub.po . . .
o.sup.N-k+1=1 for some (i.sub.k+2, . . . i.sub.N). By this
reasoning, variables of the form z.sub.OOi.sub.k+2 .sub.. . .
i.sub.N.sup.N-k+1 can be assigned a value of one in the recovery
problem only if .omega..sub.oo.sup.2=1. However, variables
z.sub.ooi.sub.k+2 .sub.. . . i.sub.N.sup.N-k+1 will be treated
differently in the recovery procedure in that they can be assigned
either zero or one independent of the value .[.of
.omega..sub.oo.sup.2.]. .Iadd.w.sub.00.sup.2.Iaddend.. This
increases the feasible set of the recovery problem, leading to a
potentially better solution.
Let
{(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+j)}i.sub.k+1.sup.L.sup.k+1-0
be an enumeration of indices {l.sub.k, i.sub.k+1} of
.[..omega..sup.2.gamma..omega..sup.N-k+1.]. .Iadd.w.sup.2 (or the
first 2 indices of w.sup.N-k+1 constructed in .Iaddend.equation
.[.(5).]. .Iadd.(4.11).Iaddend.) such that
.[..omega..sub.i.sub.k.sub.(l.sub.k+1.sub.),i.sub.k+1.sub.(l.sub.k+1.sub.-
).sup.2=1.].
.Iadd.w.sup.2.sub.l.sub.k.sub.(l.sub.k+1.sub.),i.sub.k+1.sub.(l.sub.k+1.s-
ub.)=1 .Iaddend.for
(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1)).noteq.(0,0) and
(l.sub.k(l.sub.k+1), i.sub.k+1(l.sub.k+1))=(0,0) for l.sub.k+1=0
regardless of whether .[..omega..sub.oo.sup.2=1.].
.Iadd.w.sup.2.sub.oo=1 .Iaddend.or not. To define the .[.N-k
dimensional.]. .Iadd.(N-k)-dimensional .Iaddend.assignment problem
that .[.resores.]. .Iadd.restores .Iaddend.feasibility, let
.times..times..times..times..function..times..function..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..function..times..function..times..times..times..times..times-
..function..times..times..times..times..function..times..times..times..tim-
es..function..times..times..times..times..times..times..function..times..t-
imes..times..function..times..times..times..times..times..times..times..gt-
oreq..times..times..times..times..times..times..times. ##EQU00107##
where l.sub.m . . . k+1=l.sub.mol.sub.m+1o . . .
ol.sub.k(l.sub.k+1)=l.sub.m(l.sub.m 1( . . . (l.sub.l(l.sub.k+1)) .
. . )).[.(3.13).]. .Iadd.(4.19) .Iaddend. for m=2,k+1 and where
.[.o.]. .Iadd."o" .Iaddend.denotes function composition. For
example l.sub.23=l.sub.2(l.sub.3) and l.sub.234=l.sub.2o
l.sub.3(l.sub.4)=l.sub.2(l.sub.3(l.sub.4)).
Then the (N-k)-dimensional assignment problem that restores
feasibility is
.times..times. ##EQU00108##
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times.
##EQU00108.2## Let Y be an optimal or feasible solution to this
.[.N-k dimensional.]. .Iadd.(N-k)-dimensional .Iaddend.assignment
problem. The recovered feasible solution z.sup.N is defined by
.times..times..times..times..times..times..times..function..times..times.-
.times..function..times..times..times..times..function..times..times..time-
s..times..function..times..function..times..times..times..times..times..ti-
mes..times..times..times..times. ##EQU00109## Said in a different
way, the recovered .[.reasible.]. .Iadd.feasible .Iaddend.solution
z.sup.N corresponding to the multiplier set {u.sup.k+2, . . .
,u.sup.N} is then defined by
.function..times..function..times..function..times..times..times..functio-
n..times..times..times..function..times..times..times..times..times..times-
..times..times..times..times. ##EQU00110## where l.sub.m . . . k+1
is defined in .[.(3.13).]. .Iadd.(4.19) .Iaddend.and .[.o.].
.Iadd."o" .Iaddend.denotes function composition.
The next objective is to remove the .[.asumption.].
.Iadd.assumption .Iaddend.that all cost coefficients are defined
and all zero-one variables are free to be assigned. We first note
that the above construction of a reduced dimension assignment
problem .[.(3.14).]. .Iadd.(4.20) .Iaddend.is valid as long as all
cost coefficients c.sup.N-k are defined and all zero-one variables
in z.sup.N-k are free to be assigned. Modifications are necessary
for sparse problems. If the cost coefficient
c.sub.l.sub.k.sub.(l.sub.k+1.sub.)
i.sub.k+1.sub.(l.sub.k+1.sub.)i.sub.k+2 .sub.. . .
i.sub.N.sup.N-k+1 is well-defined and the zero-one variable
Z.sub.i.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.)i.sub.k+2
. . . i.sub.N.sup.N-k+1 is free to be assigned to zero or one, then
define c.sub.l.sub.k+1.sub.i.sub.k+2 .sub.. . .
l.sub.N.sup.N-k=c.sub.l.sub.k.sub.(l.sub.k+1.sub.)l.sub.k+1.sub.(l.sub.k+-
1.sub.)l.sub.k+2 .sub.. . . i.sub.N.sup.N-k+1 as in .[.(3.12).].
.Iadd.(4.18) .Iaddend.with z.sub.l.sub.k+1.sub.i.sub.k+2 .sub.. . .
l.sub.N.sup.N-k being free to be assigned. Otherwise,
z.sub.i.sub.k+1.sub.l.sub.k+2 .sub.. . . l.sub.N.sup.N-k is
preassigned to zero or the corresponding arc is not allowed in the
feasible set of arcs. To ensure that a feasible solution exists, we
now only need ensure that the variables z.sub.i.sub.k+1.sub.o
o.sup.N-k are free to be assigned for l.sub.k+1=0.1, . . .
,L.sub.k+1 with the corresponding cost coefficients being
well-defined. (Recall that all variables of the form
z.sub.i.sub.k.sub.o . . . o.sup.N-k+1 and z.sub.o o.sub.p.sub.o . .
. o.sup.N-k+1 are free (to be assigned) and all coefficients of the
form c.sub.l.sub.k.sub.o . . . o.sup.N-k+1 and c.sub.o . . .
oi.sub.p.sub.O.sup.N-k+1 are well-defined for l.sub.k=0, . . .
,L.sub.k and i.sub.p=0, . . . ,M.sub.p for p=k+1, . . . ,N.) This
is accomplished as follows: If the cost coefficient
c.sub.k.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.)o .
. . o.sup.N-k+1 is well-defined and
z.sub.l.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.)o .
. . o.sup.N-k+1 is free, then define c.sub.i.sub.k+1.sub.o . . .
o.sup.N-k=c.sub.l.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.-
)o . . . o.sup.N-k+1 with z.sub.i.sub.k+1.sub.o . . . o.sup.N-k
being free. Otherwise, since all variables of the form
z.sub.l.sub.k.sub.o . . . o.sup.N-k+1 and z.sub.ol.sub.k+1.sub.o .
. . o.sup.N-k+1 are free to be assigned with the corresponding
costs being well-defined, set c.sub.l.sub.k+1.sub.o . . .
o.sup.N-k=c.sub.l.sub.k.sub.(l.sub.k+1.sub.)oo . . .
o.sup.N-k+c.sub.ol.sub.k+1.sub.(l.sub.k+1.sub.)o . . . o.sup.N-k+1
where the first term is omitted if l.sub.k(I.sub.k+1)=0 and the
second, if i.sub.k+1(I.sub.k+1)=0. For l.sub.k(l.sub.k+1)=0 and
i.sub.k+1(l.sub.k+1)=0, define c.sub.o . . . o.sup.N-k=c.sub.o . .
. o.sup.N-k+1.
.Iadd.IV.3.4. .Iaddend.The last step k=N-2.
The description of the algorithm ends with k=N+2. The resulting
recovery problem defined in .[.(3.14).]. .Iadd.(4.20) .Iaddend.with
k=N-2 is
.PHI..ident..times..times..times..times..times..times..times..times..iden-
t..function..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00111##
Let Y be an optimal or feasible solution to this
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem. The recovered feasible solution Z.sup.N is defined by
.times..times..times..times..times..times..times..function..times..times.-
.times..function..times..times..times..function..times..times..times..time-
s..times..function..times..function..times..times..times..times..times..ti-
mes..times..times..times..times..times. ##EQU00112## Said in a
different way, the recoered feasible solution z.sup.N is then
defnied by
.times. ##EQU00113##
.function..times..times..times..times..function..times..times..times..tim-
es..function..times..times..times..times..times..times..function..times..t-
imes..function..times..times..times..times..times. ##EQU00113.2##
where l.sub.m . . . N-1 is defined in .[.(3.13).]. .Iadd.(4.19)
.Iaddend.and .[.o.]. .Iadd."o" .Iaddend.denotes function
composition. To complete the description, let
{(l.sub.n-1(l.sub.N),i.sub.N (l.sub.N)}.sub.l.sub.N.sup.L.sup.N=0
be an enumeration of indices {l.sub.N-1,i.sub.N} of Y such that
Y.sub.l.sub.N-1.sub.(l.sub.N.sub.),i.sub.N.sub.(l.sub.N.sub.)=1 for
(l.sub.N-1(l.sub.N),i.sub.N(l.sub.N)).noteq.(0,0) and
(l.sub.N-1(l.sub.N), i.sub.N(l.sub.N)=(0,0) for l.sub.N=0
regardless of whether Y.sub.oo=1 or not. Then the recovered
feasible solution can be written as
.times..times..times..times..times..times..times..times..function..times.-
.times..times..function..times..times..times..function..times..times..time-
s..times..function..times..function. ##EQU00114##
.Iadd.IV.3.5. .Iaddend.The Upper and Lower .[.Bound.].
.Iadd.Bounds.Iaddend.. The upper bound on the feasible solution is
given by
.function..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times. ##EQU00115## and the
lower by .PHI..sub.N.gamma..sup.3 . . . ,u.sup.N) for any
multiplier value (u.sup.3, . . . , u.sup.N). In particular, we have
.PHI..sub.N(u.sup.3, . . . ,u.sup.N).ltoreq..PHI..sub.N( .sup.3, .
. . , .sup.N).ltoreq.V.sub.N( z.sup.N).ltoreq.V.sub.N({circumflex
over (z)}.sup.N) .[.(3.22).]. .Iadd.(4.27) .Iaddend. where
(u.sup.3, . . . ,u.sup.N) is any multiplier value, (.sup.-3 . . .
,.sup.-N.DELTA. maximizer of .PHI..sub.N.gamma..sup.3 . . . ,
u.sup.N), .sup.-N multidimensional assignment problem .[.(3.1).].
.Iadd.(4.7) .Iaddend.and {circumflex over (z)}.sup.N is the
recovered feasible solution defined by .[.(3.20).].
.Iadd.(4.25).Iaddend.. Finally, we conclude with the observation
that V.sub.N({circumflex over (z)})=V.sub.2(Y) where Y is the
optimal solution of .[.(3.17).]. .Iadd.(4.23) .Iaddend.used in the
construction of {circumflex over (z)} in equations .[.(3.18).].
.Iadd.(4.24).Iaddend.-.[.(3.20).]. .Iadd.(4.26).Iaddend..
.Iadd.IV.3.6. .Iaddend.Reuse of Multipliers.[...].
Since the most computationally expensive part of the algorithm
occurs in the maximization of .PHI..sub.N-k+1 starts or multipliers
close to the optimal are fundamentally important for real-time
speed. The purpose of this section is to demonstrate that the
multiplier set obtained at stage k.gtoreq.1 provide good starting
values for those obtained at step k+1. Theorem .[.3.xx.].
.Iadd.4.4.Iaddend.. Let (u.sup.3, . . . ,u.sup.N) denote a
multiplier set obtained in the .[.maximum.]. .Iadd.maximization
.Iaddend.of .PHI..sub.N.gamma..gamma..sup.3 . . . ,u.sup.N). Then
this multiplier set satisfies the string of inequalities
.PHI..sub.N(u.sup.3, . . . ,u.sup.N).ltoreq..PHI..sub.N-1(u.sup.4,
. . . u.sup.5).ltoreq. . . .
.ltoreq..PHI..sub.3(u.sup.N).ltoreq..PHI..sub.2.ltoreq.V.sub.N({circumf-
lex over (z)}), .Iadd.(4.28) .Iaddend. where after the first step
in the maximization of .PHI..sub.N multipliers are not changed in
the remaining steps. Furthermore, to improve this inequiality, let
(u.sup.2+kN-k+1, . . . ,u.sup.N,N-k+1) denote a maximizer of
.PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N). Then we have
.PHI..function..times..ltoreq..times..PHI..times..times..ltoreq..times..P-
HI..function..times..ltoreq..times..PHI..function..times..ltoreq..times..t-
imes..ltoreq..PHI..function..ltoreq..PHI..ltoreq..function..times.
##EQU00116##
.[.Inequalities depend on solution of 2d assignment problem..].
Proof: Suppose we have a value of the multipliers (u.sup.k+2, . . .
,u.sup.N) obtains in the maximization of
.PHI..function..times..ident..times..times..PHI..function..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times.
##EQU00117## where
.PHI..function..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times. ##EQU00118## These need not be the maximizer; however,
we do assume that we have solved the above minimization problem
optimally to evaluate .PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N).
Just as in the definition of the .[.earlier.]. recovery problem
.Iadd.disussed earlier.Iaddend., let {(l.sub.k(l.sub.k+1),i.sub.k+1
{(l.sub.k+1)}l.sub.k+1.sup.L.sup.k+1=0 be an enumeration of indices
{l.sub.k,i.sub.k+1) of the optimal solution w.sup.2 of problem
.[.(3.4).]. .Iadd.(4.10) .Iaddend.(or the first 2 indices of the
solution .[.w.sup.N=k+1.]. .Iadd.w.sup.N-k+1 .Iaddend.of the
relaxed problem .[.(3.2).]. .Iadd.(4.8) .Iaddend.constructed in
equation .[.(3.5).]. .Iadd.(4.11).Iaddend.) such that
w.sub.l.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.).sup.2=1
for (l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1)).noteq.(0,0) and
(l.sub.k(l.sub.k+1),i.sub.k+1(l.sub.k+1))=(0,0) for l.sub.k+1=0
regardless of whether w.sub.oo.sup.2=1 or not.
Then, the final value of the objective function can be expressed
as
.PHI..function..times..ident..times..times..PHI..function..times..times..-
times..times..times..times..times..times..times..function..times..function-
..times..times..times..times..times. ##EQU00119## where
.PHI..function..times..times..times..times..times..times..times..function-
..times..function..times..times..times..times..times..times..times..functi-
on..times..function..times..times..times..times..times..times..times..time-
s. ##EQU00120## If now another constrain set, say the (k+2).sup.th
set, is added to this problem, one has
.PHI..function..times..times..ident..times..times..PHI..function..times..-
ident..times..times..times..times..times..times..times..times..function..t-
imes..function..times..times..times..times..times..times..function..times.-
.function..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..function..times..f-
unction..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..function..times..function..times..-
times..times..times..times..times..times. ##EQU00121##
Since the constraint .SIGMA..sub.i.sub.k+1.sub.i.sub.k+3 .sub.. . .
i.sub.N
Z.sub.i.sub.k.sub.(l.sub.k+1.sub.)i.sub.k+1.sub.(l.sub.k+1.sub.)i-
.sub.k+2 .sub.. . . i.sub.N.sup.N-k+1=1 for l.sub.k+2=1, . . .
,M.sub.k2 is now imposed, the feasible region is smaller, so that
one has
.PHI..function..times..ltoreq..times..PHI..function..times..ident..times.-
.PHI..function..times..ident..times..PHI..function..times.
##EQU00122## where the fact that .PHI..sub.N=k+1 does not depend on
the multiplier set u.sup.k+2 due the added constraint set. Also,
the last equivalence follows from the fact that
.PHI..sub.N=k+1.gamma..sup.k+3 . . . ,u.sup.N) is the relaxed
problem .[.(3.xx).]. (with k replaced by k+1) for the recovery
problem .[.(3.xx).]. in step k of the above algorithm. Continuing
this way, one can compute (u.sup.3, . . . ,u.sup.N) at the first
step (k=1), fix them thereafter, and perform no optimization at the
subsequent steps to obtain .PHI..sub.N(u.sup.3, . . .
,u.sup.N).ltoreq..PHI..sub.N-1(u.sup.4, . . . u.sup.N).ltoreq. . .
.
.ltoreq..PHI..sub.3(u.sup.N).ltoreq..PHI..sub.2.ident.V.sub.N({circumflex
over (Z)}) where .PHI..sub.2 .gamma..DELTA. is defined in
(4.22).
To explain how to improve this inequality, let (u.sup.2+k,N-k+1, .
. . , u.sup.N,N-k+1) denote a maximizer of
.PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N). Then by the same
reasoning one has .[.(3.xx).]. .Iadd.the result .Iaddend.as stated
in the theorem. .[.Q.E.D..].
.Iadd.V. .Iaddend.Summary of the Lagrangian Relaxation Problem.
(APB)
Given the multidimensional assignment problem
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times.
##EQU00123## where c.sub.o . . o.sup.N is arbitrarily defined to be
zero and is included for notational convenience and where the
superscript N on both c and z is not an exponent, but denotes the
dimension of the subscripts and the assignment problem as stated in
.[.(2.5).]. .Iadd.(3.5).Iaddend.. In relaxed and recovery problems
c.sub.o . . . o.sup.N need not be zero! In this problem, we assume
that all zero-one variables z.sub.i.sub.1.sub.. . . i.sub.N.sup.N
with precisely one nonzero index are free to be assigned and that
the corresponding cost coefficients are well-defined. (This is a
valid assumption in the tracking environment .Iadd.(A. B. Poore.
Multidimensional assignments and multitarget tracking, partitioning
data sets, I. J. Cox, P. Hansen, and B. Julesz, editors, DIMACS
Series in Discrete Mathematics and Theoretical Computer Science,
American Mathematical Society, Providence, R.I., 19:169-198, 1995;
A. B. Poore. Multidimensional assignment formulation of data
association problems arising from multitarget tracking and
multisensor data fusion. Computational Optimization and
Applications, 3:27-57, 1994).Iaddend..) Although not required,
these cost coefficients with exactly one nonzero index can be
translated to zero by cost shifting .Iadd.(A. B. Poore and N.
Rijavec. A lagrangian relaxation algorithm for multidimensional
assignment problems arising from multitarget tracking. SIAM Journal
of Optimization, 3, No. 3:544-563, 1993) .Iaddend.without changing
the optimal assignment.
Having explained many of the relaxation features, it is now
appropriate to present the Lagrangian relaxation algorithm, which
iteratively constructs a feasible solution to the N-dimensional
assignment problem .[.(4.1).]. .Iadd.(5.1).Iaddend..
Algorithm .[.4.1.]. .Iadd.5.1 .Iaddend.(Lagrangian Relaxation
Algorithm).[...]. To construct a high quality feasible solution,
.[.enoted.]. .Iadd.denoted .Iaddend.by {circumflex over (z)}.sup.N,
of the assignment problem .[.(4.1).]. .Iadd.(5.1).Iaddend., proceed
as follows:
0. Initialize the multipliers (u.sup.k+2, . . . ,u.sup.N), e.g.,
(u.sup.k+2, . . . , u.sup.N)=(0, . . . ,0).
For k-1, . . . ,N-2, do
1. For the Lagrangian relaxed problem .[.(3.2).]. .Iadd.(4.8)
.Iaddend.from the problem .[.(3.1).]. .Iadd.4.7) .Iaddend.by
relaxing the last (N-k-1) sets of constraints.
2. Use a nonsmooth optimization technique to solve Maximize
{.PHI..sub.N-k+1|u.sup..GAMMA..epsilon.R.sup.M.sup.p+1 for p=k+2, .
. . ,N with u.sub.0.sup.p=0 being fixed} .[.(4.2).]. .Iadd.(5.2)
.Iaddend. where .PHI..sub.N-k+1(u.sup.k+2, . . . ,u.sup.N) is
defined by equation .[.(3.2).]. .Iadd.(4.8).Iaddend.. The algorithm
.[.following problem (3.9).]. .Iadd.in Section IV.3.2
.Iaddend.provides one method for computing a function value and a
subgradient out of the subdifferential at (u.sup.k+2, . . .
,u.sup.N), as required in most nonsmooth optimization
techniques.
3. Given an approximate or optimal maximizer of .[.(4.2).].
.Iadd.(5.2).Iaddend., say (u.sup.l+2, . . . ,u.sup.N), let w.sup.2
assignment problem .[.(3.4).]. .Iadd.(4.10) .Iaddend.corresponding
to this maximizer of .PHI..sub.N-k+1(u.sup.k+2, . . .
,u.sup.N).
4. Formulate the recovery (N-k)-dimensional problem .[.(3.13).].
.Iadd.(4.19).Iaddend., modified as discussed at the end of
subsection .[.(3.3).]. .Iadd.(IV.3.2) .Iaddend.for sparse problems.
At this stage, {circumflex over (z)}.sup.N as defined in
.[.(3.15).]. .Iadd.(4.21), .Iaddend.contains the alignment of the
indices
Formulate the final two-dimensional problem .[.(3.17).].
.Iadd.(4.22) .Iaddend.and complete the final recovered solution
{circumflex over (z)}.sup.N as in .[.(3.20).].
.Iadd.(4.25).Iaddend..
To explain the lower and upper bounds, let .PHI..sub.N defined in
.[.(3.2).]. .Iadd.(4.8) .Iaddend.with k=1, let V.sub.N(z.sup.N) be
the objective function value of the N-dimensional assignment
problem in equation .[.(2.5).]. .Iadd.(3.5) .Iaddend.corresponding
to a feasible solution z.sup.N of the constraints in .[.(2.5).].
.Iadd.(3.5).Iaddend., and let z.sup.N-k+1 .[.(2.5).].
.Iadd.(3.5).Iaddend.. Then, .PHI..sub.N(u.sup.3, . . .
,u.sup.4).ltoreq.V.sub.N( z.sup.N).ltoreq.V.sub.N({circumflex over
(z)}.sup.N) .[.(4.3).]. .Iadd.(5.3) .Iaddend. is the desired
inequality.
COMMENTS
1. Step 2 is the computational part of the algorithm. Evaluating
.PHI..sub.N-k+1 search procedure requires 99% of the computing time
in the algorithm. This part uses .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment algorithms, a search over
a large number of indices, and a nonsmooth optimization algorithm.
It is the second part (the search) that consumes 99% of the
computational time and this is almost entirely parallelizable.
2. In track maintenance, the warm starts for the initial
multipliers for the first step are available. For the relaxation
procedure, initial multipliers are available in step 2 from the
prior step in the algorithm.
3. There are many variations on the above algorithm. For example,
one can compute a good solution on the first pass (k=1) and not
perform the optimization in (2) thereafter. This yields a great
solution. Thus one can continue the optimization at the first pass,
and immediately compute quality feasible solutions to the
problem.
.Iadd.VI. .Iaddend.Lagrangian Relaxation Algorithm for the
3-Dimensional Algorithm. .[.(ABP).].
In this section, we illustrate the relaxation algorithm for the
.[.three dimensional.]. .Iadd.three-dimensional .Iaddend.assignment
problem. Having discussed the general relaxation scheme,
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es..times..times..times..times..times..times..di-elect
cons..times..times..times..times..times..times..times..times..times..time-
s. ##EQU00124## To ensure that a feasible solution of .[.(5.1).].
.Iadd.(6.1) .Iaddend.always exists for a sparse problem, all
variables with exactly one nonzero index (i.e., variables of the
formz.sub.l.sub.100, Z.sub.Oi.sub.20 and z.sub.OOi.sub.3 for
i.sub.p=1, . . . , M.sub.p and p=1,2,3 are assumed free to be
assigned with the corresponding cost coefficients being
well-defined. This assumption is valid in the tracking environment
.[.[**refPoa, **refPob].]. .Iadd.(A. B. Poore. Multidimensional
assignments and multitarget tracking, partitioning data sets, I. J.
Cox, P. Hansen, and B. Julesz, editors, DIMACS Series in Discrete
Mathematics and Theoretical Computer Science, American Mathematical
Society, Providence, R.I., 19:169-198, 1995; A. B. Poore.
Multidimensional assignment formulation of data association
problems arising from multitarget tracking and multisensor data
fusion. Computational Optimization and Applications, 3:27-57,
1994).Iaddend..
.Iadd.VI.1. .Iaddend.The Lagrangian Relaxed Assignment
Problem.[...].
The .[.three dimensional.]. .Iadd.three-dimensional
.Iaddend.assignment problem .[.(5.1).]. .Iadd.(6.1) .Iaddend.has
three sets of constraints and one can describe the relaxation by
relaxing any of the three sets of constraints, the description here
is based on relaxing the last set of constraints. A
(M.sub.3+1)-dimensional multiplier vector (i.e., u.sup.3
.epsilon.R.sup.M.sup.3.sup.+1) associated with the 3.sup.th
constraint set will be denoted by u.sup.3=(u.sub.0.sup.3,
u.sub.1.sup.3 . . . ,u.sub.M.sub.3.sup.3).sup.T with
u.sub.o.sup.3.ident.0 being fixed for notational convenience. (The
zero multiplier u.sub.0 .sup.3 is used for notational convenience.)
Now, the .[.three dimensional.]. .Iadd.three-dimensional
.Iaddend.assignment problem .[.(5.1).]. .Iadd.(6.1) .Iaddend.is
relaxed to a two-dimensional assignment problem by incorporating
last set of constraints into the objective function via the
Lagrangian. Although any constraint set can be relaxed, we choose
the last set of constraints for convenience. The .Iadd.relaxed
.Iaddend.problem is
.PHI..function..times..ident..times..times..PHI..function..ident..times..-
times..times..times..times..times..times..times..times.
.times..times..function..times..times..times..times..times..iden-
t..times..times..times..times..times..times..times..times..times..times..t-
imes..times..times..times..times..times..times..times..times..times..times-
..times. .times..times..times..times..times..times..times..times.
##EQU00125##
One of the major steps in the algorithm is the maximization of
.[..PHI..sub.3.gamma..sup.3.DELTA. u.sup.3.].
.Iadd..PHI..sub.3(u.sup.3) with respect to the multiplier vector
u.sup.3. .Iaddend.It turns out that .PHI..sub.3 .Iadd.is a concave,
continuous, and .Iaddend.piecewise affine function of the
multiplier vector u.sup.3, so that the maximization of .PHI..sub.3
.Iadd.is a problem of nonsmooth optimization. .Iaddend.Since many
of these algorithms require a function value and a subgradient of
.[..PHI..sub.3 .gamma..sup.3.DELTA..]. .Iadd..PHI..sub.3 at any
required multiplier value (u.sup.3), we .Iaddend.address this
problem in the next subsection. We note, however, that there are
other ways to maximize .PHI..sub.3 .Iadd.and the next
.Iaddend.subsection addresses but one such method.
.Iadd.VI.2. .Iaddend.Properties Lagrangian Relaxed Assignment
Problem
For a function evaluation of .[..PHI..sub.3 .gamma..].
.Iadd..PHI..sub.3.Iaddend., .Iadd.we show that an optimal (or
.Iaddend.suboptimal) solution of this relaxed problem .[.(5.2).].
.Iadd.(6.2) .Iaddend.can be constructed from that of a
two-dimensional assignment problem. Then, the nonsmooth
characteristics of .PHI..sub.3 .Iadd.are addressed,
.Iaddend.followed by a method for computing a subgradient.
Evaluation of .PHI..sub.3. .Iadd.Define for each .Iaddend.(i.sub.1,
i.sub.2) j.sub.3=j.sub.3(i.sub.1,i.sub.2) and a new cost function
c.sub.i.sub.1.sub.i.sub.2.sup.2 by:
.ident..function..times..times..times..times..times..times..times..times.-
.times..times..times..function..times..times..times..times..times..noteq..-
times..times..times..times..times..times..times..times.
##EQU00126## Given an index pair (i.sub.1,i.sub.2), j.sub.3 need
not be unique, resulting in the potential generation of several
subgradients .[.(5.11).]. .Iadd.(6.11).Iaddend.. (We further
discuss this issue at the end of .Iadd.the .Iaddend.Subsection
.[.5.2.3..].)
Now, define
.PHI..function..times..times..times..PHI..function..ident..times..times..-
times..times..times..times..times..times..times..times..times..times..time-
s..times..times. .times..times..times..times.
.times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00127##
As an aside, two observations are in order. The first is that the
search procedure needed for the computation of the relaxed cost
coefficients in .[.(5.3).]. .Iadd.(6.3) .Iaddend.is the most
computationally intensive part of the entire relaxation algorithm.
The second is that a feasible solution z.sup.3 of a sparse problem
.[.(5.1).]. .Iadd.(6.1) .Iaddend.yields a feasible solution z.sup.2
of .[.(5.4).]. .Iadd.(6.4) .Iaddend.via the construction
.times..times..times..times..times..times..times..times..times..times.
##EQU00128## Thus the two-dimensional assignment problem
.[.(5.4).]. .Iadd.(6.4) .Iaddend.has feasible solutions other than
those with exactly one nonzero index if the original problem
.[.(5.1).]. .Iadd.(6.1) .Iaddend.does.
The following Theorem .[.5.1.]. .Iadd.6.1 .Iaddend.states that an
optimal solution of .[.(5.2).]. .Iadd.(6.2) .Iaddend.can be
computed from that of .[.(5.4).]. .Iadd.(6.4).Iaddend..
Furthermore, if the solution of either of these two problems is
.epsilon. .epsilon. then so is the other. The converse is contained
in Theorem .[.5.2.]. .Iadd.6.2.Iaddend..
Theorem .[.5.1.]. .Iadd.6.1.Iaddend.. Let .[..omega..sup.2.].
.Iadd.w.sup.2 .Iaddend.dimensional assignment problem .[.(5.4).].
.Iadd.(6.4) .Iaddend.and define w.sup.3 by
w.sub.i.sub.1.sub.i.sub.2.sub.i.sub.3.sup.3=w.sub.i.sub.1*2.sup.2
if i.sub.3=j.sub.3 and (i.sub.1,i.sub.2).noteq.(0,0);
w.sub.i.sub.1.sub.i.sub.2.sub.i.sub.3.sup.3=0 if
i.sub.3.noteq.j.sub.3 and (i.sub.1,i.sub.2).noteq.(0,0);
w.sub.ooi.sub.3.sup.3=1 if
c.sub.ooi.sub.3.sup.3+u.sub.i.sub.3.sup.3.ltoreq.0;
w.sub.ooi.sub.3.sup.3=01 if
c.sub.ooi.sub.3.sup.3+u.sub.i.sub.3.sup.3>0. .[.(5.5).].
.Iadd.(6.5) .Iaddend. Then w.sup.3 is a feasible solution of the
Lagrangian relaxed problem .Iadd.(6.2) .Iaddend.and .[..PHI.
.gamma..sup.3 3.DELTA. .PHI. .gamma..sup.2 3.DELTA. .epsilon.
.epsilon..sup.2.]. .Iadd..phi..sub.3(w.sup.3;u.sup.3)={circumflex
over (.phi.)}.sub.3(w.sup.2;u.sup.3). If, in addition, w.sup.2 is
.Iaddend.optimal for the .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.problem, then w.sup.3 is an optimal
solution of the relaxed problem and .PHI..sub.3(u.sup.3)=
.PHI..sub.3(u.sup.3).
With the exception of one equality being converted to an
inequality, the following theorem is a converse of Theorem
.[.5.1.]. .Iadd.6.1.Iaddend..
Theorem .[.5.2.]. .Iadd.6.2.Iaddend.. Let w.sup.2 be a feasible
solution to problem .[.(5.2).]. .Iadd.(6.2).Iaddend.and define
w.sup.2 by
.times..times..times..times..times..times..times..times..noteq..times..ti-
mes. ##EQU00129## w.sub.oo.sup.2=0 if (i.sub.1,i.sub.2)=(00) and
c.sub.ooi.sub.3.sup.3+u.sub.i.sub.3.sup.3>0 for all i.sub.3
w.sub.oo.sup.2=1 if (i.sub.1,i.sub.2)=(00) and
c.sub.ooi.sub.3.sup.3+u.sub.i.sub.3.sup.3.ltoreq.0 for some
i.sub.3. Then w.sup.2 is a feasible solution of the problem
.[.(5.4).]. .Iadd.(6.4) .Iaddend.and
.PHI..sub.3(w.sup.3;u.sup.3).gtoreq..PHI..sub.3(w.sup.2;u.sup.3).
If, in addition, w.sup.3 is optimal for .[.(5.2).].
.Iadd.(6.2).Iaddend., then w.sup.2 is an optimal solution of
.[.(5.4).]. .Iadd.(6.4).Iaddend.,
.PHI..sub.3(w.sup.3;u.sup.3)=.PHI..sub.3(w.sup.2;u.sup.3) and
.PHI..sub.3(u.sup.3)= .phi..sub.3(u.sup.3).
The Nonsmooth Optimization Problem
An Algorithm for Computing .PHI..sub.3 and a Subgradient
Given u.sup.3 the problem of computing
.[..PHI..sub.3.gamma..sup.3.DELTA..]. .Iadd..PHI..sub.3(u.sup.3)
and a .Iaddend.subgradient of .[..PHI..sub.3.gamma..sup.3.DELTA..].
.Iadd..PHI..sub.3(u.sup.3) .Iaddend.
1. Starting with problem .[.(5.1).]. .Iadd.(6.1).Iaddend., form the
relaxed problem .[.(5.2).]. .Iadd.(6.2). .Iaddend.
2. To solve .[.(5.2).]. .Iadd.(6.2) .Iaddend.optimally, define the
.[.two dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem .[.(5.4).]. .Iadd.(6.4) .Iaddend.via the transformation
.[.(5.3).]. .Iadd.(6.3). .Iaddend.
3. Solve the two-dimensional problem .[.(5.4).]. .Iadd.(6.3)
.Iaddend.optimally.
4. Reconstruct the optimal solution, say
w.sup.3.epsilon..gamma..DELTA. equation .[.(5.6).]. .Iadd.(6.6)
.Iaddend.as in Theorem .[.5.1.]. .Iadd.6.1.Iaddend..
5. Then
.PHI..function..ident..PHI..function..times..times..times..times..times..-
times..times..times..times..times..times..times..times..times..times.
##EQU00130##
6. A subgradient is given by substituting w.sup.3 objective
function .[.(5.2).]. .Iadd.(6.2).Iaddend., erasing the minimum
sign, and taking the partial with respect to u.sub.i.sub.3.sup.3.
The result is
.times..times. ##EQU00131##
.differential..PHI..function..differential..times..times..times..times..t-
imes..times..times..times..times..times..times. ##EQU00131.2##
.Iadd.VI.3. .Iaddend.The Recovery Procedure.[...].
The next objective is to explain a recovery procedure, i.e., given
a feasible (optimal or suboptimal) solution w.sup.2 of .[.(5.4).].
.Iadd.(6.4) .Iaddend.(or w.sup.3 of .[.(5.2).]. .Iadd.(6.2)
.Iaddend.constructed in Theorem .[.5.1.]. .Iadd.6.1.Iaddend.),
generate a feasible solution z.sup.3 of equation .[.(5.1).].
.Iadd.(6.1) .Iaddend.which is close to w.sup.2 in a sense to be
specified. We first assume that no variables in .[.(5. 1).].
.Iadd.(6.1) .Iaddend.are preassigned to zero; this assumption will
be removed shortly. The difficulty with the solution w.sup.3 in
Theorem .[.5.1.]. .Iadd.6.1 .Iaddend.is that it need not satisfy
the third set of constraints in .[.(5.1).]. .Iadd.(6.1).Iaddend..
(Note, however, that if w.sup.2 is an optimal solution for
.[.(5.4).]. .Iadd.(6.4) .Iaddend.and w.sup.3, as constructed in
Theorem .[.5.1.]. .Iadd.6.1.Iaddend., satisfies the relaxed
constraints, then w.sup.3 is optimal for .[.(5.1).].
.Iadd.(6.1).Iaddend..) The recovery procedure described here is
designed to preserve the 0-1 character of the solution w.sup.2 of
.[.(5.4).]. .Iadd.(6.4) .Iaddend.as far as possible: If
w.sub.i.sub.1.sub.i.sub.23.sup.2=1 and i.sub.1.noteq.0 or
i.sub.2.noteq.0, the corresponding feasible solution z.sup.3 of
.[.(5.1).]. .Iadd.(6.1) .Iaddend.is constructed so that
z.sub.i.sub.1.sub.i.sub.2.sub.i.sub.3.sup.3=1 for some
(i.sub.1,i.sub.2,i.sub.3) . By this reasoning, variables of the
form Z.sub.OOi.sub.3.sup.3.[.,.]. can be assigned a value of one in
the recovery problem only if w.sub.OO.sup.2=1. However, variables
z.sub.OOi.sub.3.sup.3 will be treated differently in the recovery
procedure in that they can be assigned either zero or one
independent of the value w.sub.OO.sup.2. This increases the
feasible set of the recovery problem, leading to a potentially
better solution.
Let {(i.sub.1(l.sub.2),i.sub.2(l.sub.2)}.sub.l.sub.2.sup.L=0 be an
enumeration of indices {i.sub.1,i.sub.2} of w.sup.2 (or the first 2
indices of w.sup.3 constructed in equation.[.(5).].
.Iadd.(6.5).Iaddend.) such that
w.sub.i.sub.1.sub.(l.sub.2.sub.),i.sub.2.sub.(l.sub.2).sup.2=1 or
(i.sub.1(l.sub.2)i.sub.2).noteq.(0,0) and
(i.sub.1(l.sub.2),i.sub.2(l.sub.2))=(0,0) for l.sub.2=0 regardless
of whether w.sub.00.sup.2=1 or not. (The latter can only improve
the quality of the feasible solution.)
Next to define the two-dimensional assignment problem that restores
feasibility, let
c.sub.l.sub.2.sub.l.sub.3.sup.2=c.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.s-
ub.(l.sub.2.sub.)i.sub.3.sup.3 for l.sub.2=0, . . . ,L.sub.1
.[.(5.12).]. .Iadd.(6.9) .Iaddend. Then the two-dimensional
assignment problem that restores feasibility is
.times.
.times..times..times..times..times..times..times..times..times.-
.times.
.times..times..times..times..times..times..times..times..times..-
times..di-elect
cons..times..times..times..times..times..times..times.
##EQU00132##
The next objective is to remove the assumption that all cost
coefficients are defined and all zero-one variables are free to be
assigned. We first note that the above construction of a reduced
.[.dimension.]. .Iadd.two-dimensional .Iaddend.assignment problem
.[.(5.13).]. .Iadd.(6.11) .Iaddend.is valid as long as all cost
coefficients c.sup.2 are defined and all zero-one variables in
z.sup.2 are free to be assigned. Modifications are necessary for
sparse problems. If the cost coefficient
c.sub.i.sub.1.sub.(l.sub.2)i.sub.2.sub.(l.sub.2.sub.)i.sub.3.sup.3
is well-defined and the zero-one variable
z.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.)i.sub.3.sup.3
is free to be assigned to zero or one, then define
c.sub.l.sub.2.sub.i.sub.3.sup.2=c.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.s-
ub.(l.sub.2.sub.)i.sub.3.sup.3 as in .[.(5.12).]. .Iadd.(6.10)
.Iaddend.with
z.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.)i.sub.3.sup.3
being free to be assigned. Otherwise,
z.sub.l.sub.2.sub.i.sub.3.sup.2 is preassigned to zero or the
corresponding arc is not allowed in the feasible set of arcs. To
ensure that a feasible solution exists, we now only need ensure
that the variables z.sub.l.sub.2.sub.O.sup.2 are free to be
assigned for l.sub.2=0,1, . . . ,L.sub.1 with the corresponding
cost coefficients being well-defined. .[.(.]. Recall that all
variables of the form z.sub.i.sub.1.sub.OO.sup.3,
z.sub.Oi.sub.2.sub.O.sup.3 and z.sub.OOi.sub.3.sup.3 are free (to
be assigned) and all coefficients of the corresponding form
c.sub.i.sub.1.sub.OO.sup.3,c.sub.Oi.sub.2.sub.O.sup.3 and
c.sub.OOi.sub.3.sup.3.Iadd.need to be defined.Iaddend.. .[.).].
This is accomplished as follows: .[.If.]. .Iadd.if .Iaddend.the
cost coefficient
c.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.)O.sup.3
is well-defined and
z.sub.i.sub.1.sub.(l.sub.2.sub.)i.sub.2.sub.(l.sub.2.sub.)O.sup.3
is free, then define
c.sub.l.sub.2.sub.O.sup.2=c.sub.i.sub.2.sub.(l.sub.2.sub.)i.sub.2.sub.(l.-
sub.2.sub.)O.sup.3 with .sup.z.sub.l.sub.2.sub.O.sub.3.sup.2 being
free. Otherwise, since all variables of the form
z.sub.i.sub.1.sub.O.sup.3 and z.sub.Ol.sub.2.sub.O.sup.3 are free
to be assigned with the corresponding costs being well-defined, set
c.sub.l.sub.2.sub.O.sup.2=c.sub.i.sub.1.sub.(l.sub.2.sub.)OO.sup.3+c.sub.-
Oi.sub.2.sub.(l.sub.2.sub.)O.sup.3 where the first term is omitted
if i.sub.1(1.sub.2)=0 and the second, if i.sub.2(l.sub.2)=0. For
i.sub.1(l.sub.2)=0 and i.sub.2(l.sub.2)=0, define
c.sub.oo.sup.2=c.sub.OOO.sup.3.
.Iadd.VI.4. .Iaddend.The final recovery problem.[...].
The recovery problem for the case N=3 is
.times.
.times..times..times..times..times..times..times..times..times.-
.times.
.times..times..times..times..times..times..times..times..times..-
times..di-elect
cons..times..times..times..times..times..times..times..times.
##EQU00133##
Let Y be an optimal or feasible solution to this
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem. The recovered feasible solution z.sup.3 is defined by
.times..times..times. ##EQU00134##
.times..times..times..times..function..function..times..times..times..tim-
es..times. ##EQU00134.2## Said in another way, let
{(l.sub.2(l.sub.3),i.sub.3(l.sub.3)}.sub.l.sub.3.sup.L=0 be an
enumeration of indices {l.sub.2,i.sub.3} of Y .sub.such that
Y.sub.l.sub.2.sub.(l.sub.3.sub.),i.sub.3.sub.(l.sub.3.sub.)=1 for
(l.sub.2(l.sub.3), i.sub.3(l.sub.3).Iadd.).Iaddend..noteq.(0,0) and
(l.sub.2(l.sub.3),i.sub.3(l.sub.3.Iadd.).Iaddend.)=(0,0) for
l.sub.3=0 regardless of whether Y.sub.OO=1 or not. Then the
recovered feasible solution can be written as
.times..times. ##EQU00135##
.times..times..times..times..function..function..times.
##EQU00135.2## The upper bound on the feasible solution is given
by
.function..times..times..times..times..times..times..times..times..times.-
.times..times. ##EQU00136## and the lower by
.[..PHI..sub.3.gamma..sup.3.DELTA..gamma..sup.3.DELTA..].
.Iadd..PHI..sub.3(u.sup.3).Iaddend..
In particular, we have .PHI..sub.3(u.sup.3).gtoreq..PHI..sub.3(
.sup.3).ltoreq.V.sub.3( z.sup.3).ltoreq.V.sub.3({circumflex over
(z)}.sup.3) where u.sup.3 is any multiplier value, .[.(
.DELTA..sup.3.PHI..sub.3.gamma..sup.3.DELTA..epsilon. z.sup.3.].
.Iadd.( .sup.3) is a maximizer of .PHI..sub.3(u.sup.3), z.sup.3 is
an optimal solution of the multidimensional .Iaddend.assignment
problem and {circumflex over (z)}.sup.3 is the recovered feasible
solution defined by .[.(5.16).]. .Iadd.(6.13).Iaddend..
.Iadd.VII. .Iaddend.Other Relaxations for the Multidimensional
Assignment Problem. .[.(ABP).].
In this section, we briefly describe other possible relaxations and
their implications. These include linear programming relaxations
and the corresponding duals.
Recall .Iadd.from Section II .Iaddend.that one starts with the
definition of the zero-one variable z.sub.1.sub.N.sub.. . .
i.sub.N=1 and cost coefficients to form the problem
.times.
.times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times..times..times..times..times..times..times..times..tim-
es.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..di-elect
cons..times..times..times..times..times..times..times. ##EQU00137##
where c.sub.O . . . O is arbitrarily defined to be zero. Here, each
group of sums in the constraints represents the fact that each
non-dummy report occurs exactly once in a "track of data". One can
modify this formulation to include multiassignments of one, some,
or all the actual reports. The assignment problem .[.(2.5).].
.Iadd.(3.5) .Iaddend.is changed accordingly. For example, if
z.sub.i.sub.k.sup.l is to be assigned no more than, exactly, or no
less than n.sub.i.sub.k.sup.k times, then the "=1" in the
constraint .[.(2.5).]. .Iadd.(3.5) .Iaddend.is changed to
respectively. Modifications for group tracking and multiresolution
features of the surveillance region will be addressed in future
work. In making these changes, one must pay careful attention to
the independence assumptions that need not be valid in many
applications.
Again, the recent work of Poore and Drummond .Iadd.(A. B. Poore and
O. E. Drummond. Track initiation and maintenance using
multidimensional assignment problems. In D. W. Hearn, W. W. Hager,
and P. M. Pardalos, editors, Network Optimization, volume 450,
pages 407-422, Boston, 1996. Kluwer Academic Publishers B.V.)
.Iaddend.significantly extends the formulation of the track
maintenance and initiation to new approaches. The discussions of
this section apply equally to those formulations.
.Iadd.VII.1. .Iaddend.The Linear Programming Relaxation.[...].
In the linear programming relaxation, one replaces the zero-one
variables constraints z.sub.i.sub.1 .sub.. . .
i.sub.N.epsilon.{0,1} for all i.sub.1, . . . ,i.sub.N .[.(6.2).].
.Iadd.(7.2) .Iaddend. with the constraint
0.ltoreq.z.sub.i.sub.1.sub.. . . i.sub.N.ltoreq.1 for all i.sub.1,
. . . i.sub.N .[.(6.2).]. .Iadd.(7.3) .Iaddend. Then, the problem
.[.(6.1).]. .Iadd.(7.1) .Iaddend.can be formulated as a linear
programming problem with the contraint .[.(6.2).]. .Iadd.(7.2)
.Iaddend.in .[.(6.1).]. .Iadd.(7.1) .Iaddend.replaced by
.[.(6.3).]. .Iadd.(7.3) .Iaddend.with a special block structure to
which the Dantzing-Wolfe decomposition is applicable. Of course,
after solving this problem, one must now recover the zero-one
character of the variables in .[.(6.1).]. .Iadd.(7.1) .Iaddend.and
there are many ways to do this, such as using the .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment problems.
Commercial software is also available.
.Iadd.VII.2. .Iaddend.The Linear Programming Dual and Partial
Lagrangian Relaxations.[...].
Given the linear programming relaxation, one can formulate the dual
problem or the partial Lagrangian relaxation duals with respect to
any number of constraints. In particular. this is precisely what is
done in Section 3 on the Lagrangian relaxation algorithm presented.
The much broader class of algorithms provided in the U.S. Pat.
No.II.2 5,537,119 (Aubrey B. Poore. Jr., Method and System for
Tracking Multiple Regional Objects by .Iadd.MultiDimensional
.Iaddend.Relaxation, U.S. Pat. No. 5,537,119, filed Mar. 14, 1995,
issue date of Jul. 16, 1996) can be modified to remove the zero-one
character when one relaxes M sets of constraints to an .[.N-M
dimensional.]. .Iadd.(N-M)-dimensional .Iaddend.problem and
recovers with an .[.N-M+1 dimensional.]. .Iadd.(N-M+1)-dimensional
.Iaddend.problem. This avoids the problems associated with the
NP-hard .[.N-M.]. .Iadd.(N-M)- .Iaddend.and .[.N-M+1 dimensional.].
.Iadd.(N-M+1).Iaddend.-dimensional problems. However, to restore
the zero-one character, one can do it sequentially with an
assignment problem or with one of the many zero-one rounding
techniques. These formulations are easy to work out and thus the
details are omitted.
The complete dual problem is another way of solving the LP problem
and may indeed be more efficient in certain applications. In
addition, the solution of this dual problem may provide excellent
initial approximations to the multipliers for Lagrangian
relaxations.
.Iadd.VIII. .Iaddend.Hot Starts for Track Maintenance and
Initiation: Bundle Modifications .[.(ABP).].
Thus reuse of multipliers and the first proof that this reuse is
actually valid was presented in this section. The reuse in track
maintenance is demonstrated and discussed in the work of Poore and
Drummond .[.[xx].]. .Iadd.(A. B. Poore and O. E. Drummond. Track
initiation and maintenance using multidimensional assignment
problems. In D. W. Hearn, W. W. Hager, and P. M. Pardalos, editors,
Network Optimization, volume 450, pages 407-422, Boston, 1996.
Kluwer Academic Publishers B.V.).Iaddend.. The only item left is
the modification of the bundle of subgradients for the use with the
multiplier values as one goes through the recovery problem as in
Section .[.3.6.]. .Iadd.IV.3.6 .Iaddend.or as one moves the window
in track maintenance. It is this aspect of the nonsmooth
optimization that adds an order of magnitude to the speed of the
algorithms. This work is based on both a mathematical proof as in
Section .[.3.6.]. .Iadd.IV.3.6 .Iaddend.as well as computational
experience and heuristics.
.[.Adaptive Window Size Selection (ABP).]. .Iadd.IX. ADAPTIVE AND
VARIABLE WINDOW SIZE SELECTION .Iaddend.
These data association algorithms, based on the multidimensional
assignment problem, range from sequential processing to many frames
of data processed all at once! The data association problem for
.[.multisensor multitarget.]. .Iadd.multisensor-multitarget
.Iaddend.tracking is formulated using a window sliding over the
frames of data from the sensors. This discussion focuses on the
work of Poore and Drummond and not on the formulations in Section 2
.Iadd.III.Iaddend.. Firm data association decisions for existing
track are made at, say frame k, with the most recent frame being
k+M. Decisions after frame k are soft decisions. Reports not in the
confirmed tracks are used to initiate tracks over frames numbered
k-I to k+M.
The length of these windows varies from sequential processing,
which corresponds precisely to I=0 and M=1, to user defined large
values of I and M. .[.(In the case of sequential processing. we
also have a temporary track file of dynamically feasible tracks,
but incorrect data association.).]. There is a marked change in
performance over this range. As the two windows become exceedingly
long, there is little statistically significant information gained
and indeed performance can degenerate slightly. Thus, one must
manually find the correct window lengths for performance in a given
scenario and the algorithms do not change for a changing
environment. Thus we propose an adaptive method for adjusting the
window lengths. (The method has been highly successful for
selecting the correct window length for multistep methods in
ordinary differential equations.)
1. Compute the solution for different window lengths that overlap
and differ by one or two frames.
2. Compare the solution quality, i.e., the value of the objective
function, for two adjacent window lengths.
3. If there is a predefined improvement in either direction, we
then, for stability, repeat the exercise for a shorter or longer on
the first try. If there is consistent information, we adjust the
window size in the indicated direction. This can be given a well
defined mathematical formula in terms of the assignment problems of
different dimensions.
Algorithm Enhancements due to Data Structures (ABP)
Construction and enumeration of the Best K-Solutions.
New Nonsmooth Optimization Techniques.
Sensitivity Analysis
.Iadd.X. SENSITIVITY ANALYSIS .Iaddend.
For sequential processing in which the .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment algorithm is used, one
can use the LP sensitivity analysis theory and easily obtain the
corresponding answers. For the multiframe processing, the optimal
solution is not available; however, there are still two approaches
one can use. First, the basic algorithm can perform the same
sensitivity analysis at each stage (loop) of the algorithm as is
done in the .[.two dimensional.]. .Iadd.two-dimensional
.Iaddend.assignment problem since the evaluation of function
.PHI..sub.N-k+1 is equivalent to a .[.two dimensional.].
.Iadd.two-dimensional .Iaddend.assignment: problem. Alternately,
one could use an LP relaxation of the assignment problem and base
the sensitivity on the resulting LP problem. We currently see this
as an important step in finding even better solutions to the
assignment problem if so desired.
.[.New Auction Algorithms (ABP & PS).]. .Iadd.XI. NEW AUCTION
ALGORITHMS.Iaddend..
In this section we present a new auction algorithm for the .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem.
An important step in solving N-dimensional assignment problems for
N.gtoreq.3.[.,.]. is finding the optimal solution of the
.[.2-dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
problem. In particular, we wish to solve the .[.2-dimensional.].
.Iadd.two-dimensional .Iaddend.problem which includes the
zero-index. .[.T3.pically.]. .Iadd.Typically .Iaddend.this problem
can be thought of as trying to find either a minimum cost or
maximum utility in assigning person to objects, tenants to
apartments, or teachers to classrooms. We will follow the work of
Bertsekas and call the first index set persons and the second index
set objects. The statement of this problem is given below .[.(1).].
.Iadd.(11.1) .Iaddend.when the number of persons is m and the
number of objects is n.
.times.
.times..times..times..times..times..times..times..times..times.-
.times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times..times..times.-
.times..times..times. ##EQU00138##
There are a couple of assumptions which we will make about
.[.(1).]. .Iadd.(11.1).Iaddend.. First of all, we assume that
c.sub.iO and c.sub.Oj are well-defined and the corresponding
variables .chi..sub.iO .chi..sub.Oj free to be assigned. Second if
a cost c.sub.ij happens to be undefined, then the corresponding
variable .chi..sub.ij .chi. to 0. In effect if c.sub.ij is
undefined, we simply remove this cost and variable from inclusion
in the problem.
Notice that there are no constraints on the number of times person
0 and object 0 can be assigned. But notice that the first
constraint requires that each non-zero person i must be assigned
exactly once. Similarly, the second constraint forces each non-zero
object j to be assigned exactly one time. Finally, the last
constraint gives an integer solution, although we will see shortly
that this constraint can be relaxed to admit any solution
.chi..sub.ij.gtoreq.0. One reason for requiring that all of the
costs of the form c.sub.iO and c.sub.Oj be defined is so that we
are guaranteed a feasible solution exists for the given
problem.
.Iadd.XI.1. .Iaddend.Relaxation of One Constraint
Relaxing the last constraint, we obtain the following problem:
.times.
.times..times..times..times..times..times..times..times..times.-
.times..function..times..times..times..ident..times..times..times..times..-
times..times..times..times..times..times..times..times..times.
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..di-elect cons. ##EQU00139##
This lower bound is achieved and the second constraint in the
original problem is satisfied if the following conditions are
met:
.times..times..times..chi..times..times..times..times..times..times..time-
s..ltoreq..times..times. ##EQU00140##
.times..times..noteq..chi..times..times..times..times..times..times..time-
s..times..times..times. ##EQU00140.2##
.times..times..times..times..times..times..times..times..times..times..ti-
mes..times..times..times..times..times..times..times..times.
##EQU00140.3##
The relaxation of the first constraint is analogous and would lead
to similar results with i and j exchanged and the introduction of
the multiplier u.sup.1.
.Iadd.XI.2. .Iaddend.Relaxation of Both Constraints
This time we will relax both constraints and .[.using.]. .Iadd.use
.Iaddend.duality theory .Iadd.to .Iaddend.obtain a relationship
between the multipliers u.sup.1 and u.sup.2.
Definition: An assignment S and a multiplier set (u.sup.1,u.sup.2)
are said to satisfy e-complementary slackness (.epsilon.-CS) if
c.sub.ij+u.sub.i.sup.1+u.sub.j.sup.2=0 for all (i,j).epsilon.S,
c.sub.ij+u.sub.i.sup.1+u.sub.j.sup.2.gtoreq.-.epsilon. for all
(i,j).epsilon.A.
Forward Auction Algorithm
(1) Select any unassigned person i
(2) Determine the following quantities: j.sub.i=arg min
{c.sub.ik+u.sub.k.sup.2|k.epsilon.A(i)}
v.sub.i=c.sub.lj.sub.i+u.sub.j.sub.i.sup.2 w.sub.i=min
{c.sub.ik+u.sub.k.sup.2|k.epsilon.A(i), k.noteq.j.sub.i}
In the selection of j.sub.1 .[...]. above, if a tie occurs between
0 and any non-zero index k, then select j, as k. Otherwise, if
there is a tie between two or more non-zero indices, the choice of
j, is arbitrary. Also if A(i) consists of only one element, then
set w.sub.i=.infin..
(3) Update the multipliers and the assignment:
If j.sub.l=0, then (a) Add (i,0) to the assignment. (b) Update
u.sub.i.sup.1:+-c.sub.iO. If j.sub.i.noteq.0, then (a) Add
(i,j.sub.l) to the assignment. (b) Remove (i',j.sub.2) from the
assignment if j.sub.i was previously assigned. (c) Update
u.sub.ji.sup.2:
=u.sub.ji.sup.2+(w.sub.l-v.sub.l)+.epsilon.=w.sub.l-c.sub.lj.sub.i+.epsil-
on.. (d) Update u,.sub.i.sup.1:
=-(c.sub.lj.sub.i+u.sub.j.sub.i.sup.2)=-w.sub.i-.epsilon..
Reverse Auction Algorithm (1) Select any unassigned object j, such
that c.sub.Oj+u.sub.j.sup.2>0. (2) Determine the following
quantities: i.sub.j=arg min {c.sub.jk+u.sub.k.sup.1|k.epsilon.B(j)}
.beta..sub.j=c.sub.ij.sub.j+u.sub.i.sub.j.sup.1 .gamma..sub.j=min
{c.sub.kj+u.sub.k.sup.1|k.epsilon.B(j), k.noteq.1.sub.j}
In the selection of i.sub.j, above, if a tie occurs between 0 and
any non-zero index k, then select j, as k. Otherwise, if there is a
tie between two or more non-zero indices, the choice of j.sub.i is
arbitrary. Also if B(j) consists of only one element, then set
.gamma..sub.j .infin..
(3) Update the multipliers and the assignment: If i.sub.j=0, then
(a) Add (0, j) to the assignment. (b) Update u.sub.j.sup.2:
=-c.sub.Oj. If i.sub.j.noteq.0, then (a) Add (i.sub.j,j) to the
assignment. (b) Remove (i.sub.j,j') from the assignment if
(i.sub.j,j) was previously assigned. (c) Update
u.sub.l.sub.j.sup.1:
=u.sub.l.sub.j.sup.1+(.gamma..sub.j-.beta..sub.j)+.epsilon.=.gamma..sub.j-
-c.sub.ij+.epsilon.. (d) Update u.sup.j.sup.2:
=-(c.sub.ij+u.sub.i.sub.j.sup.1)=-.gamma..sub.j-.epsilon..
Combined Forward/Reverse Auction Algorithm
1. Assume that u.sup.2 is given as an arbitrary multiplier.
2. Adjust the value of u.sup.2 for each object j as follows: If
c.sub.Oj+u.sub.j.sup.2<0, then set u.sub.j.sup.2:
=-c.sub.Oj.
3. Run iterations of the Forward Auction Algorithm until all
persons become assigned.
4. Run iterations of the Reverse Auction Algorithm until all of the
objects become assigned.
Note at the completion of the Forward auction step we have the
following conditions satisfied: c.sub.oj+u.sub.j.sup.2.gtoreq.0 for
all objects j. c.sub.ij+u.sub.i.sup.1+u.sub.j.sup.2=0 for all
(i,j).epsilon.S.
c.sub.ij+u.sub.j.sup.2.ltoreq..sub.k.sup.min{c.sub.ik+u.sub.k.sup.2}
for all (i,j).epsilon.S.
Thus we can prove the following proposition.
Proposition: If we assume that c.sub.oj+u.sub.j.sup.2.gtoreq.0 at
the start of the Forward Auction Algorithm and all of the persons
are assigned via a forward step, then we have:
c.sub.ij+u.sub.i.sup.1+u.sub.j.sup.2.gtoreq.-.epsilon. for all
(i,j).epsilon.A. c.sub.ij+u.sub.i.sup.1+u.sub.j.sup.2=0 for all
(i,j).epsilon.S.
c.sub.ij+u.sub.j.sup.2.ltoreq..sub.k.sup.min{c.sub.ik+u.sub.k.sup.2}+.eps-
ilon. for all (i,j).epsilon.S.
Optimality of the Algorithm
Theorem: .epsilon.-CS preserved during every forward and reverse
iteration.
Theorem: If a feasible solution exists, then the resulting solution
is with m.epsilon. of being optimal for the Combined Forward
Reverse Algorithm.
.[.Implementation Specifics.].
.[.Parallelization.].
.[.Here are but a few comments..].
.Iadd.XII. Some Concluding Comments .Iaddend.
Although the algorithm appears to be serial in nature, its primary
computational requirements are almost entirely parallelizable. Thus
parallelization is planned.
Step 2 is the computational part of the algorithm. Evaluating
.PHI..sub.N-k+1 .[..chi..chi..chi..]. .Iadd.and computing a
subgradient use in the search .Iaddend.procedure requires 99% of
the computing time in the algorithm. This part uses .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment
algorithms, a search over a large number of indices, and a
nonsmooth optimization algorithm. It is the second part (the
search) that consumes 99% of the computational time and this is
almost entirely parallelizable. Indeed, there are .[.two
dimensional.]. .Iadd.two-dimensional .Iaddend.assignment solvers
that are highly parallelizable. Thus, we need but parallelize the
nonsmooth optimization solver to have a reasonably complete
parallelization.
If a sensitivity analysis is desired or if one is interested in
computing several near-optimal solutions, a parallel processor with
a few powerful processors and good communication such as on the
Intel Paragon would be most beneficial.
The foregoing discussion of the invention has been presented for
purposes of illustration and description. Further, the description
is not intended to limit the invention to the form disclosed
herein. Consequently, variation and modification commensurate with
the above teachings, within the skill and knowledge of the relevant
art, are within the scope of the present invention. The embodiment
described hereinabove is further intended to explain the best mode
presently known of practicing the invention and to enable others
skilled in the art to utilize the invention as such, or in other
embodiments, and with the various modifications required by their
particular application or use of the invention. It is intended that
the appended claims be construed to include alternative embodiments
of the invention to the extent permitted by the prior art.
* * * * *