U.S. patent application number 16/669933 was filed with the patent office on 2020-10-01 for signature generation and object detection that refer to rare scenes.
The applicant listed for this patent is Cortica Ltd.. Invention is credited to Adam Harel, Igal Raichelgauz.
Application Number | 20200311484 16/669933 |
Document ID | / |
Family ID | 1000005087284 |
Filed Date | 2020-10-01 |
![](/patent/app/20200311484/US20200311484A1-20201001-D00000.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00001.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00002.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00003.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00004.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00005.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00006.png)
![](/patent/app/20200311484/US20200311484A1-20201001-D00007.png)
United States Patent
Application |
20200311484 |
Kind Code |
A1 |
Raichelgauz; Igal ; et
al. |
October 1, 2020 |
SIGNATURE GENERATION AND OBJECT DETECTION THAT REFER TO RARE
SCENES
Abstract
Systems, and method and computer readable media that store
instructions for calculating signatures, utilizing signatures and
the like.
Inventors: |
Raichelgauz; Igal; (Tel
Aviv, IL) ; Harel; Adam; (Tel Aviv, IL) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Cortica Ltd. |
Tel Aviv |
|
IL |
|
|
Family ID: |
1000005087284 |
Appl. No.: |
16/669933 |
Filed: |
October 31, 2019 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62827112 |
Mar 31, 2019 |
|
|
|
62827117 |
Mar 31, 2019 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06K 9/6219 20130101;
G06K 9/6256 20130101; G06K 9/6262 20130101 |
International
Class: |
G06K 9/62 20060101
G06K009/62 |
Claims
1. A method for updating an object detector, the method comprises:
receiving an image of a rare scene and a request to update the
object detector to detect a rare object that appears in the rare
scene; calculating, by a signature generator, a signature of the
image of the rare scene; wherein the signature of the rare scene
includes a certain number of object identifiers; calculating, by
the signature generator, reference signatures of a large number of
reference images; searching for reference signatures that match the
signature of the rare scene to provide matching reference
signatures; generating a cluster that comprises the signature of
the rare scene and the matching reference signatures; generating
one or more cluster identifiers, each cluster identifier comprises
a set of object identifiers that are shared by at least a
predefined number of signatures of the cluster; wherein a number of
object identifiers per set is smaller than the certain number; and
associating the one or more cluster identifiers with the object
detector.
2. The method according to claim 1 wherein the object identifiers
are retrieval information for retrieving the significant
portions.
3. The method according to claim 1 wherein each cluster identifier
comprises do not care object identifiers and object identifiers of
the set of object identifiers that are shared by at least the
predefined number of signatures of the cluster.
4. The method according to claim 1 comprising evaluating object
detection capabilities of the concept.
5. The method according to claim 1 comprising: receiving an
indication that a first image that has a first signature was
falsely detected, by the object detector, as including the rare
object; wherein the false detection occurred after the associating
of the one or more cluster identifiers with the object detector;
searching for first reference signatures that match the first
signature to provide first reference signatures; generating a first
false positive cluster that comprises the first signature and the
first reference signatures; generating one or more false positive
cluster identifiers, each false positive cluster identifier
comprises a set of object identifiers that are shared by at least a
predefined number of signatures of the false positive cluster;
removing from the cluster at least one part of at least one
signature based on the one or more false positive cluster
identifiers.
6. The method according to claim 5 wherein the removing comprises
removing at least a part of a signature of the compressed concept
that comprises at least one set of object identifiers that are
shared by the at least predefined number of signatures of the false
positive cluster.
7. The method according to claim 5 wherein the removing comprises
removing at least a part of a signature of the compressed concept
that comprises at least one subset out of a set of object
identifiers that are shared by the at least predefined number of
signatures of the false positive cluster.
8. The method according to claim 5 comprising validating object
detection capabilities of the cluster.
9. The method according to claim 1 comprising compressing the
cluster to provide a compressed cluster by replacing the signature
of the cluster by the one or more cluster identifiers; and wherein
the associating of the one or more cluster identifiers with the
object detector comprises associating the compressed cluster with
the object detector.
10. The method according to claim 9 comprising: receiving an
indication that a first image that has a first signature was
falsely detected, by the object detector, as including the rare
object; wherein the false detection occurred after the associating
of the one or more cluster identifiers with the object detector;
searching for first reference signatures that match the first
signature to provide first reference signatures; generating a first
false positive cluster that comprises the first signature and the
first reference signatures; generating one or more false positive
cluster identifiers, each false positive cluster identifier
comprises a set of object identifiers that are shared by at least a
predefined number of signatures of the false positive cluster; and
removing from the compressed cluster at least a subset of the one
or more sets of object identifiers that are shared by at least the
predefined number of signatures of the false positive cluster.
11. The method according to claim 1 comprising: receiving a
miss-detection indication that indicates that an image that
comprises the rate object and has a second signature was not
detected, by the object detector; wherein the miss-detection
occurred after the associating of the one or more cluster
identifiers with the object detector; searching for second
reference signatures that match the second signature to provide
second reference signatures; generating a second false negative
cluster that comprises the second signature and the second
reference signatures; generating one or more false negative cluster
identifiers, each false negative cluster identifier comprises a set
of object identifiers that are shared by at least a predefined
number of signatures of the false positive cluster; adding to the
cluster at least one false negative cluster identifier.
12. The method according to claim 11 comprising validating object
detection capabilities of the cluster.
13. The method according to claim 1 comprising: compressing the
cluster to provide a compressed cluster by replacing the signature
of the cluster by the one or more cluster identifiers; wherein the
associating of the one or more cluster identifiers with the object
detector comprises associating the compressed cluster with the
object detector; receiving a miss-detection indication that
indicates that an image that comprises the rate object and has a
second signature was not detected, by the object detector; wherein
the miss-detection occurred after the associating of the one or
more cluster identifiers with the object detector; searching for
second reference signatures that match the second signature to
provide second reference signatures; generating a second false
negative cluster that comprises the second signature and the second
reference signatures; generating one or more false negative cluster
identifiers, each false negative cluster identifier comprises a set
of object identifiers that are shared by at least a predefined
number of signatures of the false positive cluster; adding to the
compressed cluster at least one false negative cluster
identifier.
14. The method according to claim 13 comprising validating object
detection capabilities of the compressed cluster.
15. The method according to claim 1 further comprising receiving an
input image; calculating by a signature generator an input image
signature; searching for at least one matching concept structure;
and determining that the input image comprises an object identified
by a matching concept--when a matching concept is found.
16. A non-transitory computer readable medium for updating an
object detector, the non-transitory computer readable medium stores
instructions for: receiving an image of a rare scene and a request
to update the object detector to detect a rare object that appears
in the rare scene; calculating, by a signature generator, a
signature of the image of the rare scene; wherein the signature of
the rare scene includes a certain number of object identifiers;
calculating, by the signature generator, reference signatures of a
large number of reference images; searching for reference
signatures that match the signature of the rare scene to provide
matching reference signatures; generating a cluster that comprises
the signature of the rare scene and the matching reference
signatures; generating one or more cluster identifiers, each
cluster identifier comprises a set of object identifiers that are
shared by at least a predefined number of signatures of the
cluster; wherein a number of object identifiers per set is smaller
than the certain number; and associating the one or more cluster
identifiers with the object detector.
17. The non-transitory computer readable medium according to claim
16 wherein the object identifiers are retrieval information for
retrieving the significant portions.
18. The non-transitory computer readable medium according to claim
16 wherein each cluster identifier comprises do not care object
identifiers and object identifiers of the set of object identifiers
that are shared by at least the predefined number of signatures of
the cluster.
19. The non-transitory computer readable medium according to claim
16 that stores instructions for evaluating object detection
capabilities of the concept.
20. The non-transitory computer readable medium according to claim
16 that stores instructions for: receiving an indication that a
first image that has a first signature was falsely detected, by the
object detector, as including the rare object; wherein the false
detection occurred after the associating of the one or more cluster
identifiers with the object detector; searching for first reference
signatures that match the first signature to provide first
reference signatures; generating a first false positive cluster
that comprises the first signature and the first reference
signatures; generating one or more false positive cluster
identifiers, each false positive cluster identifier comprises a set
of object identifiers that are shared by at least a predefined
number of signatures of the false positive cluster; removing from
the cluster at least one part of at least one signature based on
the one or more false positive cluster identifiers.
21. The non-transitory computer readable medium according to claim
20 wherein the removing comprises removing at least a part of a
signature of the compressed concept that comprises at least one set
of object identifiers that are shared by the at least predefined
number of signatures of the false positive cluster.
22. The non-transitory computer readable medium according to claim
20 wherein the removing comprises removing at least a part of a
signature of the compressed concept that comprises at least one
subset out of a set of object identifiers that are shared by the at
least predefined number of signatures of the false positive
cluster.
23. The non-transitory computer readable medium according to claim
20 that stores instructions for validating object detection
capabilities of the cluster.
24. The non-transitory computer readable medium according to claim
16 that stores instructions for compressing the cluster to provide
a compressed cluster by replacing the signature of the cluster by
the one or more cluster identifiers; and wherein the associating of
the one or more cluster identifiers with the object detector
comprises associating the compressed cluster with the object
detector.
25. The non-transitory computer readable medium according to claim
24 that stores instructions for: receiving an indication that a
first image that has a first signature was falsely detected, by the
object detector, as including the rare object; wherein the false
detection occurred after the associating of the one or more cluster
identifiers with the object detector; searching for first reference
signatures that match the first signature to provide first
reference signatures; generating a first false positive cluster
that comprises the first signature and the first reference
signatures; generating one or more false positive cluster
identifiers, each false positive cluster identifier comprises a set
of object identifiers that are shared by at least a predefined
number of signatures of the false positive cluster; and removing
from the compressed cluster at least a subset of the one or more
sets of object identifiers that are shared by at least the
predefined number of signatures of the false positive cluster.
26. The non-transitory computer readable medium according to claim
16 that stores instructions for: receiving a miss-detection
indication that indicates that an image that comprises the rate
object and has a second signature was not detected, by the object
detector; wherein the miss-detection occurred after the associating
of the one or more cluster identifiers with the object detector;
searching for second reference signatures that match the second
signature to provide second reference signatures; generating a
second false negative cluster that comprises the second signature
and the second reference signatures; generating one or more false
negative cluster identifiers, each false negative cluster
identifier comprises a set of object identifiers that are shared by
at least a predefined number of signatures of the false positive
cluster; adding to the cluster at least one false negative cluster
identifier.
27. The non-transitory computer readable medium according to claim
26 that stores instructions for validating object detection
capabilities of the cluster.
28. The non-transitory computer readable medium according to claim
16 that stores instructions for: compressing the cluster to provide
a compressed cluster by replacing the signature of the cluster by
the one or more cluster identifiers; wherein the associating of the
one or more cluster identifiers with the object detector comprises
associating the compressed cluster with the object detector;
receiving a miss-detection indication that indicates that an image
that comprises the rate object and has a second signature was not
detected, by the object detector; wherein the miss-detection
occurred after the associating of the one or more cluster
identifiers with the object detector; searching for second
reference signatures that match the second signature to provide
second reference signatures; generating a second false negative
cluster that comprises the second signature and the second
reference signatures; generating one or more false negative cluster
identifiers, each false negative cluster identifier comprises a set
of object identifiers that are shared by at least a predefined
number of signatures of the false positive cluster; adding to the
compressed cluster at least one false negative cluster
identifier.
29. A system that comprises a communication unit and a processor,
wherein the communication unit is configured to receive an image of
a rare scene and a request to update the object detector to detect
a rare object that appears in the rare scene; calculate, by a
signature generator of the processor, a signature of the image of
the rare scene; wherein the signature of the rare scene includes a
certain number of object identifiers; calculate, by the signature
generator, reference signatures of a large number of reference
images; search for reference signatures that match the signature of
the rare scene to provide matching reference signatures; generate a
cluster that comprises the signature of the rare scene and the
matching reference signatures; generate one or more cluster
identifiers, each cluster identifier comprises a set of object
identifiers that are shared by at least a predefined number of
signatures of the cluster; wherein a number of object identifiers
per set is smaller than the certain number; and associate the one
or more cluster identifiers with the object detector.
30. The system according to claim 31 comprising a sensing unit that
is configured to obtain an input image; wherein the signature
generator is configured to calculate an input image signature;
wherein the processor is configured to search for at least one
matching concept structure; and determine that the input image
comprises an object identified by a matching concept--when a
matching concept is found.
Description
CROSS REFERENCE
[0001] This application claims priority from U.S. provisional No.
62/827,117 filing date Mar. 31, 2019 and from U.S. provisional
patent 62/827,122 filing date Mar. 31, 2019, both are incorporated
herein by reference.
BACKGROUND
[0002] Object detection has extensive usage in variety of
applications, starting from security, sport events, automatic
vehicles, and the like.
[0003] Vast amounts of media units are processed during object
detection and their processing may require vast amounts of
computational resources and memory resources.
[0004] Furthermore-many object detection process are sensitive to
various acquisition parameters such as angle of acquisition, scale,
and the like.
[0005] There is a growing need to provide robust and efficient
object detection methods.
SUMMARY
[0006] There may be provided systems, methods and computer readable
medium as illustrated in the specification.
[0007] Any combination of any subject matter disclosed in any one
of U.S. provisional No. 62/827,117 and U.S. provisional patent
62/827,122 may be provided.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The embodiments of the disclosure will be understood and
appreciated more fully from the following detailed description,
taken in conjunction with the drawings in which:
[0009] FIG. 1 illustrates an example of a method;
[0010] FIG. 2 illustrates an example of a signature;
[0011] FIG. 3 illustrates an example of a dimension expansion
process;
[0012] FIG. 4 illustrates an example of a matching process;
[0013] FIG. 5 illustrates an example of a method;
[0014] FIG. 6 illustrates an example of a method; and
[0015] FIG. 7 illustrates an example of a system.
DESCRIPTION OF EXAMPLE EMBODIMENTS
[0016] In the following detailed description, numerous specific
details are set forth in order to provide a thorough understanding
of the invention. However, it will be understood by those skilled
in the art that the present invention may be practiced without
these specific details. In other instances, well-known methods,
procedures, and components have not been described in detail so as
not to obscure the present invention.
[0017] The subject matter regarded as the invention is particularly
pointed out and distinctly claimed in the concluding portion of the
specification. The invention, however, both as to organization and
method of operation, together with objects, features, and
advantages thereof, may best be understood by reference to the
following detailed description when read with the accompanying
drawings.
[0018] It will be appreciated that for simplicity and clarity of
illustration, elements shown in the figures have not necessarily
been drawn to scale. For example, the dimensions of some of the
elements may be exaggerated relative to other elements for clarity.
Further, where considered appropriate, reference numerals may be
repeated among the figures to indicate corresponding or analogous
elements.
[0019] Because the illustrated embodiments of the present invention
may for the most part, be implemented using electronic components
and circuits known to those skilled in the art, details will not be
explained in any greater extent than that considered necessary as
illustrated above, for the understanding and appreciation of the
underlying concepts of the present invention and in order not to
obfuscate or distract from the teachings of the present
invention.
[0020] Any reference in the specification to a method should be
applied mutatis mutandis to a device or system capable of executing
the method and/or to a non-transitory computer readable medium that
stores instructions for executing the method.
[0021] Any reference in the specification to a system or device
should be applied mutatis mutandis to a method that may be executed
by the system, and/or may be applied mutatis mutandis to
non-transitory computer readable medium that stores instructions
executable by the system.
[0022] Any reference in the specification to a non-transitory
computer readable medium should be applied mutatis mutandis to a
device or system capable of executing instructions stored in the
non-transitory computer readable medium and/or may be applied
mutatis mutandis to a method for executing the instructions.
[0023] Any combination of any module or unit listed in any of the
figures, any part of the specification and/or any claims may be
provided.
[0024] The specification and/or drawings may refer to an image. An
image is an example of a media unit. Any reference to an image may
be applied mutatis mutandis to a media unit. A media unit may be an
example of sensed information. Any reference to a media unit may be
applied mutatis mutandis to sensed information. The sensed
information may be sensed by any type of sensors-such as a visual
light camera, or a sensor that may sense infrared, radar imagery,
ultrasound, electro-optics, radiography, LIDAR (light detection and
ranging), etc.
[0025] The specification and/or drawings may refer to a processor.
The processor may be a processing circuitry. The processing
circuitry may be implemented as a central processing unit (CPU),
and/or one or more other integrated circuits such as
application-specific integrated circuits (ASICs), field
programmable gate arrays (FPGAs), full-custom integrated circuits,
etc., or a combination of such integrated circuits.
[0026] Any combination of any steps of any method illustrated in
the specification and/or drawings may be provided.
[0027] Any combination of any subject matter of any of claims may
be provided.
[0028] Any combinations of systems, units, components, processors,
sensors, illustrated in the specification and/or drawings may be
provided.
[0029] The analysis of content of a media unit may be executed by
generating a signature of the media unit and by comparing the
signature to reference signatures. The reference signatures may be
arranged in one or more concept structures or may be arranged in
any other manner. The signatures may be used for object detection
or for any other use.
[0030] The signature may be generated by creating a
multidimensional representation of the media unit. The
multidimensional representation of the media unit may have a very
large number of dimensions. The high number of dimensions may
guarantee that the multidimensional representation of different
media units that include different objects is sparse--and that
object identifiers of different objects are distant from each
other--thus improving the robustness of the signatures.
[0031] The term "rare scene" may mean a scene that is seldom
occurring or seldom found or a scene that includes any secne
portion that is seldom occurring or seldom found. A scene portion
may be an object captured in the scene, a combination of one of
more object, an event or a part of an event (for example a dog that
jumps in front of a vehicle) captured by the scene.
[0032] Any reference in the application to a scene it is
applicable, mutatis mutandis, to a scene portion. A rare scene may
be a scene that includes a rare scene portion.
[0033] The occurrence of the rare scene may me much lower (for
example below a certain threshold such as 1, 2, 3, 4, 5, 6, 7, 8,
9, 10%) of an occurrence of other non-rare scenes in one or more
collections of scenes. The scene may be rare if a machine learning
process is unable to identify the rare scene. The scene may be rare
if a detection of the rare scene by a machine learning process, is
substantially less reliable (for example by a factor of at least
ten, twenty, fifty, one hundred and more) than a detection of a
common scene. A scene may be regarded as a rare scene by a human or
by an automated process. Any method for classifying a scene as a
rare scene may be used. For example--a rare scene may be a scene
that is expected to surprise a human driver.
[0034] The rarity of the scene may be determined based on the
cluster structures and/or may be based on a complexity of the
scene. For example--a complex scene that should be represented by
cluster having a first number (N1) of signatures may be regarded
rare if the cluster has only a fraction of the first number of
signatures and the lack of sufficient signatures will result in a
substantial reduction in the ability to detect the complex
scene.
[0035] The term "scene" may include a situation, a place of
occurrence or action, a single object, a combination of objects, a
portion of a scene, and the like.
[0036] There may be provided a method for updating an object
detector, the method may include receiving an image of a rare scene
and a request to update the object detector to detect a rare object
that appears in the rare scene; calculating, by a signature
generator, a signature of the image of the rare scene; wherein the
signature of the rare scene includes a certain number of object
identifiers; calculating, by the signature generator, reference
signatures of a large number of reference images; searching for
reference signatures that match the signature of the rare scene to
provide matching reference signatures; generating a cluster that
may include the signature of the rare scene and the matching
reference signatures; generating one or more cluster identifiers,
each cluster identifier may include a set of object identifiers
that may be shared by at least a predefined number of signatures of
the cluster; wherein a number of object identifiers per set may be
smaller than the certain number; and associating the one or more
cluster identifiers with the object detector.
[0037] The object identifiers may be retrieval information for
retrieving the significant portions.
[0038] Each cluster identifier may include do not care (don't care)
object identifiers and object identifiers of the set of object
identifiers that may be shared by at least the predefined number of
signatures of the cluster.
[0039] The method may include evaluating object detection
capabilities of the concept.
[0040] The method may include receiving an indication that a first
image that has a first signature was falsely detected, by the
object detector, as including the rare object; wherein the false
detection occurred after the associating of the one or more cluster
identifiers with the object detector; searching for first reference
signatures that match the first signature to provide first
reference signatures; generating a first false positive cluster
that may include the first signature and the first reference
signatures; generating one or more false positive cluster
identifiers, each false positive cluster identifier may include a
set of object identifiers that may be shared by at least a
predefined number of signatures of the false positive cluster;
removing from the cluster at least one part of at least one
signature based on the one or more false positive cluster
identifiers.
[0041] The removing may include removing at least a part of a
signature of the compressed concept that may include at least one
set of object identifiers that may be shared by the at least
predefined number of signatures of the false positive cluster.
[0042] The removing may include removing at least a part of a
signature of the compressed concept that may include at least one
subset out of a set of object identifiers that may be shared by the
at least predefined number of signatures of the false positive
cluster.
[0043] The method may include validating object detection
capabilities of the cluster.
[0044] The method may include compressing the cluster to provide a
compressed cluster by replacing the signature of the cluster by the
one or more cluster identifiers; and wherein the associating of the
one or more cluster identifiers with the object detector may
include associating the compressed cluster with the object
detector.
[0045] The method may include receiving an indication that a first
image that has a first signature was falsely detected, by the
object detector, as including the rare object; wherein the false
detection occurred after the associating of the one or more cluster
identifiers with the object detector; searching for first reference
signatures that match the first signature to provide first
reference signatures; generating a first false positive cluster
that may include the first signature and the first reference
signatures; generating one or more false positive cluster
identifiers, each false positive cluster identifier may include a
set of object identifiers that may be shared by at least a
predefined number of signatures of the false positive cluster; and
removing from the compressed cluster at least a subset of the one
or more sets of object identifiers that may be shared by at least
the predefined number of signatures of the false positive
cluster.
[0046] The method may include receiving a miss-detection indication
that indicates that an image that may include the rate object and
has a second signature was not detected, by the object detector;
wherein the miss-detection occurred after the associating of the
one or more cluster identifiers with the object detector; searching
for second reference signatures that match the second signature to
provide second reference signatures; generating a second false
negative cluster that may include the second signature and the
second reference signatures; generating one or more false negative
cluster identifiers, each false negative cluster identifier may
include a set of object identifiers that may be shared by at least
a predefined number of signatures of the false positive cluster;
adding to the cluster at least one false negative cluster
identifier.
[0047] The method may include validating object detection
capabilities of the cluster.
[0048] The method may include compressing the cluster to provide a
compressed cluster by replacing the signature of the cluster by the
one or more cluster identifiers; wherein the associating of the one
or more cluster identifiers with the object detector may include
associating the compressed cluster with the object detector;
receiving a miss-detection indication that indicates that an image
that may include the rate object and has a second signature was not
detected, by the object detector; wherein the miss-detection
occurred after the associating of the one or more cluster
identifiers with the object detector; searching for second
reference signatures that match the second signature to provide
second reference signatures; generating a second false negative
cluster that may include the second signature and the second
reference signatures; generating one or more false negative cluster
identifiers, each false negative cluster identifier may include a
set of object identifiers that may be shared by at least a
predefined number of signatures of the false positive cluster;
adding to the compressed cluster at least one false negative
cluster identifier.
[0049] The method may include validating object detection
capabilities of the compressed cluster.
[0050] There may be provided a method for object detection, the
method may include receiving an input image; calculating by a
signature generator an input image signature; searching for at
least one matching concept structure; and determining that the
input image may include an object identified by a matching
concept--when a matching concept may be found; wherein the
receiving of the input image may be preceded by any of the steps
mentioned above.
[0051] There may be provided non-transitory computer readable
medium for updating an object detector, the non-transitory computer
readable medium stores instructions for receiving an image of a
rare scene and a request to update the object detector to detect a
rare object that appears in the rare scene; calculating, by a
signature generator, a signature of the image of the rare scene;
wherein the signature of the rare scene includes a certain number
of object identifiers; calculating, by the signature generator,
reference signatures of a large number of reference images;
searching for reference signatures that match the signature of the
rare scene to provide matching reference signatures; generating a
cluster that may include the signature of the rare scene and the
matching reference signatures; generating one or more cluster
identifiers, each cluster identifier may include a set of object
identifiers that may be shared by at least a predefined number of
signatures of the cluster; wherein a number of object identifiers
per set may be smaller than the certain number; and associating the
one or more cluster identifiers with the object detector.
[0052] The object identifiers may be retrieval information for
retrieving the significant portions.
[0053] Each cluster identifier may include do not care object
identifiers and object identifiers of the set of object identifiers
that may be shared by at least the predefined number of signatures
of the cluster.
[0054] The non-transitory computer readable medium that may store
instructions for evaluating object detection capabilities of the
concept.
[0055] The non-transitory computer readable medium that may store
instructions for receiving an indication that a first image that
has a first signature was falsely detected, by the object detector,
as including the rare object; wherein the false detection occurred
after the associating of the one or more cluster identifiers with
the object detector; searching for first reference signatures that
match the first signature to provide first reference signatures;
generating a first false positive cluster that may include the
first signature and the first reference signatures; generating one
or more false positive cluster identifiers, each false positive
cluster identifier may include a set of object identifiers that may
be shared by at least a predefined number of signatures of the
false positive cluster; removing from the cluster at least one part
of at least one signature based on the one or more false positive
cluster identifiers.
[0056] The removing may include removing at least a part of a
signature of the compressed concept that may include at least one
set of object identifiers that may be shared by the at least
predefined number of signatures of the false positive cluster.
[0057] The removing may include removing at least a part of a
signature of the compressed concept that may include at least one
subset out of a set of object identifiers that may be shared by the
at least predefined number of signatures of the false positive
cluster.
[0058] The non-transitory computer readable medium that may store
instructions for validating object detection capabilities of the
cluster.
[0059] The non-transitory computer readable medium that may store
instructions for compressing the cluster to provide a compressed
cluster by replacing the signature of the cluster by the one or
more cluster identifiers; and wherein the associating of the one or
more cluster identifiers with the object detector may include
associating the compressed cluster with the object detector.
[0060] The non-transitory computer readable medium that may store
instructions for receiving an indication that a first image that
has a first signature was falsely detected, by the object detector,
as including the rare object; wherein the false detection occurred
after the associating of the one or more cluster identifiers with
the object detector; searching for first reference signatures that
match the first signature to provide first reference signatures;
generating a first false positive cluster that may include the
first signature and the first reference signatures; generating one
or more false positive cluster identifiers, each false positive
cluster identifier may include a set of object identifiers that may
be shared by at least a predefined number of signatures of the
false positive cluster; and removing from the compressed cluster at
least a subset of the one or more sets of object identifiers that
may be shared by at least the predefined number of signatures of
the false positive cluster.
[0061] The non-transitory computer readable medium that may store
instructions for receiving a miss-detection indication that
indicates that an image that may include the rate object and has a
second signature was not detected, by the object detector; wherein
the miss-detection occurred after the associating of the one or
more cluster identifiers with the object detector; searching for
second reference signatures that match the second signature to
provide second reference signatures; generating a second false
negative cluster that may include the second signature and the
second reference signatures; generating one or more false negative
cluster identifiers, each false negative cluster identifier may
include a set of object identifiers that may be shared by at least
a predefined number of signatures of the false positive cluster;
adding to the cluster at least one false negative cluster
identifier.
[0062] The non-transitory computer readable medium that may store
instructions for validating object detection capabilities of the
cluster.
[0063] The non-transitory computer readable medium that may store
instructions for compressing the cluster to provide a compressed
cluster by replacing the signature of the cluster by the one or
more cluster identifiers; wherein the associating of the one or
more cluster identifiers with the object detector may include
associating the compressed cluster with the object detector;
receiving a miss-detection indication that indicates that an image
that may include the rate object and has a second signature was not
detected, by the object detector; wherein the miss-detection
occurred after the associating of the one or more cluster
identifiers with the object detector; searching for second
reference signatures that match the second signature to provide
second reference signatures; generating a second false negative
cluster that may include the second signature and the second
reference signatures; generating one or more false negative cluster
identifiers, each false negative cluster identifier may include a
set of object identifiers that may be shared by at least a
predefined number of signatures of the false positive cluster;
adding to the compressed cluster at least one false negative
cluster identifier.
[0064] The non-transitory computer readable medium that may store
instructions for validating object detection capabilities of the
compressed cluster.
[0065] There may be provided non-transitory computer readable
medium for object detection, the non-transitory computer readable
medium stores instructions for receiving an input image;
calculating by a signature generator an input image signature;
searching for at least one matching concept structure; and
determining that the input image may include an object identified
by a matching concept--when a matching concept may be found;
wherein the receiving of the input image may be preceded by
executing the instructions listed above.
[0066] There may be provided a system that may be configured to
execute any of the steps mentioned above.
[0067] The analysis of content of a media unit may be executed by
generating a signature of the media unit and by comparing the
signature to reference signatures. The reference signatures may be
arranged in one or more concept structures or may be arranged in
any other manner. The signatures may be used for object detection
or for any other use.
[0068] The signature may be generated by creating a
multidimensional representation of the media unit. The
multidimensional representation of the media unit may have a very
large number of dimensions. The high number of dimensions may
guarantee that the multidimensional representation of different
media units that include different objects is sparse--and that
object identifiers of different objects are distant from each
other--thus improving the robustness of the signatures.
[0069] The generation of the signature is executed in an iterative
manner that includes multiple iterations, each iteration may
include an expansion operations that is followed by a merge
operation. The expansion operation of an iteration is performed by
spanning elements of that iteration.
[0070] FIG. 1 illustrates a method 5000 for generating a signature
of a media unit.
[0071] Method 5000 may start by step S010 of receiving or
generating sensed information.
[0072] The sensed information may be a media unit of multiple
objects.
[0073] Step S010 may be followed by processing the media unit by
performing multiple iterations, wherein at least some of the
multiple iterations comprises applying, by spanning elements of the
iteration, dimension expansion process that are followed by a merge
operation.
[0074] The processing may include: [0075] Step S020 of performing a
k'th iteration expansion process (k may be a variable that is used
to track the number of iterations). [0076] Step S030 of performing
a k'th iteration merge process. [0077] Step S040 of changing the
value of k. [0078] Step S050 of checking if all required iterations
were done--if so proceeding to step S060 of completing the
generation of the signature. Else--jumping to step S020.
[0079] The output of step S020 is a k'th iteration expansion
results 5120.
[0080] The output of step S030 is a k'th iteration merge results
5130.
[0081] For each iteration (except the first iteration)--the merge
result of the previous iteration is an input to the current
iteration expansion process.
[0082] The method may include step S011 of configuring the spanning
elements. Step S011 may include, for example, the steps of method
9400 of FIG. 5. Alternatively--step S020 may be executed by
spanning elements that are configured according to method 9400.
[0083] Non-limiting examples of various steps of FIG. 1 are
illustrated in U.S. provisional patent 62/827,122.
[0084] FIG. 2 is an example of a signature 6027 of a media unit
that is an image 6000 and of an outcome 6013 of the last (K'th)
iteration.
[0085] The image 6001 is virtually segments to segments 6000(i,k).
The segments may be of the same shape and size but this is not
necessarily so.
[0086] Outcome 6013 may be a tensor that includes a vector of
values per each segment of the media unit. One or more objects may
appear in a certain segment. For each object--an object identifier
(of the signature) points to locations of significant values,
within a certain vector associated with the certain segment.
[0087] For example--a top left segment (6001(1,1)) of the image may
be represented in the outcome 6013 by a vector V(1,1) 6017(1,1)
that has multiple values. The number of values per vector may
exceed 100, 200, 500, 1000, and the like.
[0088] The significant values (for example--more than 10, 20, 30,
40 values, and/or more than 0.1%, 0.2%. 0.5%, 1%, 5% of all values
of the vector and the like) may be selected. The significant values
may have the values--but may eb selected in any other manner.
[0089] FIG. 2 illustrates a set of significant responses 6015(1,1)
of vector V(1,1) 6017(1,1). The set includes five significant
values (such as first significant value SV1(1,1) 6013(1,1,1),
second significant value SV2(1,1), third significant value
SV3(1,1), fourth significant value SV4(1,1), and fifth significant
value SV5(1,1) 6013(1,1,5).
[0090] The image signature 6027 includes five indexes for the
retrieval of the five significant values--first till fifth
identifiers ID1-ID5 are indexes for retrieving the first till fifth
significant values.
[0091] FIG. 4 illustrates an example of a clusters of a signatures
matching process.
[0092] It is assumed that there are multiple (M) cluster structures
4974(1)-4974(M). Each cluster structure includes cluster
signatures, metadata regarding the cluster signatures.
[0093] For example--first cluster structure 4974(1) includes
multiple (N1) signatures (referred to as cluster signatures CS)
CS(1,1)-CS(1,N1) 4975(1,1)-4975(1,N1) and metadata 4976(1).
[0094] Yet for another example--M'th cluster structure 4974(M)
includes multiple (N2) signatures (referred to as cluster
signatures CS) CS(M,1)-CS(M,N2) 4975(M,1)-4975(M,N2) and metadata
4976(M).
[0095] FIG. 4 also illustrates a media unit signature 4972 that is
compared to the signatures of the M cluster structures--from
CS(1,1) 4975(1,1) till CS(M,N2)-4975(M,N2).
[0096] FIG. 4 also illustrates an example of a first compressed
cluster structure in which the signatures (CS) were replaced by
cluster identifiers CI(1,1)-CI(1,M1)-4979''(1)-4979''(M1).
[0097] FIG. 5 illustrates method 9800 for updating an object
detector.
[0098] Method 9800 may include the following steps: [0099]
Receiving an image of a rare scene and a request to update the
object detector to detect a rare object (or other rare scene
portion) that appears in the rare scene 9802. The scene may be rare
if it was not detected in the past, if it includes a scene portion
that is rare, if a user indicates that it is rare, if a user
indicates that a scene portion is rare, whether its chances of
appearance are below a threshold, and the like. [0100] Calculating,
by a signature generator, a signature of the image of the rare
scene; wherein the signature of the rare scene includes a certain
number of object identifiers 9804. For example--there may be few
tens of object identifiers. [0101] Calculating, by the signature
generator, reference signatures of a large number of reference
images 9806. The large number may exceed 100, 1000, 10000, 100000,
1000000 and the like. The large number of reference images may
relate to a certain interest area such as driving scenes, medical
information, and the like. [0102] Searching for reference
signatures that match the signature of the rare scene to provide
matching reference signatures 9808. [0103] Generating a cluster
that may include the signature of the rare scene and the matching
reference signatures 9810. [0104] Generating one or more cluster
identifiers, each cluster identifier may include a set of object
identifiers that may be shared by at least a predefined number of
signatures of the cluster 9812. A number of object identifiers per
set may be smaller (for example less than 10, 20, 30, 40, 50, 60,
70%) than the certain number. [0105] Associating the one or more
cluster identifiers with the object detector. 9814. The association
allows the object detector to perform future object detection
processes while benefitting from the one or more cluster
identifiers.
[0106] The object identifiers may be retrieval information for
retrieving the significant portions.
[0107] Each cluster identifier may include do not care (don't care)
object identifiers and object identifiers of the set of object
identifiers that may be shared by at least the predefined number of
signatures of the cluster. Don't care objects identifiers may not
be taken into account when performing match operations with
incoming signatures.
[0108] The method may include evaluating (step 9816) object
detection capabilities of the concept. The evaluation is aimed to
see whether the concept mat detect the rare scene and maybe other
scenes that were detected before the amendment. This may involve a
supervised process during which the performance of the object
detector are measured (the performance may be measured, for
example, by false positives, false negatives, true positives, true
negative and the like). If the amendments deteriorates (at least
above a threshold) the capabilities of the object detector than the
amendment may be reconsiders, another amendment may be made, and
the like.
[0109] Method 9800 may include a false positive correction process
9818.
[0110] Examples of this process are provided below: [0111]
Receiving an indication that a first image that has a first
signature was falsely detected, by the object detector, as
including the rare object; wherein the false detection occurred
after the associating of the one or more cluster identifiers with
the object detector. [0112] Searching for first reference
signatures that match the first signature to provide first
reference signatures. [0113] Generating a first false positive
cluster that may include the first signature and the first
reference signatures. [0114] Generating one or more false positive
cluster identifiers, each false positive cluster identifier may
include a set of object identifiers that may be shared by at least
a predefined number (at least 5, 10, 15, 20, 25, 30, 35, 40, 45,
50, 55, 60, 65, 70, 75, 80, 85 and 90%) of signatures of the false
positive cluster. [0115] Removing from the cluster at least one
part (at least one object identifier) of at least one signature
based on the one or more false positive cluster identifiers. [0116]
The removing may include removing at least a part of a signature of
the compressed concept that may include at least one set of object
identifiers that may be shared by the at least predefined number of
signatures of the false positive cluster. [0117] The removing may
include removing at least a part of a signature of the compressed
concept that may include at least one subset out of a set of object
identifiers that may be shared by the at least predefined number of
signatures of the false positive cluster. [0118] Validating object
detection capabilities of the cluster. [0119] Compressing the
cluster to provide a compressed cluster by replacing the signature
of the cluster by the one or more cluster identifiers; and wherein
the associating of the one or more cluster identifiers with the
object detector may include associating the compressed cluster with
the object detector. [0120] Receiving an indication that a first
image that has a first signature was falsely detected, by the
object detector, as including the rare object; wherein the false
detection occurred after the associating of the one or more cluster
identifiers with the object detector. [0121] Searching for first
reference signatures that match the first signature to provide
first reference signatures. [0122] Generating a first false
positive cluster that may include the first signature and the first
reference signatures. [0123] Generating one or more false positive
cluster identifiers, each false positive cluster identifier may
include a set of object identifiers that may be shared by at least
a predefined number of signatures of the false positive cluster.
[0124] Removing from the compressed cluster at least a subset of
the one or more sets of object identifiers that may be shared by at
least the predefined number of signatures of the false positive
cluster.
[0125] Method 9800 may include a false positive correction process
9820.
[0126] Examples of this process are provided below: [0127]
Receiving a miss-detection indication that indicates that an image
that may include the rate object and has a second signature was not
detected, by the object detector; wherein the miss-detection
occurred after the associating of the one or more cluster
identifiers with the object detector. [0128] Searching for second
reference signatures that match the second signature to provide
second reference signatures [0129] Generating a second false
negative cluster that may include the second signature and the
second reference signatures. [0130] Generating one or more false
negative cluster identifiers, each false negative cluster
identifier may include a set of object identifiers that may be
shared by at least a predefined number of signatures of the false
positive cluster. [0131] Adding to the cluster at least one false
negative cluster identifier. [0132] Validating object detection
capabilities of the cluster. [0133] Compressing the cluster to
provide a compressed cluster by replacing the signature of the
cluster by the one or more cluster identifiers; wherein the
associating of the one or more cluster identifiers with the object
detector may include associating the compressed cluster with the
object detector. [0134] Receiving a miss-detection indication that
indicates that an image that may include the rate object and has a
second signature was not detected, by the object detector; wherein
the miss-detection occurred after the associating of the one or
more cluster identifiers with the object detector. [0135] Searching
for second reference signatures that match the second signature to
provide second reference signatures. [0136] Generating a second
false negative cluster that may include the second signature and
the second reference signatures. [0137] Generating one or more
false negative cluster identifiers, each false negative cluster
identifier may include a set of object identifiers that may be
shared by at least a predefined number of signatures of the false
positive cluster. [0138] Adding to the compressed cluster at least
one false negative cluster identifier. [0139] Validating object
detection capabilities of the compressed cluster.
[0140] FIG. 6 illustrates method 9830.
[0141] Method 9830 may include the steps of: [0142] Receiving an
input image 9832. [0143] Calculating by a signature generator an
input image signature 9834. [0144] Searching for at least one
matching concept structure 9836. [0145] Determining that the input
image may include an object identified by a matching concept--when
a matching concept may be found 9838.
[0146] Method 9830 may include or may preceded by method 9800.
[0147] FIG. 7 illustrates an example of a system capable of
executing one or more of the mentioned above methods.
[0148] The system include various components, elements and/or
units.
[0149] A component element and/or unit may be a processing
circuitry may be implemented as a central processing unit (CPU),
and/or one or more other integrated circuits such as
application-specific integrated circuits (ASICs), field
programmable gate arrays (FPGAs), full-custom integrated circuits,
etc., or a combination of such integrated circuits.
[0150] Alternatively, each component element and/or unit may
implemented in hardware, firmware, or software that may be executed
by a processing circuitry.
[0151] System 4900 may include sensing unit 4902, communication
unit 4904, input 4911, processor 4950, and output 4919. The
communication unit 4904 may include the input and/or the
output.
[0152] Input and/or output may be any suitable communications
component such as a network interface card, universal serial bus
(USB) port, disk reader, modem or transceiver that may be operative
to use protocols such as are known in the art to communicate either
directly, or indirectly, with other elements of the system.
[0153] Processor 4950 may include at least some out of [0154]
Multiple spanning elements 4951(q). [0155] Multiple merge elements
4952(r). [0156] Object detector 4953. [0157] Cluster manager 4954.
[0158] Controller 4955. [0159] Selection unit 4956. [0160] Object
detection determination unit 4957. [0161] Signature generator 4958.
[0162] Movement information unit 4959. [0163] Identifier unit
4960.
[0164] Any of the methods illustrated in the specification may be
executed by a processing circuitry that may be implemented as a
central processing unit (CPU), and/or one or more other integrated
circuits such as application-specific integrated circuits (ASICs),
field programmable gate arrays (FPGAs), full-custom integrated
circuits, etc., or a combination of such integrated circuits.
[0165] Alternatively, each component element and/or unit may
implemented in hardware, firmware, or software that may be executed
by a processing circuitry.
[0166] While the foregoing written description of the invention
enables one of ordinary skill to make and use what is considered
presently to be the best mode thereof, those of ordinary skill will
understand and appreciate the existence of variations,
combinations, and equivalents of the specific embodiment, method,
and examples herein. The invention should therefore not be limited
by the above described embodiment, method, and examples, but by all
embodiments and methods within the scope and spirit of the
invention as claimed.
[0167] In the foregoing specification, the invention has been
described with reference to specific examples of embodiments of the
invention. It will, however, be evident that various modifications
and changes may be made therein without departing from the broader
spirit and scope of the invention as set forth in the appended
claims.
[0168] Moreover, the terms "front," "back," "top," "bottom,"
"over," "under" and the like in the description and in the claims,
if any, are used for descriptive purposes and not necessarily for
describing permanent relative positions. It is understood that the
terms so used are interchangeable under appropriate circumstances
such that the embodiments of the invention described herein are,
for example, capable of operation in other orientations than those
illustrated or otherwise described herein.
[0169] Furthermore, the terms "assert" or "set" and "negate" (or
"deassert" or "clear") are used herein when referring to the
rendering of a signal, status bit, or similar apparatus into its
logically true or logically false state, respectively. If the
logically true state is a logic level one, the logically false
state is a logic level zero. And if the logically true state is a
logic level zero, the logically false state is a logic level
one.
[0170] Those skilled in the art will recognize that the boundaries
between logic blocks are merely illustrative and that alternative
embodiments may merge logic blocks or circuit elements or impose an
alternate decomposition of functionality upon various logic blocks
or circuit elements. Thus, it is to be understood that the
architectures depicted herein are merely exemplary, and that in
fact many other architectures may be implemented which achieve the
same functionality.
[0171] Any arrangement of components to achieve the same
functionality is effectively "associated" such that the desired
functionality is achieved. Hence, any two components herein
combined to achieve a particular functionality may be seen as
"associated with" each other such that the desired functionality is
achieved, irrespective of architectures or intermedial components.
Likewise, any two components so associated can also be viewed as
being "operably connected," or "operably coupled," to each other to
achieve the desired functionality.
[0172] Furthermore, those skilled in the art will recognize that
boundaries between the above described operations merely
illustrative. The multiple operations may be combined into a single
operation, a single operation may be distributed in additional
operations and operations may be executed at least partially
overlapping in time. Moreover, alternative embodiments may include
multiple instances of a particular operation, and the order of
operations may be altered in various other embodiments.
[0173] Also for example, in one embodiment, the illustrated
examples may be implemented as circuitry located on a single
integrated circuit or within a same device. Alternatively, the
examples may be implemented as any number of separate integrated
circuits or separate devices interconnected with each other in a
suitable manner.
[0174] However, other modifications, variations and alternatives
are also possible. The specifications and drawings are,
accordingly, to be regarded in an illustrative rather than in a
restrictive sense.
[0175] In the claims, any reference signs placed between
parentheses shall not be construed as limiting the claim. The word
`comprising` does not exclude the presence of other elements or
steps then those listed in a claim. Furthermore, the terms "a" or
"an," as used herein, are defined as one or more than one. Also,
the use of introductory phrases such as "at least one" and "one or
more" in the claims should not be construed to imply that the
introduction of another claim element by the indefinite articles
"a" or "an" limits any particular claim containing such introduced
claim element to inventions containing only one such element, even
when the same claim includes the introductory phrases "one or more"
or "at least one" and indefinite articles such as "a" or "an." The
same holds true for the use of definite articles. Unless stated
otherwise, terms such as "first" and "second" are used to
arbitrarily distinguish between the elements such terms describe.
Thus, these terms are not necessarily intended to indicate temporal
or other prioritization of such elements. The mere fact that
certain measures are recited in mutually different claims does not
indicate that a combination of these measures cannot be used to
advantage.
[0176] While certain features of the invention have been
illustrated and described herein, many modifications,
substitutions, changes, and equivalents will now occur to those of
ordinary skill in the art. It is, therefore, to be understood that
the appended claims are intended to cover all such modifications
and changes as fall within the true spirit of the invention.
[0177] It is appreciated that various features of the embodiments
of the disclosure which are, for clarity, described in the contexts
of separate embodiments may also be provided in combination in a
single embodiment. Conversely, various features of the embodiments
of the disclosure which are, for brevity, described in the context
of a single embodiment may also be provided separately or in any
suitable sub-combination.
[0178] It will be appreciated by persons skilled in the art that
the embodiments of the disclosure are not limited by what has been
particularly shown and described hereinabove. Rather the scope of
the embodiments of the disclosure is defined by the appended claims
and equivalents thereof
* * * * *