U.S. patent application number 17/156581 was filed with the patent office on 2022-03-17 for systems and methods for organizing an image gallery.
This patent application is currently assigned to PURE MEMORIES LTD. The applicant listed for this patent is PURE MEMORIES LTD. Invention is credited to Ophir ALMOG, Daniel KASHMIR, Lev OREN.
Application Number | 20220083587 17/156581 |
Document ID | / |
Family ID | 1000005355564 |
Filed Date | 2022-03-17 |
United States Patent
Application |
20220083587 |
Kind Code |
A1 |
ALMOG; Ophir ; et
al. |
March 17, 2022 |
SYSTEMS AND METHODS FOR ORGANIZING AN IMAGE GALLERY
Abstract
A system and method for selecting a subset of images may
include: obtaining a plurality of image files, each image file
relating to a digital image of the plurality of digital images,
each file including image data and metadata, the metadata including
a first set of features, and a second set of features; clustering
the plurality of images based on at least one of the first set of
features to generate clusters of images; selecting a set of M
largest clusters of images, wherein M is a positive integer;
scoring the images of each of the selected clusters based on at
least one of the second set of features; and selecting a set of N
images with a highest score from the selected clusters, wherein N
is a positive integer.
Inventors: |
ALMOG; Ophir; (Ramat Gan,
IL) ; OREN; Lev; (Ramat Hasharon, IL) ;
KASHMIR; Daniel; (Tel Aviv, IL) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
PURE MEMORIES LTD |
Ramat Hasharon |
|
IL |
|
|
Assignee: |
PURE MEMORIES LTD
Ramat Hasharon
IL
|
Family ID: |
1000005355564 |
Appl. No.: |
17/156581 |
Filed: |
January 24, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
17023510 |
Sep 17, 2020 |
10909167 |
|
|
17156581 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 16/58 20190101;
G06F 16/535 20190101; G06F 16/538 20190101; G06F 16/55
20190101 |
International
Class: |
G06F 16/55 20060101
G06F016/55; G06F 16/538 20060101 G06F016/538; G06F 16/535 20060101
G06F016/535; G06F 16/58 20060101 G06F016/58 |
Claims
1. A method for selecting a subset of digital images from a
plurality of digital images, the method comprising: obtaining image
data and metadata of each digital image of the plurality of digital
images, the metadata comprising a first set of features, and a
second set of features; clustering the plurality of images based on
at least one of the first set of features to generate clusters of
images; selecting a set of M largest clusters of images, wherein M
is a positive integer; obtaining content of each of the images
pertaining to the selected clusters; deriving user preferences from
the content to obtain inferred preferences; scoring the images of
each of the selected clusters based at least on the inferred
preferences; and selecting a set of N images with a highest score
from the selected clusters, wherein N is a positive integer.
2. The method of claim 1, wherein the score of an image is
calculated to reflect the level of agreement between the inferred
preferences and the content of the image.
3. The method of claim 1, wherein the second set of features
comprises image quality features, wherein scoring the images of
each of the selected clusters is performed based on the image
quality features.
4. The method of claim 1, comprising: obtaining user preferences,
wherein scoring the images of each of the selected clusters is
performed to reflect the level of agreement between the user
preferences and the metadata features.
5. The method of claim 1, comprising: presenting the selected
images to a user.
6. The method of claim 1, wherein the first set of features
comprises time features related to a time in which the image was
taken, and location features related to a location in which the
image was taken, wherein clustering comprises: clustering the
plurality of images based on the location features to generate
location-based clusters; and clustering the plurality of images
based on the time features to generate time-based clusters.
7. The method of claim 6, wherein clustering comprises: clustering
the location-based clusters based on time features to generate
location-time clusters.
8. The method of claim 6, wherein clustering the plurality of
images based on the time the image was taken is repeated for a
plurality of time scales.
9. The method of claim 1, wherein clustering is performed by:
obtaining feature values of each image from the metadata; setting
an influence radius; assigning an influence function to each image,
wherein the influence function is maximal at a position of the
image and decays along the influence radius around the position of
the image; for each image position, summing values of all the
influence functions of all the images; and assigning images to
clusters based on the influence values.
10. The method of claim 9, wherein assigning images to clusters
comprises: repeating until all images are assigned to a cluster:
selecting an image with the highest summed influence value, that is
not yet assigned to a cluster, to be a center of a cluster; and
assigning all images with a distance lower than the influence
radius from the selected image to the cluster.
11. The method of claim 9, wherein the influence function decays
linearly.
12. A method for selecting digital images from a plurality of
digital images, the method comprising: obtaining image data and
characteristics of each digital image from the plurality of digital
images; grouping the plurality of digital images to time-based
groups and location-based groups; selecting at least one group
having the most images; obtaining content of each of the images
pertaining to the selected clusters; categorizing frequent content
as preferred to obtain inferred preferences; ranking the images of
each of the at least one selected group based at least on the
inferred preferences; and selecting N highest ranked image from the
at least one selected group, wherein N is a positive integer.
13. A system for selecting a subset of digital images from a
plurality of digital images, the system comprising: a memory; and a
processor configured to: obtain a plurality of image files, each
image file relating to a digital image of the plurality of digital
images, each file comprising image data and metadata, the metadata
comprising a first set of features, and a second set of features;
cluster the plurality of images based on at least one of the first
set of features to generate clusters of images; select a set of M
largest clusters of images, wherein M is a positive integer; obtain
content of each of the images pertaining to the selected clusters;
derive user preferences from the content to obtain inferred
preferences; score the images of each of the selected clusters
based at least on the inferred preferences; and select a set of N
images with a highest score from the selected clusters, wherein N
is a positive integer.
14. The system of claim 13, wherein the processor is configured to:
calculate the score of an image to reflect the level of agreement
between the user preferences and the content of the image.
15. The system of claim 13, wherein the second set of features
comprises image quality features, wherein the processor is
configured to score the images of each of the selected clusters
based on the image quality features.
16. The system of claim 13, wherein the processor is configured to:
obtain user preferences; and score the images of each of the
selected clusters to reflect the level of agreement between the
user preferences and the metadata features.
17. The system of claim 13, wherein the first set of features
comprises time features related to a time in which the image was
taken, and location features related to a location in which the
image was taken, wherein the processor is configured to cluster the
plurality of images by: clustering the plurality of images based on
the location features to generate location-based clusters; and
clustering the plurality of images based on the time features to
generate time-based clusters.
18. The system of claim 13, wherein the processor is configured to
cluster the plurality of images by: clustering the location-based
clusters based on time features to generate location-time clusters;
and repeating clustering the plurality of images based on the time
the image was taken for a plurality of time scales.
19. The system of claim 13, wherein the processor is configured to
cluster the plurality of images by: obtaining feature values of
each image from the metadata; setting an influence radius;
assigning an influence function to each image, wherein the
influence function is maximal at a position of the image and decays
along the influence radius around the position of the image; for
each image position, summing values of all the influence functions
of all the images; and assigning images to clusters based on the
influence values.
20. The system of claim 19, wherein the processor is configured to
assign images to clusters by: repeating until all images are
assigned to a cluster: selecting an image with the highest summed
influence value, that is not yet assigned to a cluster, to be a
center of a cluster; and assigning all images with a distance lower
than the influence radius from the selected image to the cluster.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. patent
application Ser. No. 17/023,510, filed Sep. 17, 2020, which is
incorporated herein by reference in its entirety.
FIELD OF THE INVENTION
[0002] The present invention relates generally to organizing an
electronic or digital image gallery, and specifically to selecting
images from an image gallery of a user.
BACKGROUND
[0003] A typical electronic or digital image gallery of a user,
e.g. stored on a computer file system, may contain thousands of
images in different qualities and meaning, including huge amounts
of disposable and worthless images, alongside images containing the
most valuable and precious digital memories of the user. The
easiness of taking and storing images using current mobile devices
has greatly increased the number of images stored. The task of
navigating and managing those huge amounts of images may be tedious
and time consuming, instead of enjoyable. Some users may find the
task of managing and sorting their image gallery overwhelming,
while others may spend considerable time selecting images from the
mass of images in their gallery.
[0004] Current cameras typically provide metadata, e.g. data
separate from but related to the substantive image data, that is
related to or describing the images. The metadata may include for
example location features, time features, image quality features,
and other features. Some solutions may assist users by using the
metadata for filtering out (e.g. removing) lower quality images.
While this may provide some help, it may not be enough for large
collections of images. Other solutions may use content detections
algorithms, provided in a computer vision library of mobile
devices, e.g., proprietary or commercial off-the-shelf (COTS)
computer vision tools, to detect the content of images. However,
scanning a large volume of images may be a computationally
intensive task, and thus may take long time to complete on a
typical mobile device (e.g., a smartphone). Thus, it may not be
feasible to sort images based on content locally on the mobile
device, e.g., sorting may require processing images stored on the
"cloud" (typically remote, server-based storage systems).
SUMMARY
[0005] According to embodiments of the invention, a system and
method for selecting a subset of images may include: obtaining a
plurality of image files, each image file relating to a digital
image of the plurality of digital images, each file including image
data and metadata, the metadata including a first set of features,
and a second set of features; clustering the plurality of images
based on at least one of the first set of features to generate
clusters of images; selecting a set of M largest clusters of
images, wherein M is a positive integer; scoring the images of each
of the selected clusters based on at least one of the second set of
features; and selecting a set of N images with a highest score from
the selected clusters, wherein N is a positive integer.
[0006] Embodiments of the invention may include obtaining content
of each of the images pertaining to the selected clusters; and
obtaining user preferences, and calculating the score to reflect
the level of agreement between the user preferences and the
content.
[0007] According to embodiments of the invention, the second set of
features may include image quality features, where scoring the
images of each of the selected clusters may be performed based on
the image quality features.
[0008] Embodiments of the invention may include obtaining user
preferences, where scoring the images of each of the selected
clusters may be performed to reflect the level of agreement between
the user preferences and the metadata features.
[0009] Embodiments of the invention may include presenting the
selected images to a user.
[0010] According to embodiments of the invention, the first set of
features may include time features related to a time in which the
image was taken, and location features related to a location in
which the image was taken, where clustering may include: clustering
the plurality of images based on the location features to generate
location-based clusters; and clustering the plurality of images
based on the time features to generate time-based clusters.
[0011] According to embodiments of the invention, clustering may
include clustering the location-based clusters based on time
features to generate location-time clusters.
[0012] According to embodiments of the invention, clustering the
plurality of images based on the time the image was taken may be
repeated for a plurality of time scales.
[0013] According to embodiments of the invention, clustering may be
performed by: obtaining feature values of each image from the
metadata; setting an influence radius; assigning an influence
function to each image, wherein the influence function is maximal
at a position of the image and decays along the influence radius
around the position of the image;
[0014] for each image position, summing values of all the influence
functions of all the images; and assigning images to clusters based
on the influence values.
[0015] According to embodiments of the invention, assigning images
to clusters may include: repeating until all images are assigned to
a cluster: selecting an image with the highest summed influence
value, that is not yet assigned to a cluster, to be a center of a
cluster; and assigning all images with a distance lower than the
influence radius from the selected image to the cluster.
[0016] According to embodiments of the invention, the influence
function may decay linearly.
[0017] According to embodiments of the invention, a system and
method for selecting digital images from a plurality of digital
images, may include: obtaining image data and characteristics of
each digital image from the plurality of digital images; grouping
the plurality of digital images to time-based groups and
location-based groups; selecting at least one group having the most
images; ranking the images of each of the at least one selected
group based on at least one quality characteristic of the images;
and selecting N highest ranked image from the at least one selected
group, wherein N is a positive integer.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The subject matter regarded as the invention is particularly
pointed out and distinctly claimed in the concluding portion of the
specification. Embodiments of the invention, however, both as to
organization and method of operation, together with objects,
features and advantages thereof, may best be understood by
reference to the following detailed description when read with the
accompanied drawings. Embodiments of the invention are illustrated
by way of example and not limitation in the figures of the
accompanying drawings, in which like reference numerals indicate
corresponding, analogous or similar elements, and in which:
[0019] FIG. 1 is a flowchart of a method for selecting a subset of
digital images, according to embodiments of the invention;
[0020] FIG. 2 is a flowchart of a method for selecting a subset of
digital images using image clusters, according to embodiments of
the invention;
[0021] FIG. 3 is a flowchart of a method for clustering digital
images, according to embodiments of the invention;
[0022] FIG. 4 depicts an example of clustering, according to
embodiments of the invention;
[0023] FIG. 5, is a flowchart of a method for adding new images to
the plurality of digital images, according to embodiments of the
invention;
[0024] FIG. 6, which is a flowchart of a method for selecting a
subset of digital images from a plurality of digital images based
on frequent content, according to embodiments of the invention;
[0025] FIG. 7, which is a flowchart of a method for selecting a
subset of digital images from a plurality of digital images based
on user tagging, according to embodiments of the invention; and
[0026] FIG. 8 illustrates an example computing device according to
an embodiment of the invention.
[0027] It will be appreciated that for simplicity and clarity of
illustration, elements shown in the figures have not necessarily
been drawn to scale. For example, the dimensions of some of the
elements may be exaggerated relative to other elements for clarity.
Further, where considered appropriate, reference numerals may be
repeated among the figures to indicate corresponding or analogous
elements.
DETAILED DESCRIPTION
[0028] In the following description, various aspects of the present
invention will be described. For purposes of explanation, specific
configurations and details are set forth in order to provide a
thorough understanding of the present invention. However, it will
also be apparent to one skilled in the art that the present
invention may be practiced without the specific details presented
herein. Furthermore, well known features may be omitted or
simplified in order not to obscure the present invention.
[0029] Although some embodiments of the invention are not limited
in this regard, discussions utilizing terms such as, for example,
"processing," "computing," "calculating," "determining,"
"establishing", "analyzing", "checking", or the like, may refer to
operation(s) and/or process(es) of a computer, a computing
platform, a computing system, or other electronic computing device
that manipulates and/or transforms data represented as physical
(e.g., electronic) quantities within the computer's registers
and/or memories into other data similarly represented as physical
quantities within the computer's registers and/or memories or other
information transitory or non-transitory or processor-readable
storage medium that may store instructions, which when executed by
the processor, cause the processor to execute operations and/or
processes. Although embodiments of the invention are not limited in
this regard, the terms "plurality" and "a plurality" as used herein
may include, for example, "multiple" or "two or more". The terms
"plurality" or "a plurality" may be used throughout the
specification to describe two or more components, devices,
elements, units, parameters, or the like. The term "set" when used
herein may include one or more items unless otherwise stated.
Unless explicitly stated, the method embodiments described herein
are not constrained to a particular order or sequence.
Additionally, some of the described method embodiments or elements
thereof can occur or be performed in a different order from that
described, simultaneously, at the same point in time, or
concurrently.
[0030] Embodiments of the invention may provide a system and method
for selecting a subset of digital images from a plurality of
digital images. For example, the plurality of digital images may
form an image gallery of a user. A typical image gallery of a user
may contain thousands of images with different content and
qualities. Embodiments of the invention may provide an efficient
method for scoring images and selecting a subset of images from the
plurality of digital images, based on location, time content and
quality of the images. According to embodiments of the invention,
selection of the subset of images may consider the content of the
images, but may still require relatively low computational power
and may be performed relatively quickly on a mobile device.
[0031] A digital image, also referred to herein simply as an image,
may include visual (e.g., optical) representation of physical
objects provided in any applicable digital format. Images may
include simple two-dimensional (2D) array or matrix of pixels,
e.g., values representing one or more light wavelengths or one or
more ranges of light wavelength, within or beyond the visible
light, in specified locations, or any other digital representation,
provided in any applicable digital format such as jpg, bmp, tiff,
etc. images may also include panchromatic images, multispectral
images and explicit point cloud generated using laser, such as
Lidar (Light detection and ranging), e.g., embedded in IPAD pro, or
to a multi spectral cube including optical information in various
wavelengths in the range of: 400-2500 nano meters.
[0032] A digital image may be provided in a digital image file
containing image data and metadata. The metadata may describe or
include descriptors or values for a plurality of features
describing characteristics, qualities or attributes of the image.
For example, metadata features or descriptions may include or
represent time features, e.g., features related to a time the image
was taken, captured or created (e.g. image capture time and date
from a camera), location features, e.g., features related to a
location in which the image was taken, and quality features, e.g.,
features related to measures of quality of the image, e.g.,
brightness, sharpness, illumination, focal length, rear\front
camera etc. Some of the metadata features may be measured and
logged during image acquisition (e.g., in exif file embedded in the
image file). Other metadata features may be provided by the system
(e.g., other applications of the smartphone) and from connected
devices that may provide information about the image (e.g. air-pods
that may provide the capture spatial direction of the image).
According to embodiments of the invention, the metadata may be
enriched with features calculated based on user preferences and on
the image data, e.g., using standard or proprietary image
processing tools to extract features, as disclosed herein.
[0033] According to embodiments of the invention, images in an
image gallery may be divided into groups based on common or similar
characteristics, e.g., clustered based on metadata features,
specifically time and location features. It may be assumed that a
set of images captured within a daily, hourly or minute-based time
frame may have content closer to each other relative to images
captured at larger time intervals. Similarly, a set of images taken
few meters from each other are more likely to contains similar or
related content. Additionally, a specific location in which images
have been captured several times may indicate a point of interest
to the user.
[0034] Current work in the field of computer vision and artificial
intelligence (AI) for understanding an image (e.g., where it is
located, what is the content of the image, what is being captured,
relevancy, etc.) usually require supervised learning that relies on
manual tagging, at least for a learning phase. However, clustering
the images into similarity groups based on location and time may
eliminate the need of manual tagging and significantly increase the
information that can be extracted from the images autonomously by
the system. Adding user preferences and general tagging (e.g.,
identification of presence of persons, pets and objects in the
images and adding such information to image files) may improve the
comprehension of what images are likely to be of value to the
user.
[0035] According to embodiments of the invention, the entire image
gallery may first be divided into clusters or groups. Clustering or
grouping may be performed in a first stage based only on the
metadata provided by the camera, and specifically on location
features and time features. For example, clustering may be
performed based on location features to provide location-based
clusters, e.g., clusters of images taken or captured at a single
location (e.g., at a specified geographical location or within a
specific distance from one another). Additionally or alternatively,
clustering may be performed based on timing features to provide
time-based clusters, e.g., clusters of images taken or captured at
a close time intervals. Additionally or alternatively, clustering
may be performed based on timing and location features to provide
time-location clusters.
[0036] According to embodiments of the invention, clustering may
divide the image gallery into subgroups or clusters with common or
related theme or meaning, e.g., location clusters may include
images taken at home, at work etc., time based clusters may include
images taken at a single trip or party etc. Specially, while not
all clusters may have a related theme or meaning, it is believed
that at least the largest clusters (e.g., the location-based
clusters, time based clusters, or any other clusters containing the
most images, or a number of images above a threshold number) may
have common or related theme or meaning, and that the theme or
meaning of images in the largest clusters is important or
meaningful to the user.
[0037] Thus, according to embodiments of the invention, the first
stage in selecting the subset of images may include selecting a set
of M largest clusters (where M is a positive integer, e.g., a whole
number equal or larger than 1) and discarding smaller clusters. The
set of M largest clusters may include one or more clusters
containing the most images, larger number of images with relation
to other clusters, or a number of images above a threshold number.
For example, the clusters may be ordered based on size, e.g., based
on the number of images each cluster contains and the set of
clusters having the most images may be selected. According to
embodiments of the invention, large clusters (e.g., clusters with
image number above a threshold) may be further clustered to provide
smaller clusters. For example, location-based clusters may be
further clustered based on time the image was taken to generate
location-time clusters. Clustering images based on metadata may
require relatively low computational power, in comparison to the
computational power required for content detection. Thus, a first
stage of selection may be performed quickly.
[0038] According to embodiments of the invention, quality features
in the metadata may be used to filter out low quality images. For
example, in many smartphone cameras, quality features may include
features related to sharpness, red eyes, illumination etc. For
example, a quality score may be calculated for each image based on
the quality features in the metadata, and images with quality score
below a threshold may be filtered out or removed from the selected
clusters.
[0039] According to embodiments of the invention, each of the
images pertaining to the M largest clusters (e.g. those containing
the most images, or a number of images above a threshold number)
may be analyzed to detect content of the images. For example,
standard or proprietary image processing tools may be used to
detect and identify objects and features in the images. For
example, tagging algorithms provided as part of a mobile device
computer vision library (e.g. COTS and other open source libraries)
may be applied on each image. The tagging process may detect or
recognize faces including, eyes, mouths, nose, state of the eyes
e.g., red eye, blinking etc., state of feeling such as smiling and
more, pets, common objects such as bowls, papers, tables, cars,
plants and more. Eventually, a set of content tags may be provided
and added to the metadata.
[0040] According to embodiments of the invention, the content tags
may be used in various ways. For example, the content tags may be
used to further cluster the images to content-based clusters.
According to some embodiments, images may be selected from the M
largest clusters based on the content. For example, it may be
assumed that the most valuable content will be the most frequent
content. Thus, images with the most frequent content may be
selected from the M largest image clusters.
[0041] According to some embodiments, images in the M largest
clusters may be scored or rated. For example, the score, grade or
rate may be calculated based on the image quality, the image
content and the user preferences. A subset of the images may be
selected, e.g., based on the score. The subset may include images
that are assumed to have value to the user. The selected images may
be presented to the user. Thus, embodiments of the invention may
improve the technologies of digital storage, computer file
management, and of image gallery management by providing an
efficient tool for sorting and selecting images based on content,
quality and user preferences.
[0042] Reference is made to FIG. 1, which is a flowchart of a
method for selecting a subset of digital images from a plurality of
digital images, according to embodiments of the invention. An
embodiment of a method for selecting a subset of digital images
from a plurality of digital images may be performed, for example,
by the example system shown in FIG. 8. As with other method
flowcharts, operations of FIG. 1 may be used with operations in
other flowcharts shown herein in some embodiment.
[0043] In operation 110, image data and metadata related to digital
images of the plurality of digital images may be obtained, e.g., in
a plurality of image files (e.g., image files 732). Each of the
image files may relate to a single digital image (e.g., digital
image 733), and may include image data and metadata (e.g., metadata
734), as provided by the system. In typical applications the
metadata may include location features, time features, image
quality features, and other features.
[0044] In operation 120 the plurality of images may be clustered
based on at least one of the features of the metadata to generate
clusters of images. According to embodiments of the invention,
clustering may be performed using any applicable method or
algorithm. Clustering may refer to dividing the plurality of images
into groups, also referred to as clusters, so that images in a
cluster would be more similar to each other than to images from
other clusters, with relation to the features used for clustering.
It is believed, however, that in most cases, images within the same
cluster are more similar to each other with relation content as
well, in comparison to images that were assigned into other
clusters. Clustering may be performed based on any subset of one or
more features from the metadata, using any applicable algorithm The
measure of similarity may depend on the specific algorithm used.
Clustering methods may include for example, K-means, Gaussian
mixture model (GMM), mean shift, spectral clustering, density-based
spatial clustering of applications with noise (DBSCAN),
hierarchical clustering, etc., or proprietary algorithms as
disclosed herein.
[0045] For example, the time the image was taken may be used to
cluster the plurality of images based on acquisition time, to
generate time-based clusters, and location data may be used to
cluster the plurality of images based on the acquisition location,
to generate location-based clusters. According to some embodiments,
time-based clustering and location-based clustering may be repeated
several times with different time and location scales. For example,
time-based clustering may be repeated or performed separately for
time scales or ranges of minutes, hours and days. According to
embodiments of the invention, clustering may be repeated if the
metadata is enriched, to generate new clusters using the new
metadata features.
[0046] As noted, clustering of the plurality of images may be
performed several times, each time using different features for
performing the clustering, to obtain a plurality of divisions into
clusters. For example, clustering may be performed based on time
(or several time scales) and based on location (or several location
radiuses). Thus, the same plurality of images may be divided, once
to a plurality of time-based clusters, and then to a plurality of
location-based clusters. For example, each single image may be
included in a plurality of time-based clusters and in a plurality
of location-based clusters.
[0047] In operation 130, the M largest clusters of images may be
selected, where M is a positive integer equal or larger than one.
Other clusters, e.g., non-selected clusters, may discarded from the
selection process. The non-selected clusters may be saved, however,
for future uses. Selecting the larges clusters may help selecting
images that are meaningful to the user, assuming that the user may
take more photos or images at location or times that are more
important for the user. Thus, by selecting the M largest clusters,
the most important images are selected. For example, in some
embodiments 10-30 largest location-based clusters and 10-30 largest
time-based clusters may be selected. In some embodiments 20%
largest location-based clusters and/or time-based clusters may be
selected. Other selection criteria may be used.
[0048] In operation 140, images in the selected clusters may be
scored. For example, scoring may be performed based on features of
the metadata, or features of the metadata other than those used for
clustering. For example, scoring may be performed based on quality
features, e.g., brightness, sharpness, illumination, focal length,
rear\front camera etc. For example, a grade or a score may be
provided to each image for each quality feature and the quality
score or grade may be calculated as an average or weighted sum of
the grades or scores. Other algorithms for grading or scoring
images based on quality metadata may be used.
[0049] In some embodiments, images may be scored based on user
preferences and content. Image content may be obtained using any
applicable method, as disclosed herein. For example, if user
preferences are obtained or otherwise known, a score may be
calculated to reflect the level of agreement between the user
preferences and the metadata features and or the content. For
example, images with content that is preferred by the user may
obtain higher scores comparing with images without content that is
preferred by the user. Similarly, if the user prefers selfies,
images with metadata feature indicating that the image is a selfie
may obtain higher scores comparing with images with metadata
feature indicating that the image is not a selfie.
[0050] According to some embodiments, a score may be calculated
based on content of the image. For example, content of images may
be ordered by frequency of appearances in images, and images with
more frequent content may get a higher score. In some embodiments,
images with more detected content, e.g., images with more people in
them, may be assigned a higher score than images with less content.
In some embodiments, the score or grade may be calculated based on
a combination of one or more of the quality features, the content,
and the user preferences, e.g., as an average, weighted sum or
using any other function.
[0051] In operation 150, images may be selected from the selected
clusters. For example, a predetermined number of images or a
predetermined fraction or percent of images may be selected.
According to some embodiments, the N highest ranked images or N
images with the highest scores may be selected from the M selected
clusters, e.g., a plurality of images each with a score above a
threshold or each having a score higher than other images in the
selected clusters may be selected, where N is a positive integer,
e.g., a whole number equal or larger than one. According to some
embodiments, selection is performed based on the scores as well as
other criteria such as the distribution of the images among the
clusters or the content of the images. For example, the number of
selected images from a single cluster may be limited, so even if a
single cluster includes all the N highest ranked images, images may
be selected from other clusters as well. In some embodiments, the N
highest ranked images from at least two or more clusters may be
selected. In some embodiments, the selected images may all include
specified content. For example, the best images (e.g., images with
the highest scores) that include inferred or declared content may
be selected. In some embodiments, the user may specify the required
content for the selected images, thus, the best images (e.g.,
images with the highest scores) that include content specified by
the user may be selected.
[0052] According to embodiments of the invention, a second stage of
clustering may be performed for the selection of images from large
clusters, e.g., clusters with a number of images that is above a
threshold. For example, in a second stage of clustering, images
pertaining to a single cluster (clustered based on a first feature)
may be clustered based on the second feature. For example,
location-based clustering may be performed in operation 120, and
for selecting images from a large location-based cluster, the
location-based cluster may be further clustered based on time thus
generating time-location-based clusters. When selecting images from
the large location-based cluster, the N highest ranked images from
the time-location-based clusters that are closer in time to the
selection date may be selected.
[0053] In operation 160, the selected images may be presented to
the user. Thus, the user may obtain a selection of best images
(e.g., images with the highest scores) quickly, without having to
spend time navigating and choosing images from his entire image
gallery. In some embodiments, the selected images may be used to
generate an image album (e.g., a printable image album) for the
user. Thus, if the user has specified required content for the
selected images, e.g., in operation 150, the album may include a
selection of best images (e.g., images with the highest scores)
that include the required content.
[0054] Reference is made to FIG. 2, which is a flowchart of a
method for selecting a subset of digital images from a plurality of
digital images using image clusters, according to embodiments of
the invention. An embodiment of a method for selecting a subset of
digital images from a plurality of digital images using image
clusters may be performed, for example, by the systems shown in
FIG. 8. Some operations of embodiments of the method presented in
FIG. 2 are identical to operations of embodiments of the method
presented in FIG. 1. Those operations will be mentioned only
briefly and will not be described again in detail.
[0055] Operations 110-130 are similar to operation 110-130 of FIG.
1. In operation 200, low quality images may be removed or discarded
from the image gallery. For example, images may be scored based on
quality features provided in the metadata e.g., brightness,
sharpness, illumination, focal length, rear\front camera etc. For
example, a quality grade or a score may be provided to each image
for each quality feature and the quality score or grade of an image
may be calculated as an average or weighted sum of the grades or
scores. Other algorithms for grading or scoring images based on
quality features may be used. In some embodiments, images with
quality score below a threshold may be removed or discarded from
the plurality of images.
[0056] In operation 210, content of each of the images pertaining
to the M largest clusters may be obtained. For example, proprietary
or standard content detections algorithms, also referred to as
tagging algorithms, may be used to detect or identify content of
images, e.g., to detect and identify objects and features in the
images. The content detections algorithms may include the content
detections algorithms provided in current mobile device computer
vision library, e.g., COTS and other open source libraries. For
example, the tagging algorithms provided as part of a mobile device
computer vision library may be applied on each image of the
selected clusters. The tagging process may detect or recognize
people, faces including, eyes, mouths, nose, state of the eyes
e.g., red eye, blinking etc., state of feeling such as smiling and
more, pets, common objects such as bowls, papers, tables, cars,
plants and more. While tagging is relatively computationally
intensive process, performing tagging to relatively low number of
images may be performed quickly and provide instant results to the
user. In addition, images taken close to one another by time or
location may include same or similar objects and content, so after
detecting objects and content in one image, it may be much easier
to detect the same objects and content in the close images, e.g.,
images from the same cluster.
[0057] In operation 215, frequent content may be detected. Frequent
content may include content that is detected in large number of
images, e.g., in a number of images that is above a threshold or in
a percentage or fraction of the images that is above a
threshold.
[0058] In operation 220 user preferences may be obtained. The user
preferences may include information specifying content that is
valuable to the user. The user preferences may be obtained using a
questionnaire, e.g., by presenting questions to the user and
obtaining user answers. User preferences that are obtained from
answers of the user may be referred to herein as declared
preferences. The questions may define the preferred content, for
example the user may be asked about his hobbies and family, or
about visual or composition preferences, for example, a user may be
asked "Do you like to see teeth in your images?"
[0059] However, research shows that declared preferences tend to
poorly describe the real preferences of the user. Therefore, user
preferences may be learned automatically or derived from the
content of the images. User preferences that are derived from the
content of the images may be referred to herein as inferred
preferences. For example, frequent content, e.g., content detected
as frequent (e.g. frequently occurring in a user's image
collection) in operation 215, may be categorized as preferred. For
example, if cats appear in a percent of the images that is above a
threshold, cats may be recognized as preferred content. In case of
contradiction between the declared preferences and the inferred
preferences, either both options may be recognized as preferred, or
the inferred preferences may override the declared preferences.
[0060] In operation 230, the metadata may be enriched. The metadata
may be enriched with content information obtained in operation 210,
for example, a set of content tags may be added to the metadata.
The metadata may be further enriched with user preferences obtained
in operation 220.
[0061] In operation 140 images of each of the M largest clusters
may be scored, in operation 150 N images with the highest scores
may be selected, where N is a positive integer equal or larger than
one, and in operation 160 the selected images may be presented to
the user. In operation 260, feedback or input may be obtained from
the user. The user may indicate whether the selected images are
valuable to him In some embodiments, the user may provide
indication of valuable images that were not selected. The feedback
from the user may be used to calibrate the user preferences.
[0062] As noted, clustering of images may be performed using any
applicable method. A common clustering method is k-means. To use
k-means, the system may obtain as input a set of unlabeled
(explicit) list of observations (images), where each observation is
a multi-dimensional real vector (e.g., an ordered list of values),
specifically, x-y coordinates of where an image was captured for
location-based clustering, and a time an image was captured for
time-based clustering. The system may randomly guess the number of
potential clusters k, choose k random observations from the list of
observations, and set each of the selected observations as a center
of a cluster. After selecting centers, the system may go over the
observations in the list of observations, calculate a distance,
e.g., Euclidian distance, between each observation j and each of
the cluster's centers, and assign each point into the cluster with
the closest center. Eventually, statistical parameters such as mean
and standard deviation may be calculated for each cluster. The
calculated mean of each cluster may be used as a cluster center in
a next iteration. In each iteration, the system may rerun the
assigning process according to the new centers and update the
statistical parameters of each cluster. The process may terminate
when there is no difference between two consecutive iterations.
K-means clustering is generally accepted and is considered
efficient to use. However, k-means clustering depends on an initial
guess about the data structure, that often leads to a consequential
cluster calculation that poorly describes the clustered data.
Moreover, the common algorithm requires an a-priori assessment
about the number of potential clusters from the user.
[0063] Embodiments of the invention may suggest an innovative
clustering approach. According to embodiments of the invention,
each observation may be perceived as producing one unit of
influence that is induced to a pre-defined radius of influence. The
influence absorbed by each point may be summed, and observations
with high amount of influence may be selected to be a potential
center of cluster.
[0064] Reference is made to FIG. 3, which is a flowchart of a
method for clustering digital images, according to embodiments of
the invention. An embodiment of a method for clustering digital
images may be an example for a clustering method that may be used
in operation 120 presented in FIGS. 1 and 2. An embodiment of a
method for clustering digital images may be performed, for example,
by the systems shown in FIG. 8.
[0065] In operation 310, feature values of each image may be
obtained from the metadata of images. For example, to perform
time-based clustering, features regarding the time the image was
taken or captured may be obtained, and to perform location-based
clustering, features regarding the location the image was taken or
captured may be obtained. Each feature may be provided in a form of
a multi-dimensional real vector (e.g. an ordered series of numbers,
each representing a feature), for example, x-y coordinates of where
an image was captured for location-based clustering, and a time an
image was captured for time-based clustering.
[0066] In operation 320, a cluster radius, also referred to as
influence radius, may be set or selected. For example, when
performing time-based clustering, the influence radius may be set
to several minutes to perform minutes-scale clustering, to a few
hours to perform hours-scale clustering, and to a few days to
perform days-scale clustering. For performing location-based
clustering, an initial influence radius of 30 meters may be used.
Other influence radiuses may be used. In some embodiments, the
influence radiuses may be adjusted based on clustering results, for
example, if clusters that are too large are obtained, e.g.,
clusters having a number of images that is above a threshold,
clustering may be repeated with smaller influence radius, and if
clusters that are too small are obtained, e.g., clusters having a
number of images that is below a threshold, clustering may be
repeated with larger influence radius. In some embodiments,
adjacent clusters may be united or combined to form a single
cluster. This may improve the clustering process since adjacent
clusters may share common content and therefore may be united. For
example, if the distance between two clusters is lower than a
threshold, e.g., 1.5 times the influence radius, than the clusters
may be united to a single cluster. The distance may be Euclidean
distance or a statistic distance such as Mahalanobis distance,
e.g., a distance of a cluster radius and the distribution of image
positions in a second cluster.
[0067] In operation 330, an influence function may be assigned to
each position of an image (each image location is considered as an
observation). In some embodiments, the influence function may be
constant (e.g., equal a constant value) long a cluster radius or
influence radius around the position of the image, and be set to
zero elsewhere. Other influence functions may be used. For example,
the influence function may be maximal at a position of the image
and may decay gradually, e.g., linearly, exponentially, etc., along
a cluster radius or influence radius around the position of the
image. For example, when performing location-based clustering, the
influence function may be maximal at the x-y location of the image,
and for time-based clustering, the influence function may be
maximal at the time the image was captured.
[0068] In operation 340, values of all the influence functions of
all the images may be summed or otherwise combined per position. In
some embodiments, values of all the influence functions of all the
images may be summed or otherwise combined for each image position.
In operation 350, images may be assigned to clusters based on the
combined influence values. For example, assigning images to
clusters may include an iterative process that may be repeated
until all images are assigned to a cluster. In each iteration, a
position, e.g., of an image or not, with the highest summed
influence value, that is not yet assigned to a cluster, may be
selected to be a center of a cluster, and all images with a
distance lower than the influence radius from the selected image
may be assigned to the cluster.
[0069] Reference is now made to FIG. 4 which depicts an example of
clustering according to embodiments of the invention. The
non-limiting example in FIG. 4 includes a single dimension (1D)
feature, e.g., time. However, as noted, the same clustering method
may be performed in two-dimensions for location-based clustering
and with as many dimensions as required. FIG. 4 includes capturing
time of four images, image A taken at 60 seconds, image B at 95
seconds, image C at 105 seconds and image D at 150 seconds. Each
image A, B, C and D is represented in FIG. 4 by an influence
function 410, 420, 430 and 440, respectively. As can be seen, the
influence function is maximal at the time the image was taken and
decays linearly along the influence radius 450, 460 around the
position of the image on the time scale. The maximal value of the
influence function may be determined arbitrarily to any value,
e.g., to one. The influence radius 450, 460 may be determined
according to the scale of the clustering (here .about.50 seconds).
The bars in each image location represent the accumulated influence
functions in each image location. For example, the accumulated
influence functions in the position of image A includes a
contribution of the influence function of images A, B and C. In the
first iteration of assigning images to clusters, no image is
assigned to a cluster, therefore, the image with the highest summed
influence value, here image B, is selected to be a center of a
first cluster. Next all images with a distance lower than the
influence radius from the selected image, here images A and C, may
be assigned to the first cluster. Then, an image with the highest
summed influence value, that is not yet assigned to a cluster, may
be selected to be a center of a second cluster. Here the only image
that was not assigned to the first cluster is image D, which is
assigned to the second cluster.
[0070] Embodiments of the method for selecting a subset of digital
images from a plurality of digital images may be performed on an
existing plurality of digital images, e.g., on an existing image
gallery by an application downloaded to a smartphone. However,
after being performed once, the selection of images may be
dynamically updated whenever a new image is captured, or whenever a
plurality of new images is captured, every predetermined time
period, or when required by the user. According to embodiments of
the invention, adding a new image to the plurality of digital
images may be performed efficiently, without repeating the entire
clustering process, by determining if the new image pertains to an
existing cluster or not, as disclosed herein.
[0071] Reference is made to FIG. 5, which is a flowchart of a
method for adding new images to the plurality of digital images,
according to embodiments of the invention. An embodiment of a
method for adding new images to the plurality of digital images may
be performed, for example, by the systems shown in FIG. 8. As with
other flowcharts provided herein, some embodiments of the method
presented in FIG. 5 may be combined with other embodiments
described herein such as those presented in FIG. 1 and FIG. 2. An
embodiment of a method for adding new images to the plurality of
digital images may be performed, for example, whenever a new image
is captured, or a plurality of new images are captured or added to
the image gallery, or every predetermined time period, or when
required by the user. However, in case more than one image is
added, embodiments of the method for adding new images to the
plurality of digital images may be performed for each new
image.
[0072] In operation 510, a new image may be obtained or received.
In operation 520, it may be checked and determined whether the new
image belongs one or more of the existing clusters. For example, if
the new image is within the influence radius of an existing cluster
(e.g., within the influence radius of a cluster center), the new
image may be added to that cluster. The new image may be added to
more than one cluster, e.g., to one or more location-based clusters
with different cluster scales and to one or more time-based
clusters with different cluster scales, depending on the distance
of the position of the new image from the clusters center. In
operation 520, the new image is assigned to the clusters identified
in operation 510.
[0073] In operation 540, it may be determined if at least one of
the clusters to which the new image has been assigned to is a large
cluster, e.g., using the same criteria as in operation 130. For
example, the new image may be assigned to a cluster that was
already determined to be a large cluster, making it even larger, or
to a cluster that has become a large cluster following the addition
of the new image. If it is determined that the cluster is a large
cluster, the method may move to operation 140 in FIG. 1 or to
operation 210 in FIG. 2. If the new image was assigned to a cluster
that was already determined to be a large cluster, then in
operation 140 only the new image may be scored. If the new image
was assigned to a cluster that has become a large cluster following
the addition of the new image, then in operation 140 the entire
cluster may be scored, and the method may move to operations 150
and 160. Depending on the scores of the new image or the new
cluster, the selection of images may change. Similarly, if the new
image was assigned to a cluster that was already determined to be a
large cluster, then: [0074] in operation 210 only the content of
the new image may be obtained, [0075] in operation 220 the user
preferences may be updated based on the content of the new image,
[0076] in operation 230 the metadata of the new image may be
enriched, [0077] in operation 140, the new image may be scored, and
the method may continue to operation 150. If the new image was
assigned to a cluster that has become a large cluster following the
addition of the new image, then: [0078] in operation 210 the
content of the entire cluster may be obtained, [0079] in operation
220 the user preferences may be updated based on the content of the
entire cluster, [0080] in operation 230 the metadata of images of
the new cluster may be enriched, [0081] in operation 140, the
images of the new cluster may be scored, and the method may
continue to operation 150. Thus, the selection of images may be
updated dynamically, without having to repeat the entire clustering
operation.
[0082] Reference is made to FIG. 6, which is a flowchart of a
method for selecting a subset of digital images from a plurality of
digital images based on frequent content, according to embodiments
of the invention. An embodiment of a method for selecting a subset
of digital images from a plurality of digital images based on
frequent content may be performed, for example, by the systems
shown in FIG. 8. Some operations of embodiments of the method
presented in FIG. 6 are identical to operations of embodiments of
the method presented in FIGS. 1 and 2. Those operations will be
mentioned only briefly and will not be described again in
detail.
[0083] In operation 110, image data and metadata related to digital
images. In operation 610, each image may be scored based on quality
features in the metadata. For example, a quality grade or a score
may be provided to each image for each quality feature and the
quality score or grade of an image may be calculated as an average,
sum or weighted average or sum of the grades or scores. Other
algorithms for grading or scoring images based on quality features
may be used. In operation 620, images may be selected based on the
quality score. For example, N images (e.g., a number of images, a
fraction of the images, or a fraction of images up to a limit) with
the highest scores may be selected.
[0084] In operation 630, content of the selected images may be
obtained, in similar manner to operation 210. In operation 215,
frequent content may be detected. In operation 650, images with
frequent content may be selected. In some embodiments, the
percentage, proportion or ratio of each type of frequent content in
the images selected in operation 620 to the total number of images,
may be preserved in the images selected in operation 650. For
example, if the types of frequent content in the bulk of images
selected in operation 620 includes 30% images with cats, 40% images
with people, 10% images with sunset, and 20% images with other,
less frequent, content, the subset of images selected in operation
650 may include images of cats, people and sunsets. In some
embodiments, images including a plurality of types of selected
content (e.g., people and cats) may be preferred over images
containing less types of frequent content. The sum of all ratios
may equal more than 100% since a single image may include more than
one type of content, e.g., a single image may include a person, and
an animal, etc. In operation 160, the selected images may be
presented or displayed to the user or used to generate an image
album. In operation 260, feedback may be obtained or received from
the user.
[0085] Reference is made to FIG. 7, which is a flowchart of a
method for selecting a subset of digital images from a plurality of
digital images based on user tagging, according to embodiments of
the invention. An embodiment of a method for selecting a subset of
digital images from a plurality of digital images based on frequent
content based on user tagging may be performed, for example, by the
systems shown in FIG. 8. Some operations of embodiments of the
method presented in FIG. 7 are identical to operations of
embodiments of the method presented in FIGS. 1 and 2. Those
operations will be mentioned only briefly and will not be described
again in detail.
[0086] In operation 110, image data and metadata related to digital
images. In operation 710, user preferences may be obtained from
images tagged by the user. For example, the user may be requested
to select images with preferred content and tag the preferred
content in the user selected images. In addition, user preferences
may be obtained using a questionnaire. In operation 712, content of
images of the plurality of images may be obtained (in similar way
to operation 210). In operation 215, frequent content may be
detected. In operation 714 images containing frequent content and
preferred content may be selected. In operation 160, the selected
images may be presented to the user. In operation 260, feedback may
be obtained from the user. The user feedback may be used to adjust
the user preferences and refine the selection of images.
[0087] FIG. 8 illustrates an example computing device according to
an embodiment of the invention. For example, a computing device 700
with a processor 705 may be used to select a subset of digital
images from a plurality of digital images, according to embodiments
of the invention.
[0088] Computing device 700 may include a processor 705 that may
be, for example, a central processing unit processor (CPU), a chip
or any suitable computing or computational device, an operating
system 715, a memory 720, a storage 730, input devices 735 and
output devices 740. Processor 705 may be or include one or more
processors, etc., co-located or distributed. Computing device 700
may be for example a smart device, a smartphone, workstation or a
personal computer, a laptop, or may be at least partially
implemented by one or more remote servers (e.g., in the
"cloud").
[0089] Operating system 715 may be or may include any code segment
designed and/or configured to perform tasks involving coordination,
scheduling, arbitration, supervising, controlling or otherwise
managing operation of computing device 700, for example. Operating
system 715 may be a commercial operating system. Memory 720 may be
or may include, for example, a Random Access Memory (RAM), a read
only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM
(SD-RAM), a double data rate (DDR) memory chip, a Flash memory, a
volatile memory, a non-volatile memory, a cache memory, a buffer, a
short term memory unit, a long term memory unit, or other suitable
memory units or storage units. Memory 720 may be or may include a
plurality of, possibly different memory units.
[0090] Executable code 725 may be any executable code, e.g., an
application, a program, a process, task or script. Executable code
725 may be executed by processor 705 possibly under control of
operating system 715. For example, executable code 725 may be or
include code for selecting a subset of digital images from a
plurality of digital images, according to embodiments of the
invention.
[0091] Storage 730 may be or may include, for example, a hard disk
drive, a floppy disk drive, a Compact Disk (CD) drive, a
CD-Recordable (CD-R) drive, a universal serial bus (USB) device or
other suitable removable and/or fixed storage unit. In some
embodiments, some of the components shown in FIG. 8 may be omitted.
For example, memory 720 may be a non-volatile memory having the
storage capacity of storage 730. Accordingly, although shown as a
separate component, storage 730 may be embedded or included in
memory 720. Storage 730 and or memory 720 may be configured to
store an electronic or digital image gallery including image files
732, including a digital image 733 and metadata 734, and any other
parameters required for performing embodiments of the
invention.
[0092] Input devices 735 may be or may include a camera 736 for
capturing images, a mouse, a keyboard, a touch screen or pad or any
suitable input device. It will be recognized that any suitable
number of input devices may be operatively connected to computing
device 700 as shown by block 735. Output devices 740 may include
one or more displays, speakers and/or any other suitable output
devices. It will be recognized that any suitable number of output
devices may be operatively connected to computing device 700 as
shown by block 740. Any applicable input/output (I/O) devices may
be connected to computing device 700 as shown by blocks 735 and
740. For example, a wired or wireless network interface card (NIC),
a modem, printer or facsimile machine, a universal serial bus (USB)
device or external hard drive may be included in input devices 735
and/or output devices 740. Network interface 750 may enable device
700 to communicate with one or more other computers or networks.
For example, network interface 750 may include a Wi-Fi or Bluetooth
device or connection, a connection to an intranet or the internet,
an antenna etc.
[0093] Embodiments described in this disclosure may include the use
of a special purpose or general-purpose computer including various
computer hardware or software modules, as discussed in greater
detail below.
[0094] Embodiments within the scope of this disclosure also include
computer-readable media, or non-transitory computer storage medium,
for carrying or having computer-executable instructions or data
structures stored thereon. The instructions when executed may cause
the processor to carry out embodiments of the invention. Such
computer-readable media, or computer storage medium, can be any
available media that can be accessed by a general purpose or
special purpose computer. By way of example, and not limitation,
such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM
or other optical disk storage, magnetic disk storage or other
magnetic storage devices, or any other medium which can be used to
carry or store desired program code means in the form of
computer-executable instructions or data structures and which can
be accessed by a general purpose or special purpose computer. When
information is transferred or provided over a network or another
communications connection (either hardwired, wireless, or a
combination of hardwired or wireless) to a computer, the computer
properly views the connection as a computer-readable medium. Thus,
any such connection is properly termed a computer-readable medium.
Combinations of the above should also be included within the scope
of computer-readable media.
[0095] Computer-executable instructions comprise, for example,
instructions and data which cause a general purpose computer,
special purpose computer, or special purpose processing device to
perform a certain function or group of functions. Although the
subject matter has been described in language specific to
structural features and/or methodological acts, it is to be
understood that the subject matter defined in the appended claims
is not necessarily limited to the specific features or acts
described above. Rather, the specific features and acts described
above are disclosed as example forms of implementing the
claims.
[0096] As used herein, the term "module" or "component" can refer
to software objects or routines that execute on the computing
system. The different components, modules, engines, and services
described herein may be implemented as objects or processes that
execute on the computing system (e.g., as separate threads). While
the system and methods described herein are preferably implemented
in software, implementations in hardware or a combination of
software and hardware are also possible and contemplated. In this
description, a "computer" may be any computing system as previously
defined herein, or any module or combination of modulates running
on a computing system.
[0097] For the processes and/or methods disclosed, the functions
performed in the processes and methods may be implemented in
differing order as may be indicated by context. Furthermore, the
outlined steps and operations are only provided as examples, and
some of the steps and operations may be optional, combined into
fewer steps and operations, or expanded into additional steps and
operations.
[0098] The present disclosure is not to be limited in terms of the
particular embodiments described in this application, which are
intended as illustrations of various aspects. Many modifications
and variations can be made without departing from its scope.
Functionally equivalent methods and apparatuses within the scope of
the disclosure, in addition to those enumerated, will be apparent
to those skilled in the art from the foregoing descriptions. Such
modifications and variations are intended to fall within the scope
of the appended claims. The present disclosure is to be limited
only by the terms of the appended claims, along with the full scope
of equivalents to which such claims are entitled. It is also to be
understood that the terminology used in this disclosure is for the
purpose of describing particular embodiments only, and is not
intended to be limiting.
[0099] This disclosure may sometimes illustrate different
components contained within, or connected with, different other
components. Such depicted architectures are merely exemplary, and
many other architectures can be implemented which achieve the same
or similar functionality.
[0100] Aspects of the present disclosure may be embodied in other
forms without departing from its spirit or essential
characteristics. The described aspects are to be considered in all
respects illustrative and not restrictive. The claimed subject
matter is indicated by the appended claims rather than by the
foregoing description. All changes which come within the meaning
and range of equivalency of the claims are to be embraced within
their scope.
* * * * *