U.S. patent application number 16/547905 was filed with the patent office on 2021-02-25 for data breach detection.
The applicant listed for this patent is INTERNATIONAL BUSINESS MACHINES CORPORATION. Invention is credited to PREETHI ANANTHARAMAN, NIKOLAOS ANEROUSIS, ERIC KEVIN BUTLER, DIVYESH JADAV, MU QIAO.
Application Number | 20210056203 16/547905 |
Document ID | / |
Family ID | 1000004347456 |
Filed Date | 2021-02-25 |
![](/patent/app/20210056203/US20210056203A1-20210225-D00000.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00001.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00002.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00003.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00004.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00005.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00006.png)
![](/patent/app/20210056203/US20210056203A1-20210225-D00007.png)
![](/patent/app/20210056203/US20210056203A1-20210225-M00001.png)
![](/patent/app/20210056203/US20210056203A1-20210225-M00002.png)
![](/patent/app/20210056203/US20210056203A1-20210225-M00003.png)
View All Diagrams
United States Patent
Application |
20210056203 |
Kind Code |
A1 |
QIAO; MU ; et al. |
February 25, 2021 |
DATA BREACH DETECTION
Abstract
Systems and methods are described for a data breach detection
based on snapshot analytics. The described systems and methods
identify a plurality of snapshots of a data structure, identify a
plurality of leaf nodes of the data structure for each of the
snapshots, generate a vector of data attributes for each of the
leaf nodes, assign a weight to each of the vectors to produce a set
of weighted vectors for each of the snapshots, compute a distance
metric between each pair of the snapshots based on the
corresponding sets of weighted vectors, and detect an abnormal
snapshot among the plurality of snapshots based on the distance
metrics.
Inventors: |
QIAO; MU; (BELMONT, CA)
; ANANTHARAMAN; PREETHI; (SAN JOSE, CA) ; BUTLER;
ERIC KEVIN; (SAN JOSE, CA) ; JADAV; DIVYESH;
(SAN JOSE, CA) ; ANEROUSIS; NIKOLAOS; (LOS GATOS,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
INTERNATIONAL BUSINESS MACHINES CORPORATION |
ARMONK |
NY |
US |
|
|
Family ID: |
1000004347456 |
Appl. No.: |
16/547905 |
Filed: |
August 22, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 16/27 20190101;
G06F 21/554 20130101; G06F 16/21 20190101; G06F 16/128 20190101;
G06F 21/566 20130101; G06F 16/9027 20190101 |
International
Class: |
G06F 21/55 20060101
G06F021/55; G06F 16/21 20060101 G06F016/21; G06F 16/27 20060101
G06F016/27; G06F 16/901 20060101 G06F016/901; G06F 16/11 20060101
G06F016/11; G06F 21/56 20060101 G06F021/56 |
Claims
1. A method for data breach detection, comprising: identifying a
plurality of snapshots of a data structure; identifying a plurality
of leaf nodes of the data structure for each of the snapshots;
generating a vector of data attributes for each of the leaf nodes;
assigning a weight to each of the vectors to produce a set of
weighted vectors for each of the snapshots; computing a distance
metric between each pair of the snapshots based on the
corresponding sets of weighted vectors; and detecting an abnormal
snapshot among the plurality of snapshots based on the distance
metrics.
2. The method of claim 1, wherein: each of the plurality of
snapshots corresponds to a backup of the data structure at a
different point in time.
3. The method of claim 1, further comprising: identifying a tree
structure on the data structure, wherein the leaf nodes of the data
structure correspond to a bottom level of the tree structure.
4. The method of claim 1 wherein: the tree structure corresponds to
a file directory of the data structure.
5. The method of claim 1, wherein: each of the vectors comprises
values corresponding to a path depth attribute, a file size, a file
count, a file extension attribute, a file modification attribute,
or any combination thereof.
6. The method of claim 1, wherein: the weight assigned to each of
the vectors corresponds to a number of files associated with a
corresponding leaf node.
7. The method of claim 1, wherein: the weight assigned to each of
the vectors corresponds to a cybersecurity risk associated with a
corresponding leaf node.
8. The method of claim 1, further comprising: computing a set of
connection weights, wherein each of the connection weights
comprises a first index corresponding to a first leaf node of a
first snapshot and a second index corresponding to a second leaf
node of a second snapshot, wherein a sum of connection weights
having the first index is equal to a weight assigned to the first
leaf node and a sum of connections weights having the second index
is equal to a weight assigned to the second leaf node and a sum of
the set of connection weights is equal to one; multiplying each of
the connection weights by a squared distance between a vector
corresponding to the first leaf node and a vector corresponding to
the second leaf node to produce a set of weighted distances; and
calculating the distance metrics based on a sum of the weighted
distances, wherein the set of connection weights is computed to
minimize the sum.
9. The method of claim 1, wherein: the distance metric comprises an
earth mover's distance, a Kantorovich-Mallows distance, a
Wasserstein distance, or any combination thereof.
10. The method of claim 1, further comprising: computing a local
reachability density for each of the snapshots based on the
computed distance metrics; and determining whether the local
reachability density for each of the snapshots is below a threshold
based on neighboring snapshots, wherein the abnormal snapshot is
identified based on the determination.
11. An apparatus for data breach detection method in which
snapshots of backup files are taken and compared to the surrounding
files, comprising: a processor and a memory storing instructions
and in electronic communication with the processor, the processor
being configured to execute the instructions to: identify a
plurality of snapshots of a data structure; identify a plurality of
leaf nodes of the data structure for each of the snapshots;
generate a vector of data attributes for each of the leaf nodes;
assign a weight to each of the vectors to produce a set of weighted
vectors for each of the snapshots; compute a distance metric
between each pair of the snapshots based on the corresponding sets
of weighted vectors; and detect an abnormal snapshot among the
plurality of snapshots based on the distance metrics.
12. The apparatus of claim 11 the processor being further
configured to execute the instructions to: identify a tree
structure on the data structure, wherein the leaf nodes of the data
structure correspond to a bottom level of the tree structure.
13. The apparatus of claim 11, the processor being further
configured to execute the instructions to: compute a set of
connection weights, wherein each of the connection weights
comprises a first index corresponding to a first leaf node of a
first snapshot and a second index corresponding to a second leaf
node of a second snapshot, wherein a sum of connection weights
having the first index is equal to a weight assigned to the first
leaf node and a sum of connections weights having the second index
is equal to a weight assigned to the second leaf node and a sum of
the set of connection weights is equal to one; multiply each of the
connection weights by a squared distance between a vector
corresponding to the first leaf node and a vector corresponding to
the second leaf node to produce a set of weighted distances; and
calculate the distance metrics based on a sum of the weighted
distances, wherein the set of connection weights is computed to
minimize the sum.
14. The apparatus of claim 11, the processor being further
configured to execute the instructions to: compute a local
reachability density for each of the snapshots based on the
computed distance metrics; and determine whether the local
reachability density for each of the snapshots is below a threshold
based on neighboring snapshots, wherein the abnormal snapshot is
identified based on the determination.
15. A non-transitory computer readable medium storing code for data
breach detection method in which snapshots of backup files are
taken and compared to the surrounding files, the code comprising
instructions executable by a processor to: identify a plurality of
snapshots of a data structure; identify a plurality of leaf nodes
of the data structure for each of the snapshots; generate a vector
of data attributes for each of the leaf nodes; assign a weight to
each of the vectors to produce a set of weighted vectors for each
of the snapshots; compute a distance metric between each pair of
the snapshots based on the corresponding sets of weighted vectors;
and detect an abnormal snapshot among the plurality of snapshots
based on the distance metrics.
16. The non-transitory computer readable medium of claim 15,
wherein: each of the plurality of snapshots corresponds to a backup
of the data structure at a different point in time.
17. The non-transitory computer readable medium of claim 15, the
code further comprising instructions executable by the processor
to: identify a tree structure on the data structure, wherein the
leaf nodes of the data structure correspond to a bottom level of
the tree structure.
18. The non-transitory computer readable medium of claim 15, the
code further comprising instructions executable by the processor
to: compute a set of connection weights, wherein each of the
connection weights comprises a first index corresponding to a first
leaf node of a first snapshot and a second index corresponding to a
second leaf node of a second snapshot, wherein a sum of connection
weights having the first index is equal to a weight assigned to the
first leaf node and a sum of connections weights having the second
index is equal to a weight assigned to the second leaf node and a
sum of the set of connection weights is equal to one; multiply each
of the connection weights by a squared distance between a vector
corresponding to the first leaf node and a vector corresponding to
the second leaf node to produce a set of weighted distances; and
calculate the distance metrics based on a sum of the weighted
distances, wherein the set of connection weights is computed to
minimize the sum.
19. The non-transitory computer readable medium of claim 15, the
code further comprising instructions executable by the processor
to: compute a local reachability density for each of the snapshots
based on the computed distance metrics; and determine whether the
local reachability density for each of the snapshots is below a
threshold based on neighboring snapshots, wherein the abnormal
snapshot is identified based on the determination.
Description
BACKGROUND
[0001] The following relates generally to data breach detection,
and more specifically to data breach detection based on snapshot
analytics.
[0002] As individuals and corporate entities become more reliant on
data, they also become vulnerable to attacks that target that data.
Cyber resiliency is the ability of an individual or organization to
continue to function with the least amount of disruption in the
face of cyber-attacks. Cyber resiliency may include information
security, business continuity, and network resilience of
enterprises to ensure organizations continue to function during
cyber-attacks and cyber outages.
[0003] In some cases, cyber attacks may be conducted in a way that
makes them hard to detect. Therefore, there is a need in the art
for systems and methods that provide improved ability to detect
sophisticated cyber attacks.
SUMMARY
[0004] A method, apparatus, and non-transitory computer readable
medium for data breach detection based on snapshot analytics are
described. The method, apparatus, and non-transitory computer
readable medium may provide for identifying a plurality of
snapshots of a data structure, identifying a plurality of leaf
nodes of the data structure for each of the snapshots, generating a
vector of data attributes for each of the leaf nodes, assigning a
weight to each of the vectors to produce a set of weighted vectors
for each of the snapshots, computing a distance metric between each
pair of the snapshots based on the corresponding sets of weighted
vectors, and detecting an abnormal snapshot among the plurality of
snapshots based on the distance metrics.
BRIEF DESCRIPTION OF THE DRAWINGS
[0005] FIG. 1 shows an example of an abnormal snapshot detection
system in accordance with aspects of the present disclosure.
[0006] FIG. 2 shows an example of a server in accordance with
aspects of the present disclosure.
[0007] FIG. 3 shows an example of a series of snapshots in
accordance with aspects of the present disclosure.
[0008] FIG. 4 shows an example of a data structure in accordance
with aspects of the present disclosure.
[0009] FIGS. 5 through 7 show examples of a process for data breach
detection method in which snapshots of backup files are taken and
compared to the surrounding files in accordance with aspects of the
present disclosure.
DETAILED DESCRIPTION
[0010] The present disclosure provides systems and methods for data
breach detection. Embodiments of the disclosure relate to a data
breach detection method in which snapshots of backup files are
taken and compared to the surrounding files to detect any
abnormalities in the structure of the files outside normal patterns
of changes.
[0011] Embodiments of the present disclosure generate a discrete
distribution of file attributes in backup snapshots (i.e., the
snapshot signature) using a weighted tree structure, which consists
of a set of weighted vectors. Then a distance metric, measured
between two sets of weighted vectors, is calculated between two
snapshot signatures. An abnormal snapshot is detected by comparing
the local density of a snapshot with the densities of the snapshots
of neighboring data sets.
[0012] Cyber-attacks are on the rise, with phishing and social
engineering, malware, spear phishing, denial of service, and
out-of-date software being the top cyber disruptions threatening
organizations today. Key cyber resiliency concerns include: lacking
the ability to remain resilient in the wake of a cyber attach,
insufficient planning and preparedness being top barriers to cyber
resilience, and ad-hoc, non-existent, or inconsistent cyber
security incident response plans.
[0013] With cyber-attacks becoming increasingly prevalent and
costly, cyber resiliency challenges are becoming more difficult.
Cyber espionage and ransomware are becoming common, with advanced,
targeted, and persistent threats focused on many organizations. The
attack landscape is constantly expanding with network and
infrastructure becoming more complex.
[0014] Attacks such as malware corrupting backup files cause
numerous issues, and recovery processes take significant time
(especially when manual operations are required to resolve the
issues). Additionally, a growing number of regulations impose
requirements on organizations that demand costly compliance
procedures. For example, data breach notification requirements
contribute to the challenges in the digital world.
[0015] A variety of data breach detection methods may be used to
determine whether data has been compromised. These detection
methods may include techniques used to detect abnormal backup
snapshots, which can suggest a cyber threat or breach of data
security. For example, a number of data breach detection methods
use single-vector anomaly detection methods. Density-based anomaly
detection methods use vector proximity to determine statistical
outliers. These methods compare the density around a point with the
density around the local neighbors of that point. The methods
assume the density around a normal data object has a similar
density to neighboring data objects.
[0016] Clustering-based anomaly detection methods arrange similar
objects into groups. These methods assume that anomalous instances
either lie in sparse areas, small clusters, far from the center of
main clusters, or that they are not assigned to a cluster at all.
Support vector machine based anomaly detection methods can provide
a non-linear classification of data through a kernel function.
[0017] However, in many cases, the assumptions of these detection
methods may not apply to sophisticated hacking attempts that make
complex changes to data. Therefore, the present disclosure provides
systems and methods for detecting a data breech that is capable of
detecting sophisticated hacking attempts.
[0018] FIG. 1 shows an example of an abnormal snapshot detection
system 100 in accordance with aspects of the present disclosure.
The example shown includes server 105, database 110, network 115,
and attacker terminal 120.
[0019] Server 100 may communicate with database 110 via network 115
to provide a data breach detection service. For example, server 100
may take periodic snapshots of database 110 in order to detect
abnormal changes in the data. If the attacker terminal 120 gains
access to the database (e.g., via network 115), it may change some
of the data in the database 110. Then one of the backup snapshots
taken by server 100 may be recognized as abnormal and a response my
be initiated to mitigate the effects of the attack.
[0020] FIG. 2 shows an example of a server 200 in accordance with
aspects of the present disclosure. Server 200 may be an example of,
or include aspects of, the corresponding element or elements
described with reference to FIG. 1. Server 200 may include
processor unit 205, memory unit 210, snapshot component 215, data
structure component 220, vector component 225, weight component
230, distance component 235, and detection component 240.
[0021] A processor unit 205 may include an intelligent hardware
device, (e.g., a general-purpose processing component, a digital
signal processor (DSP), a central processing unit (CPU), a graphics
processing unit (CPU), a microcontroller, an application specific
integrated circuit (ASIC), a field programmable gate array (FPGA),
a programmable logic device, a discrete gate or transistor logic
component, a discrete hardware component, or any combination
thereof). In some cases, the processor may be configured to operate
a memory array using a memory controller. In other cases, a memory
controller may be integrated into processor. The processor may be
configured to execute computer-readable instructions stored in a
memory to perform various functions.
[0022] A memory unit 210 may store information for various programs
and applications on a computing device. For example, the storage
may include data for running an operating system. The memory may
include both volatile memory and non-volatile memory. Volatile
memory may random access memory (RAM), and non-volatile memory may
include read-only memory (ROM), flash memory, electrically erasable
programmable read-only memory (EEPROM), digital tape, a hard disk
drive (HDD), and a solid state drive (SSD).
[0023] Snapshot component 215 may identify a set of snapshots of a
data structure. In some examples, each of the set of snapshots
corresponds to a backup of the data structure at a different point
in time.
[0024] Data structure component 220 may identify a data structure
of each snapshot, including a set of leaf nodes. For example, data
structure component 220 may identify a tree structure where the
leaf nodes of the data structure correspond to a bottom level of
the tree structure. In some examples, the tree structure
corresponds to a file directory of the data structure.
[0025] Vector component 225 may generate a vector of data
attributes for each of the leaf nodes. In some examples, each of
the vectors includes values corresponding to a path depth
attribute, a file size, a file count, a file extension attribute, a
file modification attribute, or any combination thereof.
[0026] Weight component 230 may assign a weight to each of the
vectors to produce a set of weighted vectors for each of the
snapshots. In some examples, the weight assigned to each of the
vectors corresponds to a number of files associated with a
corresponding leaf node. In some examples, the weight assigned to
each of the vectors corresponds to a cybersecurity risk associated
with a corresponding leaf node.
[0027] Distance component 235 may compute a distance metric between
each pair of the snapshots based on the corresponding sets of
weighted vectors. For example, distance component 235 may compute a
set of connection weights, where each of the connection weights
includes a first index corresponding to a first leaf node of a
first snapshot and a second index corresponding to a second leaf
node of a second snapshot, where a sum of connection weights having
the first index is equal to a weight assigned to the first leaf
node and a sum of connections weights having the second index is
equal to a weight assigned to the second leaf node and a sum of the
set of connection weights is equal to one.
[0028] The distance component 235 may then multiply each of the
connection weights by a squared distance between a vector
corresponding to the first leaf node and a vector corresponding to
the second leaf node to produce a set of weighted distances.
Distance component 235 may calculate the distance metrics based on
a sum of the weighted distances, where the set of connection
weights is computed to minimize the sum. In some examples, the
distance metric includes an earth mover's distance, a
Kantorovich-Mallows distance, a Wasserstein distance, or any
combination thereof.
[0029] Detection component 240 detects an abnormal snapshot among
the set of snapshots based on the distance metrics. Detection
component 240 may also compute a local reachability density for
each of the snapshots based on the computed distance metrics.
Detection component 240 may also determine whether the local
reachability density for each of the snapshots is below a threshold
based on neighboring snapshots, where the abnormal snapshot is
identified based on the determination.
[0030] FIG. 3 shows an example of a series of snapshots 300 in
accordance with aspects of the present disclosure. The example
shown includes snapshots 300 and an abnormal snapshot 305. Each
snapshot 300 captures the state of a database at a particular
moment in time.
[0031] By comparing the different snapshots 300, it is possible to
detect an abnormal snapshot 305 using the systems and methods
described herein. When the abnormal snapshot 305 is detected, the
system may raise an alert notifying the user that there may have
been a data breach.
[0032] FIG. 4 shows an example of a data structure in accordance
with aspects of the present disclosure. In the example shown, the
data structure corresponds to a directory tree 400, which may
include one or more nodes 405 and leaf nodes 410. Each node 405 may
be associated with one or more children nodes. The bottom level of
the directory tree 400 includes the leaf nodes 410.
[0033] According to embodiments of the present disclosure, each
leaf node may be associated with a vector that includes various
attributes, such as a path depth attribute, a file size, a file
count, a file extension attribute, a file modification attribute,
or any combination thereof. These vectors change with each
snapshot, and the combination of the vectors represents a signature
of the snapshot of the data structure at a particular moment in
time). By comparing the signatures for statistical abnormalities,
an abnormal snapshot may be detected. Specifically, embodiments of
the present disclosure describe a system for detecting an abnormal
snapshot based on comparing snapshot signatures that includes
multiple vectors corresponding to the leaf nodes.
[0034] Each leaf node may also be associated with a weight based on
the number of files it includes (or a normalized number of files, a
size of the files, or a similar value). In other examples, the leaf
node 410 can be assigned weights by a user to represent the
likelihood that the node will be targeted by a cyber attack. That
is, certain directories may be more prone to ransomware attacks,
therefore may be more heavily weighted.
[0035] FIG. 5 shows an example of a process for data breach
detection method in which snapshots of backup files are taken and
compared to the surrounding files in accordance with aspects of the
present disclosure. In some examples, these operations may be
performed by a processor executing a set of codes to control
functional elements of an apparatus. Additionally or alternatively,
the processes may be performed using special-purpose hardware.
Generally, these operations may be performed according to the
methods and processes described in accordance with aspects of the
present disclosure. For example, the operations may be composed of
various substeps, or may be performed in conjunction with other
operations described herein.
[0036] At step 500, the system identifies a set of snapshots of a
data structure. In some cases, the operations of this step may
refer to, or be performed by, a snapshot component as described
with reference to FIG. 2. Each of the set of snapshots may
correspond to a backup of the data structure at a different point
in time.
[0037] At step 505, the system identifies a set of leaf nodes of
the data structure for each of the snapshots. In some cases, the
operations of this step may refer to, or be performed by, a data
structure component as described with reference to FIG. 2. In some
embodiments, the system identifies a tree structure on the data
structure, wherein the leaf nodes of the data structure correspond
to a bottom level of the tree structure. The tree structure may
correspond to a file directory of the data structure.
[0038] At step 510, the system generates a vector of data
attributes for each of the leaf nodes. In some cases, the
operations of this step may refer to, or be performed by, a vector
component as described with reference to FIG. 2. Each of the
vectors may comprise values corresponding to a path depth
attribute, a file size, a file count, a file extension attribute, a
file modification attribute, or any combination thereof. In some
cases, categorical values may be converted into numerical or binary
values (e.g., using one hot encoding).
[0039] At step 515, the system assigns a weight to each of the
vectors to produce a set of weighted vectors for each of the
snapshots. In some cases, the operations of this step may refer to,
or be performed by, a weight component as described with reference
to FIG. 2. The weight assigned to each of the vectors may
correspond to several files associated with a corresponding leaf
node. Additionally or alternatively, the weight assigned to each of
the vectors may also correspond to a cybersecurity risk associated
with a corresponding leaf node.
[0040] In some embodiments, the system computes a set of connection
weights, wherein each of the connection weights comprises a first
index corresponding to a first leaf node of a first snapshot and a
second index corresponding to a second leaf node of a second
snapshot, wherein a sum of connection weights having the first
index is equal to a weight assigned to the first leaf node and a
sum of connections weights having the second index is equal to a
weight assigned to the second leaf node and a sum of the set of
connection weights is equal to one. Additionally, the first leaf
node and a vector corresponding to the second leaf node to produce
a set of weighted distances, and the system then calculates the
distance metric based on a sum of the weighted distances, wherein
the set of connection weights is computed to minimize the sum.
[0041] At step 520, the system computes a distance metric between
each pair of the snapshots based on the corresponding sets of
weighted vectors. In some cases, the operations of this step may
refer to, or be performed by, a distance component as described
with reference to FIG. 2. The distance metric may comprise an earth
mover's distance, a Kantorovich-Mallows distance, a Wasserstein
distance, or any combination thereof.
[0042] At step 525, the system detects an abnormal snapshot among
the set of snapshots based on the distance metrics. In some cases,
the operations of this step may refer to, or be performed by, a
detection component as described with reference to FIG. 2. For
example, the system may compute a local reachability density for
each of the snapshots based on the computed distance metrics; and
determines whether the local density for each of the snapshots is
below a threshold based on neighboring snapshots, wherein the
abnormal snapshot is identified based on the determination.
[0043] FIG. 6 shows an example of a process for data breach
detection method in which snapshots of backup files are taken and
compared to the surrounding files in accordance with aspects of the
present disclosure. In some examples, these operations may be
performed by a processor executing a set of codes to control
functional elements of an apparatus. Additionally or alternatively,
the processes may be performed using special-purpose hardware.
Generally, these operations may be performed according to the
methods and processes described in accordance with aspects of the
present disclosure. For example, the operations may be composed of
various substeps, or may be performed in conjunction with other
operations described herein.
[0044] At step 600, the system identifies a set of snapshots of a
data structure. In some cases, the operations of this step may
refer to, or be performed by, a snapshot component as described
with reference to FIG. 2. This system provides a method or system
for an improved detection of abnormal backup snapshots based on
discrete distribution data, which capture statistical distribution
of the change in user data.
[0045] At step 605, the system identifies a set of leaf nodes of
the data structure for each of the snapshots. In some cases, the
operations of this step may refer to, or be performed by, a data
structure component as described with reference to FIG. 2. For
example, in some cases, the system may identify a set of leaf nodes
for each snapshot, over a specified time period. In some cases, the
system may identify multiple sets of snapshots containing multiple
leaf nodes.
[0046] At step 610, the system generates a vector of data
attributes for each of the leaf nodes. In some cases, the
operations of this step may refer to, or be performed by, a vector
component as described with reference to FIG. 2. This generates a
weighted tree structure consisting of a set of weighted
vectors.
[0047] At step 615, the system assigns a weight to each of the
vectors to produce a set of weighted vectors for each of the
snapshots. In some cases, the operations of this step may refer to,
or be performed by, a weight component as described with reference
to FIG. 2. The node weights can be assigned by the user, as
needed.
[0048] A variety of methods may be used to determine leaf node
weights. In a first example i.e., Type I), several files at each
unique file path may be used. This may result in a vector such as
[10, 5, 15, 4, 16, 20, 12, 8, 3, 7]. In a second example Type II),
the weight may represent a cybersecurity risk value. This may
result in a vector such as [0.1, 0.05, 0.15, 0.04, 0.16, 0.2, 0.12,
0.08, 0.03, 0.07]. In some examples, a Type II weighting scheme may
result in a normalized set of weights (i.e., where the weights sum
to 1).
[0049] At step 620, the system computes a set of connection
weights, where each of the connection weights includes a first
index corresponding to a first leaf node of a first snapshot and a
second index corresponding to a second leaf node of a second
snapshot, where a sum of connection weights having the first index
is equal to a weight assigned to the first leaf node and a sum of
connections weights having the second index is equal to a weight
assigned to the second leaf node and a sum of the set of connection
weights is equal to one. In some cases, the operations of this step
may refer to, or be performed by, a distance component as described
with reference to FIG. 2.
[0050] The first leaf node and a vector corresponding to the second
leaf node produces a set of weighted distances and calculates the
distance metric based on a sum of the weighted distances, and the
set of connection weights may be computed to minimize the sum.
[0051] At step 625, the system multiplies each of the connection
weights by a squared distance between a vector corresponding to the
first leaf node and a vector corresponding to the second leaf node
to produce a set of weighted distances. In some cases, the
operations of this step may refer to, or be performed by, a
distance component as described with reference to FIG. 2.
[0052] At step 630, the system calculates the distance metrics
based on a sum of the weighted distances, where the set of
connection weights is computed to minimize the sum. In some cases,
the operations of this step may refer to, or be performed by, a
distance component as described with reference to FIG. 2. The
distance metric, i.e., Mallow/Wasserstein distance, is calculated
using two snapshot signatures; also referred to as two sets of
weighted vectors.
[0053] At step 635, the system detects an abnormal snapshot among
the set of snapshots based on the distance metrics. In some cases,
the operations of this step may refer to, or be performed by, a
detection component as described with reference to FIG. 2. Abnormal
snapshots are detected by comparing a local density of a snapshot
with the density of its neighbors.
[0054] When the node weights are not normalized (i.e., Type I), the
absolute numbers of files under each file path, the distance
between two signatures may be defined in terms of an optimal flow
described by F=(.omega..sub.i,j):
W ( .gamma. 1 , .gamma. 2 , F ) = i = 1 m 1 j = 1 m 2 .omega. i , j
z 1 ( i ) - z 2 ( j ) 2 ( 1 ) ##EQU00001##
[0055] Which is subject to:
j = 1 m 2 .omega. i , j .ltoreq. q 1 ( i ) , i = 1 , , m 1 ; ( 2 )
i = 1 m 1 .omega. i , j .ltoreq. q 2 ( j ) , j = 1 , , m 2 ; ( 3 )
i = 1 m 1 j = 1 m 2 .omega. i , j = min ( i = 1 m 1 q 1 ( i ) , j =
1 m 2 q 2 ( j ) ) ( 4 ) .omega. i , j .gtoreq. 0 , i = 1 , , m 1 ,
j = 1 , , m 2 ( 5 ) ##EQU00002##
[0056] Once the optimal w*.sub.i,j is found, the squared distance
between .gamma..sub.1 and .gamma..sub.2 is defined as:
D 2 ( .gamma. 1 , .gamma. 2 ) = i = 1 m 1 j = 1 m 2 w i , j z 1 ( i
) - z 2 ( j ) 2 i = 1 m 1 j = 1 m 2 w i , j ( 6 ) ##EQU00003##
[0057] .gamma..sub.1, .gamma..sub.2 are the backup snapshot
signatures, z.sub.1.sup.(i) and z.sub.2.sup.(j) are the
representing vectors for leaf node i and j, q.sub.1.sup.(i) and
q.sub.2.sup.(j) are the weights, m.sub.1 and m.sub.2 are the number
of leaf nodes. The above distance is referred to as the Earth
Mover's Distance (EMD).
[0058] When type II is used, the normalized values, which are
computed by the number of files in that node divided by the total
number of files. The squared distance between .gamma..sub.1 and
.gamma..sub.2 is defined as:
D 2 ( .gamma. 1 , .gamma. 2 ) = min { .omega. i , j } i = 1 m 1 j =
1 m 2 w i , j z 1 ( i ) - z 2 ( j ) 2 ( 7 ) ##EQU00004##
[0059] This equation is subject to the constraint:
j = 1 m 2 w i , j .ltoreq. q 1 ( i ) , i = 1 , , m 1 ; ( 8 ) i = 1
m 1 w i , j .ltoreq. q 2 ( j ) , j = 1 , , m 2 ; ( 9 ) i = 1 m 1 j
= 1 m 2 .omega. i , j = i = 1 m 1 q 1 ( i ) = j = 1 m 2 q 2 ( j ) =
1 ( 10 ) .omega. i , j .gtoreq. 0 , i = 1 , , m 1 , j = 1 , , m 2 (
11 ) ##EQU00005##
[0060] The above distance is referred to as the Kantorovich-Mallows
distance. Computing the distance is optimizing matching weights
between support vectors in the two distributions such that the
aggregated distance is minimized. The matching weights
.omega..sub.i,j are restricted to be non-negative and the weights
emitting from any vector z.sub.i.sup.(j) sum up to its probability
q.sub.i.sup.(j). Thus q.sub.i.sup.(j) sets the amount of influence
from z.sub.i.sup.(j) on the overall distribution distance.
[0061] Suppose D.sub.1.sup.2(.gamma..sub.1, .gamma..sub.2) is the
mallows distance between signatures .gamma..sub.1 and .gamma..sub.2
using the percentage of file number as weight, and
D.sub.2.sup.2(.gamma..sub.1, .gamma..sub.2) is the mallows distance
between signatures .gamma..sub.1 and .gamma..sub.2 using the
percentage of file size as weight, the final distance {tilde over
(D)}(.gamma..sub.1, .gamma..sub.2)=D.sub.1.sup.2+D.sub.2.sup.2.
[0062] FIG. 7 shows an example of a process for data breach
detection method in which snapshots of backup files are taken and
compared to the surrounding files in accordance with aspects of the
present disclosure. In some examples, these operations may be
performed by a processor executing a set of codes to control
functional elements of an apparatus. Additionally or alternatively,
the processes may be performed using special-purpose hardware.
Generally, these operations may be performed according to the
methods and processes described in accordance with aspects of the
present disclosure. For example, the operations may be composed of
various substeps, or may be performed in conjunction with other
operations described herein.
[0063] At step 700, the system identifies a set of snapshots of a
data structure. In some cases, the operations of this step may
refer to, or be performed by, a snapshot component as described
with reference to FIG. 2.
[0064] At step 705, the system identifies a set of leaf nodes of
the data structure for each of the snapshots. In some cases, the
operations of this step may refer to, or be performed by, a data
structure component as described with reference to FIG. 2.
[0065] At step 710, the system generates a vector of data
attributes for each of the leaf nodes. In some cases, the
operations of this step may refer to, or be performed by, a vector
component as described with reference to FIG. 2.
[0066] At step 715, the system assigns a weight to each of the
vectors to produce a set of weighted vectors for each of the
snapshots. In some cases, the operations of this step may refer to,
or be performed by, a weight component as described with reference
to FIG. 2.
[0067] At step 720, the system computes a distance metric between
each pair of the snapshots based on the corresponding sets of
weighted vectors. In some cases, the operations of this step may
refer to, or be performed by, a distance component as described
with reference to FIG. 2.
[0068] At step 725, the system computes a local reachability
density for each of the snapshots based on the computed distance
metrics. In some cases, the operations of this step may refer to,
or be performed by, a detection component as described with
reference to FIG. 2.
[0069] At step 730, the system determines whether the local
reachability density for each of the snapshots comparing with its
neighbors is below a threshold, where the abnormal snapshot is
identified based on the determination. In some cases, the
operations of this step may refer to, or be performed by, a
detection component as described with reference to FIG. 2.
[0070] At step 735, the system detects an abnormal snapshot among
the set of snapshots based on the local reachability density. In
some cases, the operations of this step may refer to, or be
performed by, a detection component as described with reference to
FIG. 2.
[0071] The following provides an example algorithm for identifying
an anomaly using a local reachability density. The term "distance"
refers to the previously defined distance, i.e. {tilde over (D)}.
The anomaly detection algorithm is as follows. Let k-distance (A)
be the distance snapshot A to the k-th nearest neighbor. Denote the
set of k nearest neighbors of A as N.sub.k(A). Define the
reachability distance between snapshot A and snapshot B as
reachability-distance.sub.k (A, B)=max{k-distance (B), {tilde over
(D)}(A, B)}. The local reachability density of a snapshot A is
defined by:
lrd k ( A ) = 1 / ( B .di-elect cons. N k ( A ) reachability -
distance k ( A , B ) N k ( A ) ) , ( 12 ) ##EQU00006##
which is the inverse of the average reachability distance of the
snapshot A from its neighbors. The local reachability densities are
then compared with those of the neighbors using:
LOF k ( A ) = B .di-elect cons. N k ( A ) lrd ( B ) N k ( A ) / lrd
( A ) , ( 13 ) ##EQU00007##
which is the average local reachability density of the neighbors
divided by the snapshot's own local reachability density. A value
of approximately 1.0 indicates that the signature is comparable to
its neighbors, and thus not an outlier. A value below 1.0 indicates
a denser region, which can be an inlier, while values significantly
larger than 1.0 indicate outliers.
[0072] Accordingly, the present disclosure includes the following
embodiments.
[0073] A method for data breach detection based on snapshot
analytics is described. The method may include identifying a
plurality of snapshots of a data structure, identifying a plurality
of leaf nodes of the data structure for each of the snapshots,
generating a vector of data attributes for each of the leaf nodes,
assigning a weight to each of the vectors to produce a set of
weighted vectors for each of the snapshots, computing a distance
metric between each pair of the snapshots based on the
corresponding sets of weighted vectors, and detecting an abnormal
snapshot among the plurality of snapshots based on the distance
metrics.
[0074] An apparatus for data breach detection method in which
snapshots of backup files are taken and compared to the surrounding
files is described. The apparatus may include a processor, memory
in electronic communication with the processor, and instructions
stored in the memory. The instructions may be operable to cause the
processor to identify a plurality of snapshots of a data structure,
identify a plurality of leaf nodes of the data structure for each
of the snapshots, generate a vector of data attributes for each of
the leaf nodes, assign a weight to each of the vectors to produce a
set of weighted vectors for each of the snapshots, compute a
distance metric between each pair of the snapshots based on the
corresponding sets of weighted vectors, and detect an abnormal
snapshot among the plurality of snapshots based on the distance
metrics.
[0075] A non-transitory computer readable medium storing code for
data breach detection method in which snapshots of backup files are
taken and compared to the surrounding files is described. In some
examples, the code comprises instructions executable by a processor
to: identify a plurality of snapshots of a data structure, identify
a plurality of leaf nodes of the data structure for each of the
snapshots, generate a vector of data attributes for each of the
leaf nodes, assign a weight to each of the vectors to produce a set
of weighted vectors for each of the snapshots, compute a distance
metric between each pair of the snapshots based on the
corresponding sets of weighted vectors, and detect an abnormal
snapshot among the plurality of snapshots based on the distance
metrics.
[0076] In some examples, each of the plurality of snapshots
corresponds to a backup of the data structure at a different point
in time. Some examples of the method, apparatus, and non-transitory
computer readable medium described above may further include
identifying a tree structure on the data structure, wherein the
leaf nodes of the data structure correspond to a bottom level of
the tree structure.
[0077] In some examples, the tree structure corresponds to a file
directory of the data structure. In some examples, each of the
vectors comprises values corresponding to a path depth attribute, a
file size, a file count, a file extension attribute, a file
modification attribute, or any combination thereof. In some
examples, the weight assigned to each of the vectors corresponds to
a number of files associated with a corresponding leaf node. In
some examples, the weight assigned to each of the vectors
corresponds to a cybersecurity risk associated with a corresponding
leaf node.
[0078] Some examples of the method, apparatus, and non-transitory
computer readable medium described above may further include
computing a set of connection weights, wherein each of the
connection weights comprises a first index corresponding to a first
leaf node of a first snapshot and a second index corresponding to a
second leaf node of a second snapshot, wherein a sum of connection
weights having the first index is equal to a weight assigned to the
first leaf node and a sum of connections weights having the second
index is equal to a weight assigned to the second leaf node and a
sum of the set of connection weights is equal to one. Some examples
may further include multiplying each of the connection weights by a
squared distance between a vector corresponding to the first leaf
node and a vector corresponding to the second leaf node to produce
a set of weighted distances. Some examples may further include
calculating the distance metrics based on a sum of the weighted
distances, wherein the set of connection weights is computed to
minimize the sum. In some examples, the distance metric comprises
an earth mover's distance, a Kantorovich-Mallows distance, a
Wasserstein distance, or any combination thereof.
[0079] Some examples of the method, apparatus, and non-transitory
computer readable medium described above may further include
computing a local reachability density for each of the snapshots
based on the computed distance metrics. Some examples may further
include determining whether the local reachability density for each
of the snapshots is below a threshold based on neighboring
snapshots, wherein the abnormal snapshot is identified based on the
determination.
[0080] The description and drawings described herein represent
example configurations and do not represent all the implementations
within the scope of the claims. For example, the operations and
steps may be rearranged, combined or otherwise modified. Also,
structures and devices may be represented in the form of block
diagrams to represent the relationship between components and avoid
obscuring the described concepts. Similar components or features
may have the same name but may have different reference numbers
corresponding to different figures.
[0081] Some modifications to the disclosure may be readily apparent
to those skilled in the art, and the principles defined herein may
be applied to other variations without departing from the scope of
the disclosure. Thus, the disclosure is not limited to the examples
and designs described herein, but is to be accorded the broadest
scope consistent with the principles and novel features disclosed
herein.
[0082] The described methods may be implemented or performed by
devices that include a general-purpose processor, a digital signal
processor (DSP), an application specific integrated circuit (ASIC),
a field programmable gate array (FPGA) or other programmable logic
device, discrete gate or transistor logic, discrete hardware
components, or any combination thereof. A general-purpose processor
may be a microprocessor, a conventional processor, controller,
microcontroller, or state machine. A processor may also be
implemented as a combination of computing devices (e.g., a
combination of a DSP and a microprocessor, multiple
microprocessors, one or more microprocessors in conjunction with a
DSP core, or any other such configuration). Thus, the functions
described herein may be implemented in hardware or software and may
be executed by a processor, firmware, or any combination thereof.
If implemented in software executed by a processor, the functions
may be stored in the form of instructions or code on a
computer-readable medium.
[0083] Computer-readable media includes both non-transitory
computer storage media and communication media including any medium
that facilitates transfer of code or data. A non-transitory storage
medium may be any available medium that can be accessed by a
computer. For example, non-transitory computer-readable media can
comprise random access memory (RAM), read-only memory (ROM),
electrically erasable programmable read-only memory (EEPROM),
compact disk (CD) or other optical disk storage, magnetic disk
storage, or any other non-transitory medium for carrying or storing
data or code.
[0084] Also, connecting components may be properly termed
computer-readable media. For example, if code or data is
transmitted from a website, server, or other remote source using a
coaxial cable, fiber optic cable, twisted pair, digital subscriber
line (DSL), or wireless technology such as infrared, radio, or
microwave signals, then the coaxial cable, fiber optic cable,
twisted pair, DSL, or wireless technology are included in the
definition of medium. Combinations of media are also included
within the scope of computer-readable media.
[0085] In this disclosure and the following claims, the word "or"
indicates an inclusive list such that, for example, the list of X,
Y, or Z means X or Y or Z or XY or XZ or YZ or XYZ. Also the phrase
"based on" is not used to represent a closed set of conditions. For
example, a step that is described as "based on condition A" may be
based on both condition A and condition B. In other words, the
phrase "based on" shall be construed to mean "based at least in
part on."
* * * * *