U.S. patent application number 13/722647 was filed with the patent office on 2013-08-08 for methods for generating a unified virtual snapshot and systems thereof.
This patent application is currently assigned to F5 NETWORKS, INC.. The applicant listed for this patent is F5 NETWORKS, INC.. Invention is credited to Jonathan Case Nicklin, Harald Skardal.
Application Number | 20130204893 13/722647 |
Document ID | / |
Family ID | 47428018 |
Filed Date | 2013-08-08 |
United States Patent
Application |
20130204893 |
Kind Code |
A1 |
Nicklin; Jonathan Case ; et
al. |
August 8, 2013 |
METHODS FOR GENERATING A UNIFIED VIRTUAL SNAPSHOT AND SYSTEMS
THEREOF
Abstract
A method, computer readable medium, and system for generating a
unified virtual snapshot in accordance with embodiments of the
present invention includes invoking with a file virtualization
system a capture of a plurality of physical snapshots. Each of the
physical snapshots comprises content at a given point in time in
one of the plurality of data storage systems. A unified virtual
snapshot is generated with the file virtualization system based on
the captured plurality of the physical snapshots.
Inventors: |
Nicklin; Jonathan Case;
(Newburgh, MA) ; Skardal; Harald; (Nashua,
NH) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
F5 NETWORKS, INC.; |
Seattle |
WA |
US |
|
|
Assignee: |
F5 NETWORKS, INC.
Seattle
WA
|
Family ID: |
47428018 |
Appl. No.: |
13/722647 |
Filed: |
December 20, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12334281 |
Dec 12, 2008 |
8352785 |
|
|
13722647 |
|
|
|
|
61013539 |
Dec 13, 2007 |
|
|
|
Current U.S.
Class: |
707/769 |
Current CPC
Class: |
G06F 16/148 20190101;
G06F 2201/82 20130101; G06F 16/188 20190101; G06F 2201/815
20130101; G06F 11/1448 20130101; G06F 2201/84 20130101 |
Class at
Publication: |
707/769 |
International
Class: |
G06F 17/30 20060101
G06F017/30 |
Claims
1. A method for utilizing a virtual snapshot, the method
comprising: associating a request relating to a file in one of a
plurality of data storage systems to one of one or more physical
snapshots of one of the plurality of data storage systems in a
virtual snapshot; invoking an execution of the request relating to
the file at the one of the plurality of data storage systems based
on the associating; and providing a response based on the invoked
execution of the request.
2. The method as set forth in claim 1 further comprising searching
a virtual snapshot listing directory to identify the virtual
snapshot based on one or more factors.
3. The method as set forth in claim 2 wherein the one or more
factors comprise a date range for the virtual snapshot.
4. The method as set forth in claim 1 wherein the associating is
based on a search of virtualization metadata to associate the
request to the one or one or more physical snapshots or a search in
the one or more physical snapshots for a target identified in the
request to associate the request to the one or one or more physical
snapshots
5. The method as set forth in claim 1 further comprising
translating the request for the invocation of the execution of the
request at the one of the plurality of data storage systems based
on the associating.
6. The method as set forth in claim 5 wherein the providing the
response based on the invoked execution of the request further
comprises: receiving the response from the invoked execution of the
request; and translating the received response to another format
suitable for a requesting system which submitted the request; and
providing the translated response to the requesting system.
7. A non-transitory computer readable medium having stored thereon
instructions for utilizing a virtual snapshot comprising machine
executable code which when executed by at least one processor,
causes the processor to perform steps comprising: associating a
request relating to a file in one of a plurality of data storage
systems to one of one or more physical snapshots of one of the
plurality of data storage systems in a virtual snapshot; invoking
an execution of the request relating to the file at the one of the
plurality of data storage systems based on the associating; and
providing a response based on the invoked execution of the request.
20
8. The medium as set forth in claim 7 further comprising searching
a virtual snapshot listing directory to identify the obtained
virtual snapshot based on one or more factors.
9. The medium as set forth in claim 8 wherein the one or more
factors comprise a date range for the virtual snapshot.
10. The medium as set forth in claim 7 wherein the associating is
based on a search of virtualization metadata to associate the
request to the one or one or more physical snapshots or a search in
the one or more physical snapshots for a target identified in the
request to associate the request to the one or one or more physical
snapshots.
11. The medium as set forth in claim 7 further comprising
translating the request for the invocation of the execution of the
request at the one of the plurality of data storage systems based
on the associating.
12. The medium as set forth in claim 11 wherein the providing the
response based on the invoked execution of the request further
comprises: receiving the response from the invoked execution of the
request; and translating the received response to another format
suitable for a requesting system which submitted the request; and
providing the translated response to the requesting system.
13. A file virtualization system, comprising: at least one of
configurable hardware logic configured to be capable of
implementing or a processor coupled to a memory and configured to
execute programmed instructions stored in the memory comprising:
associating a request relating to a file in one of a plurality of
data storage systems to one of one or more physical snapshots of
one of the plurality of data storage systems in a virtual snapshot;
invoking an execution of the request relating to the file at the
one of the plurality of data storage systems based on the
associating; and providing a response based on the invoked
execution of the request.
14. The system as set forth in claim 13 wherein the identification
system searches a virtual snapshot listing directory stored in one
or more memory systems to identify the obtained virtual snapshot
based on one or more factors.
15. The system as set forth in claim 14 wherein the one or more
factors comprise a date range for the virtual snapshot.
16. The system as set forth in claim 13 wherein the association
system associates based on a search of virtualization metadata to
associate the request to the one or one or more physical snapshots
or a search in the one or more physical snapshots for a target
identified in the request to associate the request to the one or
one or more physical snapshots.
17. The system as set forth in claim 13 wherein the invocation
system translates the request for the invocation of the execution
of the request at the one of the plurality of data storage systems
based on the associating.
18. The system as set forth in claim 17 wherein the invocation
system receives and translates the response to another format
suitable for a requesting system which submitted the request and
the communication system provides the translated response to the
requesting system.
Description
[0001] This application is a continuation of prior U.S. patent
application Ser. No. 12/334,281, filed Dec. 12, 2008, and claims
the benefit of U.S. Provisional Patent Application Ser. No.
61/013,539, filed Dec. 13, 2007, and this application is also
related to U.S. patent application Ser. No. 12/334,294, filed Dec.
12, 2008, now abandoned, each of which is hereby incorporated by
reference in its entirety.
FIELD OF THE INVENTION
[0002] This invention relates generally to methods and systems for
capturing snapshots of file systems and, more particularly, to
methods for generating a unified virtual snapshot from a plurality
of physical snapshots of a heterogeneous network storage system and
systems thereof.
BACKGROUND
[0003] Often files and associated data in computer systems are
remotely stored on one or more network storage devices. In
anticipation of a possible restore request from a user computer
system coupled to a network storage device, a physical snapshot of
the content in the network storage device may be captured at a
recorded time. If the user computer system has a need for and
requests a restore, the captured physical snapshot can be used to
recover contents from the network storage device as of the recorded
time.
[0004] File virtualization systems provide methods for managing and
presenting a plurality of network storage devices as a single,
unified file system. Basically, file virtualization decouples the
presentation of a file system from its' physical composition.
Unfortunately, when file virtualization is implemented, there is no
method or system for generating and providing a unified virtual
snapshot in a heterogeneous storage network system.
SUMMARY
[0005] A method for generating a unified virtual snapshot in
accordance with embodiments of the present invention includes
invoking with a file virtualization system a capture of a plurality
of physical snapshots. Each of the physical snapshots comprises
content at a given point in time in one of the plurality of data
storage systems. A unified virtual snapshot is generated with the
file virtualization system based on the captured plurality of the
physical snapshots.
[0006] A computer readable medium having stored thereon
instructions for methods for generating a unified virtual snapshot
in accordance with other embodiments of the present invention
comprising machine executable code which when executed by at least
one processor, causes the processor to perform steps including
invoking with a file virtualization system a capture of a plurality
of physical snapshots. Each of the physical snapshots comprises
content at a given point in time in one of the plurality of data
storage systems. A unified virtual snapshot is generated with the
file virtualization system based on the captured plurality of the
physical snapshots.
[0007] A system that generates a unified virtual snapshot in
accordance with other embodiments of the present invention includes
an invocation system and a virtual snapshot system in a file
virtualization system. The invocation system invokes a capture of a
plurality of physical snapshots. Each of the physical snapshots
comprises content in one of the plurality of data storage systems
at a given point in time. The virtual snapshot system generates a
unified virtual snapshot based on the captured plurality of the
physical snapshots.
[0008] The present invention provides a number of advantages
including providing a unified virtual snapshot from a plurality of
physical snapshots of contents of file systems distributed across
several independent, network storage devices. Additionally, the
present invention provides a method and system which enables the
use of snapshots in environments that implement file
virtualization. Further, the present invention captures and
generates snapshots which can be utilized to re-assemble contents
of file systems with or without the file virtualization system.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] FIG. 1 is a block diagram of an example of a system that
generates and uses a virtual snapshot from a plurality of physical
snapshots of a heterogeneous network storage system;
[0010] FIG. 2A is a functional block diagram of an example of a
method for processing requests with file virtualization;
[0011] FIG. 2B is a flow chart of the example of the method for
processing requests with file virtualization illustrated in FIG.
2A;
[0012] FIG. 3A is a functional block diagram of an example of a
method for generating one or more unified virtual snapshots;
[0013] FIG. 3B is a flow chart of the example of the method for
generating one or more unified virtual snapshots illustrated in
FIG. 3A;
[0014] FIG. 4A is a functional block diagram of a method for
processing requests with file virtualization after the creation of
one or more unified virtual snapshots;
[0015] FIG. 4B is a flow chart of the example of the method for
processing requests with file virtualization after the creation of
one or more unified virtual snapshots illustrated in FIG. 4A;
[0016] FIG. 5A is a functional block diagram of the hierarchy of
unified virtual snapshots and physical snapshots;
[0017] FIG. 5B is a flow chart of the example of the method for
recovering content in a heterogeneous storage system;
[0018] FIG. 6 is a diagram of an example of a virtual snapshot
configuration record; and
[0019] FIG. 7 is a diagram of an example of a snapshot command on a
network storage device.
DETAILED DESCRIPTION
[0020] An example of a system 10 that generates and uses a virtual
snapshot of a heterogeneous network storage system is illustrated
in FIG. 1, although the present invention can be utilized in
homogeneous network storage systems with one or more storage
devices. This system 10 includes a client system 12, a file
virtualization system 14, data storage systems 16(1) and 16(2), and
metadata storage system 18, although this system 10 can include
other numbers and types of systems, devices, equipment, parts,
components, and/or elements in other configurations. The present
invention provides a number of advantages including providing a
unified virtual snapshot from a plurality of physical snapshots of
contents of file systems distributed across several independent,
network storage devices.
[0021] Referring more specifically to FIG. 1, the client system 12
utilizes the file virtualization system 14 to conduct one or more
operations with one or more of the data storage systems 16(1),
16(2), and 18, such as to store a file, delete a file, create a
file, and restore a file by way of example only, although other
numbers and types of network systems could be utilizing these
resources and other types and numbers of functions could be
performed. The client system 12 includes a central processing unit
(CPU) or processor, a memory, user input device, a display, and an
interface system, and which are coupled together by a bus or other
link, although the client system 12 can include other numbers and
types of components, parts, devices, systems, and elements in other
configurations. The processor in the client system 12 executes a
program of stored instructions as described and illustrated herein,
although the processor could execute other numbers and types of
programmed instructions.
[0022] The memory in the client system 12 stores these programmed
instructions for one or more aspects of the present invention as
described and illustrated herein, although some or all of the
programmed instructions could be stored and/or executed elsewhere.
A variety of different types of memory storage devices, such as a
random access memory (RAM) or a read only memory (ROM) in the
system or a floppy disk, hard disk, CD ROM, or other computer
readable medium which is read from and/or written to by a magnetic,
optical, or other reading and/or writing system that is coupled to
one or more processors, can be used for the memory in the client
system 12.
[0023] The user input device in the client system 12 is used to
input selections, such as to store a file, delete a file, create a
file, and restore a file, although the user input device could be
used to input other types of data and interact with other elements.
The user input device can include a computer keyboard and a
computer mouse, although other types and numbers of user input
devices can be used. The display in the client system 12 is used to
display information, such as a file or directory, although other
types and amounts of information can be displayed in other manners.
The display can include a computer display screen, such as a CRT or
LCD screen, although other types and numbers of displays could be
used.
[0024] The interface system in the client system 12 is used to
operatively couple and communicate between the client system 12 and
the file virtualization system 14 via a communications network 20,
although other types and numbers of communication networks or
systems with other types and numbers of configurations and
connections to other systems and devices can be used.
[0025] The file virtualization system 14 manages file
virtualization and the generation of unified virtual snapshots,
although other numbers and types of systems can be used and other
numbers and types of functions can be performed. The file
virtualization system 14 includes a central processing unit (CPU)
or processor, a memory, and an interface system which are coupled
together by a bus or other link, although other numbers and types
of components, parts, devices, systems, and elements in other
configurations and locations can be used. The processor in the file
virtualization system 14 executes a program of stored instructions
for one or more aspects of the present invention as described and
illustrated by way of the embodiments herein, such as managing file
virtualization and the generation of unified virtual snapshots,
although the processor in file virtualization system 14 could
execute other numbers and types of programmed instructions.
[0026] The memory in the file virtualization system 14 stores these
programmed instructions for one or more aspects of the present
invention as described and illustrated herein, although some or all
of the programmed instructions could be stored and/or executed
elsewhere. A variety of different types of memory storage devices,
such as a random access memory (RAM) or a read only memory (ROM) in
the system or a floppy disk, hard disk, CD ROM, DVD ROM, or other
computer readable medium which is read from and/or written to by a
magnetic, optical, or other reading and/or writing system that is
coupled to one or more processors, can be used for the memory in
the file virtualization system 14.
[0027] The interface system in the file virtualization system 14 is
used to operatively couple and communicate between the file
virtualization system 14 and the client system 12, the data storage
system 16(1), the data storage system 16(2), and the metadata
storage system 18 via the communications networks 20, although
other types and numbers of communication networks or systems with
other types and numbers of connections and configurations can be
used.
[0028] Each of the data storage systems 16(1) and 16(2) is a
network storage device for files, directories, and other data,
although other numbers and types of storage systems which could
have other numbers and types of functions and store other data
could be used. In this example, data storage system 16(1) is a
different type of storage device, e.g. different make and/or model,
from the data storage system 16(2) to form a heterogeneous network
storage system, although the present invention can work with other
numbers and types of storage systems, such as a homogeneous
system.
[0029] Each of the data storage systems 16(1) and 16(2) include a
central processing unit (CPU) or processor, a memory, and an
interface system which are coupled together by a bus or other link,
although other numbers and types of components, parts, devices,
systems, and elements in other configurations can be used. By way
of example only, the storage systems may not have their own
separate processing capabilities. In this example, the specialized
processor in each of the data storage systems 16(1) and 16(2)
executes a program of stored instructions for one or more aspects
of the present invention as described and illustrated by way of the
embodiments herein, such as to capture a physical snapshot by way
of example only, although the processor in each of the data storage
system could execute other numbers and types of programmed
instructions.
[0030] The memory in each of the data storage systems 16(1) and
16(2) store these programmed instructions for one or more aspects
of the present invention as described and illustrated herein,
although some or all of the programmed instructions could be stored
and/or executed elsewhere. A variety of different types of memory
storage devices, such as a random access memory (RAM) or a read
only memory (ROM) in the system or a floppy disk, hard disk, CD
ROM, DVD ROM, or other computer readable medium which is read from
and/or written to by a magnetic, optical, or other reading and/or
writing system that is coupled to one or more processors, can be
used for the memory in each of the data storage systems 16(1) and
16(2).
[0031] The interface system in the data storage system 16(1) and
the interface in the data storage system 16(2) are each used to
operatively couple and communicate between the data storage system
16(1) and the file virtualization system 14 and between the data
storage system 16(2) and the file virtualization system 14 via
communication network 20, although other types and numbers of
communication networks or systems with other types and numbers of
configurations and connections to other systems and devices can be
used.
[0032] The metadata storage system 18 is another type of network
storage device to store and manage global file virtualization
metadata from data storage systems 16(1) and 16(2), although other
numbers and types of storage systems which could have other numbers
and types of functions, which is connected in other manners, and
which could store other types of data and information could be
used. In this particular example, the metadata storage system 18 is
external to the file virtualization system 14, although the
metadata storage 18 could be located in the file virtualization
system 14. The metadata storage system 18 includes a central
processing unit (CPU) or processor, a memory, and an interface
system which are coupled together by a bus or other link, although
other numbers and types of components, parts, devices, systems, and
elements in other configurations can be used for the storage
system. By way of example only, the storage system may not have its
own separate processing capabilities. In this example, the
specialized processor in the metadata storage system 18 executes a
program of stored instructions for one or more aspects of the
present invention as described and illustrated by way of the
embodiments herein, although the processor in metadata storage
system 18 could execute other numbers and types of programmed
instructions.
[0033] The memory in the metadata storage system 18 stores these
programmed instructions for one or more aspects of the present
invention as described and illustrated herein, although some or all
of the programmed instructions could be stored and/or executed
elsewhere. A variety of different types of memory storage devices,
such as a random access memory (RAM) or a read only memory (ROM) in
the system or a floppy disk, hard disk, CD ROM, DVD ROM, or other
computer readable medium which is read from and/or written to by a
magnetic, optical, or other reading and/or writing system that is
coupled to the processor in the metadata storage system 18.
[0034] The interface system in the metadata storage system 18 is
used to operatively couple and communicate between the metadata
storage system 18 and the file virtualization system 14 via the
communications network 20, although other types and numbers of
communication networks or systems with other types and numbers of
configurations and connections to other systems and devices can be
used.
[0035] Although embodiments of the client system 12, the file
virtualization system 14, the data storage systems 16(1) and 16(2),
and the metadata storage system 18 are described herein, each of
these systems can be implemented on any suitable computer system or
computing device. It is to be understood that the devices and
systems of the embodiments described herein are for exemplary
purposes, as many variations of the specific hardware and software
used to implement the embodiments are possible, as will be
appreciated by those skilled in the relevant art(s).
[0036] Furthermore, each of the systems of the embodiments may be
conveniently implemented using one or more general purpose computer
systems, microprocessors, digital signal processors, and
micro-controllers, programmed according to the teachings of the
embodiments, as described and illustrated herein, and as will be
appreciated by those ordinary skill in the art.
[0037] In addition, two or more computing systems or devices can be
substituted for any one of the systems in any embodiment of the
embodiments. Accordingly, principles and advantages of distributed
processing, such as redundancy and replication also can be
implemented, as desired, to increase the robustness and performance
of the devices and systems of the embodiments. The embodiments may
also be implemented on computer system or systems that extend
across any suitable network using any suitable interface mechanisms
and communications technologies, including by way of example only
telecommunications in any suitable form (e.g., voice and modem),
wireless communications media, wireless communications networks,
cellular communications networks, G3 communications networks,
Public Switched Telephone Network (PSTNs), Packet Data Networks
(PDNs), the Internet, intranets, and combinations thereof.
[0038] The embodiments may also be embodied as a computer readable
medium having instructions stored thereon for one or more aspects
of the present invention as described and illustrated by way of the
embodiments herein, as described herein, which when executed by a
processor, cause the processor to carry out the steps necessary to
implement the methods of the embodiments, as described and
illustrated herein.
[0039] An overview of the present invention is set forth below.
With this example of the present invention, a file virtualization
layer (FV1) provided by file virtualization system 14 exists
between the application in client system 12 or CL1 and the data
storage systems 16(1) and 16(2) (also referred to as DS1 and DS2),
although other numbers and types of client systems and storage
systems could be used. This file virtualization layer provided by
file virtualization system 14 manages metadata in data storage 18
that tracks the location of files and directories that are
distributed across data storage systems 16(1) and 16(2) in this
particular example.
[0040] To generate or create a unified virtual snapshot, file
virtualization system 14 erects an I/O barrier to substantially
suspend data storage communications between client system 12 and
data storage systems 16(1) and 16(2) and metadata storage system
18. This suspension permits either an administrator or application
at client system 12 or file virtualization system 14 to request or
invoke a capture of physical snapshots of content on data storage
systems 16(1) and 16(2) and metadata storage system 18 using an
application programming interface (API) or command line interface
(CLI), although other manners for invoking a capture of physical
snapshots, such as a periodic automated invocation could be used.
Once all of the physical snapshots have been captured or otherwise
completed by file virtualization system 14, the unified virtual
snapshot is generated by file virtualization system 14 and the I/O
barrier is removed to allow storage data communications to resume.
The unified virtual snapshot comprises the captured physical
snapshots which are mapped together by file virtualization system
14 to form the virtual snapshot.
[0041] In this example, the I/O barrier is implemented by the file
virtualization system 14 at the application protocol level, such as
NFS or CIFS by way of example, although the I/O barrier could be
implemented in other manners. Packets are accepted at a transport
level, such as UDP or TCP by way of example, but are not proxied by
file virtualization system 14 to the data storage systems 16(1) and
16(2) and traffic to the metadata storage system 18 is halted while
the I/O barrier is asserted. As implemented, the barrier operation
is substantially transparent to the operator at the client system
12 and at most the file system seems momentarily slow, although
system can be arranged in other manners, such as to provide notice
of the implementation of the barrier if desired.
[0042] To create a persistent record of the location of files and
directories in physical snapshots, once the I/O barrier is asserted
and before a physical snapshot occurs the file virtualization
system 14 initiates copying or writing of a virtual snapshot
configuration record into data storage systems 16(1) and 16(2) and
metadata storage system 18. The virtual snapshot configuration
record is a unique record written in each of the data storage
systems 16(1) and 16(2) and metadata storage system 18 that allows
an operator or program to locate components, e.g. a file of a
virtual snapshot, although other types and amounts of information
could be included. More specifically, the snapshot configuration
record records the members of a unified virtual snapshot, i.e. in
this particular example the members are data storage systems 16(1)
and 16(2) and metadata storage system 18, although the snapshot
configuration record can store other types and amounts of data. By
way of example only, a virtual snapshot configuration record is
illustrated in FIG. 6. In this example, the virtual snapshot
configuration records are made unique by a field in the header of
each record, although other manners for providing a unique
identifier can be used.
[0043] The virtual snapshot configuration record is included in the
physical snapshots to aid in recovery. With the snapshot
configuration record and the stored metadata on the file
virtualization system 14, the file virtualization system 14 can
locate a particular file or directory. Additionally, by including
the virtual snapshot configuration record in the physical snapshot,
an external application that knows the format of the stored
metadata can use that metadata and the snapshot configuration
record to locate a file without file virtualization.
[0044] Once a unique virtual snapshot configuration record is
copied to each data storage systems 16(1) and 16(2) and metadata
storage system 18, the data storage systems 16(1) and 16(2) and
metadata storage system 18 are invoked by file virtualization
system 14 to capture physical snapshots which will contain this
virtual snapshot configuration record, although the data storage
systems 16(1) and 16(2) and metadata storage system 18 can be
invoked to capture physical snapshots by other systems in other
manners. The data storage systems 16(1) and 16(2) and metadata
storage system 18 take a physical snapshot in response to this
invocation.
[0045] By way of example only, a snapshot command which can be used
by data storage systems 16(1) and 16(2) and metadata storage system
18 is illustrated in FIG. 7, although other types of commands could
be used. Again, this method effectively embeds the unique virtual
snapshot configuration record into each of the physical snapshots
themselves.
[0046] Generation of unified virtual snapshots is implemented by
the file virtualization layer in file virtualization system 14,
although the generation can be implemented by other systems.
Virtual directories are dynamically created that contain a list of
available virtual snapshots at different points in time in file
virtualization system 14. Each virtual snapshot subdirectory
contains files and directories that exist in the physical snapshots
of the contents of the file systems on data storage systems 16(1)
and 16(2) and metadata storage system 18 in this example.
[0047] Referring now to FIGS. 2A and 2B, an example of a method for
processing requests with a file virtualization is described below.
In step 22, client system 12 (also known as CL1) issues a request
CL-REQ-1-1 for a file creation operation of a file `a` to file
virtualization system 14 (also known as FV1), although other types
and numbers of requests could be issued from other types and
numbers of systems.
[0048] In step 24, file virtualization system 14 receives the
request CL-REQ-1-1 from client system 12. Using the stored
metadata, the file virtualization system 14 translates the request
CL-REQ-1-1 into a file virtualization request FV-REQ-1-1 which is
suitable for execution on data storage system 16(1) (also known as
DS1) in which the file is actually located, although other types of
requests for other systems could be received.
[0049] In step 26, data storage system 16(1) receives the request
FV-REQ-1-1 from the file virtualization system 14. In response to
the received request FV-REQ-1-1 the data storage system 16(1)
performs the creation of file `a` and issues reply DS-RSP-1-1 back
to file virtualization system 14, although the data storage system
16(1) could perform other types and numbers of operations based on
the received request.
[0050] In step 28, file virtualization system 14 receives the reply
DS-RSP-1-1 from the data storage system 16(1). In response to the
reply DS-RSP-1-1, the file virtualization system 14 generates
metadata about the file creation operation and transmits a
FV-REQ-1-2 request to metadata storage system 18 (also known as
MD1) to record this generated metadata.
[0051] In step 30, metadata storage system 18 receives the
FV-REQ-1-2 request and stores the generated metadata. Once the
FV-REQ-1-2 request is processed, the metadata storage system 18
issues a reply MD-RSP-1-1 to the file virtualization system 14.
[0052] In step 32, file virtualization system 14 receives the reply
MD-RSP-1-1 from the metadata storage system 18. Next, the file
virtualization system 14 using information gathered from the reply
MD-RSP-1-1 and the reply DS-RSP-1-1 generates a file virtualization
reply FV-RSP-1-1 and issues the reply FV-RSP-1-1 back to client
system 12. The file virtualization system 14 also updates the
stored file virtualization configuration record to reflect this
completed operation.
[0053] Referring now to FIGS. 3A and 3B, an example of a method for
generating one or more unified virtual snapshots is described
below. In step 50, client system 12 issues request CL-REQ-2-1 for a
file deletion operation of file `a` to file virtualization system
14, although other types and numbers of requests could be issued
from other types and numbers of systems.
[0054] In step 52, file virtualization system 14 accepts the
request CL-REQ-2-1 from the client system 12, although other types
and numbers of requests could be received. Since at this time an
I/O barrier is asserted, the file virtualization system 14 performs
no action at this time on the request CL-REQ-2-1 from the client
system 12, although once the I/O barrier is removed the file
virtualization system 14 will process the request.
[0055] In step 54, while the I/O barrier is asserted, the file
virtualization system 14 generates and transmits a write request
WRITE_REQ_2_* to each of the data storage systems 16(1) and 16(2)
and the metadata storage system 18 to write the virtualization
snapshot configuration record persistent storage, although other
types and numbers of requests can be transmitted to other types and
numbers of systems. More specifically, in this particular example
the file virtualization system 14 generates and transmits a write
request WRITE_REQ_2_1 to data storage system 16(1), a write request
WRITE_REQ_2_2 to data storage system 16(2), and a WRITE_REQ_2_3 to
metadata storage system 18 to each write the virtualization
snapshot configuration record in persistent storage.
[0056] Once the virtualization snapshot configuration record is
written in persistent storage, each of the data storage systems
16(1) and 16(2) and the metadata storage system 18 generates and
transmits a response WRITE_RSP_2_* to the file virtualization
system 14, although other types and numbers of responses can be
transmitted to other types and numbers of systems. More
specifically, in this particular example data storage system 16(1)
generates and transmits a WRITE_RSP_2_1, the data storage system
16(2) generates and transmits a WRITE_RSP_2_2, and the metadata
storage system 18 generates and transmits a WRITE_RSP_2_3 to the
file virtualization system 12 once the virtualization snapshot
configuration record is written in persistent storage in each
storage system.
[0057] In step 56, file virtualization system 14 optionally flushes
metadata changes, write ahead logs, and any other information
required to ensure consistency with the file virtualization
metadata snapshot, although the file virtualization system 14 may
perform other types and numbers of operations.
[0058] In step 58 once the optional flush operations described
above in step 56 are completed, the file virtualization system 14
invokes the execution of snapshot operations on data storage
systems 16(1) and 16(2) and metadata storage system 18 by
generating and transmitting snapshot requests SNAP_REQ_2_*,
although the snapshot operations can be invoked in other manners
and physical snapshots can be taken in other types and numbers of
systems. More specifically, in this particular example file
virtualization system 14 generates and transmits request
SNAP_REQ_2_1 to data storage systems 16(1) to take a physical
snapshot, request SNAP_REQ_2_2 to data storage systems 16(2) to
take a physical snapshot, and request SNAP_REQ_2_3 to data storage
systems 18 to take a physical snapshot.
[0059] In step 60, the data storage systems 16(1) and 16(2) and the
metadata storage system 18 each receive and process the requests
SNAP_REQ_2_1, SNAP_REQ_2_3, and SNAP_REQ_2_3, respectively, to
perform a physical snapshot operation to capture a physical
snapshot in each of the data storage systems 16(1) and 16(2) and
the metadata storage system 18.
[0060] Once the physical snapshots have been taken, the data
storage systems 16(1) and 16(2) and the metadata storage system 18
each generate and transmit a response SNAP-RSP-2-* when each of the
physical snapshots at the data storage systems 16(1) and 16(2) and
the metadata storage system 18 have been taken, although other
types and numbers of responses can be transmitted to other types
and numbers of systems. More specifically, in this particular
example data storage system 16(1) generates and transmits a
SNAP-RSP-2-1, the data storage system 16(2) generates and transmits
a SNAP-RSP-2-2, and the metadata storage system 18 generates and
transmits a SNAP-RSP-2-3 to the file virtualization system 12 when
each of the physical snapshots at the data storage systems 16(1)
and 16(2) and the metadata storage system 18 have been taken.
[0061] In step 62, file virtualization system 14 receives
completion notifications from data storage systems 16(1) and 16(2)
and metadata storage system 18 indicating that the physical
snapshots are completed, i.e. the data and metadata are consistent
as of the point of time the I/O barrier has been asserted, and then
records completion of the unified virtual snapshot. Once all of the
responses SNAP-RSP-2-* have been received, the file virtualization
system 14, lowers the asserted I/O barrier and processes request
CL-REQ-2-1 as well as any other requests.
[0062] Referring to FIGS. 4A and 4B, an example of a method for
processing requests with a file virtualization after the creation
of one or more unified virtual snapshots is described below. In
step 70, client system 12 issues request CL-REQ-3-1 for a file
deletion operation of file `a` to file virtualization system 14,
although other types and numbers of requests could be issued from
other types and numbers of systems.
[0063] In step 72, file virtualization system 14 receives the
request CL-REQ-3-1 from client system 12. Using the stored
metadata, the file virtualization system 14 translates the request
CL-REQ-3-1 into a file virtualization request FV-REQ-3-1 which is
suitable for execution on data storage system 16(1) in which the
file is actually located, although other types of requests for
other systems could be received.
[0064] In step 74, data storage system 16(1) receives the request
FV-REQ-3-1 request from the file virtualization system 14. In
response to the received request FV-REQ-3-1, the data storage
system 16(1) performs the deletion of file `a` and issues reply
DS-RSP-3-1 back to file virtualization system 14, although the data
storage system 16(1) could perform other types and numbers of
operations based on the received request. Although deleted by this
operation, file `a` remains in the unified virtual snapshot
generated as described with reference to FIGS. 3A and 3B.
[0065] In step 76, file virtualization system 14 receives the reply
DS-RSP-3-1 from the data storage system 16(1). In response to the
reply DS-RSP-3-1, the file virtualization system 14 generates
metadata about the file deletion operation and transmits a
FV-REQ-3-2 request to metadata storage system 18 to record this
generated metadata
[0066] In step 78, metadata storage system 18 receives the request
FV-REQ-3-2 and updates the metadata stored on metadata storage
system 18 to reflect the deletion of file `a`, although other types
and numbers of updates could be recorded. Once the FV-REQ-3-2
request is processed, the metadata storage system 18 issues a reply
MD-RSP-3-1 to the file virtualization system 14.
[0067] In step 80, file virtualization system 14 receives the reply
MD-RSP-3-1 from the metadata storage system 18. Next, the file
virtualization system 14 using information gathered from the reply
MD-RSP-3-1 and the reply DS-RSP-3-1 generates and issues a file
virtualization reply FV-RSP-3-1 back to client system 12.
[0068] An example of the hierarchy of unified virtual snapshots and
physical snapshots is illustrated in the functional block diagram
in FIG. 5A and is described below. As set forth in functional block
90, each virtual directory contains a virtual snapshot listing
directory (VSLD). As set forth in functional block 92, each VSLD
contains a list of virtual snapshots (VSN). As set forth in
functional block 94, each VSN is an aggregation of physical
snapshots (PSNs). As set forth in functional block 96, each
Physical directory contains a physical snapshot listing directory
(PSLD). As set forth in functional block 98, each PSLD contains a
list of physical snapshots (PSN). As set forth in functional block
100, each PSN is a "point in time" image of the file system, such
as of data storage system 16(1), data storage system 16(2), or
metadata storage system 18, by way of example only.
[0069] An example of a method for recovering content in a
heterogeneous storage system is illustrated in FIG. 5B and is
described below. In step 120, the client system 12 generates and
issues a request CL-REQ-2-1 to file virtualization system 14 to
access a file in the unified virtual snapshot, although other types
and numbers of requests could be issued.
[0070] In step 122, the file virtualization system 14 receives the
request CL-REQ-2-1 from client system 12, although other types and
numbers of requests could be received. The file virtualization
system 14 processes the request CL-REQ-2-1 which includes a marker
indicating a traversal of a virtual snapshot listing directory,
although in response to the request the file virtualization system
14 could have other types and numbers of indicators. By way of
example only, the request could have a marker which indicated a
search of the virtual snapshot listing directory was needed to
identify the virtual snapshot or the absence of a marker could
indicate the need for a search. If a search is indicated by
processing the request CL-REQ-2-1, the file virtualization system
14 identifies the virtual snapshot in the virtual snapshot listing
based on one or more factors, such as a particular date range in
the request, although other manners for identifying the virtual
snapshot can be used.
[0071] In step 124, the file virtualization system 14 associates
request CL-REQ-2-1 with the identified virtual snapshot VSN-1 based
on data in the processed request CL-REQ-2-1, such as a specific
identification of the virtual snapshot VSN-1, although other
manners for identifying the virtual snapshot can be used.
[0072] In step 126, based on data in the request CL-REQ-2-1, the
file virtualization system 14 determines which of two methods for
associating the request CL-REQ-2-1 with one of the captured
physical snapshots of data storage system 16(1), data storage
system 16(2), and metadata storage system 18 to use, although the
file virtualization system 14 could determine which method to use
in other manners and could select from other types and numbers of
methods In this particular example, one of these methods searches
virtualization metadata (cached or persistent) to map the request
to a captured physical snapshot and the other method searches the
captured physical snapshot for each of the storage systems for the
target of the request.
[0073] If in step 126 the file virtualization system 14 determines
that the method which searches virtualization metadata should be
used, then the file virtualization system 14 proceeds to step 128.
In step 128, the file virtualization system 14 searches stored
virtualization metadata (cached or persistent) to map the target
identified in the request CL-REQ-2-1 to one of the captured
physical snapshots of one of data storage system 16(1), data
storage system 16(2), and metadata storage system 18. Based on the
search, the file virtualization system 14 identifies one of these
captured physical snapshots, although the file virtualization
system can perform other operations based on the result of this
search, such as generating and transmitting a message to client
system 12 that the request CL-REQ-2-1 can not be completed.
[0074] If in step 126 the file virtualization system 14 determines
that the method which searches the captured physical snapshots
should be used, then the file virtualization system 14 proceeds to
step 130. In step 130, the file virtualization system 14 searches
the captured physical snapshots for each of the data storage
systems 16(1) and 16(2) and metadata storage system 18 for a target
identified in the request CL-REQ-2-1. Based on the search, the file
virtualization system 14 either identifies one of these captured
physical snapshots, although the file virtualization system 14 can
perform other operations based on the result of this search, such
as generating and transmitting a message to client system 12 that
the request CL-REQ-2-1 can not be completed.
[0075] In step 132, once the captured physical snapshot has been
identified, the file virtualization system 14 translates the
request CL-REQ-2-1 in a format suitable for execution on the data
storage system 16(1) or data storage system 16(2) from which the
identified captured physical snapshot was taken. Once the request
CL-REQ-2-1 has been translated, the file virtualization system 14
forwards the translated request CL-REQ-2-1 to the data storage
system 16(1) or the data storage system 16(2) from which the
identified captured physical snapshot was taken. The data storage
system 16(1) or data storage system 16(2) from which the identified
captured physical snapshot was taken processes the translated
request CL-REQ-2-1, executes any operations, and generates and
transmits a response back to the file virtualization system 14,
although other types and numbers of operations could be performed
based on the received translated request.
[0076] In step 134, the file virtualization system 14 translates
the response from the data storage system 16(1), data storage
system 16(2), or metadata storage system 18 which processed the
translated request CL-REQ-2-1 and issues a reply back to the client
system 12, although the file virtualization system 14 could perform
other types and numbers of operations based on the received
response.
[0077] Accordingly, as illustrated by the description herein the
present invention provides a number of advantages including
providing a unified virtual snapshot from a plurality of physical
snapshots of contents of file systems distributed across several
independent, network storage devices of dissimilar make and model.
Additionally, the present invention provides a method and system
which enables the use of snapshots in environments that implement
file virtualization. Further, the present invention captures and
generates snapshots which can be utilized to re-assemble contents
of file systems with or without the file virtualization system.
[0078] Having thus described the basic concept of the invention, it
will be rather apparent to those skilled in the art that the
foregoing detailed disclosure is intended to be presented by way of
example only, and is not limiting. Various alterations,
improvements, and modifications will occur and are intended to
those skilled in the art, though not expressly stated herein. These
alterations, improvements, and modifications are intended to be
suggested hereby, and are within the spirit and scope of the
invention. Additionally, the recited order of processing elements
or sequences, or the use of numbers, letters, or other designations
therefore, is not intended to limit the claimed processes to any
order except as may be specified in the claims. Accordingly, the
invention is limited only by the following claims and equivalents
thereto.
* * * * *