U.S. patent application number 09/940215 was filed with the patent office on 2002-10-17 for method, system and program product for detecting lost sequences within an exchange on fibre channel.
Invention is credited to Elliott, Joseph C., Flanagan, John R., Frazier, Giles R., Huang, Catherine C., Ricci, Louis W..
Application Number | 20020152338 09/940215 |
Document ID | / |
Family ID | 22628795 |
Filed Date | 2002-10-17 |
United States Patent
Application |
20020152338 |
Kind Code |
A1 |
Elliott, Joseph C. ; et
al. |
October 17, 2002 |
Method, system and program product for detecting lost sequences
within an exchange on fibre channel
Abstract
The present invention provides fiber channel networks the
ability to detect lost information units transmitted be nodes. Each
information unit includes a new count field that identifies the
unit as a sequential event. When the information unit is received,
the count field is compared with the count field saved from the
previous unit and an error is asserted if the count is not the
expected value.
Inventors: |
Elliott, Joseph C.; (North
Creek, NY) ; Flanagan, John R.; (Poughkeepsie,
NY) ; Frazier, Giles R.; (Austin, TX) ; Huang,
Catherine C.; (Poughkeepsie, NY) ; Ricci, Louis
W.; (Hyde Park, NY) |
Correspondence
Address: |
Floyd A. Gonzalez
IBM Corporation - MS P386
2455 South Road
Poughkeepsie
NY
12601
US
|
Family ID: |
22628795 |
Appl. No.: |
09/940215 |
Filed: |
August 27, 2001 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09940215 |
Aug 27, 2001 |
|
|
|
09481345 |
Jan 12, 2000 |
|
|
|
09481345 |
Jan 12, 2000 |
|
|
|
09172695 |
Oct 14, 1998 |
|
|
|
6185631 |
|
|
|
|
Current U.S.
Class: |
710/34 |
Current CPC
Class: |
G06F 13/122
20130101 |
Class at
Publication: |
710/34 |
International
Class: |
G06F 013/00; G06F
013/14; G06F 013/28 |
Claims
We claim:
1. A method in Fibre Channel communication to detect lost
information units comprising the steps of: a) setting an initial
count value in a count field of a first information unit to be
sent; b) incrementing said count value for a subsequent second
information unit to be sent; and c) transmitting said information
unit over a Fibre Channel link.
2. The method according to claim 1 wherein a separate count value
is provided for each exchange.
3. The method according to claim 1 wherein said first information
unit is an open exchange information unit.
4. The method according to claim 1 wherein said initial count is
`0`.
5. The method according to claim 1 wherein said count field wraps
to `0` when incremented from it's maximum value.
6. The method according to claim 1 wherein said count field
includes byte 18 of the information unit.
7. A method in Fibre Channel communication to detect lost
information units comprising: a) setting an initial count value in
a count field of a first information unit to be sent; b)
decrementing said count value for a subsequent second information
unit to be sent; and c) transmitting said information unit over a
Fibre Channel link.
8. The method according to claim 7 wherein a separate count value
is provided for each exchange.
9. The method according to claim 7 wherein said first information
unit is an open exchange information unit.
10. The method according to claim 7 wherein said initial count is
`0`.
11. The method according to claim 7 wherein said count field wraps
to it's maximum value when decremented from `0`.
12. The method according to claim 7 wherein said count field
includes byte 18 of the information unit.
13. A method in Fibre Channel communication to detect lost
information units comprising the steps of: a) receiving said
information unit over a Fibre Channel link. b) saving the count
field of each information unit received; c) checking the count
field of each information unit received against the saved value of
the previous count field received; and d) asserting an error
indication when said received count field is not the expected
incremented value.
14. A system in Fibre Channel communication to detect lost
information units, the system comprising: a count setter, setting
an initial count value in a count field of a first information unit
to be sent; an incrementor, incrementing said count value for a
subsequent second information unit to be sent; and a transmitter,
transmitting said information unit over a Fibre Channel link.
15. The system according to claim 14 wherein a separate count value
is provided for each exchange.
16. The system according to claim 14 wherein said first information
unit is an open exchange information unit.
17. The system according to claim 14 wherein said initial count
18. The system according to claim 14 wherein said count field wraps
to `0` when incremented from it's maximum value.
19. The system according to claim 14 wherein said count field
includes byte 18 of the information unit.
20. A system in Fibre Channel communication to detect lost
information units, the system comprising: a setter, setting an
initial count value in a count field of a first information unit to
be sent; a decrementor decrementing said count value for a
subsequent second information unit to be sent; and a transmitter
transmitting said information unit over a Fibre Channel link.
21. The system according to claim 20 wherein a separate count value
is provided for each exchange.
22. The system according to claim 20 wherein said first information
unit is an open exchange information unit.
23. The system according to claim 20 wherein said initial count is
`0`.
24. The system according to claim 20 wherein said count field wraps
to it's maximum value when decremented from `0`.
25. The system according to claim 20 wherein said count field
includes byte 18 of the information unit.
26. A system in Fibre Channel communication to detect lost
information units, the system comprising: a receiver receiving said
information unit over a Fibre Channel link. a saver, saving the
count field of each information unit received; a checker, checking
the count field of each information unit received against the saved
value of the previous count field received; and an asserter,
asserting an error indication when said received count field is not
the expected incremented value.
27. A computer program product comprising a computer useable medium
having compute readable program code means therein in a computing
system in Fibre Channel communication to detect lost information
units, the computer readable program code means in said computer
program product comprising: a) computer readable program code means
for setting an initial count value in a count field of a first
information unit to be sent; b) computer readable program code
means for incrementing said count value for a subsequent second
information unit to be sent; and c) computer readable program code
means for transmitting said information unit over a Fibre Channel
link.
28. The computer program product according to claim 27 wherein
separate a count value is provided for each exchange.
29. The computer program product according to claim 27 wherein said
first information unit is an open exchange information unit.
30. The computer program product according to claim 27 wherein said
initial count is `0`.
31. The computer program product according to claim 27 wherein said
count field wraps to `0` when incremented from it's maximum
value.
32. The computer program product according to claim 27 wherein s
aid count field includes byte 18 of t he information unit.
33. A computer program product comprising a computer useable medium
having compute readable program code means therein in a computing
system in Fibre Channel communication to detect lost information
units comprising: a) computer readable program code means for
setting an initial count value in a count field of a first
information unit to be sent; b) computer readable program code
means for decrementing said count value for a subsequent second
information unit to be sent; and c) computer readable program code
means for transmitting said information unit over a Fibre Channel
link.
34. The computer program product of claim 33 wherein a separate
count value is provided for each exchange.
35. The computer program product of claim 33 wherein said first
information unit is an open exchange information unit.
36. The computer program product of claim 33 wherein said initial
count is `0`.
37. The computer program product of claim 33 wherein said count
field wraps to it's maximum value when decremented from `0`.
38. The computer program product of claim 33 wherein said count
field includes byte 18 of the information unit.
39. A computer program product comprising a computer useable medium
having compute readable program code means therein in a computing
system in Fibre Channel communication to detect lost information
units comprising: a) computer readable program code means for
receiving said information unit over a Fibre Channel link. b)
computer readable program code means for saving the count field of
each information unit received; c) checking the count field of each
information unit received against the saved value of the previous
count field received; and d) computer readable program code means
for asserting an error indication when said received count field is
not the expected incremented value.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This is a divisional application of Ser. No. 09/481345 filed
on Jan. 12, 2000 which is a divisional of Ser. No. 09/172,695 now
U.S. Pat. No. 6,185,631 filed on Oct. 14, 1998.
FIELD OF INVENTION
[0002] This invention relates to concept of transferring
information in a computer program product for use with a computer
system having a main storage device in processing communication
with a plurality of input/output devices.
BACKGROUND OF THE INVENTION
[0003] In a network computing environment, multitudes of commands
and requests for retrieval and storage of data are processed every
second. To properly address the complexity of routing these
commands and requests, a number of different resolutions have been
implemented. In some data processing architectures, such as
International Business Machines Enterprise System Architecture/390
(Enterprise System Architecture/390 is a registered trademark of
International Business Machines Corporation), a channel subsystem
is utilized to pass information between the main storage and
input/output (I/O) devices. The channel subsystem includes one or
more channel paths, each including one or more channels and one or
more control units. Recently developed technologies such as the
International Business Machines ESCON switch (ESCON is a registered
trademark of International Business Machines Corporation), connect
the I/O devices to the main memory through the control units using
legacy channels to support the data transfer there between.
[0004] But as the technology improves, the performance of new
system processors will require many more legacy channels than are
presently in use to support the resulting increase of information
transfer in the data processing systems. Current architectural
constraints make the addition of such legacy channels an expensive
proposition. A further challenge is to provide the link data rate
required to support the data rates of new I/O devices such as DASDs
and Tapes. In this case, simply adding more legacy channels does
not adequately address the problem. A new architecture is needed
that can scale up to the higher link speeds needed for normal
transaction processing. Therefore, any new proposed architecture,
must include a capacity to accommodate higher bandwidth channel
links such as Fibre Channel links while providing better data rates
and higher link speeds.
[0005] This application incorporates by reference U.S. Pat. No.
5,526,484 to Casper et. al.
SUMMARY OF THE INVENTION
[0006] In a Fibre Channel Communication system, wherein a Channel
is in communication with a countrol unit over a Fibre Channel
fabric, blocks of information are transmitted as multiple units.
These information units (IU's) are time division multiplexed over a
shared link(s) in windows called "exchanges". For data integrity,
it is important that all of the data reaches the targeted receiver
and that it be arranged in the sequence such that it can be
re-constructed by the receiving entity. If an information unit is
lost (sent but not received) and the loss is un-detected, data
integrity is lost and the lost data can not be recovered in an
efficient way.
[0007] The present invention solves the problem by assigning count
values to information units within an exchange window.
[0008] It is therefore an object of the present invention to
provide a method to detect lost transmitted information units in a
Fibre Channel communication network.
[0009] It is a further object of the invention to provide a Fibre
Channel information unit with a unique count value when transmitted
and further, to check the count value when received.
[0010] It is yet another object of the present invention to
maintain separate count values for each Fibre Channel exchange.
[0011] It is a further object of the present invention to
initialize a counting sequence in information units on a Fibre
channel with an open exchange information unit.
[0012] It is another object of the present invention to assert an
error indication when a count value indicates a lost information
unit.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The subject matter which is regarded as the invention is
particularly pointed out and distinctly claimed in the concluding
portion of the specification. The invention, however, both as to
organization and method of practice, together with further objects
and advantages thereof, may best be understood by reference to the
following description taken in connection with the accompanying
drawings in which:
[0014] FIG. 1 is an illustration a network computing environment
utilizing one embodiment of the present invention;
[0015] FIG. 2 is an illustration of different Fibre Channel
hierarchies; and
[0016] FIG. 3 is an illustration of prior art systems demonstrating
communication paths.
DETAILED DESCRIPTION OF THE INVENTION
[0017] In a network computing system environment 100 having a data
processing system architecture such as the one depicted in FIG. 1,
information is passed between a program storage device or a main
storage such as the one shown at 110, and one or more input/output
devices (hereinafter I/O devices) 190, using channel subsystems
150. Through a fabric that can include one or more optical fiber
and switches 160, channel paths are established, comprising
channels 155 and one or more control units shown at 180. These
channel paths are the communication links established between the
I/O devices 190 and the main storage for processing and exchange of
information. The channel subsystem, control units and the fiber are
all part of an information transfer interface mechanism 195 which
enables the main storage remain in processing communication with
the input/output devices. The information transfer interface
mechanism can include other or different components. The fabric can
be to simply replaced by one or more switches. In one embodiment of
the present invention, the interface mechanism does not even
include the fabric or any switches.
[0018] The main storage 110 stores data and programs which are
input from I/O devices 190. Main storage is directly addressable
and provides for high speed processing of data by central
processing units and one or more I/O channel subsystem devices. One
example of a main storage is a customer's storage area and a system
area (not shown). I/O devices 190 receive information or store
information in main storage. Some examples of I/O devices include
magnetic-tape units, direct-access storage devices (DASD),
displays, keyboards, printers, teleprocessing devices,
communication controllers and sensor-based equipment.
[0019] The main storage is coupled to the Storage Control Element
(SCE) 120 which in turn is coupled to one or more central
processing units (CPU) 130. The central processing unit(s) is the
control center of the data processing system and typically
comprises sequencing and processing facilities for instruction
execution, initial program loading and other related functions. The
CPU is usually coupled to the SCE via a bi-directional or
uni-directional bus. The SCE, which controls the execution and
queuing of requests made by the CPU and channel subsystem, is
coupled to the main storage, CPUs and the channel subsystem via
different busses.
[0020] The channel subsystem directs the flow of information
between I/O devices and main storage and relieves the CPUs of the
task of communicating directly with the I/O devices so that data
processing operations directed by the CPU can proceed concurrently
with I/O processing operations. The channel subsystem uses one or
more channel paths as the communication links in managing the flow
of information to or from I/O devices. Each channel path consists
of one or more channels, located within channel subsystem, and one
or more control units. As can be seen in FIG. 1, it is also
possible to have one or more dynamic switches or even a switching
fabric (network of switches) included as part of the path, coupled
to the channel(s) and the control unit(s). Each control unit is
further attached via a bus to one or more I/O device(s).
[0021] The subchannel is the means by which channel subsystem
provides information about associated I/O devices to the central
processing units, the CPUs obtain this information by executing I/O
instructions. The subchannel consists of internal storage that
contains information in the form of a channel command word (CCW)
address, channel path identifier, device number, count, status
indications, and I/O interruption subclass code, as well as
information on path availability and functions pending or being
performed. I/O operations are initiated with devices by executing
I/O instructions that designate the subchannel associated with the
device.
[0022] The execution of input/output operations is accomplished by
the decoding and executing of channel command words or CCWs by the
channel subsystem and input/output devices. A chain of CCWs
(input/output operations) is initiated when the channel transfers
to the control unit the command specified by the first channel
command word. During the execution of the specified chain of I/O
operations, data and further commands are transferred between the
channel(s) and the control unit(s). In one embodiment of the
present invention, the CCWs are pipelined and the associated data
is streamed to the Control Unit(s) meaning multiple commands and
data are sent to the Control Unit without need for interlocked
responses from the Control Unit.
[0023] The present invention uses fiber optic channels to establish
its channel path links and utilizes the beneficial features of
fiber-based channels such as increased bandwidth, higher transfer
rates and size advantages. FIG. 2 depicts an overview of the
structure and concepts and mechanism used in the Fibre Channel
interface (FC-PH) alluded to previously. The Fibre Channel (FC) is
logically a bi-directional point-to-point serial data channel
structured for high performance capability. Physically, the Fibre
Channel can be an interconnection of multiple communication points,
called N_Ports, interconnected by a switching network, called a
Fabric, or a point-to-point link. Fiber is a general term used to
cover all physical media supported by the Fibre Channel, such as
optical fiber, twisted pair and coaxial cable.
[0024] Fibre Channel is structured as a set of hierarchical
functions as illustrated in FIG. 2. Fibre Channel Physical and
Signalling interface (FC-PH, 270) consists of related functions
FC-0, FC-1 and FC-2 each of which is described as a layer or a
level. As shown in FIG. 2, FC-0 200 defines the physical portions
of the Fibre Channel including the fiber, connectors, and optical
and electrical parameters for a variety of data rates and physical
media. Coaxial cable and twisted pair versions of the Fibre Channel
are defined for limited distance applications. FC-0provides the
point-to-point physical portion of the Fibre Channel and a variety
of physical media is supported to address variations in cable
plants. FC-1 210 defines the transmission protocol which includes
the serial encoding, decoding, and error control. FC-2 220 defines
the signalling protocol which includes the frame structure and byte
sequences. FC-3 230 defines a set of services which are common
across multiple ports of a node. FC-4 240 is the highest level in
Fibre Channel standards set and defines the mapping between the
lower levels of the Fibre Channel and the Intelligent Peripheral
Interface (IPI) and Small Computer System Interface (SCSI) command
sets, the High-Performance parallel Interface (HIPPI) data framing,
IP and other Upper level Protocols (ULPs) 280.
[0025] There are also three classes of service associated with
FC's. Class 1 is a service which establishes dedicated connections,
that is once a connection is established it is retained and
guaranteed by the fabric. This service guarantees maximum bandwidth
available between two N_Ports. Class 2 service is a connectionless
service with the fabric multiplexing frames at frame boundaries.
Class 3 is also a connectionless service with the fabric
multiplexing frames at frame boundaries but class 3 supports only
unacknowledged deliveries.
[0026] The present invention uses FC-0, FC-1 and FC-2 layers as
transport for a higher layer FC-4. The present invention also
utilizes the FC-2 capability for time division multiplexing of
multiple exchanges over the same fiber optic link. Multiple
exchanges, in this case, means multiplexing multiple channel
programs at the same time with different devices. Taking advantage
of this capability in prior art was not feasible. Hence the prior
art required the dedication of a given channel to a particular
channel program whereas the present invention enables interleaving
multiple channel program starts to separate devices on one
channel.
[0027] Existing systems use a dedicated channel to communicate with
the control unit. FIG. 3 depicts the prior art example. Normally
the channel A 310 establishes processing communication with control
unit A 320 along path M 330. In order for channel A to establish
processing communication with control unit B 350, however, channel
A has to end processing communications with control unit A, wait
for a disconnect from control unit A and then establish processing
communication with control unit B along path N 370. Each channel,
in other words, can be in communication with one control unit at a
time.
[0028] There are several advantages associated with using the
present invention. 1) Multiple control units can concurrently
access and remain in processing communication with the subsystem.
2) By multiplexing while pipelining and data streaming, the number
of sequences required to execute typical I/O operation is reduced.
3) Many functions previously handled by the channels are now
delegated to the control units for processing and issuance. 4) By
interleaving multiple starts each to a separate device on one
channel and pipelining the CCWs and streaming the associated data
to the control unit, which in turn has enhanced functionality for
processing and issuance, combined commands and data are sent to the
control units without need for interlocked responses from the
Control Unit.
[0029] The combination of the foregoing features eliminates the
need for many functions which had been required in prior art
systems. For example, in the present invention most command
responses from the Control Unit to the channels are eliminated and
command and data may be combined into a single sequence.
Furthermore, all intermediate status sequences for chained CCW
commands are eliminated. Command re-execution for some commands may
be performed at the control unit rather than being initiated by the
channel. By allowing several control units to concurrently access
the subsystem, the need for most disconnect and reconnect sequences
are also eliminated.
[0030] The present invention provides major changes in the
following areas; each of which will be discussed separately:
[0031] a) initiation of channel programs;
[0032] b) execution of channel programs;
[0033] c) CCW and data pacing
[0034] d) incorrect length and chaining of flags;
[0035] e) interrupts and recovery--program controlled
interrupts
[0036] f) Re-execution of search commands at the control unit;
[0037] g) Disconnects and reconnects;
[0038] h) channel program synchronization;
[0039] i) abnormal conditions at the channel;
[0040] j) residual count;
[0041] k) reset functions;
[0042] l) lost frames and framing structure;
[0043] m) pipelining of sequences to control unit;
[0044] n) class 2 and 3 log-ins and initialization;
[0045] I. Allowing both Command and Data to be Combined into one
Sequence
[0046] The writing of data from main storage and to I/O devices and
the reading of data from I/O devices to main storage is
accomplished by executing I/O operations. At start I/O time the
channel builds the Fibre Channel sequence with the command and the
command data if the command is a write. If the CCW count is greater
than 4k on a write CCW then each subsequent 4K bytes of data are
sent as a separate data sequence. Each Read CCW is sent as one
sequence.
[0047] Read:
1 +---------+ OUTBOUND EXCHANGE +---------+ CHANNEL
-------------------------> CONTROL
<--------------------------- UNIT +---------+ INBOUND EXCHANGE
+---------+ -----CMD-DATA--------(CC,E)-----(DX PARMS 16B)----(EX
A)-----> -----CMD-DATA-------(CH,CC,E)---(LOC FARMS 16B)---(EX
A)-----> -----CMD-------------(CH,RD-4KB)------------------(EX
A)-----> <----CNTL------------(CMR)-----------------
--------(EX A')----- -----CNTL------------(ACR)------------
-------------(EX A)-----> <----DATA------------(E)---
------(DATA-4KB)(EX A')----- <----STATUS----------(CE
DE)----------------------(EX A')-----
-----CNTL------------(ACC)------------------------(EX A)----->
<----CNTL------------(DACK)-----------------------(EX
A')-----
Example 1--READ--is an example for a Define Extend, Locate Record
followed by a read CCW for one command chain
[0048] Write:
2 +---------+ OUTBOUND EXCHANGE +---------+ CHANNEL
----------------------------> CONTROL
<--------------------------- UNIT +---------+ INBOUND EXCHANGE
+---------+ -----CMD-DATA-------(CC,E)------(DX PARMS 16B)-----(EX
B)-----> -----CMD-DATA--------(CH,CC,E)--(LOC PARMS16B)-----(EX
B)-----> -----CMD-DATA--------(CH,WD,E)--(DATA 4KB)---------(EX
B)-----> <----CNTL------------(CMR-
)-------------------------(EX B')-----
-----CNTL------------(ACR)-------------------------(EX B)----->
<----STATUS----------(CE/DE)-----------------------(EX B')-----
-----CNTL------------(ACC) ------------------------(EX B)----->
<----CNTL------------(DACK)------------------------(EX
B')-----
Example 2--WRITE--This table is an example of a Define extend,
Locate Record followed by a Write CCW
Example 3A--MULTIPLE READS AND WRITES
[0049]
3 +---------+ OUTBOUND EXCHANGE +---------+ CHANNEL
------------------------------------- CONTROL
<------------------------- UNIT +---------+ BOUND EXCHANGE
+---------+ -----CMD-DATA--------(CC,E)-------------(DX PARMS
16B)-----(EX A)-> -----CMD-DATA--------(CH,CC,E)-----------(LOC
PARMS 16B)---(EX A)-----> -----CMD-DATA--------(CC,E)--
------------(DX PARMS 16B)-----(EX B)----->
-----CMD-DATA--------(CH,CC,E)----------(LOC PARMS16B)-----(EX
B)-----> -----CMD-DATA--------(CC,E)--------------(DX PARMS
16B)----(EX C)-----> -----CMD-DATA--------(CH,CC-
,E)-----------(LOC PARMS 16B)---(EX C)----->
-----CMD-------------(CH,RD-4KB)---------------------------(EX
A)-----> -----CMD-DATA--------(CH,WD,E)----------(DATA
4KB)---------(EX B)-----> -----CMD-------------(CH,RD--
4KB)---------------------------(EX C)----->
<----CNTL------------(CMR)---------------------------------(EX
A')----- <----CNTL------------(CMR)--------------------
--------------(EX C')----- -----CNTL------------(ACR)------
----------------------------(EX A)----->
<----CNTL------------(CMR)---------------------------------(EX
B')----- -----CNTL------------(ACR)-----------------------
-----------(EX C)-----> -----CNTL------------(ACR)------
----------------------------(EX B)----->
<----DATA------------(E)-----------------(DATA-4KB)--------(EX
C')----- <----STATUS----------(CE/DE)------------------
--------------(EX B')----- <----STATUS----------(CE/DE)-
-------------------------------(EX C')-----
-----CNTL------------(ACC) --------------------------------(EX
B)-----> <----CNTL------------(DACK)----------------
-----------------(EX B')----- -----CNTL------------(ACC)---
-------------------------------(EX C)----->
<----DATA------------(E)-----------------(DATA-4KB)--------(EX
A')----- <----STATUS----------(CE/DE)------------------
--------------(EX A')----- <----CNTL------------(DACK)--
-------------------------------(EX C')-----
-----CNTL------------(ACC)---------------------------------(EX
A)-----> <----CNTL------------(DACK)----------------
-----------------(EX A')-----
Example 3A--Multiple Exchanges
[0050] This is an example of multiple command chains consisting of
Define Extend, Locate Record, a read or a write in each CCW chain.
The exchange ID is shown on each line. Each exchange is totally
independent of any other exchange.
Example 3B--WRITE Example, One CCW
Present Invention
[0051]
4 ----CMD-DATA--------(4K
BYTES--------------------------------->- ;
-----CNTL------------(CMR)------------------------------
--------> <----CNTL------------(ACR)-----------------
--------------------- <----STATUS----------(CE/DE)------
------------------------------ -----CNTL------------(ACC)--
------------------------------------>
<----CNTL------------(DACK)-------------------------------------
Prior Art -----CMD------------------------------
-------------------------> <----CNTL------------(CMR-
)------------------------------------
-----CNTL------------(ACR)------------------------------------>
-----DATA--------------------------------(DATA 1KB)---------->
-----DATA------------------------------ ---(DATA 1KB)---------->
-----DATA---------------------- -----------(DATA 1KB)---------->
-----DATA--------------------------------(DATA 1KB)---------->
<----STATUS----------(CE/DE)---------------------------------
-----CNTL------------(ACC)-------------------------------- --->
<----CNTL------------(DACK)---------------------
--------------
Example 3B--WRITE--Write Example, one CCW for present invention
versus prior art
Present Invention
[0052]
5 -----CMD-DATA--------(4K
BYTES---------------------------------&g- t;
-----CMD-DATA--------(4K BYTES-------------------------
---------> -----CNTL------------(CMR)-------------------
-------------------> <----CNTL------------(ACR)------
--------------------------------- <----STATUS-----------
(CE/DE)------------------------------------
-----CNTL------------(ACC) ------------------------------------>
<----CNTL------------(DACK)----------------------------------
---- Prior Art -----CMD--------------------
------------------------------> <----CNTL------------
-(CMR)--------------------------------
-----CNTL------------(ACR)------------------------------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
-----CMD-------------(CC)-------------- ------------------->
<----STATUS----------(CE/DE)---- ---------------------------
<----CNTL------------(CMR)-- -------------------------------
-----CNTL------------(ACR)- ------------------------------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
-----DATA----------------------------(DATA 1KB)---------->
<----STATUS----------(CE/DE)--------------------------- ----
-----CNTL------------(ACC)---------------------------- ---->
<----CNTL------------(DACK)-------------------- ------------
Example 3C--WRITE--Write Example, two CCW for present invention
versus prior art
Abbreviation and word Definitions
[0053] Following are definitions of abbreviations, used in the
above examples.
6 1. CMR ----- Command Response 2. ACR -----
Accept-command-response function 3. B ------- Bytes 4. CC ------
Command Chaining 5. CH ------ CCW Command Chained into 6. Cmd -----
Command Type sequence 7. Cmd-Data Command-Data Type sequence 8.
Cntl ---- Control Type sequence 9. Data ---- Data Type sequence 10.
Status -- Status Type sequence 11. ACC ----- Status accepted 12.
DACK ---- Device-level-ACK 13. CE ------ Channel-end status bit 14.
DE ------ Device-end status bit 15. DX ------ Define Extent Command
16. LOC ----- Locate Record Command 17. E ------- End bit indicates
the last data sequence 18. KB ------ Kilo-bytes 19. RD ------ Read
Data Command 20. WD ------ Write Data Command 21. EX( ) --
Exchange.At the FC-2 layer facilities exist that allow multiple
operations to be running concurrently each with its own exchange
ID. For system 390 this is used to run multiple concurrent I/O
operations. 22. EX( ') The second exchange of an Exchange pair. Two
exchanges are linked together at the FC4 layer. FC-2 architecture
only allows one end or the other the ability to send frames at any
one time. To get around this, an exchange pair is defined and bound
together at the FC4 layer so that within an exchange pair frames
can be sent by both sides at the same time, each using its own
exchange ID.
[0054] II. Fibre Channels and Information Units
[0055] As mentioned earlier, the present invention utilizes the
FC-4 layer of Fibre Channel. Furthermore, the present invention
utilizes the definition of an SB-2 information unit which includes
information associated which is with the execution of an I/O
operation and with the operation of a device. This information is
transferred between the channel and control unit (as information
units). The definition of the SB-2 FC-4 is based upon the FC-4
independent Information Unit construct described in FC-PH.
[0056] SB-2 Information Units (IUs) contain SB-2 device-level
commands, status, data, or control information or SB-2 link-level
control information. All SB-2 IUs are sent as FC-4 device-data
frames. There are five basic SB-2 IU types that correspond to five
of the FC-PH information categories. They are:
[0057] 1. Unsolicited-command IU
[0058] 2. Solicited-data IU
[0059] 3. Unsolicited-data IU
[0060] 4. Solicited-control IU
[0061] 5. Unsolicited-control IU.
[0062] For brevity these IUs may be referred to respectively as the
command IU, data IU, and control IU in the remainder of this
document.
SB-2 IU Structure
[0063] The structure of SB-2 IUs is shown in Example 4 below.
[0064] FC-SB-2 general IU payload structure
7 +----------------------------------------------------------+ SB-2
IU Data LRC Data PAD CRC HeaderHeader Header
+---------------------------------------------- -------------+ (8)
(8) (12) (4) (0-4224) (0-2) (0 4) <--Not present in every SB-2
structure->
Example 4
IU Payload Structure
[0065] All SB-2 IUs contain the following:
[0066] 1. Eight-byte SB-2 header
[0067] 2. Eight-byte IU header.
[0068] 3. Device-Information Block (DIB)." A DIB contains a minimum
of 16 bytes; consisting of a 12 byte DIB header followed by four
bytes of longitudinal-redundancy check (LRC).
[0069] 4. Data Field. For certain DIB types, a variable length DIB
data field may immediately follow the LRC. The maximum length of
the DIB data field is 4224 bytes. Six different DIB types are
defined; they are:
[0070] a. Data DIB
[0071] b. Command DIB
[0072] c. Status DIB
[0073] d. Control DIB
[0074] e. Command-data DIB
[0075] f. Link control DIB
[0076] The type and structure of the DIB is determined by bits in
the information unit identifier (IUI) field of the IU header. (See
"Information-Unit Identifier.")
SB-2 Header
[0077] The SB-2 header is the first eight bytes of every IU.
8 +-------------------------------------------+ 0 Reserved CH
Reserved CU Image ID Image ID
---------------------+--------------------- 1 Device Address
Reserved +---------------------'-------- --------------+ 0 16
31
Example 5
SB-2 Header
[0078] The SB-2 header provides the FC-4 addressing information
needed to identify the logical path and the I/O device for which
the exchange is being initiated. Bits 0-7 and 16-23 of word 0 and
bits 16-31 of word 1 are reserved and set to zeros by the sender
and ignored by the recipient.
[0079] CHANNEL IMAGE ID Bits 8-15 of word 0 contain the eight-bit
ID of the channel image for which the exchange is being
initiated.
[0080] CONTROL UNIT IMAGE ID Bits 24-31 of word 0 contain the 8-bit
ID of the control-unit image for which the exchange is being
initiated.
[0081] DEVICE ADDRESS When the AS bit in the IU header is set to
one, bits 0-15 of word 1 contain the address of the device for
which this exchange is being initiated.
IU Header
[0082] The information unit (IU) header is eight bytes, immediately
following every SB-2 header and preceding the DIB header. The IU
header provides SB-2 control flags and necessary information to
associate an IU to a specific CCW.
9 +-------------------------------------------+ 0 IUI DH CCW Flags
Number -----------+------------------------------- 1 Reserved Token
+-------------------------------------------+ 0 31
Example 6
IU Header
[0083] The IU header consists of four fields:
[0084] 1. IUI field
[0085] 2. Device Header (DH) flags field,
[0086] 3. CCW number field
[0087] 4. Token fields
[0088] Byte 0 of word 1 is reserved and set to zero.
Information-Unit Identifier
[0089] Byte 0 of word 0 of the IU header is the information-unit
identifier (IUI). The IU type and the functions that affect field
formats and interpretation are identified by bits in the IUI, which
has the following format:
10 +-----------------------+ 0 0 0 ASEST3T2T1
+-----------------------+ 0 3 4 5 6 7
[0090] Bits 0-2 of the IUI are reserved and are set to zeros.
[0091] ADDRESS SPECIFIC (AS): The AS bit, bit 3 of the IUI, when
set to one, indicates that the IU is associated with the specific
device identified by the device-address field of the SB-2 header.
When the AS bit is set to zero, the IU is not associated with a
specific device, and the device address is not used.
[0092] SUPPLEMENTAL STATUS (ES): The ES bit is used to control the
transfer of supplemental status. For a data IU containing a status
DIB, the ES bit may be set to either one or zero. If the ES bit is
set to one, supplemental status is present in the supplemental
status field of the DIB. If the ES bit is set to zero, supplemental
status is not present in the DIB.
[0093] DIB TYPE (T3,T2,T1): Bits 5-7 of the IUI define the type of
DIB contained in the IU. The DIB type also determines the format of
the DIB. Example 7 summarizes the setting of the T3-T1 bits.
11 +----------------------------------------------+ DIB T3 T2 T1
Type DIB Format --------+-------------+----------------------- 0 0
0 Data Data 0 0 1 Command Command Header 0 1 0 Status Flags,
status, count, supplemental status 0 1 1 Control Control function
and parameters 1 0 0 Command- Command Header & Data Data 1 0 1
Link Control Link Control Functions 1 1 0 Reserved 1 1 1 Reserved
+----------------------------------- ------------+
Example 7
DIB-Type Settings
Device-Header Flags
[0094] Byte 1 of word 0 of the IU header is the device-header flags
(DHF) field. The device-header flag bits are used individually or
collectively to invoke certain SB-2 protocols to control the
execution of an I/O operation. The format of the device-header flag
field and the description of each of its flag bits follow. Bits 5-7
of the device-header flag field are reserved.
12 +-------------------------+ E ECRDYCHEE 0 0 0
+-------------------------+ 0 1 2 3 4 5 7
[0095] END (E) The E bit, bit 0, when set to one for either a
command-data DIB or data DIB, is used to indicate that the data
sent exactly satisfies the CCW count and that the CRC field is
present in the DIB. Also, the E bit when set to one for a status
DIB is used to indicate that the quantity of data sent by the
channel with an early end indication (See "Early End (EE)."),
exactly satisfied the record length. The CCW number and token in
the IU header, collectively and in a model dependent manner,
identify the CCW to which the E bit applies.
[0096] END CONNECTION (EC): The EC bit, bit 1, when set to one, for
a status DIB, indicates that the control unit is logically
disconnecting from the channel and requesting that the exchange
pair be closed. When the EC bit is set to zero, the control unit,
while still logically disconnecting from the channel, is requesting
that the exchange pair remain open. The control unit then
reconnects on the same exchange pair that was left open.
[0097] READY (RDY): The RDY bit is set to a one by the channel in
the multipurpose control function, accept command response. For all
other control functions, command IU's and data IU's the ready bit
is meaningless and ignored.
[0098] CHAINING (CH): The CH bit, bit 3, when set to one, is used
by the channel to signal its intention to chain or to confirm that
chaining is continuing.: If the CH bit is set to one, the command
IU is a command update as a result of either command chaining or
data chaining and the DIB contains information from the current
CCW. Whether the command update is for data chaining or command
chaining is indicated by the DU bit in the command flag field. If
the CH bit is set to zero, the command IU is not a command update.
The CH bit is set to one by the channel when chaining is still
indicated for the I/O operation and set to zero when chaining is
not or no longer indicated for the I/O operation or when a retry
request is not accepted.
[0099] EARLY END (EE): The EE bit, bit 4, when set to one for
either the multi-purpose control function in a control DIB (See
"Control End."), a data DIB, or a command-data DIB is used to
indicate that the quantity of data sent for the CCW indicated by
the CCW number is less than the CCW count specified for that CCW
and for the data DIB and command-data DIB, the CRC field is
present. If the EE bit is set to zero, an early end condition is
not being indicated and the CRC field is not present in the
DIB.
[0100] CCW Number: The CCW number, byte 2 and 3 of word 0, of the
IU header contains two 8 bit binary values that are assigned by the
channel to the CCW associated with the IU being sent. The left most
8 bits (high-order 8 bits) of the CCW number represent an 8-bit
binary number that is assigned by the channel to each CCW which
contains a command. The right most eight bits (low-order bits) of
the CCW number represent a model-dependent value assigned by the
channel to every CCW. This value, in combination with the
high-order 8-bits of the CCW number identifies uniquely, every CCW
executed by the channel.
Token
[0101] The token field, bytes 1-3 of word 1, of the IU header
contains a 24-bit binary value that is provided by the channel. The
value X`000000` is not a valid token and is used only in those IUs
that do not require a valid token.
Device Information Block (DIB) Structure
[0102] A DIB consists of a DIB header, LRC, and for some DIB types
a DIB-data field. The DIB type is identified by bits in the IU
header. EXAMPLE 8 shows the DIB structure common to all DIB
types.
13 +---------------------------------------------+ DIB LRC DIB Data
Header +------------------------------ ----------------+ (12) (4)
(0-4224)
Example 8
Basic DIB Structure
DIB Header
[0103] The DIB header is the first 12 bytes of every DIB type and
immediately follows the IU header. The contents of the DIB header
and the DIB data field are determined by the DIB type.
[0104] Example 9 shows the basic DIB structure common to all DIB
types.
14 +---------------------------------------------+ 0 DIB Type
Specific --------------------------------------------- 1 DIB Type
Specific --------------------------- ------------------- 2 Reserved
IU DIB Data Count Byte Count +--------------------------
--------------------+ 0 7 15 31
Example 9
DIB Header Structure
[0105] The format and contents of the DIB header fields are
described in later sections as well as they relate to those fields
in a DIB that are common to more than one DIB type.
IU Count
[0106] Byte 1 of word 2, of each DIB header type contains the IU
count. The IU count is an eight-bit binary number that is included
in every IU sent during an open exchange.
[0107] The IU count value is initialized to X`00` when an exchange
is opened and the first IU sent. The value of zero is inserted in
the IU count field of the DIB header for the sequence sent to open
an exchange. After each IU is sent the IU count is advanced by one
and the new value included in the IU count field of the next IU
sent. When the IU count reaches X`FF` and additional sequences are
to be sent, the IU count is permitted to wrap and the next value
used is X`00`.
DIB Data Byte Count
[0108] Bytes 2 and 3 of word 2, of the DIB header contain the DIB
data byte count. The DIB data byte count is a 16 bit binary number
that represents the number of bytes of data contained in the
DIB-data field. Pad bytes and CRC are not included in the data byte
count.
Longitudinal-Redundancy-Check Field
[0109] The longitudinal-redundancy-check (LRC) field contains a
32-bit redundancy-check code, immediately following the DIB header
in bytes 28 to 31 of the 32 byte FC-4 header. LRC is provided only
on the bytes consisting of the SB-2 header, IU header, and DIB
header.
[0110] The LRC generator/checker is initialized to X`A55AA55A`
before the header fields are processed a full word at a time.
DIB Data Field
[0111] The DIB Data field, if present, is a variable length field
which contains either data, data and pad bytes, data and pad bytes
and a CRC, or just a CRC.
[0112] The quantity of data contained in the DIB data field is
indicated in the DIB-data count field of the DIB header. All DIB
types can be sent with a DIB data field of zero bytes. A command
DIB and a control DIB are always sent with a DIB data field of zero
bytes. The maximum number of bytes that can be sent in the DIB data
field is 4224 bytes.
15 +--------------------------------------------+ Data PAD CRC
+-------------------------- -------------------+ (0-4224) (0-3)
(0/4)
[0113] When the data field contains the maximum of 4223 bytes of
data, the PAD field and CRC field both contain 0 bytes. When CRC is
present in the DIB, the maximum number of bytes in the DIB data
field is 4220.
Pad
[0114] Pad bytes, if present, are contained in the last word of the
DIB data field and are used to pad the DIB data field to the next
word boundary.
Cyclic-Redundancy-Check Field
[0115] The cyclic-redundancy-check (CRC) field, if present in an
IU, contains a word-aligned 32-bit redundancy-check code. The CRC
field is present in a command-data DIB, data DIB, status DIB, and
link control DIB.
[0116] For a command-data DIB and a data DIB, the CRC field is
present only when either the E bit or the EE bit in the
device-header flags field is set to one. For a status DIB, the CRC
field is present only when the supplement status (ES) bit is set to
one. For a link control DIB containing a link payload, the CRC
field is present only when the E bit is set to one.
[0117] The processing of CRC, in terms of generation and checking,
follows the definition as provided in Annex N of the FIBRE CHANNEL,
Physical and Signaling Interface (FC-PH). Additional information
can be found in Fiber Distributed Data Interface (FDDI) Media
Access Control (MAC) (ISO/IEC 9314-2:1989) appendix B. The
initialized value of the CRC generator is X`IFFFFFFFF`.
Command DIB Structure
[0118] A command DIB is sent only in a command IU by the channel.
It is used to transfer information associated with the current CCW
to the control unit. At the beginning of an I/O operation, a
command IU containing a command DIB may be used to initiate an
operation with an I/O device. When data chaining is performed, a
command IU containing a command DIB may be used to update the
information held about the current command at the control unit.
Each command DIB contains a command header and LRC.
Command Header
[0119] The command header is the first 12 bytes of either a command
DIB or command-data DIB. The command header has the following
format:
16 +---------------------------------------------+ 0 CCW CCW CCW
Command Flags Count --------------------------------------------- 1
Reserved Command Flags
--------------------------------------------- 2 Reserved IU Data
Count Byte Count +---------------------------------------------+ 0
7 15 31
[0120] The command header contains both information from the
current CCW and information supplied by the channel necessary for
the control unit to execute the current command. The command header
describes to the control unit how execution of the I/O operation
being initiated is to be performed and additionally may also
specify how a subsequent I/O operation is to be performed.
[0121] Bytes 0-2 of word 1 and byte 0 of word 2 are reserved and
set to zero.
Command Field
[0122] Byte 0 of word 0 of the command header contains the command
specifying the I/O operation to be executed with an I/O device. The
basic operations are specified by the following commands: read,
read backward, write, control, and sense.
[0123] When a command IU is used to update the count and flags
during data chaining (the DU flag of the command DIB is set to
one), the contents of the command field is unpredictable and is
ignored by the control unit receiving the command IU.
CCW Flag Field
[0124] Byte 1 of word 0 contains the CCW flags. Bits 3 and 5-7 are
reserved and are ignored by the control unit receiving the command
IU. The value to which bits 3 and 5-7 are set is unpredictable. The
format of the CCW flag field is as follows:
17 +-------------------------------+ CD CC SLI PCI
+-------------------------------+ 0 1 2 3 4 5 6 7
[0125] CHAIN DATA (CD): The CD flag, bit 0, when set to one,
specifies an intent to perform chaining of data. It causes the CCW
flags, CCW count, and command flags designated in the next command
IU in which the DU flag and the CH bit are set to ones to be used
with the current command.
[0126] CHAIN COMMAND (CC): The CC flag, bit 1, when set to one
while the CD flag is set to zero, specifies an intent to perform
chaining of commands. Upon normal completion of the current I/O
operation and after receiving device end from the device, chaining
of commands will cause the command code specified in the command
header of the next command IU to be initiated.
[0127] SUPPRESS LENGTH INDICATION (SLI): The SLI bit, bit 2,
controls whether or not command chaining is to occur on an
incorrect-length condition. When the SLI bit is set to one and an
incorrect-length condition exists for the current command, command
chaining, if indicated, is permitted. If the SLI bit is set to zero
and an incorrect-length condition exists for the current command,
command chaining, if indicated, may or may not occur depending on
the setting of the COC flag bit. If the COC flag bit is set to zero
command chaining is not permitted and ending status with the
residual count and appropriate status flag bits are transferred to
the channel. (See "Continue on Command Immediate (COC).")
[0128] PROGRAM-CONTROLLED-INTERRUPTION (PCI): The PCI flag, bit 4,
when set to one for the current command, causes the control unit to
generate and transfer to the channel a PCI indication. The control
unit indicates that it has started execution of a command with the
PCI bit set to one by sending a command response IU or, only for
the first command of the channel program that is executed as an
immediate operation, a solicited data IU containing a status DIB.
When a command response is received, the channel recognizes that
the control unit has started execution of the CCW identified by the
CCW number in the IU header. If the PCI bit is set to one in the
identified CCW, the channel sub-system recognizes an intermediate
interruption condition.
[0129] CCW Count Field : Bytes 2 and 3 of word 0 contain the byte
count specified in the current CCW. The CCW count field is a 16-bit
field indicating the number of bytes to be transferred between the
channel and control unit during execution of the CCW. The field is
interpreted as a 16-bit unsigned binary integer. The value in the
field can range from 0 to 65,535.
[0130] Command-Flag Field : Byte 3 of word 1, contains the command
flags. The channel uses the command flags to provide the control
unit with additional information on how the I/O operation,
specified by the contents of the CCW in the command header, is to
be executed. Bits 0-2 are reserved, are set to zeros by the
channel, and are ignored by a control unit. The format of the
command flag field is as follows:
18 +-------------------------------+ DU COCSYRREXSSS
+-------------------------------+ 0 3 4 5 6 7
[0131] DATA-CHAINING UPDATE (DU) : The DU flag, bit 3, when set to
one, indicates that the CC flag, the CD flag, and the count sent in
this command IU are associated with a new CCW used during data
chaining. The CH bit in the device header is set to one when the DU
flag is set to one.
[0132] CONTINUE ON COMMAND IMMEDIATE (COC): The COC flag, bit 4,
when set to one along with the CC flag also set to one indicates
that suppression of the incorrect length condition is recognized by
the channel and command chaining to the next command is permitted
at the end of execution of a command immediate operation. When a
control unit executes a command as an immediate operation and the
CCW count for the command is nonzero, an incorrect length condition
is recognized and command chaining, if indicated, is under control
of the COC flag or SLI flag. If the COC flag or the SLI flag is set
to one, command chaining, if indicated, is permitted. If the COC
flag and the SLI flag are both set to zero, command chaining is not
permitted. (See "Suppress Length Indication (SLI).")
[0133] SYNCHRONIZE RESPONSE (SYR): The SYR flag, bit 5, when set to
one, indicates to the control unit that the command IU contains a
command being sent in response to status received for the
immediately preceding command. The channel may set the SYR bit to
one only when the CH bit is set to one and the DU bit is set to
zero in the command header.
[0134] REPEAT EXECUTE (REX): The REX flag, bit 6, when set to one,
indicates that the channel is requesting the control unit to
perform the transfer-in-channel function by repetitively executing
the command in the command header. The channel sets the REX bit for
a command only when the CCW containing the command has the CC bit
set to one, the CD bit set to zero, and the next CCW (the one to
which chaining will occur) contains a transfer-in-channel command
and an address of the previous CCW. When the control unit
recognizes this indication, re-execution of the current command is
performed until one of the following conditions occur:
[0135] 1. Execution of the command results in status of other than
channel end and device end alone being recognized.
[0136] 2. Execution of the command has been performed 128
times.
[0137] 3. Execution of the command is terminated by a cancel,
selective reset, or system reset.
[0138] When an execution of the command results in status of
channel end, device end, and status modifier alone being
recognized, chaining to the next command occurs at the control
unit.
[0139] When re-execution of the command is performed 128 times
without status other than channel end and device end alone being
recognized, re-execution of the command is terminated and the
status recognized for the last execution of the command is sent to
the channel.
[0140] When execution of the command results in status of other
than channel end and device end alone or channel end, device end,
and status modifier being recognized, re-execution of the command
is terminated at the control unit and status is sent to the
channel.
[0141] SYNCHRONIZE SEND STATUS (SSS): The SSS flag, bit 7, when set
to one, indicates that the channel has recognized a condition
requiring synchronization with I/O execution at the control unit
and is requesting the control unit to generate and present status
at the completion of the I/O operation for this command IU. In the
case of normal completion and when channel-end status is not
accompanied by device-end status, the control unit sends a status
DIB containing device-end status when the I/O operation has been
concluded at the device. The following are the conditions for which
the channel sets the SSS flag to one for the current command:
[0142] 1. The next command to be sent to the control unit is a NOOP
without chaining.
[0143] 2. The prefetch control bit is set to one, the synchronize
control bit in the ORB is set to zero, the current command being
sent to the control unit is for a write operation, and the next
command to be sent to the control unit is for a read operation.
[0144] 3. The SSS flag may be set to one depending on the setting
of the prefetch control bit and conditions at the channel.
Command-Data DIB Structure
[0145] A command-data DIB is sent only in a command IU by the
channel. It is used to transfer both information associated with
the current CCW for write commands with a nonzero CCW count and all
or some portion of the data associated with the command.
[0146] A command-data DIB contains a command header, LRC, and a
DIB-data field. The format of the command header is the same as
that of a command DIB. The DIB data field is a maximum of 4224
bytes. If the CCW count for the command is 4224 bytes or less, all
of the data for the command is sent in the DIB-data field. If the
CCW count for the command is greater than 4224 bytes, 4224 bytes
are sent in the DIB-data field and the remaining data is sent in
one or more data IUs.
[0147] The data byte count, bytes 2 and 3 of word 2, of the command
header specify the number of data bytes contained in the DIB-data
field. (See "DIB Data Byte Count.")
Data DIB Structure
[0148] A data DIB is sent only in a solicited data IU by either the
channel or control unit. For a write operation, the channel uses
one or more data IUs containing a data DIB to transfer any
remaining portion of data not sent in the command-data DIB. When
the quantity of data to be sent for a write operation exceeds the
maximum number of bytes that can be sent in an IU, the channel
sends a command IU containing a command-data DIB for the command
and the maximum number of bytes allowed; followed by one or more
data IUs until either the quantity of data specified by the CCW
count is satisfied or an early end condition is recognized.
[0149] For a read operation, the control unit uses one or more data
IUs containing a data DIB to transfer to the channel the data for
the read command. When the quantity of data to be sent requires
more than one data IU, the control unit sends a data IU followed by
one or more additional data IUs until one of the following occurs:
1) the quantity of data specified by the CCW count is satisfied, 2)
an early end condition is recognized, or 3) a condition requiring
status to be sent is recognized. All data IUs contain the maximum
number of bytes, 4224 , except the last data IU sent for a CCW
which may have fewer than 4224 bytes. When the last data IU for the
current CCW contains more than 4220 bytes of data, CRC for the data
can not be included in the data DIB and must be sent in a data DIB
by itself. This data DIB containing the CRC is sent in a data IU
following the last data IU containing data. The byte count is set
to zero and the DIB data field contains only the four-byte CRC.
[0150] The CCW number and token provided in the IU header of each
data IU sent is the same as that provided in the command IU.
Data Header
[0151] The data header is the first 12 bytes of a data DIB. The
data header has the following format:
19 +---------------------------------------------+ 0 Reserved
---------------------------------------- ------ 1 Reserved
--------------------------------------------- 2 Reserved IU Data
Count Byte Count +---------------------------------------------+ 0
16 31
[0152] Word 0, word 1, and byte 0 of word 2 are reserved and set to
zero by the sender and ignored by the recipient.
Status DIB Structure
[0153] An unsolicited data IU containing a status DIB is used to
present asynchronous status or any status that initiates a
connection and is not sent in direct response to an IU received
from the channel.
[0154] A status DIB in a data IU is sent only in either a solicited
data IU or unsolicited data IU by a control unit. It is used to
transfer status, with or without supplemental status, to the
channel. A solicited data IU containing a status DIB may be sent in
direct response to a command IU, or certain control IUs with one of
the following control functions specified: request status, accept
command response, selective reset, and a cancel that initiates an
exchange pair. Additionally, a solicited data IU containing a
status DIB may be sent during the data transfer portion of an
exchange.
[0155] After sending a data IU containing a status DIB, the control
unit discards all command and data IUs associated with that device
address and control unit image, either queued at the control unit,
or received from the channel, with the SYR bit set to zero.
Acceptance may be indicated explicitly when the control unit
receives a status accepted IU or it may occur implicitly when the
control unit receives a command IU with the SYR bit in the command
header set to one. When any of the following conditions exist,
status is sent to the channel at the completion of the current I/O
operation at the control unit:
[0156] 1. The SSS command flag bit is set to one.
[0157] 2. The CC flag bit is set to zero
[0158] 3. The first command of the channel program is executed as
an immediate operation.
[0159] 4. The CC flag bit is set to one and the SLI flag bit is set
to zero and an incorrect length condition is detected.
[0160] 5. The CC flag bit is set to one, both the SLI flag bit and
COC command flag bit are set to zero, the CCW count is nonzero, and
the command is executed as an immediate operation.
[0161] 6. When the REX bit is set to zero, and status other than
channel end (with or without device end) is recognized.
[0162] 7. When the REX bit is set to one, and the command has been
executed 128 times.
[0163] 8. When the REX bit is set to zero, and status of channel
end (with or without device end) is recognized, and conditions at
the control unit require a disconnection from the channel to
suspend the transferring of command IUs and data Ius.
[0164] If none of the above conditions exist, status is not sent
and chaining at the control unit is permitted. Additionally, status
is not sent if during an existing exchange any previously sent
status has not yet been accepted by the channel.
Status Header
[0165] The status header is the first 12 bytes of a status DIB. The
status header has the following format:
20 +--------------------------------------------+ 0 Status Status
Status Flags Parameters
-----------------------+-------------------- 1 Queuing Time
Defer-Time Parameter -----------------------+-------------------- 2
Reserved IU Supplemental Count Status Byte Count
+--------------------------------------------+ 0 16 31
Status-Flag Field
[0166] Byte 0 of word 0 of the status header contains the status
flags. The status-flag field is used to provide additional
information to the channel concerning the conditions that were
present at the control unit when status was generated and
conditions that pertain to the status DIB. These flags assist the
channel in determining how to handle the status DIB and what
status, if any, to report. The status-flag field has this
format:
21 +-----------------------+ FFC CISACRLRRV
+-----------------------+ 0 3 7
[0167] Flag-Field Code (FFC): The FFC, bits 0-2, is a three-bit
encoded field that either in conjunction with or independent of the
other status flag bits further describes the status information
contained in the status byte or the status-parameter field or both.
The flag-field code assignments are:
[0168] 000 No Function. The status byte and status-parameter field
are not affected by this FFC code.
[0169] 001 Queuing Information Valid. The queuing time field
contains control-unit queuing information. This code is set by the
control unit and checked by the channel only during the
presentation of device-end status during a reconnection for an I/O
operation.
[0170] 010 Resetting Event. A resetting-event exists for the
logical path and the device for which unit-check status is being
presented. This code is permitted only when unit-check status is
presented for a resetting-event condition and then only as initial
status in response to a command IU for the first command of the
channel program.
1011-111 Reserved
[0171] CHANNEL INITIATED (CI): The CI bit, bit 3, when set to one,
indicates that this solicited data IU containing a status DIB is in
direct response to a control IU indicating selective reset with
either a request for retry or a request for unit check.
[0172] SUPPLEMENTAL STATUS AVAILABLE (SA): The SA bit, bit 4, may
optionally be used by the control unit to indicate that
supplemental status associated with the current status is
available. The SA bit is used only when unit-check status is
indicated.
[0173] COMMAND RETRY (CR): The CR bit, bit 5, when set to one, is
used to request command retry if the status byte also contains
retry status.
[0174] LONG RECORD (LR) The LR bit, bit 6, when set to one,
indicates that a long-record condition was detected by the control
unit; that is, additional data would have been sent to the channel
or written to the device beyond the data provided for the current
command. The LR bit may be set to one by the control unit only when
the channel-end status bit is set to one.
[0175] RESIDUAL-COUNT VALID BIT (RV) The RV bit, bit 7, when set to
one, indicates that the status-parameter field contains the
residual count. For write commands, the residual count is equal to
the difference between the CCW count for the write command and the
number of bytes actually written to the device. For read commands,
the residual count is the difference between the CCW count and the
number of bytes actually read from the device and transferred to
the channel. The RV bit may be set to one by the control unit only
when the channel-end status bit is set to one.
[0176] Status Byte : Byte 1 of word 0 contains the status byte. The
status byte indicates device and control unit status. The status
byte has this format:
22 +----------------------------------------+ Status Bit Position
Description ----------------+-------------- ----------- 0 Attention
1 Status modifier 2 Control-unit end 3 Busy 4 Channel end 5 Device
end 6 Unit check 7 Unit exception
+----------------------------------------+
[0177] For more information on status bits, see IBM ESA/390 Common
I/O-Device Commands, SA22-7204.
Status-Parameter Field
[0178] The status-parameter field, bytes 2 and 3 of word 0 of the
status header, is a 1 may contain either a residual count or a
pacing parameter. The pacing parameter is provided in the status
parameter field when status is presented for the first command of a
channel program, or, when presenting device end in order to
reconnect when the chaining condition is set. In the former case of
presenting channel end plus device end status in response to an
immediate command, the status parameter field contains the pacing
parameter so this field is not available to carry the residual
count. If the conditions are such that the pacing parameter is not
present and the RV bit is set to zero, the control unit sets the
status-parameter field to zero, and the channel receiving the
status DIB ignores the status-parameter field.
[0179] RESIDUAL COUNT--The residual count is a 16-bit unsigned
binary number that represents the difference between the CCW count
for a command and the quantity of data actually transferred either
from or to the device for that command. The residual count is
always equal to or greater than zero.: For write commands, the
residual count is equal to the difference between the CCW count of
the write command and the number of bytes actually written to the
device. For read commands, the residual count is the difference
between the CCW count and the number of bytes actually read from
the device and transferred to the channel. The residual count is
meaningful only when the residual-count-valid (RV) bit is one.
Pacing Parameter
[0180] The pacing parameter is an eight-bit value that is carried
in the least significant byte of the status parameter field.
[0181] The pacing parameter is set by the control unit to indicate
to the channel the maximum number of IUs a channel may send per
outbound exchange before it receives a command response IU which
allows the control unit to allocate resources fairly among multiple
channel programs that may be executing concurrently.
Queue-Time Parameter
[0182] Bytes 0 and 1 of word 1 contain the queue-time parameter
(QTP). Queue time represents the time the I/O operation is queued
at the control unit. When control unit queuing is provided, I/O
operations specified by the channel program are executed up to a
point where the I/O resources are unavailable to execute an I/O
operation in that channel program (such resources would typically
be in use executing I/O operations specified in other channel
programs).
Defer-Time Parameter
[0183] Bytes 2 and 3 of word 1 contain the defer-time-parameter
(DTP). Device- defer-time is accumulated by the control unit
whenever ongoing channel program execution needs to be temporarily
delayed because a device-dependent action needs to be
performed.
Supplemental Status Field
[0184] Supplemental status, when present, provides additional
information concerning conditions at the control unit or device for
which status information is being provided. Supplemental status
exists only for unit-check status. When supplemental status is
included in a status DIB, it is sent in the supplemental-status
field. When supplemental status is not included in the status DIB
with its associated status information, the supplemental status is
held by the control unit until it is retrieved or reset by the
channel.
Control DIB Structure
[0185] A control DIB is sent only in a control IU by either a
channel or control unit. It is used to transfer control information
necessary for the execution of an I/O operation or for device-level
recovery. The structure of the control IU and its DIB is shown in
this application. The control DIB consists of a 12-byte control
header and a four-byte LRC.
Control Header
[0186] The control header is the first 12 bytes of a control DIB.
The control header contains information indicating the control
function to be performed and, for some functions, the parameters
needed to perform the specified function. The control header
consists of a one-byte control-function field, a three-byte
control-parameters field, IU count field, and reserved bytes. The
format of the control header is as follows:
23 +-------------------------------------------+ 0 Control Control
Function Parameters ------------------------------------------- 1
Reserved ------------------------------------------- 2 Reserved IU
Reserved Count +-------------------------------------------+ 0 7 15
31
Example 10
Control Header
[0187] Word 1 and bytes 0 and 2-3 of word 2 are reserved and set to
zeros by the sender and ignored by the recipient.
Control Function and Control Parameters
[0188] The first byte immediately following the IU header contain
the control function, which is interpreted in conjunction with the
bits in the device-header field of the IU header to determine the
device-level function to be performed and the format the
control-parameter field. A summary of the functions represented by
coding of bits 0-4 of the control-function field is shown in the
following Example:
24 +---------------------------------------------+ Bits* 0 1 2 3 4
Control Function ----------------+---------------- ------------- 0
0 0 0 0 Multipurpose 0 0 0 1 0 Command response 0 0 1 0 0 Stack
status 0 0 1 1 0 Cancel 0 1 0 0 0 Systemreset 0 1 0 1 0 Selective
reset 0 1 1 1 0 Request status 1 0 0 0 0 Device-level exception 1 0
1 0 0 Status accepted 1 0 1 1 0 Device-level acknowledgment 1 1 0 1
0 Purge path 1 1 0 1 0 Purge path response
--------------------------------------------- EXPLANATION: * All
combinations of bits 0-4 that are not listed are reserved.
+---------------------------------------------+
Example 11
Summary of Device-Level-Control Functions
[0189] For all control DIBs, bits 5, 6, and 7 of the
control-function field are set to zeros by the sender. The table in
Example 12 provides a definition, in summary form, of the IUI bits
and device-header flag bits for the various control functions.
Control IUs are sent as either FC-2 solicited or FC-2 unsolicited
control information categories. A control IU that initiates a
connection is sent as an unsolicited control information category.
A control IU that does not initiate a connection that is sent
during an existing connection is sent as a solicited control
information category. The sections following Example 12 describe
each control function and any control parameters required.
25
+---------------------------------------------------------------- +
Device-Header F IUI Flag Bits / 0 1 2 3 4 5 6 70 1 2 3 4 5 6 7 M
---------------+---------------- F / AETTTEERCE r L Ctrl SS321 CDHE
o Parm. Control Fnctn Y m Field
--------------+--------------+---------------+--+-------+- -----
Multipurpose Accept cmnd 0 0 0 1 z 0 1 1z z 1 z z z z zCH M z
response Control End 0 0 0 1 z 0 1 lz z z z 1 z z zCH M z Cmnd
response 0 0 0 1 z 0 1 1z z z z z z z zCU F/M @ Stack status 0 0 0
X z 0 1 1z z z z z z z zCHF&L/L z Cancel 0 0 0 1 z 0 1 1z z z z
z z z zCH F/L z System reset 0 0 0 0 z 0 1 1z z z z z z z zCH F/L z
Seltve reset 0 0 0 1 z 0 1 1z z z z z z z zCH F/L Request stat 0 0
0 1 X 0 1 1z z z z z z z zCH F/M z Device-level exception Address
exception 0 0 0 1 z 0 1 1z z z z z z z zCUF&L/L $ Stat accepted
0 0 0 X z 0 1 1z z z z z z zCHF&L/M/Lz Device-lvl ack0 0 0 X z
0 1 1z z z z z z zCUF&L/M/Lz Purge path 0 0 0 1 z 0 1 1z z z z
z z z zCH F z Purge path response 0 0 0 1 z 0 1 1z z z z z z z zCU
F&L z
-----------------------------------------------------------------
EXPLANATION: @ Includes pacing parameters in control-parameter
field Includes modifier bits in control-parameter field $ Includes
exception code in control-parameter field F Start new exchange
pair. (First IU of exchange pair.) M Middle IU of exchange.
(Exchange pair remains.) L Last TU of exchange-exchange pair
removed F&L Both first and last IU of exchange. F/M/L First,
Middle, or Last sequence of exchange. CH Channel sends IU to
control unit. CU Control unit sends IU to channel. X Bit is set to
one or zero as appropriate for conditions Z Bit or field is always
set to zero by the sender and is ignored by the recipient. 0 Bit is
always set to zero and is checked for zero. 1 Bit is always set to
one and is checked for one.
+---------------------------------------------------------------+
Example 12
Summary of Control IUs.
Multipurpose Control Function
[0190] The multipurpose control function is used to perform a
device-level control function that is determined solely by the
device-header flag bits, in conjunction with the context in which
the IU is received. The following functions are indicated by a
multipurpose control function:
[0191] 1. Accept command response
[0192] 2. Control End
[0193] For a multipurpose control function, the control parameter
field of the control header is set to zero by the sender. The
setting of the bits in the IUI field and the device-header flag
field depend on the function being indicated by the multipurpose
control function and the context in which it was received. The
following are descriptions of the different control functions that
can be indicated using the multipurpose control function:
Accept Command Response
[0194] When the channel uses a multipurpose control function in
response to a command response for the first command of the channel
program, the channel is indicating that the command response is
accepted and the control unit can proceed with execution of the I/O
operation. For brevity, the multipurpose-control function when used
in this way is referred to as an accept command response and a
control IU indicating accept command response is referred to as an
accept-command-response IU.
[0195] CONTROL END: The channel uses the multipurpose control
function with the EE bit set to one, to indicate control end as the
result of the channel being unable to perform a command update for
data chaining. For brevity, the multipurpose control function when
used in this way is referred to as a control end and a control IU
indicating control end is referred to as a control-end IU.
[0196] The AS bit is always set to one for a control-end IU.
[0197] When the channel sends a control-end IU, it is indicating
that, as a result of an abnormal condition, the transfer of the
command update for data chaining was suppressed at the channel and
that no more data will be sent for the current write operation. A
control-end IU sent by the channel requires a status response from
the control unit. The sending of additional IUs by the channel is
held in abeyance until the required response is received.
[0198] A control unit that receives a control-end IU continues to
execute the write operation with the device until the conditions
for ending an I/O operation are met and then it transfers status to
the channel. The status DIB carries the residual count for the
current CCW being executed when the control end IU was
received.
[0199] A control-end IU requires both a valid CCW number and valid
token. The CCW number used is the value the channel would have sent
in the command update had it been able to perform the data chain.
The token assigned is model dependent.
Command Response
[0200] A control unit uses a command-response function to indicate
that a command IU it received has been accepted for execution. For
brevity, a control IU indicating command response is referred to as
a command-response IU. A command-response IU is sent only by a
control unit. A control unit sends a command response IU only under
the following conditions:
[0201] When the control unit intends to start execution of the
first command of the channel program and it is to be executed as a
non-immediate command.
[0202] When the control unit has started execution of a command,
other than the first command of the channel program, for which the
PCI bit in the command header is set to one.
[0203] A command response IU always requires a valid CCW number and
token, which is the CCW number and token received with the command
for which the command response is being sent.
[0204] When the command response IU is sent in response to the
first command of a channel program, the least significant byte of
the control parameters field contains the eight-bit pacing
parameter. (See "Pacing Parameter.")
Stack Status
[0205] The channel uses the stack-status function to indicates that
status is not accepted. The control unit or I/O device is to hold
the status information (status flags, status byte, and status
parameter) and associated supplemental status, if any, until the
status is requested by a request-status IU, until the status is
presented as the response to a command IU, or until the status
information is cleared or withdrawn. For brevity, a control IU
indicating stack status is referred to as a stack-status IU.
Cancel IU
[0206] The channel uses the cancel function to cause the control
unit to terminate or nullify the current I/O operation, if any, for
the specified device. (See "Cancel Function.") For brevity, a
control IU indicating cancel, is referred to as a cancel IU.
[0207] A cancel IU is sent only by the channel and when recognized
by a control unit the response sent is a DACK.
[0208] A cancel IU requires a valid token but does not require a
valid CCW number.
System Reset
[0209] The channel uses the system reset function to cause the
control unit and associated devices to be reset with respect to the
logical path on which the function was received. (See "System-Reset
Function.") For brevity, a control IU indicating system reset is
referred to as a system-reset IU.
[0210] A system reset IU is sent only by the channel and when
recognized by the control unit the response sent is a DACK.
[0211] A system reset IU requires a valid token but does not
require a valid CCW number. The CCW number field for a system reset
IU is meaningless and not checked. The token field for a system
reset IU is not checked by the recipient. The token value received
in the system reset IU, is returned in the DACK response.
Selective Reset
[0212] The channel uses the selective-reset function to cause the
control unit to end execution of the current operation, if any, for
the specified device, and, depending on the bits within the
control-parameter field, to perform one of the following
functions:
[0213] Selective reset
[0214] Channel-initiated retry
[0215] Channel-initiated unit check
[0216] A selective reset IU is sent only by the channel.
Request Status
[0217] The channel uses the request-status function to indicate
that the channel is prepared to have status information presented
for the addressed device for which status is pending. (See
"Stacking Status Function" and "Request-Status Function.") For
brevity, a control IU indicating request status is referred to as a
request-status IU.
[0218] A request-status IU is sent only by the channel. The AS bit
is set to one in a request-status IU.
Device Level Exception
[0219] A control unit uses the device-level-exception function to
indicate that an abnormal condition was recognized in the IU
received for which the sending of status is inappropriate or not
permitted. The abnormal condition is indicated by the exception
code present in the first byte of the control-parameter field. For
brevity, a control IU indicating device-level exception is referred
to as a device-level-exception IU.
[0220] A device-level-exception IU is sent only by a control
unit.
Status Accepted
[0221] The channel uses the status-accepted function to indicate
that the status received in a status DIB was accepted. For brevity,
a control IU indicating status accepted is referred to as a
status-accepted IU.
[0222] A status-accepted IU is sent only by the channel and when
recognized by the control unit the response sent is a DACK.
Device-Level Acknowledgment (DACK)
[0223] A control unit uses a device-level acknowledgment (DACK)
function to indicate acceptance of a stack-status IU, cancel IU,
system reset IU, selective reset IU, request-status IU, or
status-accepted IU and that the requested function was or will be
performed. For brevity, a control IU indicating a device level ACK
is referred to as a DACK IU.
[0224] A device level acknowledgment is sent only by a control
unit.
[0225] A DACK IU sent in response to a request-status IU which
initiated a connection indicates that the requested function was or
is to be performed. If the channel recognizes a DACK IU in response
to a request-status IU which does not initiate a connection, the
channel detects a device-level protocol error.
Purge Path
[0226] A channel uses the purge path function as part of
device-level recovery in order to purge the transmit-receive path
of any residual IUs associated with the logical path and device for
which recovery is being performed. For brevity, a control IU
indicating purge path is referred to as a purge path IU.
[0227] A purge path IU is sent only by a channel and when
recognized by a control unit the response is a purge path response
IU.
[0228] A purge path IU requires a valid token but does not require
a valid CCW number. The CCW number field for a purge path IU is
meaningless and not checked. The token value received in the purge
path IU is returned in the token field of the purge path response
IU.
[0229] A purge path IU is sent as the first sequence of an outbound
exchange as an unsolicited control FC-2 information category.
Purge Path Response
[0230] A control unit uses the purge path response function to
indicate receipt and recognition of the purge path function. For
brevity, a control IU indicating purge path response is referred to
as a purge path response IU.
[0231] A purge path response IU is sent only by a control unit in
response to a purge path.
[0232] A purge path response IU is sent as a solicited control FC-2
information category, and is sent as the first and last sequence of
the exchange.
Link-Control DIB Structure
[0233] A link-control DIB is sent in a control IU by the channel or
control unit. It is used to transfer SB-2 link-level information. A
control IU containing a link-control DIB is referred to as a
link-control IU. A link-control IU is used to perform the following
link-level functions:
[0234] establish a logical path,
[0235] remove a logical path,
[0236] exchange ID,
[0237] test initialization,
[0238] and to transfer incident information.
[0239] A link-control DIB contains a link header, an LRC, and for
some functions a link payload with its associated CRC.
Link Header
[0240] The link header is generated by the channel or a control
unit and it appears immediately after the IU header of either a
solicited or unsolicited control IU. The link header contains
information indicating the link-control function to be performed
and, for some functions, the parameters needed to perform the
specified function. The link header consists of a one-byte link-
control-function field, a two-byte link-control-information field,
a one-byte IU count, a two-byte link payload byte count, and
reserved bytes. The format of the link header is as follows:
26 +--------------------------------------------+ 0 Reserved Link
Link Control Control Information
-------------------------------------------- 1 Reserved
--------------------------------------- ------ 2 Reserved IU Link
Payload Count Byte Count +---------------------------------------
------+ 0 16 31
Example 13
Link Header
[0241] Byte 0 of word 0, word 1, and byte 0 of word 2 are reserved
and set to zero.
[0242] A link-control function when sent as a request by the
channel requires a valid token and when sent as a request by a
control unit, the token value is meaningless and not checked by the
channel. A link control response sent by a control unit uses the
token and CCW number from the IU to which it is being sent in
response. A link control response sent by a channel does not use
the token and CCW number from the IU to which it is being sent in
response.
Link-Control Information
[0243] The link-control information field, bytes 2 and 3 of word 0,
contain additional parameters necessary in order to perform the
link-control function specified by bits 0-4 of byte 1 of word 0.
The parameters are determined by the link-control function
specified.
[0244] The parameters associated with a link-control function are
contained in the first byte of the link-control information field,
byte 2 of word 0. The second byte of the link-control information
field, byte 3 of word 0, is always set to zero.
Link Payload
[0245] The link payload is a variable length field with a maximum
length of 4220 bytes.
[0246] III. Device-Level Functions and Protocols
[0247] This section describes the functions and protocols necessary
for the execution of an I/O operation and the exchange of control
information. These functions depend on the successful completion of
all FC-PH protocols.
[0248] A. Device-Level Operations
[0249] Initiation of an I/O operation, the transfer of data, and
the ending of an I/O operation are device-level operations which
rely on SB-2 device-level functions and protocols. The device-level
functions discussed in this section have all been defined in "SB-2
Information Units." This section describes how these functions are
used in order to perform device-level operations.
[0250] B. Channel Program Execution
[0251] A channel may concurrently execute one or more channel
programs on the same or different logical paths. Each channel
program executed by a channel is for a different device and
requires a different exchange pair to be used. The number of
channel programs that may be concurrently executed by a channel is
model dependent.
[0252] Notes
[0253] 1. The fetching of CCWs for the purpose of initiating
multiple I/O operations with a device requires that the prefetch
control (P) bit in the ORB be set to one.
[0254] 2. When the channel fetches from storage and sends to the
control unit the information contained in a CCW, that CCW is
considered the current CCW at the channel. This process is referred
to as the execution of the current CCW.
[0255] 3. CCWs that are executed by the channel with the sending of
a command IU to the control unit are considered active at the
channel until an indication is received that CCW has been executed
at the device. The term "active" in this case means that the
channel must be capable of making that CCW current again. Once the
channel has received a positive indication that a CCW has been
executed at the device, that CCW can be retired and considered no
longer active.
[0256] 4. The CCW number assigned to a CCW that is considered no
longer active at the channel may be reassigned.
[0257] C. Initiating an I/O Operation
[0258] An I/O operation is initiated with an device when the
channel transfers the command from the current CCW to that device.
The command is transferred in the command header of a command IU.
(See "Command Header.") A channel executing a channel program may
concurrently initiate multiple I/O operations with the same device.
Each I/O operation consists of the transfer of a command in a
command IU containing either a command DIB or a command-data DIB
and, in the case of a write operation with a nonzero CCW count, the
transfer of data in a data IU. A control unit that receives one or
more command IUs, each initiating an I/O operation, executes each
command in the order in which it is received from the channel.
Execution of the command first requires acceptance of the command
at the control unit. When a command is accepted, that command is
considered current at the control unit until execution at the
device is considered completed.
Data-Transfer Protocol
[0259] Data is transferred between the channel and the control unit
as part of the execution of an I/O operation. The transfer of data
from the channel to the control unit is a write operation. The
transfer of data from the control unit to the channel is a read
operation. The operation, read or write, is determined by the
current command. Read commands have bit 7 of the command byte set
to zero, and write commands have bit 7 of the command byte set to
one.
[0260] Some read and write commands do not result in the transfer
of data. These commands are executed as immediate operations, are
designed to be executed without transferring data, have counts of
zero in the CCW, or have data-record lengths of zero. (See
"Initiating an I/O Operation" for the protocols used when the
command is executed by the control unit as an immediate operation.
See "Zero CCW Count" for the protocols used when the CCW count is
zero.)
[0261] When, for first command of a channel program with a nonzero
CCW count, the data-record length at the control unit is zero but
the command is not accepted as an immediate operation, a command
response IU is sent to the channel. Following receipt of the accept
command response, status containing channel end with or without
device end is sent to the channel, the residual-count-valid bit is
set to one, and the residual count field contains the CCW count
received with the command. For this case, if the command is a read
command, no data is transferred to the channel. If the command is a
write command and data was sent with the command, the data is
discarded by the control unit. When, for a command other than the
first command of a channel program with a nonzero CCW count, the
data record length at the control unit is zero, a command response
IU is sent to the channel only if the PCI flag is set to one for
the command. If conditions at the control unit require the sending
of status to the channel, status is sent after any command response
that may be required. In this case the residual counts and data
transfer rules for the first command of a channel program apply. If
the conditions at the control unit do not require the sending of
status and chaining is to take place at the control unit, only a
command response, if requested is sent; otherwise, nothing is sent
to the channel.
[0262] Data is transferred from the channel to the control unit in
a command-data DIB sent in a command IU or data DIB sent in a data
IU. Data is transferred from the control unit to the channel only
in a data DIB. See "Command-Data DIB Structure" and "Data DIB
Structure" for the maximum DIB sizes. The quantity of data
transferred for an I/O operation is determined by the count in the
CCW for the operation or the data required by the device for the
current command, when less than the CCW count. For each CCW that
results in a command IU being sent to the control unit, the CCW
count is included in the command header. For a write operation, the
CCW count is used by a control unit to determine the quantity of
data to expect from the channel. For a read operation, the CCW
count is used by the control unit to determine the quantity of data
that can be sent to the channel. For both a read and write
operation the CCW count is used by the control unit to determine
the residual count to be returned to the channel, if status is to
be presented, and it is used in conjunction with the CCW flags and
command flags, to determine if chaining is to occur.
Write Operation
[0263] A write operation is the transfer of data from the channel
to the control unit as part of the execution of a command that has
bit 7 set to one.
[0264] For a write operation with a nonzero CCW count, the channel
sends the command and the data for the command in a command data
DIB. If the quantity of data specified by the count for the current
CCW exceeds the maximum allowable DIB size (see "Command-Data DIB
Structure"), the remaining data is sent in data DIBs (see "Data DIB
Structure") until all of the data specified by the CCW count is
sent. If all of the data specified by the CCW count is sent, the E
bit is set to one in the last IU sent containing the CRC. If all of
the data specified by the CCW count is not sent, the EE bit is set
to one in the last IU sent containing the CRC.
[0265] When, during a write operation, the channel is able to send
some but not all of the data for a CCW, the EE bit is set to one in
the last IU sent. The channel, for the affected channel program,
suspends sending IUs to the control unit until status is
received.
[0266] When status is received, the channel may or may not resume
execution of the channel program with the control unit, depending
on the contents of the status DIB and conditions at the
channel.
[0267] When, during a write operation, the channel is unable to
send any of the data specified for a CCW, the EE bit is set to one
for the command data DIB, the command header contains the required
information for the CCW, and only the initialized value of the CRC
generator is sent in the DIB data field.
[0268] When, for a write operation, the quantity of data required
by a device is equal to or greater than the CCW count, the
resulting residual count is zero. If the quantity of data required
by the device is less than the CCW count, the resulting residual
count is the difference between the data required and the CCW
count.
[0269] When, at the end of a write operation, status including
channel end is sent to the channel, the residual count calculated
is included in the status DIB. If the quantity of data required by
the device, is greater than the CCW count the LR status flag is
also set to one to indicate that the device would have used more
data had it been provided.
[0270] When the quantity of data received by the control unit for a
write operation is less than the quantity specified by the CCW
count and the EE bit is set to one in the last IU received, the I/O
operation is executed and the data received, if any, is used by the
device.
[0271] If the quantity of data required by the device is exactly
equal to the quantity of data received, the E bit is set to one for
the status DIB sent. If the quantity of data required by the device
is either less than or greater than the quantity of data received,
the E bit is not set to one for the status DIB sent; CRC is checked
on the entire amount of data received.
Read Operation
[0272] A read operation is the transfer of data from the control
unit to the channel as part of the execution of a command that has
bit 7 set to zero. For a read operation the channel sends the
command to the control unit in a command DIB in a command IU. The
CCW count and CCW flags for the command are included in the command
header.
[0273] When, for a read operation, the quantity of data sent to the
channel exactly equals the count for the current CCW, the resulting
residual count is zero and the E bit is set to one in the last data
IU sent for that CCW.
[0274] If the quantity of data sent to the channel is less than the
count for the current CCW, the resulting residual count is the
difference between the quantity of data sent to the channel and the
CCW count and the E bit is set to zero and the EE bit is set to one
in the last data IU sent for that CCW.
[0275] If the control unit would have transferred more data to the
channel had a larger count been provided or data chaining
indicated, the resulting residual count is zero and the E bit is
set one in the last data IU sent for that CCW.
[0276] When, at the end of a read operation, status including
channel end is sent to the channel, the residual count calculated
is included in the status DIB. If the quantity of data sent to the
channel by the device, is exactly equal to the CCW count but the
device would have sent more data had a larger CCW count been
provided, the LR status flag is also set to one.
Data Chaining
[0277] When there are successive CCWs to be executed by the channel
for a single I/O operation, the execution of these CCWs is referred
to as data chaining. When each CCW associated with a channel
program becomes the current CCW being executed, the channel
indicates to the control unit the initiation of a data chaining
operation by transferring a command-IU containing the chain-data
flag, CCW count, and other flags associated with the CCW.
Subsequent IUs containing data DIBs may also be transferred, either
by the channel or control unit, depending upon whether input or
output was specified in the command IU initiating the I/O
operation.
[0278] When execution of the current CCW is completed and the
chain-data flag is set to one, data chaining takes place, provided
no abnormal conditions are detected and all other conditions for
data chaining are satisfied.
[0279] Data chaining can occur only when the CD bit was set to one
for the current CCW. When data chaining takes place, the command IU
is used to update the flags and count held at the control unit, the
chaining bit (CH bit) of the device header flag field and the
data-chaining-update (DU) flag of the command-flag field are both
set to ones.
[0280] If the control unit has received a command IU with the DU
flag bit and the CH bit set to ones, if the control unit has the
data-chaining condition set, and if the chain-data bit was set to
one in the previous command IU, then the command flags and count
from the command IU are accepted and become the current flags and
count controlling the I/O operation.
[0281] The chain-data bit in the CCW-flag field of the command IU
indicates there is a subsequent count and command-flag update for
the device that will immediately follow the execution of the
current command IU, provided that no abnormal conditions are
encountered.
[0282] In some cases while data chaining, the channel may request
the transfer of status when the last CCW of a data chain is
executed. The channel does this by setting the SSS bit to one
signalling the control unit to provide ending status in an IU
containing a status DIB when the I/O operation has ended. In this
case, if command chaining was set to one in the last CCW of the
data chain, then normal command chaining would occur after the
channel received the IU containing the status DIB. The control unit
provides a status DIB if data chaining is ended and no further
chaining is specified.
[0283] When data-chaining occurs at the channel during a write
operation, a command IU with a command-data DIB containing the CCW
count, required flags for the next CCW, and data is sent to the
control unit. If the quantity of data specified for the CCW exceeds
the quantity of data that can be sent in a command data DIB, one or
more data DIBs may be sent. If the quantity of data sent for the
CCW is exactly equal to the count in the CCW, the E bit is set to
one in the last IU sent for the CCW. Data chaining continues until
all of the data is transferred for the write operation or until the
data chaining condition is not set at the end of a CCW. If the
quantity of data sent for the CCW is less than the count in the
CCW, the EE bit is set to one in the last IU sent for the CCW. For
this case, data chaining, if indicated for the CCW, is not
performed and execution of the channel program is suspended until
status is received from the control unit. If the quantity of data
used by the device is less than the total count provided by the
channel, then a status DIB is sent; the residual count field
indicates the amount of data which was not used. If the quantity of
data required by the device is greater than the total count
provided and the EE bit is not set, then status, if sent, has the
LR bit set to one.
[0284] When, at the channel for a read operation, data chaining is
indicated in the current CCW, a command IU with a command DIB
containing the CCW count and required flags for the next CCW is
sent to the control unit. The channel continues to do this for each
CCW to which data chaining was specified until the command IUs for
all of the CCWs for data chaining have been sent. For a read
operation, the sending of the command IUs indicating data chaining
does not constitute execution of the command or CCWs. The operation
and data chaining is considered to have been initiated at the
control unit. There is an indefinite period of time from the
initiation of a read operation with data chaining and when data for
that operation arrives at the channel.
[0285] When a data IU is received by the channel, the CCW number
indicates the CCW for which this data is being sent. At that point
in time, the CCW identified becomes current again at the channel
and execution of the CCW is considered to be occurring. As each
successive data IU is received for a read operation the channel
ensures that the CCW number specified corresponds to the
appropriate CCW. The quantity of data received for a CCW must
exactly equal the CCW count for that CCW in order for data chaining
to occur at the channel. If, before all of the data for a CCW is
received, the next data IU received from the control unit specifies
a different CCW than the current CCW for the read operation, a
device-level protocol error is recognized. If either status or a
data IU with the EE bit set to one is received, data chaining is
ended for the current read operation. When, at the control unit for
a read operation, the quantity of data to be sent to the channel
exceeds the CCW count in effect from the command IU currently being
executed, the E bit is set to one in the last data IU sent for the
CCW and if the data chaining condition is set, data chaining occurs
at the control unit.
[0286] When data chaining occurs at the control unit, the next CCW
goes into effect, that is, the contents of the next command IU with
the CH bit set to one and a command DIB containing the DU flag set
to one become current and go into effect for the read operation.
The remaining data is sent in data IUs under the control of the new
CCW. The CCW number from the current command IU is used in each
data DIB sent, and the CCW count determines the maximum quantity of
data that can be sent. Data chaining continues at the control unit
until all of the data is transferred for the read operation or at
the completion of the current CCW, the data chain condition is not
set. If all of the data is transferred for the read operation and
the data chaining condition is set, data chaining does not occur
and status is sent to the channel. If all of the data has not been
transferred to the channel for the read operation and the data
chaining condition is not set, data chaining does not occur and a
status DIB, if sent, contains the LR status flag bit set to one.
The device ensures that data chaining is occurring at the proper
times by recognizing a data-chaining condition. The data-chaining
condition is recognized for each device and is used to verify that
successive CCWs are being executed by the channel. The
data-chaining condition is set whenever the device accepts a
command IU and the chain-data bit is set to one in the command-flag
field. The data-chaining condition is reset whenever a control IU
is received and system reset, or selective reset, is performed;
cancel or stack status is performed. The data-chaining condition is
also reset whenever a command IU is received and the chain-data bit
is set to zero, or status containing device end has been accepted
by the channel.
[0287] When the data-chaining condition is set and the control unit
initiates the ending of the I/O operation by transferring
channel-end status to the channel, with or without device-end
status, and if the quantity of data transferred is less than the
CCW count, then the data-chaining indication is reset, and chaining
is not performed. Data chaining occurs whenever the data-chaining
condition is set, and the chaining (CH) bit in the device-header
flag field and the data-chaining-update flag (DU flag) are both set
to ones in the command IU. If the data-chaining condition is set in
the control unit and if a command IU is recognized with the
chaining bit (CH bit) or the data-chaining-update flag (DU flag)
set to zero, then a device-level protocol error is detected.
[0288] If the data-chaining and command-chaining conditions are
both not set and if a command IU is received with either the DU
flag set to one or the CH bit set to one, then a device-level
protocol error is detected. The CH bit and DU flag are tested
before the data-chaining condition is changed to conform to the new
value of the chain-data bit in the command-flag field. If the
data-chaining condition is set, the control unit ensures that the
path to the device remains available when the current count is
exhausted, until the next command IU is recognized or until the
data-chaining condition is reset.
Nullification Control During I/O Initiation
[0289] The present art has a protocol requirement that precluded
the initiation of an I/O operation until the control unit receives
notification from the channel that its status had been accepted for
the first command of a channel program. This protection was needed
for some devices to insure a proper level of recovery from errors
occurring during the initiation phase of an I/O operation; however,
this also generates a through-put performance penalty because
initiation of I/O operations are delayed awaiting the channel
signal. The penalty is primarily caused by internal channel delays
in generating the response signal and in propagation delays.
Further aggravating the situation is the fact that no software
changes are allowed during the development of systems such as
ESCON. Hence, the rule has to be enforced across ALL device types
independent of whether or not the protection was necessary.
[0290] Under the instructions described in the present invention,
the change restriction is relaxed allowing a finer granularity of
control. This allows the incorporation of four concepts:
[0291] A) System software (Device Service Exit and Input/Output
Supervisor) recognizing the set of devices not needing
nullification protection.
[0292] B) A new control function defined in the subchannel and
activated by system software (Device Service Exit) signaling the
channel that nullification protection is not in effect for the
respective subchannel.
[0293] C) A command, with appropriate parameter information,
generated by System Software (Device Service Exit) and transferred
by the channel advising the control unit that nullification
protection is not in effect.
[0294] D) A set of defined protocol rules describing how the
control unit is to function when nullification protection is not in
effect.
[0295] D. Ending an I/O Operation
[0296] The ending of an I/O operation is either channel initiated
or control-unit initiated. The channel may initiate the ending of
an I/O operation as the result of an abnormal condition or a
non-error condition. The control unit may initiate the ending of an
I/O operation as the result of the completion of the execution of
the command or the transfer of all data associated with the command
or as the result of an abnormal condition detected during the
execution of the command. When the channel initiates the ending of
the I/O operation, it sends a control IU indicating one of the
following control functions:
[0297] 1. Cancel (see "Cancel Function.")
[0298] 2. Selective reset (see "Selective-Reset Function.")
[0299] 3. System reset (see "System-Reset Function.")
[0300] When conditions require that the control unit initiate the
ending of the I/O operation with the channel, it sends status in a
status DIB and discards all command and data IUs received
subsequent to the current command. When conditions at the control
unit permit the ending of an I/O operation without the transfer of
status to the channel, the control unit starts execution of the
next command. (See section "Status DIB Structure.")
[0301] In the absence of errors, when the current command is
executed as a non-immediate operation, the control unit initiates
the ending of the I/O operation when one of the following
conditions is satisfied:
[0302] 1. All of the data specified by the CCW count has been
transferred and the data-chaining condition is not set.
[0303] 2. The data required by the device for the command has been
transferred.
[0304] When the current command is executed as an immediate
operation, the control unit initiates the ending of the I/O
operation when the conditions for the device allow channel-end
status to be generated.
Ending an I/O Operation Without Transferring Status
[0305] When execution of the current command reaches completion
with the device and the conditions at the control unit require that
status not be presented to the channel, the I/O operation is
considered to have ended and execution of the next command in the
chain may proceed.
Ending an I/O Operation with Transferring Status
[0306] When the control unit ends execution of the current command,
including the transfer of data, and the conditions at the control
unit require status to be transferred to the channel, a status DIB
indicating channel-end status is sent along with the appropriate
status flags and residual count. If the operation is ended because
of an abnormal condition, the control unit includes the appropriate
status for this condition. The control unit may or may not include
device end status. If device-end status is included, the operation
is considered ended by the device when this status is accepted by
the channel. If device-end status is not included, the I/O
operation is considered ended by the device when the control unit
later transfers device-end status and receives acknowledgment that
the status has been accepted by the channel.
[0307] If unit-check status is included, the control unit can
indicate that supplemental status is available by setting the
supplemental-status-available (SA) bit to one in the status
DIB.
[0308] If the control unit is connected when it is ready to
transfer status in order to initiate the ending of the I/O
operation with the channel, a status DIB with channel end with or
without device-end status is sent.
[0309] If the control unit is not connected when it is ready to
transfer status in order to complete the ending of the I/O
operation with the channel, a data IU containing a status DIB
indicating device-end status is sent.
[0310] When, during an existing connection, conditions at the
device require that channel-end status without device-end be sent
and the chaining condition is set, the control unit may send a
status DIB without device-end status and indicate whether or not
the exchange pair should be left open.
[0311] If the control unit wants to request that the exchange pair
be left open with the acceptance of this status, the EC bit in the
IU header is set to zero; otherwise, the EC bit is set to one and
the channel will initiate closing the exchange pair when status is
accepted.
[0312] When the control unit indicates that it wants the exchange
pair to remain open with the acceptance of status, the channel may
or may not honor that request.
[0313] If the conditions to allow chaining are satisfied, the
channel sends a status-accepted IU with the CH bit set to one and
leaves the outbound exchange open, which is the signal to the
control unit that its request has been honored. The control unit
responds with a DACK IU leaving the inbound exchange open. If
conditions at the channel do not allow chaining, the channel
responds with a status-accepted IU and closes the outbound
exchange, which signals the control unit that its request has not
been honored and to close the inbound exchange with the sending of
the DACK IU.
[0314] When the control unit's request to leave the exchange pair
open has been honored by the channel, the device is considered to
be no longer connected to the channel even though an active
exchange pair still exists. Neither the control unit nor the
channel is permitted to use this exchange pair for other operations
or other devices.
[0315] When the control unit has device-end status for the device
that was active on the exchange pair at the time of the
disconnection, it is presented on this open exchange pair.
Supplemental Status
[0316] Supplemental status provides sense information in a status
DIB to describe conditions at the control unit or device for which
status information containing unit check is being provided. The
control unit does not transfer supplemental status unless the
channel has indicated its ability to accept it. The channel
indicates whether it can accept status along with supplemental
status when the channel sends a request-status IU. The
supplemental-status (ES) bit, when set to one, indicates that the
channel will accept supplemental status. When the ES bit is set to
one in a request-status IU, the channel is indicating that it can
accept a status DIB containing supplemental status as a direct
response to the request-status IU.
Residual-Count
[0317] The control unit determines the residual count by
calculating the difference between the CCW count and the actual
number of bytes either written to a device or read from a device
and sent to the channel. The value of the residual count is
dependent on the CCW count, the number of bytes transferred between
the channel and control unit, and the number of bytes actually read
or written to the device. For a write operation the channel may
transfer more data to the control unit than the device requires,
but never more data than specified by the CCW count.
[0318] If the channel transfers all of the data specified by the
CCW count to the control unit and all of the data is used by the
device, the residual count is equal to zero for this command.
[0319] If a write command with a nonzero CCW count is executed as
an immediate I/O operation, the residual count is equal to the CCW
count.
[0320] When, for a write operation, the device requires less data
than specified by the CCW count, the residual count is equal to the
difference between the CCW count and the actual number of bytes of
data used. Alternatively, if the device requires more data than
specified by the CCW count, the residual count is zero for this
command. For the case where, for a write operation, the channel
transfers less data than specified by the CCW count (see "Control
End" and "Write Operation"), the residual count is still determined
by calculating the difference between the CCW count and the actual
number of bytes used by the device.
[0321] For a read operation the control unit transfers to the
channel only a quantity of data equal to or less than the CCW count
for the command. The residual count is the difference between the
CCW count and the actual number of bytes sent. If the control unit
transfers all of the data specified by the CCW count, the residual
count is equal to zero for this command. If, a read command with a
nonzero CCW count is executed as an immediate I/O operation, the
residual count is equal to the CCW count. When, for a read
operation, the control unit transfers less data to the channel than
specified by the CCW count, the residual count is equal to the
difference between the CCW count and the actual number of bytes
sent. If, for the read operation, all of the data specified by the
CCW count is sent but more data would have been sent to the channel
had a larger count been specified, the residual count is equal to
zero.
[0322] Note that there are only two cases in which the residual
count is greater than zero:
[0323] 1. when the record length is less than the CCW count,
and
[0324] 2. when, during a write operation, the channel is unable to
send all of the data for a CCW and has set the EE bit in the last
IU sent.
[0325] In the first case, the LR bit is set to zero, and in the
second case the LR bit is meaningless. The following abnormal
situations may require that the control unit set additional flags
in the device header or may prevent the control unit from including
a valid residual count in a status DIB:
[0326] 1. The control unit sets the E flag to one in the device
header for a status DIB only for a write operation involving an
early end indication for which the amount of data received by the
control unit and written to the device exactly equals the record
length.
[0327] 2. If an abnormal condition prevents the control unit from
determining the exact amount of data either read from or written to
a device, the control unit is unable to calculate a valid residual
count. In this case, the control unit sets the RV bit equal to
zero. It also indicates a unit check along with channel end status
or it requests command retry. See definition of CR bit in Status
section "Command Retry." If the channel accepts the command retry
request, the retry proceeds normally. If command retry is not
honored, the channel terminates the I/O operation. If the channel
receives a status DIB with the channel-end status bit set to one,
with the unit check bit set to zero, and with the residual count
valid bit set to zero, the channel detects a device-level protocol
error.
[0328] 3. During a read operation, if an abnormal condition
prevents the control unit from determining the exact amount of data
transferred to the channel, then the control unit cannot calculate
a valid residual count. In this case, it sets the RV bit equal to
zero and indicates unit check along with channel end status. The
control unit may also request retry as described above. During data
chaining, the residual count from the current CCW is reset. When
the next CCW takes effect. a new residual count is started for the
data transfer associated with the new CCW. If a status DIB is sent
as a result of the new CCW, it contains a valid residual count.
Command Chaining
[0329] When there are successive I/O operations to be executed by a
channel and a control unit for a single channel program, the
execution of these successive I/O operations is referred to as
command chaining. When an I/O operation is initiated by the
channel, it uses a command IU to transfer the command, CCW flags,
CCW count, and command flags associated with the current CCW to the
control unit. In this manner, the control unit is informed whether
or not command chaining is specified for the current CCW. When an
I/O operation is completed and the chain-command flag is set to one
and the chain-data flag is set to zero, command chaining is
permitted, provided no abnormal conditions are detected and all
other conditions for chaining are satisfied. When command chaining
takes place, the next command and associated flags and count become
current and the next I/O operation is considered to have started.
The next command and associated flags and count are found in the
next command IU received with chaining bit (CH bit) set to one and
the data-chaining-update flag (DU command flag) set to zero. The
control unit ensures that command chaining is occurring at the
proper times by recognizing a command-chaining condition. The
command-chaining condition is recognized for each device. The
command-chaining condition is set as the result of command chaining
or as a result of command retry. During command chaining, the
command chaining condition is set whenever the device accepts a
command and the CC flag is set to one and the CD flag is set to
zero in the CCW-flag field of a command IU.
[0330] The command-chaining condition is set for command retry when
the CH bit is set to one in the IU used by the channel to accept
the status that requests a command retry. (See command retry
"Command Retry.") When the command chaining condition is set and
the conditions for not sending status at the completion of
execution of the current command are all met the control unit
performs command chaining by starting execution of the command in
the next command IU with the CH bit set to one and the DU command
flag bit set to zero. (See "Status DIB Structure" for the
conditions when status is sent and when chaining occurs at the
control unit.) If command chaining condition is set but the channel
has requested that status be sent at the completion of execution of
the current command, the control unit sends status to initiate
command chaining with the channel. In this case command chaining
does not occur until the channel receives status with device end
indicated and the channel sends a command IU for the next command
to be executed.
[0331] The command-chaining condition is reset whenever:
[0332] (1) system reset or selective reset is performed;
[0333] (2) the device receives a cancel or stack-status in a
control IU;
[0334] (3) status containing channel end but not device end for an
I/O operation has been accepted by the channel and the chaining bit
is set to zero in the status-accepted IU;
[0335] (4) status containing device end for an I/O operation has
been accepted by the channel using an accept-status IU;
[0336] (5) an address-exception condition is recognized and a
control IU with a device-level-exception indication is sent; or
[0337] (6) the channel accepts a valid status DIB with the AS bit
set to one and the status contains unit check (except when command
retry is requested and the retry requested is to be honored by the
channel), unit exception, busy, attention, status modifier without
any other bits set, or control-unit end, provided it is not
control-unit end alone. A valid status DIB for which the AS bit set
to zero does not suppress command chaining.
[0338] Command chaining occurs whenever the command-chaining
condition is set at the device, and the CH bit and the DU bit are
set to one and zero, respectively. If the command-chaining
condition is set at the device, the control unit ensures that the
path to the device remains available when device-end status is
presented until the next command is recognized or until the
command-chaining condition is reset. If the command-chaining
condition is set at the device and if a command IU is recognized
with the CH bit set to zero or the DU flag set to one, a
device-level protocol error is detected. If the command-chaining
and the data-chaining conditions are both not set and if a command
IU is recognized with the CH bit set to one, then a device-level
protocol error is detected.
[0339] E. I/O Synchronization Control
[0340] The Fibre Channel interface architecture provides the
protocols necessary to allow the pipelined execution of CCWS, thus
reducing overheads required for the execution of S/390 channel
programs. However, not all channel programs can execute correctly
if prefetching is performed. For example, channel programs that
modify themselves during execution by reading CCWs into the CCW
stream will not work as intended if the CCWs are prefetched from
memory. Thus the ability for the channel to do prefetching is
explicitly enabled by the software via an existing parameter in the
Operation Request Block (ORB P bit). The definition of channel
program prefetch is expanded to include both CCWs and the
associated data (when its a write).
[0341] Even when the channel is permitted to prefetch CCWs and data
by the software, there are times when it is necessary that the
pipelining of CCWs must stop and the channel wait for the execution
of all the outstanding commands complete before resuming the
execution of the channel program and continue pipelining. This
synchronization point is needed when the CCWs being executed
transition from a read command to a write command. The
synchronization is required just in case the channel program is
reading data into the same buffers that will be used for the write
command. If the channel did not synchronize on this transition from
read to write CCWs then the wrong data may be written to the I/O
device. The cost and complexity for the channel to determine if the
same storage locations and lengths are used for the command is too
costly to implement because of the additional channel overheads and
development complexity.
[0342] There are many examples of where the software builds channel
programs that have many read/write transitions and thus would not
be able to benefit from CCW pipelining. Examples include DASD
channel programs with multiple domains (the LOCATE Record CCW looks
like a write command), printer channel programs, CTC communication
(MPC protocol), etc. In all these cases the software knows that the
same storage locations are not being used for both the read and
write buffers. A new parameter is invented for the ORB that allows
the program to override the channel read/write synchronization in
order to optimize performance. This parameter is known as the I/O
Synchronization Control. This parameter is specified by the
applications that build virtual/real channel programs. The system
can automatically set the I/O Synchronization override in cases
where virtual channel programs are converted to real channel
programs, with some degree of accuracy.
[0343] The present mechanism provides for the software (application
and operating system) to override I/O synchronization points to
reduce overheads and improve I/O response times.
[0344] F. Device-Level Controls
[0345] Device-level controls are used to modify or control the
execution of an I/O operation, or to place the control unit and
device in a known state. Most device-level controls are specified
by combinations of bits in the device-header flag field and the IUI
field in the IU header. For command and Control IUs, device-level
controls are also specified in the command or control DIB. For
other IU types, device-level controls are specified only by the
device-header flag field and IUI field.
Stacking Status Function
[0346] The stack-status control function indicates that the channel
does not accept the status; the control unit or device is to hold
the status information (status flags, status byte, and status
parameter) and associated supplemental status, if any, until
requested by the channel or until status information is cleared,
withdrawn, or unstacked.
Cancel Function
[0347] The cancel function causes the designated device to
terminate execution or perform nullification of the current
operation, if any. When the channel has initiative to send a cancel
IU during data transfer for a write operation, it stops data
transfer for the write operation. The last IU sent for the data
transfer contains either a command-data DIB or data DIB with the E
or EE bit set to one and contains valid CRC.
Request-Status Function
[0348] The request-status function indicates that the channel is
prepared to have pending status information presented for the
addressed device. The status must be associated with the logical
path on which the request-status IU was received. Request status
has no effect on status associated with logical paths other than
that on which the request status was received. The request-status
control function can be used to obtain stacked status or to obtain
supplemental status when the control unit indicates supplemental
status is available.
Status-Acceptance Function
[0349] The channel indicates that status is accepted in several
ways, depending on whether the status is for an I/O operation,
whether the channel intends to perform command chaining, and, if
command chaining is to be performed, whether the status contains
device end. If the status is considered by the channel to be
unrelated to an I/O operation or if the channel does not intend to
perform command chaining, the channel indicates that status is
accepted by sending a status-accepted IU with the chaining (CH) bit
in the device-header flag field set to zero. If the channel intends
to perform command chaining and the status DIB has the channel-end
status bit set to one and the device-end status bit set to zero,
the channel indicates that status is accepted by a sending a
status-accepted IU with the CH bit set to one. If the channel
intends to perform command chaining and the status DIB has the
device-end status bit set to one, the channel indicates implicitly
that the status is accepted by sending a command IU with the CH bit
and the SYR flag set to one. (See "command Chaining") A control
unit indicates to the channel that it recognizes the
status-accepted IU by sending a DACK IU. The CCW number and token
from the status-accepted IU is used in the DACK sent. The channel
can consider the status-accepted function to have been performed
when it receives the DACK IU. When a status-accepted IU is sent the
channel may elect to either leave open or close the outbound
exchange, depending on conditions at the channel and whether or not
the control unit had requested to leave the exchange pair open. If
with the status transfer containing channel-end status without
device-end status the control unit requests that the exchange pair
be left open and conditions at the channel allow for leaving the
exchange pair open, the status accepted IU is sent and the outbound
exchange is left open. If the control unit did not request for the
exchange pair to be left open, status other than channel end
without device end is presented, or conditions at the channel do
not allow the exchange pair to be left open, the status-accepted IU
is sent and the outbound exchange is closed.
Device-Level-Acknowledgment Function
[0350] The device-level-acknowledgment (DACK) function indicates
that the control unit has recognized a corresponding stack-status,
cancel, system-reset, selective-reset, request-status, or
status-accepted IU. For this use of the DACK function, refer
to:
[0351] "Stacking Status Function,"
[0352] "Cancel Function,"
[0353] "System-Reset Function,"
[0354] "Selective-Reset Function,"
[0355] "Request-Status Function," and
[0356] "Status-Acceptance Function."
[0357] IV. Command Reset/Recovery and Retry--Abnormal
Conditions
[0358] A. Exchange Recovery on a Fibre Channel Link
[0359] In an ESCON environment, a variety of errors can occur which
potentially affect the integrity of dynamic connection through the
ESCON switch. For this reason, the Connection Recovery procedure
was invented. This is a protocol which involved the sending of
special continuous sequences to both ends of the connection. The
protocol was designed to be robust in the presence of errors, and
to guarantee that, if completed successfully, the connection would
be known to be removed at both endpoints as well as at the switch,
and all frames associated with the removed connection would be
purged from the network. The connection recovery protocol thus
provided the dual functions of connection removal and endpoint
synchronization. After completing connection recovery, the
endpoints were returned to a known state, and could then establish
a new connection to complete the required link or device level
recovery procedures. (It should be noted that the term `endpoint
synchronization` as we are using it in this document refers to the
returning of both endpoints of a connection to a known state, and
not to the establishment of bit or character synchronization of the
fiber link).
[0360] The FC-SB2 Architecture uses two FC-2 exchanges to perform
all operations on the Fibre Channel link. The first is opened by
the channel and is used to send Information Units (IUs) from the
channel to the control unit, and the other is opened by the control
unit and is used to send IUs from the control unit to the channel.
These two exchanges are associated for the duration of the
operation, and are referred to as an exchange pair. The
architecture specifies rules for managing exchange pairs. An
exchange pair in FC-SB-2 is analogous to a connection in ESCON, a
major difference being that in FC-SB-2, multiple, independent
exchange pairs may be active concurrently between a channel and a
control unit.
[0361] As with an ESCON connection, various error conditions can
effect the integrity of an exchange pair, therefore as in ESCON, a
protocol which allows exchange pairs to be reliably removed and the
endpoints of the exchange pair to be re-synchronized after an error
is required. Since FC-SB-2 uses the FC-PH standard as its
underlying transport, the protocol must use only FC-SB-2 IUs and
existing FC-PH-defined link services.
[0362] Our invention satisfies the above requirement through the
combined use of two separate procedures: the Abort Exchange
procedure and the Purge Path procedure.
Abort Exchange
[0363] When an error occurs which affects the integrity of an
exchange pair, the first action performed by the N_Port detecting
the error is to send the FC-2 basic link service Abort Sequence
(ABTS) for BOTH exchanges of the exchange pair, not just the one
initiated by the N_Port which detected the error. If an N_Port
receives an ABTS for one of the exchanges of an exchange pair, it
is required to send ABTS to close the other exchange of the pair.
It is important that all N_Ports send or receive an ABTS for both
exchanges of the pair. This both simplifies the recovery and
assures that for all recovery windows, both exchanges will closed
at both endpoints.
Purge Path
[0364] The Abort Exchange procedure provides the `connection
removal` attribute of the ESCON connection recovery, but, by
itself, it does not provide endpoint synchronization. The following
flow diagram illustrates a problem where, under certain conditions,
recursive recovery loop could occur:
27 Channel's View Control Unit's View 1 2 3 4 5 6
[0365] The problem can potentially occur whenever the required
recovery action following ABTS is simply to re-transmit the
original IU. In the above example, the channel sends a Command on
exchange (1).The Control unit receives the command, but it's
Command Response (CMR) sent on exchange (2) is delayed (possibly
because of queuing delays, etc). The channel times out waiting for
the CMR, performs ABTS for exchange (1) as required, and re-sends
the Command on a new exchange (3). After this, the original CMR on
exchange (2) arrives at the channel, and the channel interprets
this as the response to it's command sent on (3). Consequently, it
views (2) and (3) to be an exchange pair. When the ABTS for (1)
arrives at the control unit, it sends ABTS for (2) per the recovery
rules. It then sees the re-transmitted command on exchange (3) and
generates a CMR on exchange (4), so it now considers (3) and (4) to
be a pair. When the ABTS for (2) arrives at the channel, it will
send an ABTS for (3). The channel and control unit have gotten out
of sync, and an endless repetition of the above sequence
occurs.
[0366] To achieve endpoint synchronization in the FC-SB-2
environment, it is necessary to send `something` from one of the
endpoints to the other. The receiving endpoint must return a
response to the `something` to the sender. When the sender receives
the response, it is assured that all latent IU's have been purged
from the path, and that it may then proceed with it's required
recovery actions. The required characteristics for `something` are
as follows:
[0367] It must carry enough information to be able to be identified
by the receiver as pertaining to the specific operation which is to
be recovered. Since multiple operations may be executing
concurrently between a channel and a control unit for different
logical paths, or for different devices on the same logical path,
it must be an FC-SB-2 IU which carries all the device level
information pertaining to the affected operation i.e. it must be an
FC-SB-2 device level IU).
[0368] It must not cause the operation at the device to be affected
in any way. That is, it must not change the state of the device, or
cause a reset to occur, etc.
[0369] Thus it can not be one of the existing SB-2 `termination`
functions such as Cancel, Selective Reset, etc.
[0370] To meet these criteria, two new FC-SB-2 device level control
IUs, Purge Path and Purge Path Response were defined in the FC-SB-2
architecture. The protocols for the use of these IUs are as
follows:
[0371] Purge Path is sent only by a channel. It is sent whenever an
outbound exchange is aborted at the channel (either by sending or
receiving an ABTS), and that exchange was associated with a device
level operation.
[0372] Purge Path uses the logical path and device address
information from the affected operation, and contains a unique
token.
[0373] All inbound exchanges opened for the affected logical path
and device received either before of after Purge Path is sent but
before the Purge Path Response is received are aborted (via the
transmission of ABTS) by the channel. Any IUs received on the
exchanges to be aborted are discarded.
[0374] When the control unit receives a Purge Path, it closes any
inbound or outbound exchanges open for the affected logical path
and device (via the transmission of ABTS) and sends the Purge Path
Response on a new exchange with the same token and addressing
information received in the Purge Path. The state of the addressed
device is not changed in any way.
[0375] After sending the Purge Path, the channel waits for a Purge
Path Response IU on a new inbound exchange containing the same
addressing information and Token which were sent in the Purge Path.
If the channel receives a Purge Path Response from the same logical
path and device but with a different token, this is not considered
a valid response to the Purge Path, and the inbound exchange on
which the Purge Path Response was received is aborted per the third
rule above.
[0376] When the channel receives a valid purge path response with
the expected addressing information and token, it sends a Link
Level Acknowledgement (LACK) to close the outbound exchange on
which the Purge Path was sent. After sending the LACK, exchange
recovery is complete at the channel and it may initiate any
required procedures to recover the affected operation at the
device. Similarly when the control unit receives the LACK, it
considers the exchange recovery to be complete, and it may also
initiate any additional recovery procedures at that point.
[0377] As an example of the endpoint synchronization provided by
the purge path procedure, the following is the error scenario
pictured above with the addition of Purge Path:
28 Channel's View Control Unit's View 7 8 9 10 11 12 13
[0378] In this diagram, the channel sends Purge Path instead of
re-sending the command. This allows the CMR which was held up in
transmission to be received and discarded before the purge path
response is received. After the completion of the purge path
procedure, the channel may re-send the command without the exposure
of an ambiguous response.
Purge Path in a Bridge Environment
[0379] The requirement for endpoint synchronization exists in the
FC to ESCON bridge environment also, but the implementation
requirements are different:
[0380] To achieve the desired result, the IU used for the purge
path function must be translatable into an ESCON frame and sent to
the ESCON control unit. So the Purge Path IUs defined above cannot
be used.
[0381] Because only a single operation (link or device level) may
be in progress between the channel and an ESCON control unit at a
time, logical path and device addressing is not required. Also
because of this, however, the purge path procedure must be
performed for both link and device level operations.
[0382] The same requirement that the IU used for purge path must
not affect the state of any logical paths or devices at the control
unit also exists in the bridge environment.
[0383] For these reasons, Test Initialization (TIN) IU was chosen
for the purge path IU in the bridge environment, with the Test
Initialization Response (TIR) serving as the purge path response.
The rules for sending TIN to invoke the purge path procedure at the
bridge are the same as those listed above for the native
environment, with the following additions and exceptions:
[0384] TIN is sent whenever an outbound exchange is aborted at the
channel, regardless of whether the outbound exchange was opened for
a link or device level operation.
[0385] The bridge does not abort any exchanges as a result of
receiving a TIN. It may abort an inbound exchange as the result of
the channel aborting it's outbound exchange prior to sending the
TIN.
[0386] It is not necessary to receive a TIR response to the TIN in
order to verify that the path is flushed. Any response which is
sent by the control unit, such as Link Busy or Link Reject will
provide equivalent information, as long as the token in the
response matches that in the TIN (Link Busy could be encountered in
a link level frames passing situation where the channel must
defer). Responses sent by the bridge (Port Busy or Port Reject)
must be retried.
[0387] After sending a TIN, the channel holds any initiation IU it
receives and continues to wait until it receives one of the
following, with the same token as contained in the TIN:
[0388] TIR
[0389] Bind Established
[0390] Link Busy
[0391] Link Reject
[0392] If, while waiting, the channel receives an ABTS for an
inbound exchange, it discards all IUs received on that inbound
exchange, but does NOT abort the outbound exchange on which the TIN
was sent. After receiving one of the above responses and completing
the link protocols associated with that response, the pipe may be
considered purged and the original operation may be retried.
[0393] If any of the following responses from the Bridge are
received with the same token as the TIN, the channel aborts both
the outbound exchange on which the TIN was sent, as well as any
inbound exchange open from the bridge, discards any IUs received on
that inbound exchange, and re-transmits the TIN on a new outbound
exchange with a different token:
[0394] Port Busy
[0395] Port Reject
[0396] No response (timeout)
[0397] Note that the exchange on which the PBY or PRJ is received
does not have to be aborted, since it is always a single-frame
sequence which both opens and closes its exchange.
Command Retry
[0398] A control unit may request command retry for a command
either when an error detected at the control unit prevents
execution of the command or when an error detected at the channel
results in the control unit receiving a channel-initiated-retry
request and the request is accepted. When command retry is
performed by the channel as the result of receiving a command retry
request from the control unit, the command specified by the CCW
number received with the command retry request is re-sent. The
command IU has the CH bit and the SYR bit set to one and the DU bit
set to zero. If the CCW # received is not valid or does not
correspond to a CCW that can be made current again at the channel,
the status is accepted, command retry is not performed, and
execution of the channel program terminated.
[0399] B. System-Reset Function
[0400] The system-reset function is performed by the control unit
with respect to a logical path whenever any of the following
conditions is recognized by the control unit:
[0401] A valid device-level system-reset IU is received on the
logical path. A device-level system-reset IU is always sent with
the AS bit set to zero.
[0402] The logical path is removed or established.
[0403] Certain internal recovery procedures occur within the
control unit.
[0404] C. Selective-Reset Function
[0405] The selective-reset function causes a device and its status
to be reset with respect to the particular logical path. Only the
device and certain allegiances associated with the designated
logical path are reset. The definition of the reset state of the
device is model dependent.
[0406] D. Device Level Exception
[0407] A control unit uses the device-level-exception function to
indicate that an abnormal condition was recognized in the IU
received for which the sending of status is inappropriate or not
permitted. The abnormal condition is indicated by the exception
code present in the first byte of the control-parameter field. For
brevity, a control IU indicating device-level exception is referred
to as a device-level-exception IU.
[0408] A device-level-exception IU is sent only by a control
unit.
[0409] E. Retention of Logical Paths During Transient FC-PH Link
Failures
[0410] Transient events such as noise bursts or loose cables can
cause an FC-PH N_Port to detect a link failure. This present
invention describes a procedure which allows the N_Port to retain
SB-2 upper level protocol information relating to logical paths
after the link failure ends. The invention consists of two phases:
a) transient link failure detection, b) transient link failure
recovery. Distinguishing transient link failures from more extended
link failures has two advantages: First, it allows SB-2 operations
to continue normally after transient link failures, and second, it
provides a means for a control unit to immediately release
resources associated with SB-2 logical paths during extended link
failures when there is a high probability that the logical paths
are no longer usable.
[0411] It is important to maintain SB-2 logical paths during
transient link failures to avoid these transient errors from being
escalated to the System Control Program with possible loss of
reserves at the device and possible loss of access to the
device.
[0412] An FC-PH link failure occurs if FC-PH synchronization is
lost for at least 100 ms. This can result from transient conditions
such as a loose connection or noise burst. After the transient
condition disappears, the link may continue to operate normally. In
this situation, there is no reason to reinitialize SB-2 logical
paths after the FC-PH link has been successfully resynchronized. If
loss of synchronization remains for an extended period, however,
there is a high probability that a more serious problem exists and
that reinitialization of SB-2 logical paths will be required to
resolve the problem. Thus the first concept of this disclosure is
the recognition of a distinction between an FC-PH link failure and
an SB-2 or upper level protocol link failure. An SB-2 link failure
is defined as an FC-PH link failure of a duration longer than 1.5
seconds (SB_TOV). Logical paths are unconditionally removed after
an SB-2 link failure, but logical paths may be retained after an
FC-PH link failure provided the recovery procedure below is
successfully performed.
[0413] An additional aspect of this invention in configurations
involving a switch (fabric) provides a control unit the means to
detect a remote FC-PH link failure, or a link failure which occurs
on a link to a channel which is connected to another F_Port on the
fabric. A remote link failure at a channel is detected when the
control unit receives either 1) a "registered state change
notification" extended link service command from the fabric
controller which indicates an event has occurred at the channel, or
2) an N_Port login command from the channel. Either of these events
indicates that a link failure or other problem may have caused a
channel to remove a logical path which existed on the link between
the channel and the fabric.
Transient Link Failure Recovery
[0414] In order to retain SB-2 logical paths after a transient link
failure, the following recovery procedure must be performed within
a prespecified time period:
[0415] 1) The N_Port which detected the failure must verify that it
is connected is the same N_Port or F_Port as it was before the
failure, and
[0416] 2) The existence of the logical paths which were present
before the failure must be verified after the failure within a
prespecified time limit.
[0417] To accomplish 1) above, the N_Port which detected the
failure verifies that the N_Port_Name of the attached N_Port (or
the F_Port_Name of the attached F_Port) is the same as before the
failure. This is done during N_Port (or F_Port) login. Logical
paths are only retained if the N_Port_Name (or F_Port_Name) of the
attached N_Port (or F_Port) has not changed. This is a well
understood procedure which is typically performed to determine
whether a cabling change has occurred whenever there is any
indication that a cabling change might have occurred.
[0418] To accomplish 2), a control unit which detects the link
failure is given initiative to perform the Test Initialization
(TIN) function with all channels with which it had logical paths
established on the link on which the FC-PH link failure occurred.
For point to point configurations, the link attached to the control
unit is the only link on which all logical paths exist, so the TIN
function is performed only with the attached channel. For fabric
configurations, the channels to which the TIN function is performed
depends on the link on which the failure occurred:
[0419] If the FC-PH link failure occurred on the link attached to
the control unit, the TIN function is performed with all channels
to which the control unit had a logical path established.
[0420] If the FC-PH link failure was a remote link failure, then
the TIN function is performed only with the channel which is
attached to the link on which the FC-PH link failure occurred.
[0421] A control unit performs the TIN function by sending a TIN
request and awaiting a reply. The TIN request requires the channel
to respond with an indication of the logical paths which it
considers to exist between it and the control unit. If the
destination channel responds with an indication that all logical
paths which were present before the failure are still present, then
the logical paths to the channel and the associated resources are
retained. If no response to the TIN request is received in a
pre-specified time interval (referred to as RA_TOV), then the
control unit retries the TIN function. If, after multiple retries,
either no response is received within a longer time interval
(referred to as LP_TOV) after the first detection of the link
failure, or if a response is received indicating that the logical
paths which were previously established are no longer established,
then a control unit removes all logical paths to the channel to
which the TIN was sent and frees all resources associated with the
logical paths. The value of LP_TOV used to await the response to
the TIN is of sufficient duration to allow multiple attempts to
send a TIN and await a response to be performed. This longer time
interval allows multiple intermittent FC-PH link failures to occur
during the LP_TOV time period without causing the automatic removal
of logical paths. A value of 4 seconds, or between 3 and 4 times
the SB-2 link failure time (SB_TOV) is chosen. During the LP_TOV
time period when attempts to verify the logical paths are being
made, the detection of additional FC-PH link failures on the
attached link or the remote link does not restart the LP_TOV time
period.
[0422] The advantage of this procedure is that it avoids time
consuming upper level protocol re-initialization after transient
FC-PH link failures. It also provides a method to release unused
resources for use by other operations which require them.
[0423] A different scheme using the FDISC and PDISC FC-PH extended
link services had been developed for FC-PH loops (FC-AL). These
extended link services allow one N_Port to determine whether or not
it is logged in with another N_Port after a loop initialization
without affecting existing exchanges. This provides the means for
the retention of upper level protocol information pertaining to the
exchanges after a loop initialization. It does not provide,
however, the means to verify logical paths, however, since the
existence of logical paths is independent of the existence of
exchanges. The FC-AL scheme also does not specify that resources
are to be automatically released during longer link failures caused
by intentional cable disconnections or frequent transient link
failures. The release of resources after extended link failures or
multiple intermittent link failures is an important aspect of this
disclosure because it prevents the needless retention of unused
resources from impacting operations which require the
resources.
[0424] V. Pacing Sequences to a Control Unit
[0425] A. IU Pacing
[0426] IU pacing provides a method by which a control unit, for
each channel program, can limit the number of IUs sent by the
channel without an intervening response. Each channel provides an
IU pacing credit which is initialized to a model dependent value at
the start of each channel program. The initialized value of the IU
pacing credit can be modified by the control unit at the start of a
channel program or at each time the control unit reconnects with
device-end status. At the start of a channel program the control
unit modifies the IU pacing credit by providing a pacing parameter
in either the command response or status sent in response to the
first command of the channel program. The pacing parameter defines
the maximum number of IUs that a channel can send on a given
exchange before it receives a command response IU. (See "Command
Response" and "Pacing Parameter" for information on how the control
unit sets the pacing parameter.)
[0427] If the control unit sets the pacing parameter to zero, the
IU pacing credit value of the channel is not modified. At the start
of a channel program the channel sends a model dependent number of
IUs to the control unit. The number of IUs sent can not exceed the
IU pacing credit value. After some model dependent number of IUs
are sent and prior to the transmission of the last IU that is
allowed to be sent, a channel will request a command response to be
returned by setting the PCI bit in a command or command-data DIB.
The PCI bit must be set such that the remaining IU pacing credit
does not prevent the transmission of all of the IUs for a CCW. Upon
receipt of this command response, the channel is allowed to send an
additional number of IUs as defined below.
[0428] If the control unit elects to modify the IU pacing credit at
the channel by proving a pacing parameter in the command response
or status DIB sent for the first command of the channel program,
this new value when accepted becomes the current IU pacing credit
and the remaining number of IUs the channel can send. At each
disconnection the IU pacing credit is reset to the initial model
dependent value. Subsequent command responses received by the
channel during the connection allow it to send an additional number
of IUs beyond the IU for which the command response was sent equal
to the value of the pacing parameter.
[0429] Note that if the control unit has lowered the value of the
IU pacing credit, the channel may have already sent all the IUs
which are allowed by a given command response IU. In this case, the
channel awaits additional command response IUs before sending
additional IUs. When a command response IU is received which allows
more IUs to be sent, the channel sends additional IUs as needed and
allowed, and sets the PCI bit whenever needed in order to solicit
further command responses as required. If a disconnection occurs
during the channel program and the control unit initiates a
connection to continue the channel program, the control unit may
change the default value of the IU pacing credit. The IU pacing
credit used by the channel for the continuation of the channel
program is equal to the pacing parameter value provided by the
control unit in the status DIB, if a value was set, or the model
dependent default value of the channel if the pacing parameter is
zeros.
[0430] NOTE ON THE DEFINITION: The pacing function is only defined
for transmissions from the channel to the control unit. It is not
required for control unit to channel transmissions because
resources are already allocated in the channel prior to execution
of the channel program.
[0431] B. Zero CCW Count
[0432] When a CCW is fetched containing a count field of zeros, the
CCW count field of the command IU is likewise set to zeros. When
the control unit executes a command with a CCW count of zero, the
type of response sent to the channel, if any, and whether or not
command chaining, if indicated, occurs depend on how the command is
executed and the flag bits in the command header for the
command.
[0433] If the first command of a channel program is executed as an
immediate operation, a data IU containing a status DIB is sent. The
LR bit and the RV bit are set to zero. The E-bit is meaningless for
this case.
[0434] Command chaining, if indicated, is initiated by the channel
when device-end status is received and the conditions for command
chaining at the channel are satisfied. If the first command of a
channel program would have been executed as a non-immediate
operation had a count other than zero been provided, a command
response IU is sent. If nullification is in effect at the control
unit, execution of the command and chaining to the next CCW
requires receipt of the accept-command response from the channel.
If nullification, is not in effect at the control unit execution of
the command and chaining to the next CCW can proceed after the
command has been accepted and the command response sent.
[0435] If other than the first command of a channel program is
executed and the PCI bit and SSS bit in the command header are both
set to zero, no response is sent to the channel and command
chaining, if indicated, occurs at the control unit. (See "Command
Chaining.")
[0436] If other than the first command of a channel program is
executed and the PCI bit and the SSS bit are both set to one, a
command response IU followed by a data IU containing a status DIB
are sent. The E bit in the IU header is set to zero by the sender
and ignored by the receiver, and the RV bit in the status-flags
field is set to one for a residual count of zero. If the command
would have been executed as a non-immediate operation had a count
other than zero been provided, the LR status flag bit is also set
to one; otherwise, the LR bit is set to zero.
[0437] Command chaining, if indicated, is initiated by the channel
when DE status is received and the conditions for command chaining
at the channel are satisfied. (See "Command Chaining.") If other
than the first command of a channel program is executed and the PCI
bit is set to one and the SSS bit is set to zero, a
command-response IU is sent.
[0438] Command chaining, if indicated, occurs at the control unit
and no status IU is sent. If other than the first command of a
channel program is executed and the PCI bit is set to zero and the
SSS bit is set to one, a command response is not sent but a data IU
containing a status DIB is sent. The E bit in the IU header is set
to zero by the sender and ignored by the receiver, and the RV bit
in the status-flags field is set to one for residual count of
zero.
[0439] If the command would have been executed as a non-immediate
operation had a count other than zero been provided, the LR status
flag bit is also set to one; otherwise, the LR bit is set to zero.
Command chaining, if indicated, is initiated by the channel when DE
status is received and the conditions for command chaining at the
channel are satisfied. (See "Command Chaining.")
[0440] NOTE: See IBM Enterprise Systems Architecture/390 Principles
of Operation, SA22-7201, to determine when a count of zero is valid
in a CCW.
[0441] Whether a control unit considers a count of zero to be valid
for a command is model dependent.
[0442] C. Sending Sequences to the Control Unit
[0443] Without some sort of pacing at the FC4 layer of Fibre
Channel an individual channel program could use up all the buffers
in a Control Unit. The Fibre Channel FC2 layer protocol will keep
the FC4 from losing data but does nothing for fairness among
channel programs.
[0444] In order to resolve this problem, the present invention
provides a solution to have the Fibre Channel send a limited number
of sequences to a Control Unit and then wait for the Control Unit
to response with some type of indication of the number of sequences
it has executed. The channel will then send that number of sequence
to the Control Unit. This can then continue on indefinitely.
[0445] Pacing in this case is controlled by the channel, using
existing protocol constructs at the FC4 layer. (See reference one).
The channel at initialization time assumes a preset value for
pacing. The control unit can change this value at Command Response
time or Status time of the first CCW of a chain by inserting a
value other then 00 in word one byte one in the Command response
Sequence or Status Sequence.
[0446] A value of 00 means do not change the current pacing value.
A value other then 00 will be used as the new pacing value. The
channel will default to the initial pacing value after a physical
disconnect or between starts.
[0447] For the following discussion X is equal to the number of
sequences a channel can send to a Control Unit before waiting for
Command Response. The channel will be initialized with X equal to
17 for Fibre Channel attached Control Units. At initial status or
Command Response time X can be changed by the Control Unit with a
value in the range of 17 to 255. Values of 1 to 16 would require
more rules because one write CCW could require 16 sequences to send
all the data for that CCW. For this to work at least one CCW and
its data plus one more CCW and part of it's data, if a write, has
to fit within X sequences. Thus a Control Unit can not decrease the
channels initialized value.
[0448] The channel has to keep flags on a CCW bases to sort out if
the Command Response is for initial status, pacing, system PCI or
any combination of all three. A Command Response received by the
channel means that the sequences, between the CCW a previous
command response was received for and the CCW-1 the current Command
Response was received for, are now available. Assuming that X is
equal to 32 then the channel would insert the PCI flag in a command
sequence that occurs at or close to 16 sequences sent by the
channel. The channel then sends 16 more sequences of CCWs and data
and then again set the PCI flag in a command sequence that is at or
close to 32 sequences. Thus when the Control Unit Executes the
first 16 sequences, it will send a command response back to the
channel in response to the PCI flag in the command sequence. The
control unit has 16 more sequences to execute before it runs out of
work for this CCW string. The channel on seeing the Command
Response sequence from the Control Unit will send n more sequences
to the Control Unit, assuming the CCW chain is not completed. N is
equal to the number of sequences between the CCW the command
response is for and the CCW the previous command response was for,
or on start up the first CCW.
[0449] VI. Executing Search Commands Locally Using a Control
Unit
[0450] Count Key Data (CKD) CCW chains generally contain a search
transfer in channel (TIC) back to the search command. In ESCON
architecture the control unit returns the CE, DE status back to the
channel on the search CCW and the channel then re-sends the search
CCW and its associated data back to the control unit as the next
CCW. When the search argument is satisfied at the control unit, CE,
DE and SM status is sent to the channel causing the channel to go
on to the next CCW. This procedure does not take advantage of the
function of pipelining and impacts performance since only one CCW
can be sent to the control unit at a time. This invention adds some
control functions to allow the control unit to independently manage
the search TIC operation and, as a result, take advantage of the
performance benefits of pipelining.
[0451] To address this problem, when pipelining a CCW chain with a
TIC-8, the present invention provides for the channel to send the
CCW command with a flag bit (control function) that tells the
control unit to continue to re-execute the command as long as CE,
DE status is generated by the command. This occurs without
interaction with the channel. When status of CD DE and SM is
generated, the control unit will not execute the TIC-8 again and
instead continue on to the next CCW command. If at any time status
other then CE, DE and or SM is generated the control unit sends
that status back to the channel and the operation will continue or
terminate as with prior art.
[0452] The Repeat Execute (REX) flag, bit 6, of the Command-Flag
Field in the Command Header when set to one, indicates that the
channel is requesting the control unit to perform the
transfer-in-channel function by repetitively executing the command
in the command header. The channel sets the REX bit for a command
only when the CCW containing the command has the CC bit set to one,
the CD bit set to zero, and the next CCW (the one to which chaining
will occur) contains a transfer-in-channel command and an address
of the previous CCW. When the control unit recognizes this
indication, re-execution of the current command is performed until
one of the following conditions occur:
[0453] 1. Execution of the command results in status of other than
channel end and device end alone being recognized.
[0454] 2. Execution of the command has been performed 128
times.
[0455] 3. Execution of the command is terminated by a cancel,
selective reset, or system reset.
[0456] When an execution of the command results in status of
channel end, device end, and status modifier alone being
recognized, chaining to the next command occurs at the control
unit.
[0457] When re-execution of the command is performed 128 times
without status other than channel end and device end alone being
recognized, re-execution of the command is terminated and the
status recognized for the last execution of the command is sent to
the channel.
[0458] When execution of the command results in status of other
than channel end and device end alone or channel end, device end,
and status modifier being recognized, re-execution of the command
is terminated at the control unit and status is sent to the
channel.
Search TIC Loop
[0459] The following is an example of how a Search TIC loop CCW
string is handled by Fibre Channel. This example is representative
of the seek, set sector, search ID equal case. The following Table
is how a CCW list may appear in 390 storage.
29TABLE 1 shows CCWs in a large host system such as IBM's S/390
storage .------------------------------
------------------------------. .--------------. .--------------.
SEEK 008 WRITE - CCW 1 100 READ - CCW4 '--------------'
'--------------' .--------------. .--------------. SET SECTOR 010
WRITE - CCW 2 108 READ - CCW5 '--------------' '--------------'
.--------------. SEARCH ID 018 WRITE - CCW 3 '--------------'
.--------------. 020 TIC TO 18 - TIC TO CCW 3 '--------------'
.--------------. 028 TIC TO 100 - TIC TO CCW 4 '-------------'
'-----------------------------------------------------------'
[0460] The CCWs in case one are sent to the control unit in 5
sequences back as shown below. See reference 1 for protocol
details. CCW 3 is flagged with the Reexecute flag (REX) such that
the control Unit will continue to re-execute it until either an
exception condition is reached or the CE-DE-SM condition is
reached. If an exception condition occurs, status is presented to
the channel with a CCW number of 3 in the status frame. when the
E-DE-SM condition occurs CCW 4 is then executed.
30TABLE 2 CCWs as they appear at the Control Unit.
.-------------------------------------------------- ----.
.--------------. CCW 1 WRITE FLAG = CC DATA '--------------'
.--------------. CCW 2 WRITE FLAG = CC DATA '--------------'
.--------------. CCW 3 WRITE CU RE-EXECUTES THIS CCW AS LONG FLAG =
CC & REX AS THE STATUS IS CE & DE ONLY. DATA
'--------------' .--------------. CCW 4 READ CU EXEC. THIS CCW WHEN
CE-DE-DE FLAG = CC STATUS OCCURS FOR THE ABOVE CCW '--------------'
.--------------. CCW 5 READ CU EXEC. THIS CCW WHEN CE & DE
'--------------' STATUS OCCURS FOR THE ABOVE CCW THIS IS THE LAST
CCW EXECUTED.
'-----------------------------------------------------'
[0461] This is the only case a Control Unit executes the next CCW
with status other then CE-DE.
Software Controls for CCW Pipelining
[0462] The FICON architecture is designed to execute existing
S/360, S/370, and S/390 channel programs with full compatibility to
the parallel OEMI and ESCON channel interfaces. However, certain
modifications to the system architecture, operating system,
application programs and channel programs are required in order to
obtain the maximum efficiency possible. This maximal efficiency is
achieved by pipelining the execution of the channel program. This
means that the CCWs and associated data (for output operations) are
continuously sent to the control unit with out an explicit
interlocked exchange between the channel and control unit for each
CCW. This pipelining of the CCW execution reduces the link
overheads and improves the execution efficiency of the channel
program.
[0463] Not all channel programs can be executed with CCW
pipelining. One example of a channel program that cannot be
executed in a pipelined fashion is a channel program that modifies
itself by reading additional CCWs into the CCW stream. If the
channel program was prefetched the channel would terminate
prematurely due to executing a CCW that was not modified and did
not have the command chaining flag on, or terminate in error
because the CCW did not arrive before the channel fetched the
storage and tried to execute it as a CCW before a CCW was actually
read in. Another example of a channel program that can not
automatically prefetched is one where the program changes the
channel program while it is executing. If prefetching were always
done, the timings would change such the program would always miss
the opportunity to change the channel program before it
executes.
[0464] Some channel programs may always be prefetched. When
unauthorized applications make an I/O request, the operating system
needs to translate the virtual channel program into a real channel
program. By definition, virtual channel programs cannot be self
modifying. This is because at the time the channel program is
built, the application cannot know the real location of the storage
that will contain the translated CCWs. Thus, the channel program
cannot possibly be constructed to read data into the location that
the real channel program will be constructed. Additionally, the
application cannot modify the channel program once it has started
execution since the application cannot know where their real
channel program resides once it has been passed to the operating
system for execution. Therefore, the operating system may always
specify that CCW prefetching is allowed for the execution of
virtual channel programs.
[0465] Since real channel programs may be self modifying, or the
program may modify them while executing, the I/O requester must
explicitly indicate when prefetching is allowed. The S/390
architecture allows the operating system to explicitly request CCW
prefetching via the P bit in the ORB (Operation Request Block).
Input/Output Synchronization
[0466] There are certain conditions that require the channel to
synchronize during the execution of a channel program. This
synchronization means that the fetching of CCWs and data is
suspended until status is received for a particular CCW. After the
status is received for the CCW being synchronized, execution of the
channel program resumes. One example of a required synchronization
point is whenever a channel program transitions from executing an
input operation to executing an output operation. If data that is
being read into a specific buffer is then intended to be written
out of the same buffer within the same channel program, the channel
must wait for all outstanding read requests to finish before
proceeding to the execution of the subsequent write command. Its
not practical for the channel to keep track of every buffer address
and length for all input operations so that when an output
operation is to be executed it will know whether synchronization is
required or not. The channel complexity and overhead would be
prohibitive. Thus, whenever the channel program transitions from an
input CCW to an output CCW it will automatically synchronize to
guarantee data integrity. However, there are very common channel
programs that have many input/output transitions without any
requirement to synchronize. A typical example is a DASD channel
program that reads records from multiple extents (see example
below). The Locate Record (LR) CCW is an output operation as it
writes parameters to the control unit. Locate Record must be done
to specify the domain before record can be read from a particular
extent. If multiple domains are going to be read from in a single
channel program then a synchronization event will occur at the
start of every domain. Since the LR parameters are usually always
disjoint from the data buffers, an override for the input/output
synchronization will allow the program to optimize the channel
performance.
READ/WRITE Synchronization
[0467] Channel will synchronize on input/output transitions with in
the channel program:
[0468] required to guarantee integrity
[0469] software may override
[0470] IOBNORWS
[0471] IOSNORWS
Read Multiple Records
[0472]
31 Define Extent Locate Record Read Read Read
-------------------> R/W Locate Record Transition Read Read Read
-------------------> R/W Locate Record Transition Read Read
Read
[0473] For virtual channel programs the operating system must scan
through the channel program, page fixing the buffers and
translating the CCWs into CCWs with real addresses. The translation
process keeps track of the buffer addresses as it translates the
CCWs so as to avoid fixing the same page more than once. It is
possible for the operating system to keep track of whether or not a
previous CCW was a input operation when it encounters the first
write operation. It can also know whether or not the buffer was
already on a page that was fixed. If both conditions are not true
for any CCWs in the channel program then the operating system knows
that it is safe for the channel not to synchronize on input/output
transitions. The S/390 architecture allows the operating system to
specify that the channel should not synchronize on input/output
CCWs transitions via the Y bit in the ORB. Therefore, for virtual
channel programs the operating system can automatically override
the channel synchronization on input/output CCW transitions. I/O
requesters that pass real channel programs to the operating system
are required to explicitly indicate when input/output
synchronization is not required.
PCI Synchronization
[0474] Some applications use Program Controlled Interrupts (PCI) as
a means to be notified when a portion of the channel program is
about to executed. The PCI interrupt gives the program the
opportunity to modify the channel program to execute in a different
way depending on data that may have been returned in a portion of
the channel program already executed. One example of such an
application is the system IEBCOPY utility. Unless the channel
synchronizes on the CCW following the PCI, use of CCW prefetching
would prevent the program from every successfully modifying the
channel program before it is fetched by the channel for execution.
This would cause an increase in the their CPU overheads for
additional interrupts as the channel program is not extended,
increases in channel and device overheads as additional start
subchannels are required to resume execution of the channel
program.
[0475] Other applications use PCI interrupts to notify the program
when buffers associated with a completed portion of the channel
program may be freed or reused for other purposes. This allows for
more efficient use of real storage. Programs that use PCI in this
way do not want the synchronization overhead described above. The
S/390 architecture provides for the M bit that allows the program
to request the channel not to synchronize on the CCW following the
PCI. This allows for a more efficient execution of the channel
program.
PCI Synchronization
[0476] Some I/O drivers use PCI to modify the channel program
before it is executed:
[0477] controls provided to force synchronization on next CCW
[0478] IOBEPCIS
[0479] IOSXPCIS
IEBCOPY Example
[0480]
32 SET SECTOR Search ID Equal TIC *-8 TIC *+8 Read Key/Data (PCI)
Read Count =========> TIC *+8 =========> Read Sector (CC=0)
---------- Read Key/Data (PCI) Read Count =========> TIC *+8
=========> Read Sector (CC=0) ---------- . . .
PCI Synchronization
[0481] Some I/O drivers use PCI to modify the channel program after
it is executed and do buffer management:
[0482] software default is not to synchronize, optimize
performance
Simple Buffer Management
Define Extent
[0483] Locate Record
[0484] Read
[0485] Read
[0486] Read
[0487] Locate Record (PCI)
[0488] Read
[0489] Read
[0490] . . .
[0491] Read
[0492] Locate Record (PCI)
[0493] Read
[0494] Read
[0495] Read
[0496] . . .
[0497] VII. Transmitting Fibre Channel Sequences Concurrently
Between two ULPS in Both Directions
[0498] The present Fibre Channel architecture only allows sequences
to be sent in one direction or the other within an exchange. The
side that opens an exchange, the Requester, can send sequences to
the other side, the Responder, until the Requester side transfers
initiative to the Responder in the last sequence it sends.
Presently, the Responder can send sequences to the Requester until
it returns initiative to the Requester side. This type of protocol
inhibits doing a high performance task using hosts such as IBM's
System 390 I/O FC4 using Fibre Channel as the transport.
[0499] The solution to this problem is to open an exchange in each
direction and tie the two exchange IDs together at the FC4. The
link address-logical Control Unit address and device
address-effectively ties the two exchange IDs together. Thus one
side is the Requester for exchange ID A and the Responder for
exchange ID B. while the other side is the Responder for exchange
ID A and the Requester for exchange ID B. This allows the channel
side to open an exchange ID A to a Control Unit and start streaming
out Sequences to the Control Unit. Mean while the Control Unit can
open exchange ID B and start sending responses back to the channel.
When the channel sends out the first request that opens exchange ID
A it also includes a 3 byte token in the header of this sequence.
All sequences associated with this link address-logical Control
address and Unit address--will have this same token. The Control
Unit will always return this same token to the channel in the
sequence header when responding to any requests for this link
address.logical Control Unit address.Device Address. This provides
a means for the channel to quickly associate exchange B from the
Control Unit to exchange A opened by the channel. This three byte
token is defined by the channel and is definitive for every
exchange opened by the channel.
[0500] VIII. Dis-connections and Re-connections
[0501] As described earlier, most disconnections and reconnections
have become unnecessary under the teachings of the present
invention. However, there are still some specific cases that needs
addressing.
[0502] A. Disconnecting without Closing the Exchange Pair
[0503] In existing systems such IBM's ESCON, when a Control Unit
and or device has operation to perform that may take milliseconds
of time the Control Unit disconnected from the channel with Channel
End status so that the channel can then do useful work with other
devices. However on Fibre Channel a Control Unit does not have to
physically disconnect from the Fibre Channel for the channel to do
other useful work because of the multiplexing capability of Fibre
Channel. Physically disconnecting and reconnecting is extra
overhead that is not always required for Fibre Channel.
[0504] Under the present invention, the mechanism allows the
Control Unit to inform the channel if it wants to physically
disconnect from the channel or just to logically disconnect from
the channel by the use of the End Connection bit.
[0505] END CONNECTION (EC): The EC bit, when set to one, in a
status sequence, indicates that the control unit is logically and
physically disconnecting from the channel and requesting that the
exchange pair be closed. When the EC bit is set to zero, the
control unit, while still logically disconnecting from the channel,
is requesting that the exchange pair remain open. When, upon
completion of execution of the current command at the control unit,
and conditions require a logical disconnection from the channel, CE
without device end status is generated and sent to the channel.
When the channel receives a status sequence with CE status without
device end status and with the EC bit set to zero, the channel
accepts the status and depending on conditions present, either
leaves the outbound exchange open or closes the outbound exchange.
If the channel elects to leave open the out- bound exchange, the
status accepted (ACC) Information Unit (IU) sent does not close the
exchange. In this case the control unit keeps the inbound exchange
open when the device-level acknowledgment (DACK) IU is sent. If the
channel elects to close the outbound exchange, the ACC IU sent
closes the outbound exchange. In this case the control unit also
closes the inbound exchange when the DACK IU is sent. If either the
channel elects to keep the outbound exchange open as requested but
the control unit closes the inbound exchange or the channel elects
to close the outbound exchange but the control unit doesn't close
the inbound exchange, a link-level protocol error is recognized. A
control unit that sets the EC bit to zero for a status sequence in
order to logically disconnect and leave the exchange pair open,
must, if accepted by the channel, reconnect on that exchange pair;
otherwise unpredictable results may occur. A logical reconnection
on the exchange pair occurs when the control unit sends a status
sequence containing DE status.
[0506] B. Control Units Keeping Track of Disconnect Time
[0507] In present systems such as IBM's ESCON channel, when a
control unit and or a device has to execute a function that takes
milliseconds of time and doesn't require the services of the
channel, the control unit disconnects from the channel with channel
end status. This frees the channel to work with another control
unit /device. However disconnecting and reconnecting to a channel
generates extra overhead which can impact I/O throughput.
[0508] On Fibre Channel, a control unit does not have to disconnect
to allow the channel to do other useful work because of the
multiplexing capability of Fibre Channel. When the control unit
does not require the services of the channel and when a condition
requires it to execute an independent function, communication
between the channel and control unit can be temporarily ended;
however, the connection with the channel is maintained for the
duration of time the condition at the control unit exists. This
condition typically occurs whenever the control unit requires the
device to access the physical medium for the purpose of reading or
writing data. For purposes of measuring device and channel
utilization, this time interval would be recognized by the an ESCON
channel as disconnect time since an actual disconnect from the
channel occurred. With Fibre Channel, the same condition is
encountered by the control unit; however, since no disconnect
occurs, the channel would be unable to recognize the occurrence and
likewise be unable to properly calculate the amount of disconnect
and connect time in determining resource utilization.
[0509] Under the teachings of the present invention a means is
provided for the channel to continue to properly calculate resource
utilization in the new environment. This invention describes a
method for the control unit to measure the time interval and to
report this information to the channel. An algorithm is defined
that enables the channel to properly interpret the measured time
interval reported and to establish the amount of resource
utilization that has occurred. For definition of terms and
architecture see the following reference documents:
Status Sequence
[0510] The status header is the first 12 bytes of a status
sequence. The status header has the following format: 14
Defer-Time Parameter
[0511] Bytes 2 and 3 of word 1 contain the
defer-time-parameter(DTP). Device-defer-time is accumulated by the
control unit whenever ongoing channel program execution needs to be
temporarily delayed because a device-dependent action needs to be
performed. This condition typically occurs whenever the control
unit requires the device to access the physical medium for the
purpose of reading or writing data. However, because of the
device-dependent nature of the action, other conditions may also be
recognized by the control unit creating the need for a temporary
delay of channel program execution. In all cases, device-defer time
continues to be accumulated with each successive temporary delay
until such time when either the control unit forces a disconnection
from the channel or execution of the channel program has been
completed. If the control unit disconnects from the channel, the
previously accumulated defer-time value is presented as the
device-defer-time parameter along with status containing
channel-end without device end (this status may or may not include
command retry status). When the temporary delay of channel program
execution is no longer required, the connection to the channel is
reestablished and device-end status is accepted from the control
unit. Device-defer time is not accumulated during the time interval
of a disconnection from the channel.
[0512] If no disconnection from the channel occurs during execution
of the channel program, the previously accumulated device-
defer-time is presented as the device-defer-time parameter along
with status of channel end signaling completion of channel program
execution at the channel (status of device end may also be included
with the channel--end status). In all cases, the control unit
retains the defer-time information until it receives acknowledgment
that the status transfer was accepted. When this has occurred, all
previously stored or accumulated defer time values are zeroed at
the control unit and accumulation, if any, begins anew. The
defer-time-parameter is obtained in the following manner.
Control Unit Timer
[0513] Defer Time Parameter Format.
33 3 0 1 .-------------------.-------------.-- -------. 0 . . . . .
. . 01xx xxxx xxxxxx....xx
'------------------'-------------'---------'
<----------------> n bits 1 0 4 5 5 .------.-------------.
DTUF DTU '------'-------------' DEFER-TIME PARAMETER 0 = Binary
zero 1 = Binary one X = Any binary value
Defer-Time Parameter
[0514] 0=Binary zero
[0515] 1=Binary one
[0516] x=Any binary value
[0517] The control unit, using a 4-byte timer having a 1
microsecond resolution, derives this defer-time parameter (DTP)
which contains control information and a 11-bit defer-time unit.
When the channel accepts the accompanying status, it uses the
control information, and is able to reestablish the 4-byte time
value (with low-order truncation). DEFER TIME UNIT FACTOR (DTUF):
Bits 0-4, when non-zero, specify the number bit positions the
11-bit defer time unit must be shifted. If a shift was required,
the DTUF is calculated by subtracting the value of the highest
bit-position (which was set to one) of the queue-time unit from 21
(this applies only in those cases where the bit position is equal
to or less than 21). If no adjustment was required, the DTUF value
is 00000, binary. DEFER TIME UNIT (DTU): Bits 5-15 contain the
11-bit defer- time unit accumulated by the control unit timer. The
bit-significance of the time unit is specified by the DTUF. The
tables shown below describes the relationship between the values of
DTUF and the bit-significance if DTU.
[0518] When
34 DTUF is: Bit 15 of DTP represents: 00000 1 microsecond 00001 2
microseconds 00010 4 microseconds 00011 8 microseconds ' ' ' ' ' '
10101 2,097,152 microseconds
[0519] In the example given below, the control unit timer has
accumulated a defer time of 9,464 microseconds (X`24F8`) and
calculated the defer-time parameter. When received by the channel,
a shift of the most significant bit of the DTU to bit-position 18
of the channel 4-byte-time-value, and bit 15 of the DTP represents
a value of 8 microseconds. The low-order 3-bit positions are set to
zeros. The formula used is (CHANNEL TIMER=FTU.times.2DTUF).
35 1 1 2 2 3 0 7 8 8 9 1 .----------------------.---------------
.---. 0000 0000 0000 0000 0010 0100 1111 1xxx
'----------------------'--------------'---' CONTROL UNIT 32 BIT
TIMER 1 0 4 5 5 .-----.-----------. 0001110010011111
'-----'-----------' DEFER-TIME PARAMETER 11 2 2 3 0 78 8 9 1
.----------------------.------------------. 0000 0000 0000 0000
0010 0100 1111 1000 '----------------------'------------------'
CHANNEL 32 BIT TIME VALUE
[0520] C. Registered State Change Notification--Point to Point
Configuration
[0521] The Registered State Change Notification (RSCN) Extended
Link Service (ELS) and the State Change Registration (SCR) ELS are
currently used only in conjunction with Fibre Channel fabrics. The
S/390, however, requires a method of reporting state changes in
point to point configurations as well as fabrics. The use of SCR
and RSCN in point to point configurations as well as in fabric
configurations allows a single set of microcode to provide the
state change notification function in both configurations.
[0522] The present invention provides a solution that if a channel
and control unit are directly attached to each other by a single
link, each one registers for state changes by sending an SCR ELS to
the attached channel or control unit. Although the SCR ELS is sent
to the fabric controller if a fabric is present, all other aspects
of the SCR request is identical to the SCR request which would have
gone to the fabric controller if a fabric were present. This allows
a common microcode to be used for both fabric and point to point
configurations.
[0523] In addition, S/390 needed a method of making state change
mandatory. To provide this, S/390 will require users of the SCR
function to reject any request to disable SCR. Once SCR is enabled
by sending the SCR enable request, any subsequent requests to
disable SCR will be rejected. The only means of disabling SCR is to
log out with the port with which it was enabled. In order to begin
operation after a logout, a new SCR enable request must be
sent.
[0524] When a state change is to be reported, the RSCN notification
is sent to the attached channel or control unit instead of the
fabric controller. All other aspects of the RSCN notification are
identical to the RSCN notification which would have been sent to
the fabric controller if a fabric were present.
[0525] IX. Operational Messages--Test ELS
[0526] The present Fibre Channel adapter microcode requires a
method of communicating with the control processor in a bridge
using an existing industry standard Extended Link Service (ELS),
but no ELS existed which was designed for this purpose. Therefore,
a new way of using the TEST ELS was developed. The new use of the
TEST ELS does not violate any requirements of the existing Fibre
Channel specifications.
[0527] The present invention provides a solution, especially when
using large systems such as IBM's S/390. When the microcode in the
S/390 Fibre Channel adapter needs to convey operational messages to
the control processor in the FC-ESCON bridge, it sends a TEST ELS
with a payload which contains the message. The only use currently
made of the TEST ELS is to send dummy payloads across the link to
provide system loading; the use described here extends that usage
to convey interprocessor requests and could also be used to convey
other types of requests or information. The messages defined by the
FC-390 Fibre Channel protocol currently include requests for error
recovery information related to one or more FC-ESCON server
processors in the FC-ESCON bridge. When the bridge control
processor receives the TEST ELS with a payload containing such a
request, it instructs the appropriate FC-ESCON server processors to
send the requested information to the channel.
[0528] X. PCI Fibre Channel Cards
[0529] It is possible to use the concepts of present invention in
conjunction to off the shelf PCI Fibre Channel cards. The problem
is, however, that the percentage of the logic checked in these PCI
cards is not up to the standards usually expected of other similar
cards or components. Fibre Channel architecture provides CRC for
each frame sent on the Fibre Channel link but this does not check
the logic that is gathering the sequences from storage, that meets
high system standards, and parses the sequences into one or more
frames with generated CRC on the Fibre Channel link.
[0530] To protect the FC4 sequence headers, therefore, LRC is used.
To protect the data field, CRC is used. The FC4 header is 32 bytes
with bytes 28,29,30 and 31 defined as a 4 byte LRC. The last 4
bytes of data of a data sequence, that concludes a data block, is a
4 byte CRC field. The LRC provides good protection for 28 bytes of
header data and is generated and checked by a micro. The data block
can be any length from 0 to 65535 bytes. CRC is used to provide
good protection across this much data. The LRC and CRC generation
and checking is done in well check hardware providing total
protection of data through out the paths traveled by the data. The
Fibre Channel implementation of the SB-2 Information Unit (IU)
structure is shown below.
FC-SB-2 General IU Payload Structure
[0531]
36
.------.------.--------.---.--------------------------.-----.----
-. SB-2 IU DIB LRC Data PAD CRC HeaderHeader Header
'------'------'--------'---'------------------
---------'-----'----' (8) (8) (12) (4) (0-4224) (0-2) (0 4)
<--Not present in every SB-2 structure-> IU Payload
Structure
[0532] All SB-2 IUs contain the following:
[0533] 1. Eight-byte SB-2 header
[0534] 2. Eight-byte IU header.
[0535] 3. Device-Information Block (DIB)."
[0536] A DIB contains a minimum of 16 bytes; consisting of a 12
byte DIB header followed by four bytes of longitudinal-redundancy
check (LRC). For certain DIB types, a variable length DIB data
field may immediately follow the LRC. The maximum length of the DIB
data field is 4224 bytes. The DIB data may or may not contain the
PAD and the CRC field. Longitudinal-Redundancy-Check Field. The
longitudinal-redundancy-check (LRC) field contains a 32-bit
redundancy-check code, immediately following the DIB header in
bytes 28 to 31 of the 32 byte FC-4 header. LRC is provided only on
the bytes consisting of the SB-2 header, IU header, and DIB header.
The LRC generator/checker is initialized to X`A55AA55A` before the
header fields are processed a full word at a time. DIB DATA FIELD
The DIB Data field, if present, is a variable length field which
contains either data, data and pad bytes, data and pad bytes and a
CRC, or just a CRC. The CRC when transmitted is on total data count
basis which can be up to 65535 bytes. So the DIB Data field that
transmits the last data of a data block will contain the CRC field.
It may take up to 16 DIB Data fields to transmit one data block
because the maximum number of bytes that can be sent in the DIB
data field is 4224 bytes. Pad bytes, if present, are contained in
the last word of the last DIB data field used to complete a data
transfer and are used to pad the DIB data field to the next word
boundary. Cyclic-Redundancy-Check Field The cyclic-redundancy-check
(CRC) field, if present in an IU, contains a word-aligned 32-bit
redundancy-check code. The processing of CRC, in terms of
generation and checking, follows the definition as provided in
Annex N of the FIBRE CHANNEL, Physical and Signaling Interface
(FC-PH). Additional information can be found in Fiber Distributed
Data Interface (FDDI) Media Access Control (MAC) (ISO/IEC
9314-2:1989) appendix B. The initialized value of the CRC generator
is X`IFFFFFFFF`.
Detecting Lost Sequences within an Exchange on Fibre Channel
[0537] The Fibre Channel protocol provides for the detection of
lost frames within a sequence but does not provide for a reliable
way for all PCI Fibre Channel cards to detect lost sequences within
an exchange. The present invention provides a count field in the
header of each sequence that has the follows the following
protocol. This count field is know as the Information Unit (IU)
count. A sequence is also known as an Information Unit. Each
Sequence header contains the IU count. The IU count is an eight-bit
binary number that is included in every sequence sent during an
open exchange. The IU count value is initialized to X`00` when an
exchange is opened and the first sequence is sent. The value of
zero is inserted in the IU count field of the sequence header for
the sequence sent to open an exchange. After each sequence is sent
the IU count is advanced by one and the new value included in the
IU count field of the next sequence sent. When the IU count reaches
X`FF` and additional sequences are to be sent, the IU count is
permitted to wrap and the next value used is X`00`. If an sequence
is received with an IU count value that is not one greater than the
IU count value received in the previous sequence received, or, if
the previous IU count was X`FF` and the next IU count received is
not X`00`, an IU count error is recognized and the sequence is
discarded. When an sequence is discarded because of an IU count
error, the recovery performed depends on the operation or function
being performed. In some cases a retry of the operation may be
attempted and in other cases the operation or function may be
terminated without retry.
Selective Use of Class 2 Service in a Class 3 Fibre Channel
[0538] The complexity of a Fibre Channel implementation can be
reduced and the performance enhanced by restricting it to Class 3
service if no confirmation of frame delivery is required at the
link level. During normal operations upper-level protocols insure
the proper level of communication integrity in the management of
transferring and receiving frames. However, during initialization
and some error recovery conditions, confirmation of frame delivery
by upper layer protocols is not always provided.
[0539] For example, cases may exist where its necessary to
determine if a node is attached and if it's operational. A frame is
sent to the node, and a response is awaited. If class 3 is used and
the node does not respond, no message is returned and the sender of
the frame must await a specified amount of time for a response.
Typically a large interval would be specified to account for
transmission and internal node delays as well as the accommodation
of a large variety of different kinds of configurations. This wait
time causes the procedure to take an excessive amount of time,
especially when multiple retries of the operation must be
performed. An implementation is needed which obtains the
performance benefits of class 3 while avoiding the overheads caused
by class 3 during error recovery and initialization.
[0540] The FICON implementation overcomes this problem by the
selective use of class 2 only during error recovery and
initialization. When it is necessary to send a frame to a
destination which may not be operational, class 2 is used. If the
frame is not deliverable, the switch returns a response indicating
this to the sender of the frame. This eliminates the long delay
which would have been necessary if class 3 was used. The specific
procedures in which class 2 is used are restricted to the Fibre
Channel Extended Link Services (ELSs) and FICON link-level
functions listed below.
[0541] FC-PH Fabric Login ELS
[0542] FC-PH N_Port Login ELS
[0543] FC-PH State Change Registration ELS
[0544] Request Identifier FICON link-level function
[0545] Test Initialization FICON link-level function
[0546] The above functions are only used during initialization and
error recovery. Class 3 is used for all device-level functions,
which normally constitute the vast majority of all operations.
(Most existing implementations simply allow the initialization and
error recovery procedures to take a long time since they happen so
infrequently. This is acceptable in some applications, but not most
of the critical applications in which FICON is to be used.
[0547] While the invention has been described in detail herein in
accordance with certain preferred embodiments thereof, many
modifications and changes therein may be effected by those skilled
in the art. Accordingly, it is intended by the appended claims to
cover all such modifications and changes as fall within the true
spirit and scope of the invention.
* * * * *