It is our experiance that swapping the SCSI cable(s) for new is the most common fix for SCSI I/O issues.

The driver supports fixed and variable block size (within bufferlimits). Both the auto-rewind (minor equals device number) andnon-rewind devices (minor is 128 + device number) are implemented.

In variable block mode, the byte count in write() determines the sizeof the physical block on tape. When reading, the drive reads the nexttape block and returns to the user the data if the read() byte countis at least the block size. Otherwise, error ENOMEM is returned.

In fixed block mode, the data transfer between the drive and thedriver is in multiples of the block size. The write() byte count mustbe a multiple of the block size. This is not required when reading butmay be advisable for portability.

Support is provided for changing the tape partition and partitioningof the tape with one or two partitions. By default support forpartitioned tape is disabled for each driver and it can be enabledwith the ioctl MTSETDRVBUFFER.

By default the driver writes one filemark when the device is closed afterwriting and the last operation has been a write. Two filemarks can beoptionally written. In both cases end of data is signified byreturning zero bytes for two consecutive reads.

Writing filemarks without the immediate bit set in the SCSI command block actsas a synchronization point, i.e., all remaining data form the drive buffers iswritten to tape before the command returns. This makes sure that write errorsare caught at that point, but this takes time. In some applications, severalconsecutive files must be written fast. The MTWEOFI operation can be used towrite the filemarks without flushing the drive buffer. Writing filemark atclose() is always flushing the drive buffers. However, if the previousoperation is MTWEOFI, close() does not write a filemark. This can be used ifthe program wants to close/open the tape device between files and wants toskip waiting.

If rewind, offline, bsf, or seek is done and previous tape operation waswrite, a filemark is written before moving tape.

The compile options are defined in the file linux/drivers/scsi/st_options.h.

4. If the open option O_NONBLOCK is used, open succeeds even if thedrive is not ready. If O_NONBLOCK is not used, the driver waits forthe drive to become ready. If this does not happen in ST_BLOCK_SECONDSseconds, open fails with the errno value EIO. With O_NONBLOCK thedevice can be opened for writing even if there is a write protectedtape in the drive (commands trying to write something return error ifattempted).

Minor Numbers¶

The tape driver currently supports up to 2^17 drives if 4 modes foreach drive are used.

The minor numbers consist of the following bit fields:

The non-rewind bit is always bit 7 (the uppermost bit in the lowermostbyte). The bits defining the mode are below the non-rewind bit. Theremaining bits define the tape device number. This numbering isbackward compatible with the numbering used when the minor number wasonly 8 bits wide.

Sysfs Support¶

The driver creates the directory /sys/class/scsi_tape and populates it withdirectories corresponding to the existing tape devices. There are autorewindand non-rewind entries for each mode. The names are stxy and nstxy, where xis the tape number and y a character corresponding to the mode (none, l, m,a). For example, the directories for the first tape device are (assuming fourmodes): st0 nst0 st0l nst0l st0m nst0m st0a nst0a.

Each directory contains the entries: default_blksize default_compressiondefault_density defined dev device driver. The file ‘defined’ contains 1if the mode is defined and zero if not defined. The files ‘default_*’ containthe defaults set by the user. The value -1 means the default is not set. Thefile ‘dev’ contains the device numbers corresponding to this device. The links‘device’ and ‘driver’ point to the SCSI device and driver entries.

Each directory also contains the entry ‘options’ which shows the currentlyenabled driver and mode options. The value in the file is a bit mask where thebit definitions are the same as those used with MTSETDRVBUFFER in setting theoptions.

A link named ‘tape’ is made from the SCSI device directory to the classdirectory corresponding to the mode 0 auto-rewind device (e.g., st0).

Sysfs and Statistics for Tape Devices¶

The st driver maintains statistics for tape drives inside the sysfs filesystem.The following method can be used to locate the statistics that areavailable (assuming that sysfs is mounted at /sys):

1. Use opendir(3) on the directory /sys/class/scsi_tape
2. Use readdir(3) to read the directory contents
3. Use regcomp(3)/regexec(3) to match directory entries to the extendedregular expression “^st[0-9]+$”
4. Access the statistics from the /sys/class/scsi_tape/<match>/statsdirectory (where <match> is a directory entry from /sys/class/scsi_tapethat matched the extended regular expression)

The reason for using this approach is that all the character devicespointing to the same tape drive use the same statistics. That meansthat st0 would have the same statistics as nst0.

The directory contains the following statistics files:

1. in_flight

   • The number of I/Os currently outstanding to this device.

2. io_ns

   • The amount of time spent waiting (in nanoseconds) for all I/Oto complete (including read and write). This includes tape movementcommands such as seeking between file or set marks and implicit tapemovement such as when rewind on close tape devices are used.

3. other_cnt

   • The number of I/Os issued to the tape drive other than read orwrite commands. The time taken to complete these commands uses thefollowing calculation io_ms-read_ms-write_ms.

4. read_byte_cnt

   • The number of bytes read from the tape drive.

5. read_cnt

   • The number of read requests issued to the tape drive.

6. read_ns

   • The amount of time (in nanoseconds) spent waiting for readrequests to complete.

7. write_byte_cnt

   • The number of bytes written to the tape drive.

8. write_cnt

   • The number of write requests issued to the tape drive.

9. write_ns

   • The amount of time (in nanoseconds) spent waiting for writerequests to complete.

10. resid_cnt

    • The number of times during a read or write we foundthe residual amount to be non-zero. This should mean that a programis issuing a read larger thean the block size on tape. For writenot all data made it to tape.

Note

The in_flight value is incremented when an I/O starts the I/Oitself is not added to the statistics until it completes.

The total of read_cnt, write_cnt, and other_cnt may not total to the samevalue as iodone_cnt at the device level. The tape statistics only countI/O issued via the st module.

When read the statistics may not be temporally consistent while I/O is inprogress. The individual values are read and written to atomically howeverwhen reading them back via sysfs they may be in the process of beingupdated when starting an I/O or when it is completed.

The value shown in in_flight is incremented before any statstics areupdated and decremented when an I/O completes after updating statistics.The value of in_flight is 0 when there are no I/Os outstanding that areissued by the st driver. Tape statistics do not take into account anyI/O performed via the sg device.

BSD and Sys V Semantics¶

The user can choose between these two behaviours of the tape driver bydefining the value of the symbol ST_SYSV. The semantics differ when afile being read is closed. The BSD semantics leaves the tape where itcurrently is whereas the SYS V semantics moves the tape past the nextfilemark unless the filemark has just been crossed.

The default is BSD semantics.

Buffering¶

The driver tries to do transfers directly to/from user space. If thisis not possible, a driver buffer allocated at run-time is used. Ifdirect i/o is not possible for the whole transfer, the driver bufferis used (i.e., bounce buffers for individual pages are notused). Direct i/o can be impossible because of several reasons, e.g.:

• one or more pages are at addresses not reachable by the HBA
• the number of pages in the transfer exceeds the number ofscatter/gather segments permitted by the HBA
• one or more pages can’t be locked into memory (should not happen inany reasonable situation)

The size of the driver buffers is always at least one tape block. In fixedblock mode, the minimum buffer size is defined (in 1024 byte units) byST_FIXED_BUFFER_BLOCKS. With small block size this allows buffering ofseveral blocks and using one SCSI read or write to transfer all of theblocks. Buffering of data across write calls in fixed block mode isallowed if ST_BUFFER_WRITES is non-zero and direct i/o is not used.Buffer allocation uses chunks of memory having sizes 2^n * (pagesize). Because of this the actual buffer size may be larger than theminimum allowable buffer size.

NOTE that if direct i/o is used, the small writes are not buffered. This maycause a surprise when moving from 2.4. There small writes (e.g., tar without-b option) may have had good throughput but this is not true any more with2.6. Direct i/o can be turned off to solve this problem but a better solutionis to use bigger write() byte counts (e.g., tar -b 64).

Asynchronous writing. Writing the buffer contents to the tape isstarted and the write call returns immediately. The status is checkedat the next tape operation. Asynchronous writes are not done withdirect i/o and not in fixed block mode.

Buffered writes and asynchronous writes may in some rare cases causeproblems in multivolume operations if there is not enough space on thetape after the early-warning mark to flush the driver buffer.

Read ahead for fixed block mode (ST_READ_AHEAD). Filling the buffer isattempted even if the user does not want to get all of the data atthis read command. Should be disabled for those drives that don’t likea filemark to truncate a read request or that don’t like backspacing.

Scatter/gather buffers (buffers that consist of chunks non-contiguousin the physical memory) are used if contiguous buffers can’t beallocated. To support all SCSI adapters (including those notsupporting scatter/gather), buffer allocation is using the followingthree kinds of chunks:

1. The initial segment that is used for all SCSI adapters includingthose not supporting scatter/gather. The size of this buffer will be(PAGE_SIZE << ST_FIRST_ORDER) bytes if the system can give a chunk ofthis size (and it is not larger than the buffer size specified byST_BUFFER_BLOCKS). If this size is not available, the driver halvesthe size and tries again until the size of one page. The defaultsettings in st_options.h make the driver to try to allocate all of thebuffer as one chunk.
2. The scatter/gather segments to fill the specified buffer size areallocated so that as many segments as possible are used but the numberof segments does not exceed ST_FIRST_SG.
3. The remaining segments between ST_MAX_SG (or the module parametermax_sg_segs) and the number of segments used in phases 1 and 2are used to extend the buffer at run-time if this is necessary. Thenumber of scatter/gather segments allowed for the SCSI adapter is notexceeded if it is smaller than the maximum number of scatter/gathersegments specified. If the maximum number allowed for the SCSI adapteris smaller than the number of segments used in phases 1 and 2,extending the buffer will always fail.

EOM Behaviour When Writing¶

When the end of medium early warning is encountered, the current writeis finished and the number of bytes is returned. The next writereturns -1 and errno is set to ENOSPC. To enable writing a trailer,the next write is allowed to proceed and, if successful, the number ofbytes is returned. After this, -1 and the number of bytes arealternately returned until the physical end of medium (or some othererror) is encountered.

Module Parameters¶

The buffer size, write threshold, and the maximum number of allocated buffersare configurable when the driver is loaded as a module. The keywords are:

Note that if the buffer size is changed but the write threshold is notset, the write threshold is set to the new buffer size - 2 kB.

Boot Time Configuration¶

If the driver is compiled into the kernel, the same parameters can bealso set using, e.g., the LILO command line. The preferred syntax isto use the same keyword used when loading as module but prependedwith ‘st.’. For instance, to set the maximum number of scatter/gathersegments, the parameter ‘st.max_sg_segs=xx’ should be used (xx is thenumber of scatter/gather segments).

For compatibility, the old syntax from early 2.5 and 2.4 kernelversions is supported. The same keywords can be used as when loadingthe driver as module. If several parameters are set, the keyword-valuepairs are separated with a comma (no spaces allowed). A colon can beused instead of the equal mark. The definition is prepended by thestring st=. Here is an example:

The following syntax used by the old kernel versions is also supported:

where:

• aa is the buffer size for fixed block mode in 1024 byte units
• bb is the write threshold in 1024 byte units
• dd is the maximum number of scatter/gather segments

IOCTLs¶

The tape is positioned and the drive parameters are set with ioctlsdefined in mtio.h The tape control program ‘mt’ uses these ioctls. Tryto find an mt that supports all of the Linux SCSI tape ioctls andopens the device for writing if the tape contents will be modified(look for a package mt-st* from the Linux ftp sites; the GNU mt doesnot open for writing for, e.g., erase).

The supported ioctls are:

The following use the structure mtop:

Is used for several purposes. The command is obtained from countwith mask MT_SET_OPTIONS, the low order bits are used as argument.This command is only allowed for the superuser (root). Thesubcommands are:

• 0

  The drive buffer option is set to the argument. Zero meansno buffering.

• MT_ST_BOOLEANS

  Sets the buffering options. The bits are the new states(enabled/disabled) the following options (in theparenthesis is specified whether the option is global orcan be specified differently for each mode):

  MT_ST_BUFFER_WRITES

  write buffering (mode)

  MT_ST_ASYNC_WRITES

  asynchronous writes (mode)

  MT_ST_READ_AHEAD

  read ahead (mode)

  MT_ST_TWO_FM

  writing of two filemarks (global)

  MT_ST_FAST_EOM

  using the SCSI spacing to EOD (global)

  MT_ST_AUTO_LOCK

  automatic locking of the drive door (global)

  MT_ST_DEF_WRITES

  the defaults are meant only for writes (mode)

  MT_ST_CAN_BSR

  backspacing over more than one records canbe used for repositioning the tape (global)

  MT_ST_NO_BLKLIMS

  the driver does not ask the block limitsfrom the drive (block size can be changed only tovariable) (global)

  MT_ST_CAN_PARTITIONS

  enables support for partitionedtapes (global)

  MT_ST_SCSI2LOGICAL

  the logical block number is used inthe MTSEEK and MTIOCPOS for SCSI-2 drives instead ofthe device dependent address. It is recommended to setthis flag unless there are tapes using the devicedependent (from the old times) (global)

  MT_ST_SYSV

  sets the SYSV semantics (mode)

  MT_ST_NOWAIT

  enables immediate mode (i.e., don’t wait forthe command to finish) for some commands (e.g., rewind)

  MT_ST_NOWAIT_EOF

  enables immediate filemark mode (i.e. whenwriting a filemark, don’t wait for it to complete). Pleasesee the BASICS note about MTWEOFI with respect to thepossible dangers of writing immediate filemarks.

  MT_ST_SILI

  enables setting the SILI bit in SCSI commands whenreading in variable block mode to enhance performance whenreading blocks shorter than the byte count; set this onlyif you are sure that the drive supports SILI and the HBAcorrectly returns transfer residuals

  MT_ST_DEBUGGING

  debugging (global; debugging must becompiled into the driver)

• MT_ST_SETBOOLEANS, MT_ST_CLEARBOOLEANS

  Sets or clears the option bits.

• MT_ST_WRITE_THRESHOLD

  Sets the write threshold for this device to kilobytesspecified by the lowest bits.

• MT_ST_DEF_BLKSIZE

  Defines the default block size set automatically. Value0xffffff means that the default is not used any more.

• MT_ST_DEF_DENSITY, MT_ST_DEF_DRVBUFFER

  Used to set or clear the density (8 bits), and drive bufferstate (3 bits). If the value is MT_ST_CLEAR_DEFAULT(0xfffff) the default will not be used any more. Otherwisethe lowermost bits of the value contain the new value ofthe parameter.

• MT_ST_DEF_COMPRESSION

  The compression default will not be used if the value ofthe lowermost byte is 0xff. Otherwise the lowermost bitcontains the new default. If the bits 8-15 are set to anon-zero number, and this number is not 0xff, the number isused as the compression algorithm. The valueMT_ST_CLEAR_DEFAULT can be used to clear the compressiondefault.

• MT_ST_SET_TIMEOUT

  Set the normal timeout in seconds for this device. Thedefault is 900 seconds (15 minutes). The timeout should belong enough for the retries done by the device whilereading/writing.

• MT_ST_SET_LONG_TIMEOUT

  Set the long timeout that is used for operations that areknown to take a long time. The default is 14000 seconds(3.9 hours). For erase this value is further multiplied byeight.

• MT_ST_SET_CLN

  Set the cleaning request interpretation parameters usingthe lowest 24 bits of the argument. The driver can set thegeneric status bit GMT_CLN if a cleaning request bit patternis found from the extended sense data. Many drives set one ormore bits in the extended sense data when the drive needscleaning. The bits are device-dependent. The driver isgiven the number of the sense data byte (the lowest eightbits of the argument; must be >= 18 (values 1 - 17reserved) and <= the maximum requested sense data sixe),a mask to select the relevant bits (the bits 9-16), and thebit pattern (bits 17-23). If the bit pattern is zero, oneor more bits under the mask indicate cleaning request. Ifthe pattern is non-zero, the pattern must match the maskedsense data byte.

  (The cleaning bit is set if the additional sense code andqualifier 00h 17h are seen regardless of the setting ofMT_ST_SET_CLN.)

The following ioctl uses the structure mtpos:

MTIOCPOS

Reads the current position from the drive. UsesTandberg-compatible QFA for SCSI-1 drives and the SCSI-2command for the SCSI-2 drives.

The following ioctl uses the structure mtget to return the status:

MTIOCGET

Returns some status information.The file number and block number within file are returned. Theblock is -1 when it can’t be determined (e.g., after MTBSF).The drive type is either MTISSCSI1 or MTISSCSI2.The number of recovered errors since the previous status callis stored in the lower word of the field mt_erreg.The current block size and the density code are stored in the fieldmt_dsreg (shifts for the subfields are MT_ST_BLKSIZE_SHIFT andMT_ST_DENSITY_SHIFT).The GMT_xxx status bits reflect the drive status. GMT_DR_OPENis set if there is no tape in the drive. GMT_EOD means eitherend of recorded data or end of tape. GMT_EOT means end of tape.

Miscellaneous Compile Options¶

The recovered write errors are considered fatal if ST_RECOVERED_WRITE_FATALis defined.

The maximum number of tape devices is determined by the defineST_MAX_TAPES. If more tapes are detected at driver initialization, themaximum is adjusted accordingly.

Immediate return from tape positioning SCSI commands can be enabled bydefining ST_NOWAIT. If this is defined, the user should take care thatthe next tape operation is not started before the previous one hasfinished. The drives and SCSI adapters should handle this conditiongracefully, but some drive/adapter combinations are known to hang theSCSI bus in this case.

Tandberg Scsi & Raid Devices Driver Updater

The MTEOM command is by default implemented as spacing over 32767filemarks. With this method the file number in the status iscorrect. The user can request using direct spacing to EOD by settingST_FAST_EOM 1 (or using the MT_ST_OPTIONS ioctl). In this case the filenumber will be invalid.

When using read ahead or buffered writes the position within the filemay not be correct after the file is closed (correct position mayrequire backspacing over more than one record). The correct positionwithin file can be obtained if ST_IN_FILE_POS is defined at compiletime or the MT_ST_CAN_BSR bit is set for the drive with an ioctl.(The driver always backs over a filemark crossed by read ahead if theuser does not request data that far.)

Debugging Hints¶

Debugging code is now compiled in by default but debugging is turned offwith the kernel module parameter debug_flag defaulting to 0. Debuggingcan still be switched on and off with an ioctl. To enable debug atmodule load time add debug_flag=1 to the module load options, thedebugging output is not voluminous. Debugging can also be enabledand disabled by writing a ‘0’ (disable) or ‘1’ (enable) to the sysfsfile /sys/bus/scsi/drivers/st/debug_flag.

If the tape seems to hang, I would be very interested to hear wherethe driver is waiting. With the command ‘ps -l’ you can see the stateof the process using the tape. If the state is D, the process iswaiting for something. The field WCHAN tells where the driver iswaiting. If you have the current System.map in the correct place (in/boot for the procps I use) or have updated /etc/psdatabase (for kmemps), ps writes the function name in the WCHAN field. If not, you haveto look up the function from System.map.

Tandberg Scsi & Raid Devices Drivers

Note also that the timeouts are very long compared to most otherdrivers. This means that the Linux driver may appear hung although thereal reason is that the tape firmware has got confused.