| libscan

## Function

Scans a database with a set of models that represents a family

## Description

Scans each signature, profile or HMM in a directory against a sequence database and writes a signature hits file for each one. Or scans sequences against such a library of discriminating elements and writes a library scan file for each one.

This program is part of a suite of EMBOSS applications that directly or indirectly make use of the protein structure databases pdb and scop. This program is part of an experimental analysis pipeline described in an accompanying document. We provide the software in the hope that it will be useful. The applications were designed for specific research purposes and may not be useful or reliable in contexts other than the described pipeline. The development of the suite was coordinated by Jon Ison to whom enquiries and bug reports should be sent (email jison@hgmp.mrc.ac.uk).

The advantage of screening a relatively small library of discriminating elements with a sequence is that it is sufficient for the sequence to detect its true family (discriminator) in the first rank for an effective prediction. This is in contrast to searching a larger sequence database to identify homology, where biologically significant hits may achieve statistically insignificant scores and therefore be missed. A library might help the detection of such proteins because they may still score their true discriminator higher than the others in the library regardless of statistical estimates. Further improvements to predictions are gained when multiple sources of evidence, in this case the different types of discriminating element, are considered.

The libscan application allows a protein sequence or sequences to be screened against the library of discriminating elements. The results of a screen are returned to the user in a library scan file containing a list of top-scoring SCOP domains rank-ordered by p-value for each individual type of discriminator, and also for all of the discriminator types in combination (combined prediction). For the combined prediction, the p-value is derived from an empirically derived distribution of the product of the p-values of the individual methods.

libscan runs in one of two modes either (i) database search mode or (ii) library screen mode. In database search mode libscan reads one or more directories each containing a single type of discriminating element, the permitted types are sparse sequence signature, Gribskov profile, Henikoff profile or hidden Markov model. Each directory must contain a discriminating element for each of one or more scop families; files for indivdual families should have the same name (possibly with different file extensions) in the different directories, e.g. the scop family with the identifier 46445 might be represented by files 46445.hmm, 46445.sig, 46445.gribs and 46445.henik in their respective directories. libscan scans each discriminating element against a sequence set that must also be provided by the user and generates a signature hits file (of scored hits to sequences in the set) for each individual discriminating element. In mode (i), the user must also provide a scop validation file so that a classification of hits can be given in the signature hits files. The user specifies a maximum number of false hits that are written to the signature hits files.

In library screen mode, libscan reads a sequence set, screens each sequence against the library (directories of discriminating elements) and writes a library scan file (of top-scoring families) for each sequence. In either mode, a scop classification file is required as a source of family classification data. The paths and extensions of the sequence set and scop classification files (input) different types of discriminating elements (input), and paths and extension of signature hits files or library scan file (output) are specified by the user.

## Algorithm

The approach for calculating p-values for individual discriminator-sequence matches is the same irrespective of which mode libscan is run in and is summarised as follows: (i) Each discriminating element is scanned (scored) against each sequence. (ii) The ditribution of scores is generated for each different type of element. (iii) Using these distributions, a p-value is calculated for each discriminator-sequence match. The approach for calculating the 'combined p-value' for each individual sequence is as follows: (i) p-values of the matches for the different types of discriminator versus a sequence are multiplied to yield the 'product of p-values'. (ii) A distribution of 'product of p-values' is generated for each individual sequence. (iii) Using these distributions, a so-called 'combined p-value', reflecting the overall significance of the matches of the sequence to the different types of discriminators, is generated. Importantly, two (or more) p-values are only ever multiplied together if corresponding hit is to the same region of the protein. 'Same' is defined as two hits overlapping by a user-defined number of residues.

## Usage

## Command line arguments

       Standard (Mandatory) qualifiers (* if not always prompted):
       -mode               menu       Libscan runs in one of two modes either (i)
                                      database search mode or (ii) library screen
                                      mode. In database search mode libscan reads
                                      one or more directories each containing a
                                      single type of discriminating element, the
                                      permitted types are sparse sequence
                                      signature, Gribskov profile, Henikoff
                                      profile or hidden Markov model. In library
                                      screen mode, libscan reads a sequence set,
                                      screens each sequence against the library
                                      (directories of discriminating elements) and
                                      writes a library scan file (of top-scoring
                                      families) for each one.
       -db                 seqset     In database search mode libscan scans each
                                      discriminating element against a sequence
                                      set. In library screen mode, libscan reads a
                                      sequence set and screens each sequence
                                      against the library (directories of
                                      disciminating elements)
       -[no]grib           boolean    Use Gribskov profile
       -[no]henik          boolean    Use Henikoff profile
       -[no]hmm            boolean    Use HMM profile
       -[no]sig            boolean    Use signature
    *  -hmmpath            string     Location of HMM profile files (input)
    *  -hmmextn            string     Extension of HMM profile files (input)
    *  -hmmoutpath         string     Location of signature hits/library scan
                                      files for hmm searches (output)
    *  -hmmoutextn         string     Extension of signature hits/library scan
                                      files for hmm searches (output)
    *  -gbvpath            string     Location of Gribskov profile files (input)
    *  -gbvextn            string     Extension of Gribskov profile files (input)
    *  -gbvgapo            float      Gap insertion penalty
    *  -gbvgape            float      Gap extension penalty
    *  -gbvoutpath         string     Location of signature hits/library scan
                                      files for Gribskov profile searches (output)
    *  -gbvoutextn         string     Extension of signature hits/library scan
                                      files for Gribskov profile searches (output)
    *  -hnfpath            string     Location of Hennikoff profile files (input)
    *  -hnfextn            string     Extension of Hennikoff profile files (input)
    *  -hnfgapo            float      Gap insertion penalty
    *  -hnfgape            float      Gap extension penalty
    *  -hnfoutpath         string     Location of signature hits/library scan
                                      files for Henikoff profile searches (output)
    *  -hnfoutextn         string     Extension of signature hits/library scan
                                      files for Henikoff profile searches (output)
    *  -sigpath            string     Location of signature files (input)
    *  -sigextn            string     Extension of signature files (input)
    *  -nterm              menu       Select number
    *  -sub                matrixf    Residue substitution matrix
    *  -siggapo            float      Gap insertion penalty
    *  -siggape            float      Gap extension penalty
    *  -sigoutpath         string     Location of signature hits/library scan
                                      files for signature searches (output)
    *  -sigoutextn         string     Extension for signature hits/library scan
                                      files for signature searches (output)
       -scopf              infile     In either mode, a 'scop classification file'
                                      is required as a source of family
                                      classification data. A scop classification
                                      file contains classification and other data
                                      for domains from the scop database. The file
                                      is in embl-like format and is generated by
                                      scopparse. Domain sequence information can
                                      be added to the file by using scopseqs.
    *  -targetf            infile     A 'scop validation file' contains sequence
                                      relatives (hits) for each of a number of
                                      different scop families, superfamilies and
                                      folds. The file contains the collated
                                      results from psiblast searches of a sequence
                                      database for the indvidual scop families;
                                      hits of unambiguous family assignment are
                                      listed under their respective family,
                                      otherwise a hit is assigned as relatives to
                                      a scop superfamily or fold instead. The scop
                                      validation file is generated by seqnr and
                                      is in embl-like format.
    *  -maxhits            integer    The maximum number of hits to be written to
                                      the signature hits file.
    *  -overlap            integer    When estimating the statistical significance
                                      of scored matches of different types of
                                      discriminating element to a sequence, two
                                      (or more) p-values are only ever multiplied
                                      together if corresponding hit is to the same
                                      region of the protein. 'Same' is defined as
                                      two hits overlapping by a user-defined
                                      number of residues.
       -mrgoutpath         string     Location of signature hits / library scan
                                      files for merged results (output)
       -mrgoutextn         string     Extension for signature hits / library scan
                                      files for merged results (output)

       Additional (Optional) qualifiers: (none)
       Advanced (Unprompted) qualifiers: (none)
       Associated qualifiers:

       "-db" associated qualifiers
       -sbegin              integer    Start of each sequence to be used
       -send                integer    End of each sequence to be used
       -sreverse            boolean    Reverse (if DNA)
       -sask                boolean    Ask for begin/end/reverse
       -snucleotide         boolean    Sequence is nucleotide
       -sprotein            boolean    Sequence is protein
       -slower              boolean    Make lower case
       -supper              boolean    Make upper case
       -sformat             string     Input sequence format
       -sdbname             string     Database name
       -sid                 string     Entryname
       -ufo                 string     UFO features
       -fformat             string     Features format
       -fopenfile           string     Features file name

       General qualifiers:
       -auto                boolean    Turn off prompts
       -stdout              boolean    Write standard output
       -filter              boolean    Read standard input, write standard output
       -options             boolean    Prompt for standard and additional values
       -debug               boolean    Write debug output to program.dbg
       -verbose             boolean    Report some/full command line options
       -help                boolean    Report command line options. More
                                      information on associated and general
                                      qualifiers can be found with -help -verbose
       -warning             boolean    Report warnings
       -error               boolean    Report errors
       -fatal               boolean    Report fatal errors
       -die                 boolean    Report deaths

Standard (Mandatory) qualifiers | Allowed values | Default
--- | --- | ---
-mode | Libscan runs in one of two modes either (i) database search mode or (ii) library screen mode. In database search mode libscan reads one or more directories each containing a single type of discriminating element, the permitted types are sparse sequence signature, Gribskov profile, Henikoff profile or hidden Markov model. In library screen mode, libscan reads a sequence set, screens each sequence against the library (directories of discriminating elements) and writes a library scan file (of top-scoring families) for each one. |
1 | (Database search mode)
2 | (Library screen mode)
| 1
-db | In database search mode libscan scans each discriminating element against a sequence set. In library screen mode, libscan reads a sequence set and screens each sequence against the library (directories of disciminating elements) | Readable set of sequences | Required
-[no]grib | Use Gribskov profile | Boolean value Yes/No | Yes
-[no]henik | Use Henikoff profile | Boolean value Yes/No | Yes
-[no]hmm | Use HMM profile | Boolean value Yes/No | Yes
-[no]sig | Use signature | Boolean value Yes/No | Yes
-hmmpath | Location of HMM profile files (input) | Any string is accepted | ./
-hmmextn | Extension of HMM profile files (input) | Any string is accepted | .hmm
-hmmoutpath | Location of signature hits/library scan files for hmm searches (output) | Any string is accepted | ./
-hmmoutextn | Extension of signature hits/library scan files for hmm searches (output) | Any string is accepted | .hmmout
-gbvpath | Location of Gribskov profile files (input) | Any string is accepted | ./
-gbvextn | Extension of Gribskov profile files (input) | Any string is accepted | .gribs
-gbvgapo | Gap insertion penalty | Any numeric value | 1.0
-gbvgape | Gap extension penalty | Any numeric value | 1.0
-gbvoutpath | Location of signature hits/library scan files for Gribskov profile searches (output) | Any string is accepted | ./
-gbvoutextn | Extension of signature hits/library scan files for Gribskov profile searches (output) | Any string is accepted | .gribout
-hnfpath | Location of Hennikoff profile files (input) | Any string is accepted | ./
-hnfextn | Extension of Hennikoff profile files (input) | Any string is accepted | .henik
-hnfgapo | Gap insertion penalty | Any numeric value | 1.0
-hnfgape | Gap extension penalty | Any numeric value | 1.0
-hnfoutpath | Location of signature hits/library scan files for Henikoff profile searches (output) | Any string is accepted | ./
-hnfoutextn | Extension of signature hits/library scan files for Henikoff profile searches (output) | Any string is accepted | .henikout
-sigpath | Location of signature files (input) | Any string is accepted | ./
-sigextn | Extension of signature files (input) | Any string is accepted | .sig
-nterm | Select number |
1 | (Align anywhere and allow only complete signature-sequence fit)
2 | (Align anywhere and allow partial signature-sequence fit)
3 | (Use empirical gaps only)
| 1
-sub | Residue substitution matrix | Comparison matrix file in EMBOSS data path | EBLOSUM62
-siggapo | Gap insertion penalty | Any numeric value | 10.0
-siggape | Gap extension penalty | Any numeric value | 0.5
-sigoutpath | Location of signature hits/library scan files for signature searches (output) | Any string is accepted | ./
-sigoutextn | Extension for signature hits/library scan files for signature searches (output) | Any string is accepted | .sigout
-scopf | In either mode, a 'scop classification file' is required as a source of family classification data. A scop classification file contains classification and other data for domains from the scop database. The file is in embl-like format and is generated by scopparse. Domain sequence information can be added to the file by using scopseqs. | Input file | Required
-targetf | A 'scop validation file' contains sequence relatives (hits) for each of a number of different scop families, superfamilies and folds. The file contains the collated results from psiblast searches of a sequence database for the indvidual scop families; hits of unambiguous family assignment are listed under their respective family, otherwise a hit is assigned as relatives to a scop superfamily or fold instead. The scop validation file is generated by seqnr and is in embl-like format. | Input file | Required
-maxhits | The maximum number of hits to be written to the signature hits file. | Any integer value | 50
-overlap | When estimating the statistical significance of scored matches of different types of discriminating element to a sequence, two (or more) p-values are only ever multiplied together if corresponding hit is to the same region of the protein. 'Same' is defined as two hits overlapping by a user-defined number of residues. | Any integer value | 10
-mrgoutpath | Location of signature hits / library scan files for merged results (output) | Any string is accepted | ./
-mrgoutextn | Extension for signature hits / library scan files for merged results (output) | Any string is accepted | .mrgout
Additional (Optional) qualifiers | Allowed values | Default
(none)
Advanced (Unprompted) qualifiers | Allowed values | Default
(none)

## Input file format

libscan reads any normal sequence USAs.

The format of the scop classification file is described in scopparse

The format of the signature files is explained in siggen

The format of the profile and hmm files is explained in documentation for the propechy application and the hmmer package respectively.

## Output file format

The format of the signature hits file is described in sigscan

## Data files

EMBOSS data files are distributed with the application and stored in the standard EMBOSS data directory, which is defined by the EMBOSS environment variable EMBOSS_DATA.

To see the available EMBOSS data files, run:

    % embossdata -showall

To fetch one of the data files (for example 'Exxx.dat') into your current directory for you to inspect or modify, run:

    % embossdata -fetch -file Exxx.dat

Users can provide their own data files in their own directories. Project specific files can be put in the current directory, or for tidier directory listings in a subdirectory called ".embossdata". Files for all EMBOSS runs can be put in the user's home directory, or again in a subdirectory called ".embossdata".

The directories are searched in the following order:

- . (your current directory)
- .embossdata (under your current directory)
- ~/ (your home directory)
- ~/.embossdata

libscan requires a residue substitution matrix.

## Notes

None.

## References

None.

## Warnings

The score distributions and therefore p-values are calculated only from the data that libscan is provided. Therefore, meaningful p-values are only guaranteed if the sequence set (database search mode) or number of families (library screeen mode) is sufficiently large.

## Diagnostic Error Messages

None.

## Exit status

It always exits with status 0.

## Known bugs

None.

## See also

Program name | Description
--- | ---
contacts | Reads coordinate files and writes files of intra-chain residue-residue contact data
fraggle | Removes fragment sequences from files of hits for scop families
hmmgen | Generates a hidden Markov model for each alignment in a directory by using the HMMER package
interface | Reads coordinate files and writes files of inter-chain residue-residue contact data
profgen | Generates various profiles for each alignment in a directory
rocplot | Provides interpretation and graphical display of the performance of discriminating elements (e.g. profiles for protein families). rocplot reads file(s) of hits from discriminator-database search(es), performs ROC analysis on the hits, and writes graphs illustrating the diagnostic performance of the discriminating elements
scopalign | Generate alignments for families in a scop classification file by using STAMP
scoprep | Reorder scop classificaiton file so that the representative structure of each family is given first
scopreso | Removes low resolution domains from a scop classification file
seqalign | Generate extended alignments for families in a scop families file by using CLUSTALW with seed alignments
seqsearch | Generate files of hits for families in a scop classification file by using PSI-BLAST with seed alignments
seqsort | Reads multiple files of hits and writes (i) a scop families file and (ii) a scop ambiguities file
seqwords | Generate file of hits for scop families by searching swissprot with keywords
siggen | Generates a sparse protein signature from an alignment and residue contact data
sigplot | Generates data files of signature performance
sigscan | Scans a signature against swissprot and writes a signature hits file

Hidden Markov models can be generated for scop families by using hmmgen which uses the HMMER package.

Simple frequency matrices, Gribskov profiles and Hennikoff profiles can be generated for scop families by using profgen.

A 'signature file' contains a sparse sequence signature. The files are generated by siggen.

A 'signature hits file' contains the results of a search of a discriminating element (one of protein signature, hidden Markov model, simple frequency matrix, Gribskov profile or Hennikoff profile) against a sequence database. The files are generated by sigscan and modelscan.

A 'library scan file' holds the results of a search of a sequence against a library of discriminating elements for scop families (one directory for each type of element, e.g. sparse sequence signatures, profiles or HMMs).

The library scan file contains, for each invividual type of discriminating element and for the combination of the elements, classification and other data for each of a user-defined number of top-scoring families (discriminators) from the scop database.

A 'scop classification file' contains classification and other data for domains from the scop database. The file is in embl-like format and is generated by scopparse. Domain sequence information can be added to the file by using scopseqs.

A 'scop validation file' contains sequence relatives (hits) for each of a number of different scop families, superfamilies and folds. The file contains the collated results from psiblast searches of a sequence database for the indvidual scop families; hits of unambiguous family assignment are listed under their respective family, otherwise a hit is assigned as relatives to a scop superfamily or fold instead. The scop validation file is generated by seqnr and is in embl-like format.

## Author(s)

Jon Ison (jison © rfcgr.mrc.ac.uk)
MRC Rosalind Franklin Centre for Genomics Research Wellcome Trust Genome Campus, Hinxton, Cambridge, CB10 1SB, UK

Ranjeeva Ranasinghe (rranasin © hgmp.mrc.ac.uk)
HGMP-RC, Genome Campus, Hinxton, Cambridge CB10 1SB, UK

## History

Written (2003) - Jon Ison & Ranjeeva Ranasinghe

## Target users

This program is intended to be used by everyone and everything, from naive users to embedded scripts.