Mercurial > repos > peterjc > secreted_protein_workflow
annotate README.rst @ 10:2c8931827fa5 draft
Uploaded with note about NR versioning
| author | peterjc |
|---|---|
| date | Mon, 30 Mar 2015 11:46:13 -0400 |
| parents | 3b5eecc9551e |
| children | 99209ed2ec87 |
| rev | line source |
|---|---|
| 10 | 1 Introduction |
| 2 ============ | |
| 3 | |
| 4 Galaxy is a web-based platform for biological data analysis, supporting | |
| 5 extension with additional tools (often wrappers for existing command line | |
| 6 tools) and datatypes. See http://www.galaxyproject.org/ and the public | |
| 7 server at http://usegalaxy.org for an example. | |
| 8 | |
| 9 The NCBI BLAST suite is a widely used set of tools for biological sequence | |
| 10 comparison. It is available as standalone binaries for use at the command | |
| 11 line, and via the NCBI website for smaller searches. For more details see | |
| 12 http://blast.ncbi.nlm.nih.gov/Blast.cgi | |
| 13 | |
| 14 This is an example workflow using the Galaxy wrappers for NCBI BLAST+, | |
| 15 see https://github.com/peterjc/galaxy_blast | |
| 16 | |
| 17 | |
| 18 Galaxy workflow for counting species of top BLAST hits | |
| 19 ====================================================== | |
| 20 | |
| 21 This Galaxy workflow (file ``blast_top_hit_species.ga``) is intended for an | |
| 22 initial assessment of a transcriptome assembly to give a crude indication of | |
| 23 any major contamination present based on the species of the top BLAST hit | |
| 24 of 1000 representative sequences. | |
| 25 | |
| 26 .. image:: https://raw.githubusercontent.com/peterjc/galaxy_blast/master/workflows/blast_top_hit_species/blast_top_hit_species.png | |
| 27 | |
| 28 In words, the workflow proceeds as follows: | |
| 29 | |
| 30 1. Upload/import your transcriptome assembly or any nucleotide FASTA file. | |
| 31 2. Samples 1000 representative sequences, selected uniformly/evenly though | |
| 32 the file. | |
| 33 3. Convert the sampled FASTA file into a three column tabular file. | |
| 34 4. Runs NCBI BLASTX of the sampled FASTA file against the latest NCBI ``nr`` | |
| 35 database (assuming this is already available setup on your local Galaxy | |
| 36 under the alias ``nr``), requesting tabular output including the taxonomy | |
| 37 fields, and at most one matching target sequence. | |
| 38 5. Remove any duplicate alignments (multiple HSPs for the same match). | |
| 39 6. Combine the filtered BLAST output with the tabular version of the 1000 | |
| 40 sequences to give a new tabular file with exactly 1000 lines, adding | |
| 41 ``None`` for sequences missing a BLAST hit. | |
| 42 7. Count the BLAST species names in this file. | |
| 43 8. Sort the counts. | |
| 44 | |
| 45 Finally we would suggest visualising the sorted tally table as a Pie Chart. | |
| 46 | |
| 47 | |
| 48 Sample Data | |
| 49 =========== | |
| 50 | |
| 51 As an example, you can upload the transcriptome assembly of the nematode | |
| 52 *Nacobbus abberans* from Eves van den Akker *et al.* (2015), | |
| 53 http://dx.doi.org/10.1093/gbe/evu171 using this URL: | |
| 54 | |
| 55 http://nematode.net/Data/nacobbus_aberrans_transcript_assembly/N.abberans_reference_no_contam.zip | |
| 56 | |
| 57 Running this workflow with a copy of the NCBI non-redundant ``nr`` database | |
| 58 from 16 Oct 2014 (which did **not** contain this *N. abberans* dataset) gave | |
| 59 the following results - note 609 out of the 1000 sequences gave no BLAST hit. | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
60 |
| 10 | 61 ===== ================== |
| 62 Count Subject Blast Name | |
| 63 ----- ------------------ | |
| 64 609 None | |
| 65 244 nematodes | |
| 66 30 ascomycetes | |
| 67 27 eukaryotes | |
| 68 8 basidiomycetes | |
| 69 6 aphids | |
| 70 5 eudicots | |
| 71 5 flies | |
| 72 ... ... | |
| 73 ===== ================== | |
| 74 | |
| 75 As you might guess from the filename ``N.abberans_reference_no_contam.fasta``, | |
| 76 this transcriptome assembly has already had obvious contamination removed. | |
| 77 | |
| 78 At the time of writing, Galaxy's visualizations could not be included in | |
| 79 a workflow. You can generate a pie chart from the final count file using | |
| 80 the counts (c1) and labels (c2), like this: | |
| 81 | |
| 82 .. image:: https://raw.githubusercontent.com/peterjc/galaxy_blast/master/workflows/blast_top_hit_species/N_abberans_piechart_mouseover.png | |
| 83 | |
| 84 Note the nematode count in this image was shown as a mouse-over effect. | |
| 85 | |
| 86 | |
| 87 Disclaimer | |
| 88 ========== | |
| 89 | |
| 90 Species assignment by top BLAST hit is not suitable for any in depth | |
| 91 analysis. It is particularly prone to false positives where contaminants | |
| 92 in public datasets are mislabelled. See for example Ed Yong (2015), | |
| 93 "There's No Plague on the NYC Subway. No Platypuses Either.": | |
| 94 | |
| 95 http://phenomena.nationalgeographic.com/2015/02/10/theres-no-plague-on-the-nyc-subway-no-platypuses-either/ | |
| 96 | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
97 |
| 10 | 98 Known Issues |
| 99 ============ | |
| 100 | |
| 101 Counts | |
| 102 ------ | |
| 103 | |
| 104 This workflow uses the Galaxy "Count" tool, version 1.0.0, as shipped with | |
| 105 the current stable release (Galaxy v15.03, i.e. March 2015). | |
| 106 | |
| 107 The updated "Count" tool version 1.0.1 includes a fix not to remove spaces | |
| 108 in the fields being counted. In the example above, while the top hits are | |
| 109 not affected, minor entries like "cellular slime molds" are shown as | |
| 110 "cellularslimemolds" instead (look closely at the Pie Chart key).. | |
| 111 | |
| 112 The updated "Count" tool version 1.0.1 also adds a new option to sort the | |
| 113 output, which avoids the additional sorting step in the current version of | |
| 114 the workflow. | |
| 115 | |
| 116 A future update to this workflow will use the revised "Count" tool, once | |
| 117 this is included in the next stable Galaxy release - or migrated to the | |
| 118 Galaxy Tool Shed. | |
| 119 | |
| 120 NCBI nr database | |
| 121 ---------------- | |
| 122 | |
| 123 The use of external datasets within Galaxy via the ``*.loc`` configuration | |
| 124 files undermines provenance tracking within Galaxy. This is exacerbated | |
| 125 by the lack of officially versioned BLAST database releases by the NCBI. | |
| 126 | |
| 127 This workflow assumes that you have an entry ``nr`` in your ``blastdb_p.loc`` | |
| 128 (the configuration file listing locally installed BLAST databases external | |
| 129 to Galaxy - consult the NCBI BLAST+ wrapper documentation for more details), | |
| 130 and that this points to a mirror of the latest NCBI "non-redundant" database | |
| 131 from ftp://ftp.ncbi.nlm.nih.gov/blast/db/ | |
| 132 | |
| 133 i.e. The workflow is intended to be used against the *latest* nr database, | |
| 134 and thus is not reproducible over the long term as the database changes. | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
135 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
136 |
| 9 | 137 Availability |
| 138 ============ | |
| 139 | |
| 10 | 140 This workflow is available to download and/or install from the main Galaxy Tool Shed: |
| 9 | 141 |
| 10 | 142 http://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species |
| 9 | 143 |
| 144 Test releases (which should not normally be used) are on the Test Tool Shed: | |
| 145 | |
| 10 | 146 http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species |
| 9 | 147 |
| 148 Development is being done on github here: | |
| 149 | |
| 10 | 150 https://github.com/peterjc/galaxy_blast/tree/master/workflows/blast_top_hit_species |
| 5 | 151 |
| 152 | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
153 Citation |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
154 ======== |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
155 |
| 10 | 156 Please cite the following paper (currently available as a preprint): |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
157 |
| 10 | 158 NCBI BLAST+ integrated into Galaxy. |
| 159 P.J.A. Cock, J.M. Chilton, B. Gruening, J.E. Johnson, N. Soranzo | |
| 160 bioRxiv DOI: http://dx.doi.org/10.1101/014043 (preprint) | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
161 |
| 10 | 162 You should also cite Galaxy, and the NCBI BLAST+ tools: |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
163 |
| 10 | 164 BLAST+: architecture and applications. |
| 165 C. Camacho et al. BMC Bioinformatics 2009, 10:421. | |
| 166 DOI: http://dx.doi.org/10.1186/1471-2105-10-421 | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
167 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
168 |
| 10 | 169 Automated Installation |
| 170 ====================== | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
171 |
| 10 | 172 Installation via the Galaxy Tool Shed should take care of the dependencies |
| 173 on Galaxy tools including the NCBI BLAST+ wrappers and associated binaries. | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
174 |
| 10 | 175 However, this workflow requires a current version of the NCBI nr protein |
| 176 BLAST database to be listed in ``blastdb_p.loc`` with the key ``nr`` (lower | |
| 177 case). | |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
178 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
179 |
|
6
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
180 History |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
181 ======= |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
182 |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
183 ======= ====================================================================== |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
184 Version Changes |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
185 ------- ---------------------------------------------------------------------- |
| 10 | 186 v0.1.0 - Initial Tool Shed release, targetting NCBI BLAST+ 2.2.29 |
|
6
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
187 ======= ====================================================================== |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
188 |
|
36b2c2b5051e
Uploaded v0.0.2 with updated tool versions, should fix header_lines warning from Filter1 tool.
peterjc
parents:
5
diff
changeset
|
189 |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
190 Developers |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
191 ========== |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
192 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
193 This workflow is under source code control here: |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
194 |
| 10 | 195 https://github.com/peterjc/galaxy_blast/tree/master/workflows/blast_top_hit_species |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
196 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
197 To prepare the tar-ball for uploading to the Tool Shed, I use this: |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
198 |
| 10 | 199 $ tar -cf blast_top_hit_species.tar.gz README.rst repository_dependencies.xml blast_top_hit_species.ga blast_top_hit_species.png N_abberans_piechart_mouseover.png |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
200 |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
201 Check this, |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
202 |
| 10 | 203 $ tar -tzf blast_top_hit_species.tar.gz |
|
4
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
204 README.rst |
|
b14c822a37fe
Uploaded v0.0.1d, README file with clearer citation instructions.
peterjc
parents:
diff
changeset
|
205 repository_dependencies.xml |
| 10 | 206 blast_top_hit_species.ga |
| 207 blast_top_hit_species.png | |
| 208 N_abberans_piechart_mouseover.png | |
| 209 | |
| 210 | |
| 211 Licence (MIT) | |
| 212 ============= | |
| 213 | |
| 214 Permission is hereby granted, free of charge, to any person obtaining a copy | |
| 215 of this software and associated documentation files (the "Software"), to deal | |
| 216 in the Software without restriction, including without limitation the rights | |
| 217 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | |
| 218 copies of the Software, and to permit persons to whom the Software is | |
| 219 furnished to do so, subject to the following conditions: | |
| 220 | |
| 221 The above copyright notice and this permission notice shall be included in | |
| 222 all copies or substantial portions of the Software. | |
| 223 | |
| 224 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 225 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 226 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | |
| 227 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | |
| 228 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | |
| 229 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | |
| 230 THE SOFTWARE. |