QIIME 1.9.0 bug affecting PyNAST alignment of 16S amplicons generated with non-515F/806R primers

15 04 2015

Hi all,

We discovered an issue in QIIME 1.9.0 with alignment of 16S amplicons that are generated with non-515F/806R primers when using PyNAST with the default template alignment. This bug will only be a problem for your analysis if it meets those exact criteria.

This bug stems from an issue with the template alignment that is packaged in the qiime-default-reference project, and would cause many of your sequences to fail to align with PyNAST. The issue is discussed in more detail, including a description of a work-around that we’re currently testing, on qiime-default-reference issue #14. Please follow that issue to track progress on this. We’ll follow-up on the blog and QIIME forum when a fix is ready – we expect that this will be this week.

We’d like to thank forum user Slave Trajanoski for pointing this out, and for the helpful example data which let us narrow down the problem quickly.

We know that this is very inconvenient, and we apologize for any issues this may have caused. If you have any questions about this, please don’t hesitate to follow up on the forum. If you’d like to know if this issue affected one of your runs, please post to the forum with the output of running print_qiime_config.py.



QIIME 1.9.0 is live!

30 01 2015

We’re excited to announce the release of QIIME 1.9.0! As is typical for our releases, the QIIME 1.9.0 Virtual Box and EC2 image will be ready in about a week.

For details on the new features in QIIME 1.9.0 and how to start using it, see the QIIME 1.9.0 release notes.

Greg, on behalf of the QIIME developers

QIIME 1.9.0 release candidate 1 is now live!

24 12 2014

Hi all,
We’re excited to announce that the QIIME 1.9.0 release candidate 1 is now live and ready for testing. You can find details here, which includes a discussion of what we mean by “release candidate”.

This release candidate is packed with new features, some of which are highlighted in the release notes at the above link.

As always, thanks for using QIIME!

Happy holidays and new year!

ISME 15 QIIME Workshop

11 03 2014

Hello QIIME users,
The QIIME Developers will be teaching a workshop after ISME 15 on August 30th, 2014 in Seoul, South Korea. This is the Saturday following ISME, and will be held at Seoul National University, near the ISME venue. You can find details on the workshop, including a link to the application, at:


Hope to see you at ISME 15!

– Greg, on behalf of the QIIME development group

QIIME 1.8.0 is live!

12 12 2013

Hello QIIME users,
Today we’re very excited to announce the release of QIIME 1.8.0, which is packed with new features that expand the functionality and usability of QIIME. The QIIME 1.8.0 Virtual Box and EC2 images will be ready later this week. Here are some of the highlights in 1.8.0:

First, one of the most frequent comments that we get about QIIME (which we completely agree with) is that it is difficult to install. To address this, we’ve defined a QIIME “base install” package, which is installable with pip so greatly reduces the complexity of QIIME installation. The QIIME base install package supports running the most commonly used QIIME commands with default parameters, and in most cases will be sufficient for your entire QIIME analyses. You can find discussion of this in the updated QIIME Installation Guide.

Next, our PCoA plots are no longer based on KiNG, but on the new Emperor 3D plotting package. This enables more advanced analysis of PCoA plots in the context of sample metadata, and is one step toward our larger goal of improving QIIME’s interactive visualization capabilities.

We’ve added support for assembling Illumina paired end reads in the new join_paired_ends.py script, which wraps  ea-utils and SeqPrep,  and for working with alternative barcoding schemes used on the Illumina platform with the new extract_barcodes.py. A new tutorial has been added for working with alternative barcoding schemes.

We’ve updated QIIME’s default taxonomy assigner to be the new uclust-based consensus taxonomy assigner. This was shown to be more accurate and faster than the existing methods (Bokulich, Rideout et al. (submitted)). Important: the RDP Classifier is no longer the default taxonomy assigner used in QIIME.

otu_category_significance.py has been removed in favor of group_significance.py, which supports additional types of tests, and is more maintainable and extensible than otu_category_significance.py.

core_diversity_analysis.py has a new parameter, --recover_from_failure, that allows the user to re-run the script on an existing output directory and will only re-run analyses that haven’t already been run. This supports rapid recovery from failed runs, and additionally allows the user to add categories to a previous run, which is very common and previously required a full re-run.

We’ve added new script, estimate_observation_richness.py, which implements some of the interpolation and extrapolation richness estimators in Colwell et al. (2012). Important: This script should be considered beta software; it is currently an experimental feature in QIIME.

Finally, we’ve updated our documentation on contributing to QIIME. If you’re interested in helping us make QIIME better there are lots of ways that you can get involved. See our new Contributing to QIIME document.

And this is all just scratching the surface. There are many more features in QIIME 1.8.0 – for more details you should review the full ChangeLog.


Greg (on behalf of the expanding list of QIIME developers)

PyNAST 1.2.1 release is live, and future PyNAST announcements will be posted to the QIIME Blog

16 11 2013

Hi all,

I’m happy to announce the PyNAST 1.2.1 release, available for download here. This is primarily a bug fix release, allowing PyNAST to make smarter decisions about where to store temporary files (QIIME issues #999 and #1114). This had been an issue for some PyNAST users working in cluster environments. If this hasn’t been an issue for you, it is not important to upgrade from PyNAST 1.2.0.

Also, all future news and announcements related to PyNAST will be posted here on the QIIME blog. Merging these blogs will help us reduce our administrative burden. The PyNAST blog has been very low traffic (four posts in nearly four years) so we don’t expect this to increase the number of posts here noticeably.

Thanks to the QIIME users how helped us track down this PyNAST issue!


Bug in compare_categories.py, supervised_learning.py, and detrend.py

30 10 2013
QIIME users,
A bug was recently discovered in QIIME 1.7.0 (and previous QIIME versions) that may affect some of QIIME’s scripts that wrap R functionality. These scripts include compare_categories.py, supervised_learning.py, and detrend.py.
The bug involves a discrepancy between the way that QIIME parses metadata mapping files. QIIME has two different parsers written in the Python and R programming languages. The Python parser strips leading/trailing whitespace from mapping file data fields, while the R parser does not. This can lead to the exclusion of samples, and/or incorrect grouping of samples based on a mapping file category, when the R parser is used. This bug may affect the results of compare_categories.py (only adonis, db-RDA, Moran’s I, MRPP, and PERMDISP; the other methods are fine), supervised_learning.py, and detrend.py. Mapping files that successfully passed check_id_map.py’s validation tests may still be affected by this issue.
IMPORTANT: If your mapping file has leading/trailing whitespace (with or without double quotes) in any of the mapping file fields and you used any of the aforementioned scripts, *your results may be incorrect*. It is very easy to accidentally add leading/trailing whitespace to your mapping file, especially if you are editing one by hand in a spreadsheet program such as Excel, so this bug may affect you even if you’re sure that your mapping file fields don’t have this issue.
Unfortunately, we cannot predict how your results may have changed due to this bug (e.g., we cannot predict whether statistical tests will be overly conservative, or how far off a test statistic might be from the correct value) since samples with leading/trailing whitespace will be dropped from analyses, and incorrect groupings of samples based on a categorical variable may be created based on whitespace. For example, a Treatment category may have some sample groups labeled as Control and Fast, but if a couple of the samples were labeled ‘ Fast   ‘ (without quotes), these samples would be artificially grouped together, instead of being grouped with the rest of the Fast samples. Thus, due to the random nature of sample exclusion and artificial grouping, we cannot predict how your results might be affected.
This parsing bug, as well as check_id_map.py’s validation tests and documentation, have been fixed in the latest development version of QIIME 1.7.0-dev (fixed on October 3, 2013, commit 5669b5891c26c9631c465243c046cdc33d0f8ba7). In order to avoid serious issues like this in the future, we are working on migrating the R parsing code into its own CRAN package (with extensive unit tests), as well as formally defining a metadata mapping file format so that other tools can consistently implement and validate this format.
If you are using a release version (or an older development version) of QIIME, we have put together a workaround that you can use until the next QIIME release. There is a script called strip_mapping_file_fields.py (hosted as a Gist on GitHub) that takes a mapping file as input, strips any double quotes (“), then strips any leading/trailing whitespace, and writes the resulting data to a new mapping file. This script requires that QIIME 1.7.0 is installed. If you are a MacQIIME user, you will need to first activate your MacQIIME environment by running the ‘macqiime’ command.
We highly recommend that you first use check_id_map.py to validate your mapping file and *ensure that there are no errors or warnings*. Next, use this script to “cleanse” your mapping file before using any of the aforementioned scripts (compare_categories.py, supervised_learning.py, and detrend.py).
Here’s how you can download and use the script. Assuming your mapping file is named map.txt, run:
check_id_map.py -m map.txt -o check_id_map_output
# Fix any issues with your mapping file until there are no errors or warnings.
git clone https://gist.github.com/7009545.git strip_mapping_file_fields
python strip_mapping_file_fields/strip_mapping_file_fields.py -m map.txt -o map_fixed.txt
# Continue with your analyses, using map_fixed.txt.
We apologize for any inconvenience this may cause you, and we will continue striving to prevent bugs of this nature as QIIME development progresses. As always, please get in touch with us on the QIIME forum if you have any issues or questions.