Browser Note: Chrome, Firefox, and Safari all seem to work. We are not supporting MS Internet Explorer for this unless you are a coder and want to debug the problem and suggest the fix. [Download Mozilla or Firefox]

Must have Cookies turned on. Must have Javascript turned on.

tri·age (trE-'äzh)
Etymology: French, sorting, sifting, from trier to sort, from Old French
Definition: the assigning of priority order to projects on the basis of where funds and resources can be best used or are most needed.

This document is long but contains valuable information. If you want to triage reports, you'll need to refer to this document after the first read.

Welcome to the Triage Guidelines page for Experience Report reviewing. Triage is designed to help manage the large number of reports that are submitted to Erowid. With this system, we hope to improve the rate at which publishable reports get posted. Historically, a small number of Reviewers have gone through reports, either by viewing them in chronological order, or searching on Substance, Title, or Author name. They then go through a 5-20 minute process for each report, categorizing and editing the ones that are suitable for publishing, and deleting the ones that aren't.

Triage involves picking a report, reading it, giving it a rating, flagging it for Admin Attention where applicable, and noting anything special to consider about the report where applicable. As a Triager, you provide a valuable service by helping to separate the wheat from the chaff. Reviewers can use your ratings and comments to decide more quickly which reports to make live.

One of the Big Ideas behind triaging is to have at least 2 triagers read each report before it gets reviewed. So for reports you are triaging now, a second person may have already rated it, or may rate it after you. That's normal. Having multiple ratings helps the reviewers get a sense of what reports are most likely trash or treasure.

We ask that people commit to spending at least 4 hours in the first couple of weeks when starting as a Triager because this makes it easier to learn the system and get feedback from the reviewers.


Please read Experience Report Reviewing: The Good, the Bad, and the Ugly, and a brief overview of the experience report reviewing process. It can be useful to read or re-read these after you've triaged 20 reports or so.

Please also read:

The Value of Experience, by Earth and Fire Erowid, Extracts #10

Communicating Experience, by Jon Hanna, Extracts #10

Triagers are expected to be very familiar with the content of the Erowid Experience Vaults. You will be much better at rating reports for a given substance if you have read or skimmed many reports available for that substance. Published reports can be viewed in order of Great (A) to barely Acceptable (D). To browse this way:

1. Go to the Experience Report Vaults front page.
2. pick your target substance. for example, Mushrooms
3. Click on the Show All button
4. Scroll to the bottom of the page, where you can navigate among all the pages of reports
5. Click on the very last page (in this case, page 20) -- that's the lowest end of the reports, reports that are barely Acceptable
6. The top end is the very first page, where reports have stars. Yellow stars have the highest rating, A.
7. Note that within a grouping, reports are not rated further. For instance, the first report to show a yellow star is not considered "the best report on Erowid", and the final one in that list is not "the least good gold-star report", if that makes sense...

Neat Thing to Know: Reports that are published on any given day, for example, Jan 20, 2016, are also listed from highest rating group to lowest rating group. So, if you have typically checked the New Experience Reports on the What's New page, now you know.

Some links on this page require a password: ignore that until after you've read the whole document and have decided whether or not you have the time to commit to triaging. We ask that people commit to spending about 4 hours per week in the first two weeks when starting as a Triager because this makes it easier to learn the system and get feedback from the reviewers.

Once you're done with this page: If it sounds up your alley, please contact volunteers@erowid.org and let Spoon know how many hrs/wk you think you would be able to put in [estimate conservatively], and what login name you'd like to use [10-letter limit -- see Active Review Team chart at Erowid.org/experiences/ for examples]. We'll contact you with instructions about how to log in and change your password.

---

After You Have a Login

Triagers-in-training have access to a 40-report training set of pre-rated reports.

1. To begin, read and rate all of the reports in the campaign called Training Set. Please include a short comment in the Comments field to justify each rating.
2. Then look at My Triages-->My History (linked near the top of the page of the admin index) and compare your ratings to those of the benchmark score (usually by earth, fire and spoon).
3. Once you are done with the Training Set and have read over the rating comments, please ping spoon in IRC or email volunteers at erowid.org and let us know. A manager (right now it's spoon) can give you feedback on how your ratings compare to the canon rating scale.

Triagers who have completed the Training Set may be approved to continue triaging submitted reports from sets called Campaigns. We add your email address to the triager email list; this list receives a couple of messages a month announcing new campaigns or other administrative trivia.

1. Triage reports from either of or both the campaigns, but please stick to campaigns. These are subsets of reports that match keywords for the title, substance, or report body fields. Usually they are substance-specific groupings, for example, dissociatives reports, or, 2C- reports, etc.
2. Once you've rated 25 reports from the campaigns, please ping spoon in IRC or email volunteers at erowid.org again and let us know.
3. As with the training set, you can look at My Triages-->My History (linked near the top of the page of the admin index). Once a report you've triaged has been triaged or published by an admin, their ratings and comments will show up here, and you can compare your ratings to theirs.

It is strongly suggested that you get on secure IRC, which is an efficient way to get feedback and ask questions. So if it's at all possible, please get set up with secure IRC, there are people available to help you if you run into difficulties. It's also possible to get feedback via e-mail if it's difficult to be in IRC at the same time as a manager. Next are details of what do with each report.


This is it! You're ready to read, rate, and note anything unusual.

As you begin this process, keep in mind we are interested in keeping lots of reports that may not be very exciting to read, but accurately illustrate the typical range of experiences. As long as the report is in decent English (reviewers can edit, so don't necessarily trash a report just for formatting reasons...), doesn't stray too far off the point, isn't falsified, describes the substance (dose, method of administration, effects) in a reasonable way, etc., then it's an acceptable report of the person's experience and should be kept, even if it will be rated very low. Don't trash a report just because "the same thing is said better elsewhere."

The primary complaint we receive about the Experience Vaults is that we publish too many poorly written or dull reports. Although quality of writing is one factor used in grading reports, we feel strongly that reports written by less educated or less skilled writers should not be excluded: accepting only erudite, articulate reports would introduce its own bias. This goes hand-in-hand with a decision not to completely correct and standardize grammar and spelling in reports. Writing skill and style are important elements that help convey a sense of the author. The style, tone, and word choices made by an author can help peer reviewers assess the validity of what the author describes.

We understand that some visitors would prefer not to read poorly written reports, and over time, we hope to improve filtering and sorting interfaces so that individuals can avoid reading reports that fall below their specified quality thresholds.


The majority of reports are about experiences with psychoactive plants and chemicals. A small number of reports are about non-drug experiences with meditation, yoga, or other methods. A very small number of reports describe incidents of drug testing or police encounters. These non-drug experiences are still valid reports and should be rated as such.


The main way to think about rating reports is that we grade from A to F with extra resolution around the C (both a C plus and a C minus are available). There is also an A plus, but for the most part this should not be used except in extraordinary cases. Also note that there is a special rating called "Not an Experience" that we use when someone submits something that is not within the boundaries of the normal experience report review system. Warning: The percentages mentioned below reflect the whole pool of reports submitted to the site. The Triager Training Set of reports has a slightly different distribution in order to illustrate all the different ratings. You don't need to try to fit the percentages mentioned below when you are rating the training set.

• Acceptable (C, C-, C+) : Reports that adequately communicate the subjective or technical specifics of an experience. C- reports either are not so well written or do not describe things in interesting ways, C+ reports are well written and have adequate data, but maybe they're not quite long enough or descriptive enough to be rated "Good". Think of Acceptable as the middle 50% of reports.


• Unacceptable (F) : Reports that will never be seen by the public -- and are effectively deleted -- are rated Unacceptable. They are extremely poorly written (often practically unintelligible, even with a fix of capitalization or the addition of some paragraph breaks), do not communicate anything about subjective state and exceptionally unclear about dose or time, and/or spend nearly all of the report describing unrelated details about the people, video game/movie/black light poster they are looking at, or what they read online about chakras. Around 40% of reports submitted have historically been judged Unacceptable. See: Examples of Unacceptable reports


• Good (B) : Reports that read well and include useful and/or interesting information about the substance(s) and dose they took, or communicate something well about their experience. Think of Good as the top 9% of reports. For reference, these kinds of reports have single blue stars by them when they are made live. There are probably fewer of these reports than you think.


• Great (A) : Reports that read exceptionally well and include useful, interesting and detailed information about the substance(s) and dose, and clearly communicate a state of mind. Think of Great as the top 1% of reports. For reference, these kinds of reports have double blue stars by them when they are made live. Note, Great or Good reports may include evocative descriptions of an experience that capture a feeling-tone, epiphany or "moment" that seems typical of the target substance [the way Aldous Huxley essays capture an LSD or mescaline experience]. Great reports are really really really rare. Really.


• Marginal (D) : Reports that have one or two redeeming points that should be saved for the sake of the data, but are generally poor quality. The live reports are set up so that marginal reports show up at the bottom of lists. If in doubt between giving a report a D or F rating, it's probably a D or an F+.


• Cellar (F with Data) : This is a special rating indicating that the report was not worth publishing for some reason (it was way too short, too poorly worded, too something), but did contain a unique or particularly interesting datapoint (like it's a really high-dose 2C-B report, or it's a rare report of IV Salvinorin). Not a lot of reports fall into this category. Note: Cellar reports can be included in an Advanced Search at Erowid.org/experiences/exp_search.cgi if the box "Include Cellar (reports rated too low for normal display)" is checked. They just don't appear in the normal stream of reports.


• Not Report (G): This rating is for things which are submitted that are not Experience Reports at all. It's reserved for abuses of the system, such as a news article, obvious spam, instructions on how to do something with no mention at all of use, questions, etc. Examples: reports that consist of "I submitted a report yesterday, aren't you going to publish it?", "I didn't know where else to ask this. Where can I buy meth?", www.insertobnoxiousurlhere.com, "How to Make Hash Brownies" with zero mention of ingestion, etc. Note: You may be tempted to rate G for an extremely poorly written or short experience report; these are still experience reports, not to be rated G. Or you may be tempted to rate G for a report that is not about a specific experience but more a summary or retrospective about use -- this is a Summary/Retrospective experience report and should be rated like any other report. This goes for Preparation/Recipe reports as well; as long as there is one mention of effects, it's still a report.

Flagging a report for Admin Attention is to say that there is something in the report that needs to be checked by one of the Experience Report admins before the report is published or deleted, or you think it needs special consideration. If you check this box, please type a brief note about why in the Triage Notes field. Does the report strike you as possible fabricated? Is there an unusual mention of a recent (current year) major injury, hospitalization or death that you think could be verifiable? If you're wondering if something could require admin attention, just ask.


The "Duplicate of" entry field should be used to enter the ID of another report that appears to be identical to this one. Rate the report as you would normally, but mark as duplicate using this field. Currently, this enters specially formatted text into your triage notes. Please don't edit the formatting of that or it will cause it not to work properly.

First, the Triage Notes field is used to note any problems in a report which caused it to be flagged for Admin Attention. Secondly, if there are particularly interesting things about the report and you wanted to explain why you rated it the way you did, you can use this field. New triagers are asked to please write something, even a couple words, about why they picked a particular rating. It helps us know how to give you feedback.


Rating reports is based on specific criteria, but there are a lot of fuzzy boundaries, and idiosyncratic or exceptional reports to deal with. Here are some of criteria Reviewers have noted using:

• Spelling & Grammar Errors: We mostly do not correct spelling and grammar on reports that are submitted. Occasionally we'll fix typos at the Review stage, or convert all-lower-case or all-upper-case to normal sentence case, but writing style and proficiency can help convey 'data' about the author. The main consideration for Rating based on spelling and grammar is how easy the report is to read for a native English speaker at a high school reading level. ALL CAPS, terrible punctuation, lack of capitalization, using lots of hAxx0r and very bad spelling can make it difficult to understand text communications and make them less valuable for the long term archive. Judicious use of colloquial terms and neologisms can add to the style of a report. In the end, the issue is how readable the text is.


• Telling Others What To Do: Also referred to as "you language" or "didactic" in reviewer/triager comments. The Experience Vaults are for first-hand descriptions of experiences and are not intended for people to write articles or tell other people what to do. Reports which are heavily populated with "you should", "I recommend", "always", or other sweeping declarative statements about other people's experiences (e.g. "First of all I would like to let people know that your body does NOT crave it after using it") are graded downwards and head very quickly toward Unacceptable. Please note in the Comments when a report is heavy with this sort of voice. (Very well written reports of this type should be marked for Admin Attention with a note in Triage Notes saying why, so they can be considered for inclusion elsewhere on the site. An example would be a detail-rich extraction method.)


• Death, Injury or Major Hospitalization: (Very Rare) Please flag any serious death or injury reports for Admin Attention and say why in the Triage Notes. Even Bad reports can sometimes be important to follow up on.


• Specific Products or Vendors: To avoid the Experience Vaults becoming a de facto advertising platform for vendors and commercial products, we remove specific references to product names, websites (including Erowid.org) and vendors in nearly all cases.


• Fiction & Believability: We get a small portion of reports which are falsified or fictionalized. Fictionalized reports should be marked Unacceptable. If you think that other Triagers or reviewers wouldn't catch this, note why you think it's fiction in the Triage Notes field.


• Cellar: A special rating for reports which contain some useful data but are either poorly written, too brief, seemingly unreliable, or of questionable authenticity. The Cellar is where we can store the information off for future research where it will not be displayed publicly without a special search.
  
  For Cellar-placed items, write a brief note or comment in the Notes section to explain why you want to cellarize it. Either describe why you didn't delete it (what the redeeming quality is) or why you don't want to put it in the normal stream.
  
  Examples might be like:
  "mentioned hospitalization, but had no contact info"
  "interesting mention of breathing problems, otherwise worthless"
  "rare combination, but not credible"
  
  Things that should go in the Cellar are reports that contain potentially valuable information about combinations, health problems, idiosyncratic reactions, extreme doses, or other data but do not come from people we trust, are deemed not completely believable for some reason, are extremely poorly written, or have some other reason why they should not be put in the general lists.

Rating reports as "Unacceptable" is one of the harder tasks. It can be hard to decide this for a report that someone has clearly put some energy into, but for any of a variety of reasons just doesn't make it.

Common "Unacceptable" Reports:

• Reports that contain very little beyond a mention of the drug they took and then a description of what they did. "We drove around in bud's car, then we went to the mall, then we walked to the quickie-mart and everything was really bright and we laughed a lot. Jim laughed juice out his nose and it looked all flourescent." etc. There are a lot of these and they suck. They are "Unacceptable".
  
• Reports that are extremely redundant and offer no real interest or color to the world. There are a lot of bordering-on-unacceptable or unacceptably written reports about ecstasy, LSD, nitrous, and DXM. If you are in doubt about the relative quality, open a separate window and skim a few of the bottom-end reports for that substance and see where it fits. If it's below the bottom of what's there, rate it "Unacceptable".
  
  

Q: Don't you have enough reports yet?

Some visitors have asked "Don't you have enough?" But in our view, the answer is clearly "No". Even with a large collection of reports, there are many substances, sub-sets of users, or types of use that are inadequately represented.

Of 25,000 published reports, only 112 describe absinthe use, less than 200 describe ayahuasca experiences, and only around 350 involve nitrous oxide. For truly obscure psychoactives, the numbers are tiny: only thirty-three 1P-LSD reports, three reports of Xenon use, and two reports about phendimetrazine. Although we have over 1,800 reports involving psilocybin-containing mushrooms, there is very little depth in certain areas: only 30 involve family themes and only five include the combination of fluoxetine (Prozac) and mushrooms. If the goal is collecting enough reports for each substance to allow for the meaningful study of sub-categories such as gender, dose, setting, and drug combinations, even 2,000 per substance begins to seem like a small number.

Q: Are we making a dent in the pile of reports yet?

A: Yes! In early 2016, the rate of triaging was outpacing the rate of incoming reports 2 to 1. As of February 2016, 69% of unpublished reports have been fully triaged (read by 2 or more people). For context, ten years ago it was 47%. We are gaining ground as a greater number of consistent triagers join in the effort. Eventually we hope to catch up and get close to triaging reports within a few weeks of submission. For specific classes of substances, for example research chemicals and ayahuasca, the number of unread reports is closer to zero. (Less-reported substances get some priority for campaigns.)

Note: we exclude DXM reports from default searches because there are so many of them, and they tend to be of poor quality. They were driving reviewers nuts.

Q: I tend to have a lot more experience/knowledge with some substances, would it be OK to start digging into 'reports to be triaged' for substances that I would be better prepared to score fairly and accurately? Or is it best to stick to the campaigns, even if someone may not have much knowledge about that substance?

A: A couple of things...

1. It may be easier to triage reports for substances one is familiar with, but there are a lot of substances that none of the triagers are familiar with...and they still need to be triaged.
2. It's useful to get triagers' perspectives on reports of substances they may not have experience with, because it helps test the usefulness of the report as a description of an experience.
3. Sticking to campaigns increases the chance a report will get triaged by two people within a short period, which is relevant because reviewers tend to look at "fully triaged" reports when considering what to prioritize for publishing.

It doesn't have to be all campaigns only all the time...but it really really helps us to move through the campaigns rather than having reports triaged scatter-shot. If you have a particular interest in a specific class of reports, consider suggesting it as a future campaign topic.

The point of triaging is to get reports read by two people, which gets them in a queue for a reviewer to pick from. They'll get in that queue more quickly if there's a concentration of effort otherwise, a report might get triaged by one person one year, and another person another year it also helps bring attention to keywords that might be good to triage more quickly like 'methoxetamine'... or 'hospital'.

Q: On some reports I've been on the fence, not sure whether a report is a 'C' or a 'C-' lets say. If I give it a 'C' and I see the other triager(s) gave it a 'C-', should I change it or is that difference in opinions a good thing, as long as its within +/- one rating. I find that the C and C- and C- and D get interchanged a lot. Sometimes I feel like I should have stuck with the other grade.

A: Changing a rating on a report you've already triaged isn't necessary. Your understanding of the rating system is evolving and fluctuating, especially in the training stage. You are encouraged to note your hesitation, you can always say a report is "C/C- edge" or something, and briefly mention why you picked the rating you did and not the other rating.

For older untriaged reports, there's a slightly greater chance that both the bottom-of-the-barrel and the cream-of-the-crop have been taken out of the queue already. The better reports are often spotted by their title or substance, and many have long been reviewed. And the deletes are sometimes easy to spot, too. So you might have a lot of C's! Your mileage may vary.

Over time, you might want to focus on a particular interest among reports:

• "Submitted Reports" on the index page shows all reports that have been submitted but not deleted or reviewed yet. You can search or sort these reports by Title, Author and Substance.
  
• "Sub by Good Auths" shows reports that match author names of live reports that have been rated as "Good". Obviously this isn't fool proof in the case of commonly used nicks (e.g. "Josh"), but it helps to locate good reports.
  
• "Sub by Death/Hospital" shows reports that have these words in the Title. This also catches titles like "out of body hallucination of death", but it's another way to help find reports with potentially important health information.

Personalized campaigns can be arranged once a triager has read over 300 or so reports. Please inquire for details. Getting feedback on how your ratings compare to standard ratings:

My Triage & Review History allows you to see how your ratings compare to live reports once reports you've rated have been reviewed. Note especially these links:

Reports I have Triaged

• that have been Reviewed
  Reports I triaged Acceptable or better
  
• That were Reviewed Approved
  
• That were Reviewed Trash
  Reports I triaged as Unacceptable
  
• That were Reviewed Approved
  
• That were Reviewed Trash
  

Stats:
The stats on the front page of the Experience Vaults Triage index only update every few hours, so if you do a couple reports, the numbers aren't going to change for a while. Also, even though your campaign total might be down to zero, that campaign may still be open if reports remain that are single-triaged and in need of a second reader *other* than you.

Please feel free to ask questions in IRC (email volunteers@erowid.org if you need help getting set up).

Enjoy!