Month: July 2022

Evidence-based medicine: What do we mean by ‘low quality’ evidence in Trans Healthcare?

/ growinguptransgender / 3 Comments

In 2021 the NHS (NICE) reviewed the evidence for the use of puberty blockers within trans adolescent healthcare. They concluded that the evidence was of “very low certainty”, a finding that was then repeated across national media, with the headline “Evidence for puberty blockers use very low, says NICE”. This unsurprisingly prompted increased demands, including from politicians, for withdrawal of trans adolescent healthcare.

Several people have written about the flaws in the NICE review, including this excellent article by AJ Eckert. I’ve examined parents of trans children’s concerns with the NICE approach to evidence in a recent peer reviewed article in which parents of trans children described puberty blocker Randomised Control Trials as “conversion therapy” or akin to “eugenics”.

A recent expert report from the US indirectly challenges the NICE approach, and merits further reading.

Background

In 2022 the Florida Agency for Health Care Administration published a biased and ideologically driven report claiming that medical care for gender dysphoria does not meet generally accepted medical standards. A group of experts linked to Yale University responded with an analysis and critique of the poor science underpinning Florida’s report. Their full report is here.

Evidence Quality

One section of the Expert report from Yale University focuses on evidence quality. Their analysis has relevance for conversations on trans adolescent healthcare happening in the UK (including the Cass review), so this blog will shine a spotlight onto this section of the report.

*Note: In the below quoted sections, BPW refers to sources cited in the original (and misleading) Florida report. The June 2 report refers to the original, and flawed Florida report.

The following italicised text is taken directly from the Yale report. Please take time to read this section through:

…the BPW analysis reaches the conclusion that there is little or no evidence for the benefits of medical care for gender dysphoria.

The BPW analysis is highly deceptive, because it dismisses nearly all existing studies of medical treatment for gender dysphoria as “low quality,” without explaining that this is a highly technical term and not a natural-language condemnation of the studies. By contrast, the GRADE system, which the authors purport to use, is quite clear about its quality rating systems and its limitations.

In general, only randomized controlled trials (RCTs) are coded as “high” quality evidence in the GRADE system. A randomized controlled trial is a study that divides patients randomly into a control group (no treatment) and a treatment group. In contrast, an observational study records information about patients in a real-world setting that is more reliably generalizable, e.g., a cohort of patients seen at a clinic. Under the GRADE guidelines, observational studies are coded as “low” in quality.

The key point is that “low quality” in this context is a technical term and not a condemnation of the evidence, because “low quality” studies regularly guide important aspects of clinical practice. Indeed, the GRADE system, which the BPW document claims to use, specifically notes that GRADE should not be used to dismiss observational studies or to give absolute priority to RCTs: Although higher quality evidence is more likely to be associated with strong recommendations than lower quality evidence, a particular level of quality does not imply a particular strength of recommendation. Sometimes, low or very low quality evidence can lead to a strong recommendation.

The methodology adopted by the BPW document will thus, predictably, conclude that any body of scientific literature that does not contain RCTs is “low” in quality. Had BPW begun, as they should have, with a literature review of the evidence on puberty blockers and hormones, they would have seen that the evidence consists primarily of observational studies (for the good reasons discussed below). Thus, the 30 pages that it takes the authors to lay out their methodology is misleading: a knowledgeable reader would know that if there are few or no RCTs in the literature, then the BPW technical conclusion is foregone and, as importantly, is not a sound guide for clinical recommendations.

Put in simpler terms, if we coded apples as “high quality fruit” and bananas as “low quality fruit,” then any fruit bowl that has only bananas would predictably be technically coded as “low quality.” But that technical conclusion conveys very little information without context. For example, if no apples exist, then bananas may be a nutritious choice.

The drafters of the GRADE system emphasize that technically “low quality” evidence can support a strong clinical treatment recommendation. For example, pediatricians now agree that children should not be given aspirin for fevers. This recommendation is based on observational studies that showed an association between aspirin treatment during viral illnesses and the development of Reyes syndrome (a rapid and progressive disease of neurological dysfunction that can be fatal). Based on those studies, it would be unethical to conduct an RCT giving some children aspirin, and so the strong, consensus treatment recommendation is based entirely on “low quality” studies.

The critical fact is that RCTs are not, and cannot be, the gold standard for medical research on gender dysphoria. In the context of treatments for gender dysphoria, randomized controlled trials would often be inappropriate for ethical reasons. Medical care has long been shown, by reliable scientific methods, to address gender dysphoria and improve mental health: as we have repeatedly noted, these treatments have been recommended by rigorous clinical practice guidelines issued by WPATH and the Endocrine Society and endorsed by every major medical organization. Given this medical consensus, which is based on solid scientific evidence, it would be unethical to conduct an RCT that involved denying standard medical care to a control group of individuals.

Similar ethical issues, along with practical barriers, leave many areas of consensus medicine supported by observational studies and not RCTs. Many surgical procedures, for example, are not supported by RCTs. Nor are standard protocols for lowering cholesterol using statins, one of the most widely-prescribed drugs in the United States.

It is thus simply a mistake – and a mischaracterization of medical research across fields of medicine – to conclude that the absence of RCTs means that there is “no evidence” for the efficacy of medical treatment for gender dysphoria. Medical research requires, instead, that researchers evaluate the design and conduct of specific observational studies and do so with an awareness of clinical context.

In sharp contrast to BPW, this is precisely what the authors of the Endocrine Society did in their 2017 clinical guidelines, which use the GRADE system but, in addition, carefully discuss the characteristics of the studies supporting each treatment guideline. The Endocrine Society discloses the GRADE rankings for each treatment recommendation in order to be transparent about the evidence base for each of its recommendations. Then, following National Academy of Medicine (formerly, Institute of Medicine) standards for clinical practice guidelines, they proceed to a qualitative review of the evidence, place the evidence in clinical context, and discuss openly the values at stake in making a clinical practice recommendation.

The June 2 Report repeatedly and erroneously dismisses solid studies as “low quality.” If Florida’s Medicaid program applied the June 2 Report’s approach to all medical procedures equally, it would have to deny coverage for widely-used medications like statins (cholesterol-lowering drugs taken by millions of older Americans) and common medical procedures like mammograms and routine surgeries.

In its opening words, the June 2 Report makes an error that is repeated throughout the document: “Studies presenting the benefits to mental health, including those claiming that the services prevent suicide, are either low or very low quality and rely on unreliable methods such as surveys and retrospective analyses, both of which are cross-sectional and highly biased.” As we document in Section II.B., above, it is an outright mistake to conclude that a study in the technical category of “low quality” is unreliable or poor evidence for clinical practice. Thus, it is frank error for the June 2 Report to dismiss well-done, scientifically important studies because they rank as “low quality” using specialized, technical terms.

Like the BPW document, the June 2 Report thus relies on a deceptive use of technical terminology that is at odds with the standards used in medical research. It simply is not – and cannot be – the case that all clinical recommendations must be based on RCTs. Many areas of medicine do not lend themselves to ethical and practical RCTs. It is unethical to conduct an RCT when randomizing a patient to a control group would cause harm by denying treatments of known efficacy. For example, it would be unethical to conduct an RCT on the treatment of juvenile diabetes by randomizing some participants to receive insulin and others to receive no treatment.

It is quite common for the medical community to adopt important, consensus clinical practices supported by observational studies alone. For example, observational studies, notably the famous Framingham Heart Study, provided the framework for clinical practice guidelines in prevention and treatment of cardiovascular disease. In 2013, the American College of Cardiology and the American Heart Association issued updated clinical practice guidelines on the treatment of cholesterol to reduce heart disease risk in adults (the “Cholesterol Guidelines”). These authoritative guidelines have been widely used in clinical practice but are based not only on RCTs but on a great deal of observational evidence, including studies technically ranked as “low quality.” Concretely, many of the original treatment recommendations regarding statins are based on observational studies, not RCTs. The authors of the Cholesterol Guidelines, very much like the Endocrine Society authors, are quite careful to grade their evidence. But they do not rest their treatment guidelines on a mechanical assessment of technical quality. Instead, they (like the Endocrine Society) carefully explain why particular bodies of evidence should be given weight in clinical decision-making.

The cholesterol example shows that the June 2 Report rests on a fundamental misunderstanding of medical research and clinical practice. If the Florida Medicaid program actually adopted the standard of evidence urged by the June 2 report, the program would not cover statins (drugs to lower cholesterol) for many patients, which are prescribed to 28% of adults over the age of 40 and are one of the most effective ways to prevent cardiovascular death. Other common practices that would have to be reconsidered under this logic include: post-menopausal hormone replacement therapy (which reduces lifetime risk of heart attacks and stroke) and mammography screening for breast cancer.

The same point is true of the technically “low quality” evidence base for many surgical procedures, including minimally invasive gall bladder surgery, which have long since had a foundational grounding in observational studies. We think it unlikely that Florida’s Medicaid program will begin to refuse to pay for statins, mammograms, and routine surgeries. If not, then the June 2 Report reflects an untenable and discriminatory double standard.

(I’ll emphasise in case of confusion – all the above text in italics is not my work, but a direct quote from the Yale report – the authors of which are listed here)

Relevance for the NHS

The text above prompts several important questions for the NHS:

  1. Does the NHS (NICE report and Cass review) reflect a similar “discriminatory double standard” in its characterisation of the evidence base as ‘low quality’? Does NICE use the same approach across other areas of healthcare (including e.g. abortion)?
  2. Should the NHS (NICE and the Cass review) be more careful when using technical terms like “low quality evidence” in an already polarised and politicised area of healthcare, knowing how such terms are reported in the media and knowing how such terms inflame and undermine support for healthcare?
  3. Why has the NHS (NICE and the Cass review) failed to grapple with the very well established reasons why “high quality evidence” like RCTs are infeasible and unethical? Given the known ethical and practical impossibility of RCTs, why on earth did the NICE review recommend collection of such “high quality evidence”?

I’m also left with some bigger questions for the UK:

4. The above intervention from an esteemed team of establishment professors across different disciplines is likely to hold some weight in challenging and holding to account poor healthcare sector practices. Here in the UK we do not have paediatricians, law professors, experts in effective approaches to experience-informed healthcare speaking up on the clear flaws in the NHS approach to trans adolescent healthcare. Why not?

5. Are there lessons we can learn from experts in other areas of healthcare who have challenged commitment a narrow definition of ‘evidence’ within healthcare policy?

(post-script: I’d normally include a link to the NICE report, but it appears to have disappeared from the NICE website – if someone has a functioning link please let me know so I can update it).

Ten Easy Tips for Trans Inclusive Education on Puberty and Bodies

/ growinguptransgender / Leave a comment

Schools have a duty of care to trans pupils, with a requirement to provide LGBTQI+ inclusive PSHE. However, many schools still struggle to provide trans inclusive primary and early secondary school education on puberty and bodies, with schools often drawing upon outdated and exclusionary curricula.

Research has shown that trans children can experience exclusionary curricula, especially on bodies and puberty, as upsetting, delegitimising and harmful. Schools providing trans exclusionary education are associated with high levels of minority stress for trans pupils, with minority stress leading to poor levels of mental health and increased chances of trans pupils dropping out from education.

The good news is that trans inclusive education should not be difficult, once educators become confident in a slightly different and more inclusive way of approaching the subject.

In this blog I will share 10 recommendations for trans-inclusive puberty education, with quotes from educational resources that are trans inclusive. Importantly, this blog and the content highlighted herein focuses on how to ensure mainstream education on puberty and bodies is trans inclusive (it is aimed at ensuring the education every child receives is trans-inclusive, rather than aiming to specifically cater to trans children’s unique needs). (I’d also be interested in any additional advice on ensuring trans inclusive materials like these are intersex inclusive as well as disability inclusive).

This blog will primarily focus on a new educational resource called You-ology. It is produced by the American Academy of Pediatrics (the world’s biggest organisation of paediatricians), and aims to provide puberty education tailored for EVERY body. It is available in e-book form for £8.

You-ology: A puberty guide for EVERY body

This book does a couple of important things well:

1. It acknowledges, and frames puberty according to, the important role played by hormones.

On hormones:

“During puberty there are hormones that tell your body to grow faster, hormones that tell your breasts or testes (also known as testicles or balls) to start growing, hormones that tell your hair to sprout in new places, and hormones that cause new smells to creep out of your armpits”.

Growing bigger:

“Growing is one of the first signs of puberty. A hormone called growth hormone increases a ton during puberty to make you grow fast. Your hands and feet start to grow! So when you find yourself outgrowing your trainers faster than normal, you can smile to yourself and know puberty is starting!”

Testosterone and oestrogen:

“There are hormones called oestrogen and testosterone that cause most of the changes. A tiny, pea-sized gland in the brain, called the pituitary gland, sends a chemical messenger (yep, another hormones) to the testes or ovaries to tell them to start making hormones. Testes make a lot of testosterone. Ovaries make a lot of oestrogen and a little testosterone. Throughout this book, we will tell you more about what each of these hormones does.”

2. It considers the changes that happen to a majority of bodies. It emphasises shared experiences in puberty, rather than suggesting humans have two completely distinct and separate puberties. Rather than presenting ‘girl puberty’ in one lesson and ‘boy puberty’ in another, it instead divides up pubertal changes into the different types of changes. It provides one chapter on hair changes, one chapter on body odour changes, one on emotions. The similarities are emphasised as well as the differences, emphasising within the section on hair that in puberty most bodies develop increased hair on the legs and under the arms, while some bodies, especially bodies with a lot of testosterone, also develop hair in other places. Below is a quote from the You-ology chapter on hair.

On hair:

“where you sprout hair depends on – you guessed it – hormones. EVERY body begins to make the hormone testosterone early in puberty. Testicles make a lot. Ovaries make a little. Even a little testosterone will cause EVERY body to grow darker, thicker hair on their legs, in their armpits, around their private parts; around their nipples or on their face. The amount of testosterone you have determines how much and where the hair shows up. If you have a lot of testosterone, you will grow more hair on your face. You might also (later in puberty) grow hair on your chest, abdomen (belly) and back”.

3. Where changes are significantly gendered, with most girls having a different experience to most boys, it divides the content according to the specific type of change, rather than dividing into girls vs boys. It provides one chapter on breasts and chests, one chapter on periods, one on testosterone driven changes. It manages to talk about these changes without excluding, shaming or delegitimising trans children.

On vaginas:

“If you have a vagina. So let’s get back to body parts and start with outside private parts that most girls, and some nonbinary and trans kids have. If you have these parts, you know some of the names for them. If you don’t have these parts, it’s a great time to learn more about them!

On penises:

“Let’s shift to talking about the genitals that most boys and some trans and non-binary kids have. If you were born with a penis, you’ve been looking at and touching it since you were a baby, right? And at some point (hopefully a long time ago!), you learned to hold your penis to aim it into the toilet when you pee. So if you’re like most kids with a penis, you have been pretty familiar with your genitals for a long time, and you know the names too, but let’s go through them just to be sure. If you don’t have a penis, it’s still important to learn about these parts to understand how EVERY body works!”

On uterus and ovaries:

“Most girls and some trans/non-binary kids have some pretty cool inside parts that work together. First, remember how we talked about the opening of the vagina? The vagina is actually a soft tunnel that starts at the vulva and goes inside the body to connect the inside parts to the outside world…”

On sperm & testes:

“Most boys and some trans/non-binary kids have bodies that can make sperm, and the penis is involved. Do you wonder how? We already mentioned that urine travels in the urethra as it passes through the penis and out of the body. But guess what also comes out through the uretha? Sperm. The cool thing is that urine and sperm comes from very different places inside the body. Urine is made by the kidneys and then sent into the bladder to be stored. Sperm …etc ”

On breasts:

“If you are like most girls, or some nonbinary or transgender kids, you have ovaries, which also help puberty start by making oestrogen. The very first thing oestrogen usually does is telling the breasts that it’s time to start growing”.

Where diagrams are used to show specific body parts, there is no reductive label provided of ‘boy’ or ‘girl’ bodies. This kind of title is not needed and alienates and invalidates trans pupils in front of their peers. Below is an example of a diagram of body part that does not need to have a ‘girl’ or ‘boy’ heading.

4. It recognises and is not afraid of acknowledging diversity, explicitly recognising the existence of trans people, intersex people, and the reality that not every person experiences puberty in the same way. Key to this inclusive approach is bringing in the word ‘Most’. When talking about gendered changes, this resource talks about, for example, periods being something that happens during puberty for ‘most girls, and some trans and non-binary kids’. Addition of the word ‘most’ is really important for trans, non-binary and intersex kids, making space for the reality that all bodies are different. Addition of the word ‘most’ is also important for cis girls who may not have periods for a variety of reasons.

5. I also like the way that this resource talks about puberty as inherently about moving from a child’s body towards having an adult’s body, in comparison to other puberty resources that describe puberty as building bodies ready for reproduction. De-centring fertility and reproduction during education on puberty is helpful, especially for individuals who are likely to have a non-traditional route to building (chosen) families, a category that is likely to include a significant proportion of LGBT youth.

6. Hormone driven changes. The resource is clear that the changes of puberty are driven by hormones. It finds a way to talk about hormones explicitly, talking about the changes that are caused by oestrogen, and the changes caused by testosterone. It talks about breast development being a pubertal changes that occurs in bodies with a large amount of oestrogen. It acknowledges that a majority of all bodies have some testosterone. It talks about the types of changes that are typical for bodies that have a large amount of testosterone.

7. Clear and upfront on body parts. Where body parts are integral to a particular pubertal change, this resource again is clear and upfront. When talking about periods, it talks about the changes that happen to people with a uterus, which includes most girls. Most girls, and anyone else with a uterus, when they have a large amount of oestrogen during puberty, are likely to start having periods.

8. It avoiding ruse of the reductive term ‘biological sex’. Within any trans inclusive puberty education it is important to avoid simplistic and reductive definitions of sex or ‘biological sex’. Where biological characteristics are relevant, it is important to refer to them in the plural, as ‘biological sex characteristics’. The Endocrine Society (global experts on hormones or endocrinology) advises against using the term biological sex noting that “the terms ‘biological sex’ and ‘biological male’ or ‘biological female’ are imprecise and should be avoided”. There are multiple sex characteristics (chromosomes, gonads, hormone levels etc), and humans do not fall into only two binary divisions in terms of sex characteristics. This recognition is critical to including trans and intersex people, as well as to including people who may not have a specific sex characteristic for a variety of reasons (eg cis women who have had a hysterectomy).

9. Likewise, it is important to avoid using gendered language like ‘male’ or ‘female’ to define body parts. Within this specific You-ology resource, there are a couple of references to ‘male bodies’. This type of reductive language goes against the ethos of the wider book and is rather frustrating. Let’s be clear here – using the word male to describe bodily features is just as exclusionary and harmful as talking about a ‘boy’s body’. There is no way to respectfully (or even accurately) describe a trans girl as having a ‘male body’. This type of language is delegitimising, inaccurate and disrespectful. If we care for the well-being of trans pupils, non-binary pupils or intersex pupils, we need to evolve away from inaccurate and binary language that excludes, delegitimises and stigmatises. (the image below comes from work by Sophie Labelle)

10. Being up front about the diversity amongst humans is easy to do, it simply requires educators who are themselves educated, and unafraid of talking about diversity. In the UK however, we have a generation of teachers who have themselves grown up under section 28, who have not have the opportunity to themselves receive diversity informed education. This is why inclusive educational resources like this one are so very important.

Further resources

I’ll include here a few other recommended resource, but please send me other ideas to add in.

What makes a Baby by Cory Silverberg and Fiona Smyth provides visually appealing information on babies, reproduction, fertility and families, suitable for any age. It is designed to be trans inclusive, and inclusive of all ways of building a family, including IVF, adoption, chosen families.

A guide for adults to help children engage with the book is also available here

A new resource (You know, sex) by the same team aims at a slightly older audience, covering bodies, puberty, and sex ed. with the same visually engaging and inclusive style.

The above resources are all paid-for US resources, not included in the curricula of major UK providers of PHSE education. The limited UK PHSE materials that I have seen have had woeful trans inclusivity, and would require adaptation by individual teachers to avoid harming trans pupils. This is obviously poor, and more is needed to pull up the standard of generic and off-the shelf PSHE materials, especially from PSHE specialist providers.

A downside of these resources shared above is that they do not include practical class materials or worksheets. (Mainstream UK puberty worksheets can often be trans-exclusionary and harmful, requiring individual adaptation by any teachers who care for their trans pupils). If any educators have produced free to access class worksheets or other materials that are trans inclusive, please email me (c.horton@gold.ac.uk) with links to any trans inclusive educational materials or worksheets.

Trans inclusive education is not difficult to do. It is time to make sure education is fit for every pupil.

Update: Someone has also recommended this book (that I’ve not yet read) The Every Body Book: The LGBTQ+ Inclusive Guide for Kids about Sex, Gender, Bodies, and Families by Rachel Simon and Noah Grigni