Transcriptions
Note: this content has been automatically generated.
00:00:01
okay well it's great fun to be here uh
00:00:04
i shows you know what topic that relates a bit
00:00:09
more to my night job which is the other
00:00:12
of the a. g. c. and i have a feeling that i was initially invited because of my day job and
00:00:19
quite a few years and look good in carbohydrate metabolism but uh as you'll see i've
00:00:25
wanted develop another thing because in order for us to understand the consequences of
00:00:30
but very lookers and carbohydrates and sugars we need to understand have well we know
00:00:37
so disclosures i've consulted in one time or another for just about everyone
00:00:44
a pointer company is bunch of governments a whole bunch of academics aside is
00:00:49
sometimes they paid me and so i'm completely conflict that uh you
00:00:54
can uh you you can you know fine you know uh
00:00:58
diametrically opposed companies that and i think these things sort of cancel each other out of ah
00:01:05
and this event given all of that uh an inhibition by the
00:01:09
way the travel expenses an honorary input the stock right
00:01:13
now i'm not gonna mention any commercial products because academic products can more
00:01:19
then make my case and i say that following what
00:01:23
is a completely informal and and systematic review
00:01:27
of an egg data that i've derived from more than fifteen thousand manuscripts that of course my desk
00:01:34
as the other ray j. c. n. over the last ten years and uh i would say you you know uh
00:01:40
the a. j. c. and uh does not get some of the studies that go off to the big weeklies
00:01:47
and a variety of public health you know type manuscripts but i would say that
00:01:51
we did a pretty good selection of things after that because of our
00:01:56
global position and clinical depression so i think the sample
00:02:00
even though it's an act than mine and data
00:02:04
uh i think it represents what might be considered a very good inflow of
00:02:09
a question related studies that you know that appear in the literature
00:02:15
so my thesis is that if you wanna have confidence in your beliefs about
00:02:21
the impact of liberating carbohydrates on human health you have to have confidence in the measurements
00:02:27
and evidence of a causal pathway now yesterday if you look at the
00:02:31
bottom of the slide you know a bit our son mentioned three
00:02:35
essential components and i think my slide was prepared before you
00:02:39
mention those but i added this to the bottom because
00:02:43
the conference in measurements are david's data and methods can reliability and the evidence
00:02:49
of the causal pathway is the logic that can extant conclusions and
00:02:54
i think all of us have to be aware and keep ourselves always aware of richard fine men's principle
00:03:00
at the first principles that you must not fool yourself and you are the
00:03:04
easiest person to fool and what i mean by that is that uh
00:03:10
most everyone that i know attack academics that i know that publish spend a lot of time
00:03:17
on evidence that doesn't agree with what they think
00:03:22
and they spend very little time on the nature of the evidence that does agree with what they think
00:03:27
because it agrees with what they think so we have to be very careful about fooling ourselves
00:03:34
now the grass tyson said or you know a decade or so ago
00:03:39
uh no science to choose maturity without precision data sciences about measurement
00:03:46
preferably with something that is not your own ah yes which are
00:03:50
inextricably code join with the baggage of your own brain
00:03:54
and i think that's an extraordinary statement and he's talking about
00:03:59
really biases that are introduced by preconceived ideas you know
00:04:04
various you know notions that you come up with your
00:04:07
unchecked imagination and the light now by that definition
00:04:12
nutrition human interaction is a very mature finance so whole decade
00:04:20
and a half ago i'm sitting having my sunday coffee
00:04:25
reading the new york times i just try and relax and enjoy myself and there's an article one airline food
00:04:31
no this is not going to do with nutrition these are airline
00:04:34
executives talking about having to serve food to people on airlines
00:04:39
and the southwest airlines executives uh that southwest airlines doesn't serve food but in surveys
00:04:46
is consistently rated uh serving good meals now people don't even know they ate
00:04:52
how can you expect them to tell you reliably what they eat
00:04:58
i mean you know seriously guys you know okay so when we
00:05:03
have data about dietary intake records that are matched with you
00:05:08
know energy determined by doubly label watered at the same time
00:05:12
and now there are hundreds of subjects i mean it's
00:05:16
probably close to a thousand that with this is done but certainly six or seven hundred energy intake
00:05:21
is routinely under reported by twenty percent and obese people under reported by more than twenty percent
00:05:28
we're not talking small numbers you're in the energy business this is you know this is like
00:05:32
but the malformed and energy twenty percent okay
00:05:37
and then individual nutrient takes are validated
00:05:41
with correlations in the point to the point five range now i want the word
00:05:45
validated removed from wall nutrition manuscripts and they can gonna make this rule
00:05:52
you know for the a. g. c. n. because what it means is newer methods are just as bad as the old ones
00:05:59
it doesn't mean they're right in any way shape or form okay
00:06:04
so if we look at the study that was published you know some years ago
00:06:07
from anonymous phase group and that the u. s. d. a. where they did
00:06:12
a twenty four hour recalls every ten days for people over the course of six months
00:06:19
and to estimate adequate into its you'll notice that it took many many
00:06:27
recalls you know to to estimate what accounted for eighty percent of the variance of
00:06:33
the average intake over that six month period okay now these were comparing
00:06:40
you know a dietary intake records with what the with the average i mean
00:06:46
they didn't have independent measurements of what the intake actually was right
00:06:51
so this could be much worse than it appears to be but the point here is that
00:06:56
a very large fractions that is are done with that anything like this
00:07:00
number of recalls or and or you know or record dietary records
00:07:06
recently published a paper in the ages yeah and actually two papers that were you know you're
00:07:11
really not twins and but maybe mounted to port uh in which my own workers
00:07:19
we're measuring controlled feeding studies this was that the productions enter your wanna land the and a
00:07:27
rose print the son of friday whether people so the user individuals who receive their diets
00:07:34
uh you know over a period of i think it was two weeks or something of that sort
00:07:41
and they were to buy all markers with the independent by markers were matched to
00:07:46
what their intake should be by the fact that they receive the diets
00:07:51
and if you'll notice for all of these you know there's none of them
00:07:56
really that account for more than half of the variations are these are controlled feeding studies so imagine what
00:08:04
is happening out in the real world and for some things that were important here we've talked about saturated fatty acids and
00:08:11
things like this the actual variation explain was you know it's very minor fraction of the total
00:08:21
now the other thing that we don't generally talk about in these kind of in directory records
00:08:27
that uses the changes in food composition the variation in fruit composition so here is
00:08:32
one study were very look at the variation and vitamin c.
00:08:37
uh overseas and so the the yeah the the summer is on the left side and uh
00:08:43
i think it's a summer and winter is on the right side or fall i'm sorry spring and fall
00:08:48
and what you know this is this this was paper was done by experts why
00:08:53
because they change the scale and the second thing to somebody who knew how to change the scale to
00:08:58
make this was not you know this was just not casual right they made it this way
00:09:05
yeah so look at the variation in those concentrations just look at that
00:09:10
the conference and intervals they're they're huge right and this is
00:09:15
not almost never accounted for in dietary intake studies
00:09:21
a similar kind of study was done uh about you know
00:09:25
decade or more ago now looking at individual components
00:09:29
of rustic uh so this is a good core group look assimilate in three different prospectus p. c.'s
00:09:36
and in this case what you see is there is a rough you know distribution of those response
00:09:43
and if you try to relate these responses to the
00:09:47
relative risk for cancer okay and you don't use
00:09:52
variation in the problem in the production chain of concentrations you come up with was a
00:09:58
pretty good dose response curve in the reduction in the relative risk of cancer
00:10:04
but if you also then it back in the variation introduced by the production chain just disappears entirely
00:10:12
right now i'm not saying that this is representative of all studies of this type or any but it's we don't see this
00:10:20
we hardly ever have anybody do with that i know of or at least most of
00:10:24
the papers we get and i think we're potentially you know fooling ourselves sort
00:10:33
some iterations the science of wood we can measure food intake is just bother anybody yeah i mean so i imagine
00:10:41
if you try to understand diabetes which can't accurately and
00:10:45
precisely measure blood sugar or renal disease with that
00:10:49
confidence in your area and create name measurements i mean we have a problem here and it doesn't you know
00:10:54
maybe it's only me you know i guess it doesn't seem to bother a lot of people but
00:10:58
i think this is a serious issue that we have to face and i don't think this field is gonna move
00:11:04
really forward into a much more mature science without someone facing
00:11:09
this uh and even in cases where we have
00:11:13
no one you know by market for frequently not even you just
00:11:19
so the first thing is you know we have a problem with my measurements at least i think we do
00:11:26
uh and the next thing is what is what about evidence so no one you know if
00:11:31
there's no one in this meeting is gonna stand up here and say i'm gonna
00:11:35
tell you something and i have no evidence for at all but you know i'm
00:11:38
gonna put it on okay so so that no one in any committee or
00:11:44
whatever will tell you that they're not making an evidence based decision so talking about
00:11:49
having sound science i love that one 'cause it shows up all the time
00:11:53
everybody things to doing sound science otherwise they don't get crowded right so so i don't think that's very helpful
00:11:59
so why is there so much you know fast now about evidence based depression
00:12:04
'cause everybody thinks they're giving you evidence it's in the definition of evidence
00:12:10
what am i calling it but instead how my waiting okay
00:12:14
no one act kind texting given the conversation just today i'd like to remind you that the dietary reference in
00:12:19
takes for non of the forty essential nutrients has been established by formal systematic review of the evidence
00:12:27
not okay there's been a little evidence light or something is vitamin d. in cancer and
00:12:34
some they're mean it was a little bit but okay the same is true
00:12:37
the dietary guidelines so the dietary guidelines evidence review was looking at what's happened
00:12:45
to the evidence in the last years since the less dietary guidelines
00:12:48
that's not how you doing evidence review you have to go back to the beginning and add the new stuff to all the other stuff
00:12:56
that's just not been that right and then we so every can choose who
00:13:00
was in charge of the dietary guidelines to or so guidelines ago
00:13:05
when we talk about evidence read you know review right so i i mention that i didn't think
00:13:10
it was this was a systematic review when he said well what's the systematic review white
00:13:15
okay i don't really know what that means the other part is that it's there's an inherent conflict
00:13:22
of interest here so the dietary guidelines can maybe you're supposed to give advice to the government
00:13:30
but they're using the government evidence library to establish what the advice should be
00:13:36
now if industry did that there would be an uproar right it would be an absolute uproar
00:13:42
i mean the so to me this there's an inherent conflict here and we need to find a way
00:13:47
around that for the future now ah there's there's a i get i think there's a relatively
00:13:56
prevailing opinion throughout the cushion community that somehow
00:14:00
nutrition evidence these different from other sciences
00:14:04
and i just don't believe that are all high this the principles of the scientific
00:14:10
method are the same whether you're doing physics or what you do we nutrition
00:14:16
you need an explicit question
00:14:20
we need to define an explicit outcome ahead of time you need to intervene in some way so
00:14:27
no causation without manipulation then you have to repeat it so you feel like you're
00:14:33
you you measures something real and if you're lucky and you repeat it enough times then you get predictability
00:14:41
and i don't i say nutrition is absolutely the same as every other science
00:14:45
in this respect what's different it's really hard to do the studies
00:14:51
the principles on any different but it's very hard to do long term diets that is
00:14:57
in control fashion and some of them may be unethical as with her yesterday
00:15:03
so this is the problem right and i also want to remind you that we're all
00:15:08
experts here and the bottom of the evidence based hierarchies expert opinion so that's me
00:15:13
and pretty much everybody in the audience you might disagree with what i just said right i
00:15:17
mean so we're all on the bottom of the evidence hierarchy what we need is
00:15:21
so movement up the evidence hierarchy and what we have is a lot of we
00:15:26
have a massive amount of data in the middle of the evidence or okay
00:15:32
so burner horizon burke who has who to find it but we
00:15:37
usually call the uncertainty principle set it in a different way
00:15:41
which i think applies to essentially every kind of scientific endeavour we're gonna be involved in
00:15:47
the persian or otherwise you said since the measuring devices been constructed by the observer
00:15:53
we have to remember the what we observe is not
00:15:55
nature itself but nature exposed or method of questioning
00:16:00
this is really a critical concept i mean it's an extraordinarily critical kind so
00:16:06
and you know what what what does it matter okay now i picked this example here
00:16:12
because it's a drug example that's that's safety and efficacy the jobs and so
00:16:17
we don't worry about complicated died studies a million people seventy five studies
00:16:24
and what what's right or if one looked at or or if one
00:16:30
looks at the observations bodies okay one sees a very significant hazard
00:16:38
of the axis and the hamster here
00:16:42
have a ratio is very similar to what you see in the question
00:16:45
thirty one point seven six you know that kind of range
00:16:49
and one can reduce that apparent hazard bit by a certain number of corrections right
00:16:58
but it's only when you deal with the r. c. to use the find out there's no
00:17:03
rest at all no increased risk or on your cities
00:17:07
this is been demonstrated in over and over and over again
00:17:13
and we have again numerous examples in an accretion feel so here we have
00:17:19
two different views of nature again by the method of questioning so we a beta
00:17:23
carotene intake and cardiovascular disease mortality if one looks at observation or cohorts
00:17:30
okay one sees a point estimate that's a very significant protective effect
00:17:38
but if one looks at the randomised control trials not only is there
00:17:41
no protective effect is the suggestion that there may actually be oh
00:17:50
and the if the field is full of the stuff so this is one
00:17:54
of my favourites 'cause i was on the phone look yes it uh
00:17:59
committee for reading probably guess it the grains and at the time there was
00:18:04
a prevailing opinion that well you we're going at it to grain
00:18:08
products because of neural tube defects but just think when we had for
00:18:11
what yes and kind of we put in b. twelve and relaxing
00:18:15
we're gonna lower foremost sistine levels so we're gonna get a totally additional value
00:18:21
added a fat right so that this is in the uh in german
00:18:26
okay if we reduced almost to screen level five to eleven percent
00:18:31
we can decrease my korean far trends and korea a coronary heart disease dats
00:18:38
in humans okay by you know eight to twelve percent
00:18:43
right now in addition to this massive reduction in cardiovascular
00:18:49
complications by doing this the calculation what how much money the country with saving it was immense
00:18:56
alright so mark twain as one of my fear many like they record your t. v. set supposing is good
00:19:05
finding out is better and following these theses i mean of the whole variety
00:19:13
of control trials with on the cardiovascular events in which we have
00:19:18
buffy bow group and then a group like we see you probably have
00:19:21
to beat will be relaxing some different permutations and by and large
00:19:27
nah nothing happened there are few exceptions to that
00:19:32
but by and large nothing happens or this you know money saving
00:19:36
and things just didn't come about no this is also true
00:19:39
provide in many environments say in cardiovascular disease i mean and for
00:19:44
the fact that there are now a several dozen hypotheses
00:19:51
that were derived from observations that either been tested by r. c. t.s in about
00:19:57
fifteen to twenty different publications and none of them have
00:20:02
actually supported the hypotheses generated by the observations studies
00:20:09
now can we get better i would just brought this up because it's another popular topic can
00:20:15
we get better if we have by workers because what we really wanna buy the channels
00:20:21
and that's a long term study so can we you know what can we do with bile market so
00:20:27
they would the mets has a very nice little discussion of that and this article
00:20:31
in the annals of internal medicine any points out that there's really only one
00:20:38
convincing kind of my own marker that you can use and that gives you the true estimate of the clinical outcome
00:20:45
and that's when it's right on the causal pathway any intervention
00:20:50
you know uh is also directed towards that pathway
00:20:55
uh i like a good idea you showed that yet another slide which i together
00:20:59
'cause it's too complicated to explain but he showed another series of permutations
00:21:05
of the bile markers that are not on the causal pathway okay and where the
00:21:10
by market may be affected by not the causal pathway that don't do this
00:21:15
so we have you know many we'll probably by marcus unless we
00:21:19
know they're on the causal pathway and we haven't points
00:21:22
that allow us to assess their validity and as they would
00:21:27
the mets without we have this whole list of drawbacks
00:21:32
that were approved with surrogate markers and which which all of them change the surrogate marker
00:21:40
but none of them had the clinical effect that they were supposed to have
00:21:45
okay things to increase you know lowering l. d. l. increasing h. p. l.
00:21:50
but pressure you know related roads et cetera so the bottom market business is
00:21:55
very iffy unless you have a lot of good information about it
00:22:06
as i mentioned we have two ways of looking at nature and we have very different is about you
00:22:13
know why differences of opinion about the value of some
00:22:16
of these approaches so these are three quotes
00:22:20
from very famous people whose names i have left off for that
00:22:26
protection of the innocent sort of thing and uh this
00:22:31
you know uh these are people you would all recognise if we if i put the names so in one case as with them
00:22:40
referee for the n. a. j. c. an article who said and it's return
00:22:44
to the water is coming to the authors the weather should be
00:22:48
i should note that in dietary epidemiology prospective cohort studies
00:22:52
are regarded virtually on par with randomised control trials
00:22:56
now as far as i know the only people believe that on a personal epidemiologists right
00:23:01
ah but that's what he said alright ah the other ones
00:23:06
uh you know well constructed prospect of cohort studies
00:23:10
those can be used when for cause out meeting with a high degree
00:23:13
of certainty when randomised trials or of hard endpoint some practical
00:23:19
and then another one there is really very little directory confounding going on here we have almost
00:23:25
everything in the model really not much left that could be plausible can founders now
00:23:32
uh is that true well it may have been crew in the twentieth century but if we look at
00:23:38
what big data have told us about associations of this is
00:23:43
a study that one one you know figure from uh
00:23:47
a provides raj but ellen john you need is that was in gem a couple of two thousand fourteen or so
00:23:54
in which they looked at the associations among variables and and haynes
00:24:00
and only look that associations that the strength of greater than zero point two as i recollect
00:24:07
and you know there's the inter related you know correlations among these variables
00:24:13
and in fact the most of them had associations on the
00:24:18
order of sixty other come variables in an ins
00:24:22
so if you look at beta carotene i think it was like sixty four associations with beta carotene
00:24:28
so the idea that you can you know come to conclusions about beta carotene
00:24:34
seems kind of fanciful to me you know given the level of these different kinds of associations that would set for years
00:24:41
these are lifestyle things okay i can buy that you know they can make one and
00:24:45
all that sort of stuff but we were using tools of the last century
00:24:51
to look at things that i think we need to move on and even more impressive this is from
00:24:57
we were able it's group that appeared in biotechnology the sheer okay the
00:25:03
the in addition you know this or i shoes the endings data
00:25:08
uh the the studies that were done by leeway code were and you know and and or some other
00:25:13
subjects that they study wanted to really over time and in addition to
00:25:20
the sort of physical measurements they did a variety of laboratory measurements and they did you
00:25:24
normally measurements and they measured you know a a whole bunch of series of you
00:25:29
know a blip is and things of this sort of this is a much more
00:25:31
comprehensive want to look at the the related associations here this is the cardiovascular
00:25:39
i think what they called it the little keys you know the the the neighbourhood okay
00:25:43
and what leroy quite did which is almost never done in the traditional
00:25:49
the correctional epidemiological studies the look at associations with the t. f.
00:25:54
zero point o. five after the correction for multiple comparisons
00:25:59
that's almost never done in the traditional reporting of nutritional epidemiologist
00:26:08
so you know i think i don't present all of studies but you know our seat
00:26:12
user repeatedly fail to support implications from observations that is in the christian anyway
00:26:18
and we need to remember first that in those observations that useful to the data or diets or
00:26:25
whatever the nutrients kinda flew it's they're responsible to defect those hypotheses
00:26:31
so was i policies that if they'll know there's lots of reasons why that should be so to do
00:26:37
subjects that are in randomised trials and populations that is that different groups of people right
00:26:44
uh for the r. c. t.s we could have one goes we could
00:26:47
miss the therapeutic window we may not give it long enough
00:26:51
well we maybe using the wrong surrogate endpoints or the one that you
00:26:56
know we generally don't wanna talk about is the nutrients not responsible
00:26:59
for the for the fact he could be something else in non between in the food we could be things to do with the
00:27:07
it individuals genome rabbit genome or not it no matter gene on yet another level of complexity
00:27:13
and they could also be due to just our inability to what town for
00:27:18
this huge explosives all the that i showed from so raj and
00:27:24
and uh the record so it just to be the unknown and
00:27:28
you know that done rooms fills unknown unknowns right we have
00:27:32
the roll out there we just haven't been able to deal with them because of the way we're approaching these data
00:27:37
and and if you tell us the some of my good friends they say but wait cigarette smoking as a
00:27:44
cause of cancer has never been tested with an r. c. t. v. you say well you're right
00:27:50
and the leo point out why yesterday and the reason is that the
00:27:57
relative risks of cancer was smoking on your fifty or sixty
00:28:02
so you didn't feel pretty sure that this is well above
00:28:06
the noise level of your method of finding out
00:28:10
whether someone smokes or not or how much they smoke most nutritional risks are down here below true
00:28:18
and frequently you know we see papers and i unfortunately i publish a bunch of them um it might
00:28:23
better than a bunch of them i don't wanna take single rather for that one point true
00:28:29
rest now i don't know whether those risk around so i don't know it's level at all
00:28:35
we just don't know or we don't talk about it very much
00:28:41
the other issue we have with observations thirty says something that i think
00:28:46
is really insidious like colour making a silk purse for myself here
00:28:52
so one of the conditions that's essential for mad analysis is that all the
00:28:57
primaries studies have to provide an unbiased estimate of the true with that
00:29:03
now this is generally matter if you have good or cities is the only way
00:29:07
to deal with in all the contenders and in that sort of stuff
00:29:13
but the condition is often not met with observation studies first due to a whole
00:29:18
bunch of well appreciated biases and if you haven't seen it there's a
00:29:22
article by neal barnard walker willard and being in the current jammer which they discuss a lot of the use
00:29:29
and uh up a bunch of in it you know other than the ones i
00:29:33
have here they this is a two page article which is worth reading
00:29:38
but what they don't discuss is a what i think is a seriously under appreciated problem
00:29:45
which is the fact that the initial studies themselves often don't
00:29:50
account for the force discovery rate because there are
00:29:54
have multiple observations some hundreds of observations that
00:29:57
are never corrected for the multiple observations
00:30:01
so you have an effect which is really false discovery and that leads into
00:30:08
that analyses when and you know inappropriately because it's not the initial data or by
00:30:16
now we have ways of trying to approach cause albion observation studies and i
00:30:20
think these are being more commonly applied and they're clearly very important so
00:30:25
i picked this not because is the world's greatest study but it's the simplest to look at you know so it's easy to show
00:30:31
and as many of you know there'd been a whole series of observation studies
00:30:37
which suggests that low vitamin d. levels are related to type two
00:30:41
diabetes and in fact it's a pretty impressive you know
00:30:45
rest here one point two relatively asked and very high degree of how you
00:30:51
know how strange and see if you know for false discovery later so
00:30:56
those data are you know with circulating and i recently there been a number of you know approaches using
00:31:03
i mean the only minimisation which allows which in which the
00:31:07
subjects are sorted on the basis of their genes
00:31:11
which presumably can be affected by the environment and in the nature of this
00:31:15
observation was that when we look at that the whole effect disappears
00:31:20
and this doesn't prove that vitamin d. is not somehow on the
00:31:24
causal pathway but it's very strong evidence against it in
00:31:28
and use kind of men billion minimisation studies have now cleared up at least in my mind a whole bunch of
00:31:36
a very weak hypotheses based on traditional observations studies
00:31:42
so when the and you know our final conclusion from a lot of the study should be maybe
00:31:48
right associations are uncovered and observations that is right policies
00:31:54
we may never have for the reasons we discussed earlier people talked about yesterday we
00:32:00
may never have the international studies that allows us to fully test these hypotheses
00:32:06
but the response to that deficit shouldn't be to promote the level
00:32:11
of evidence we do have because i love it which is
00:32:14
frequently done explicitly i'm very often implied
00:32:19
you know what we need to do is to demote the certainty
00:32:23
of our conclusions how strongly do we know something okay
00:32:32
now i i didn't try to get the version in my talk after yesterday's talk about shoulders
00:32:39
'cause i thought i need to throw in some things that was my original charge you know the person carbohydrates
00:32:44
so here's a can of coke and a tablespoon and a half of olive oil which has more calories
00:32:52
the olive oil is more calories right
00:32:55
now if you're doing dietary intake records which is easier to remember
00:33:01
that you have a can of coke yesterday i can't okay
00:33:05
huh what about the void when you're stir fry or in your pay yeah
00:33:13
i mean this this is a serious bias in intake records it
00:33:17
and i would argue i have a thesis that no one is tested for me yet that
00:33:22
because you can measure the cans of coke better then you can measure the hidden fat
00:33:27
the variation around your intake of solid is is now or
00:33:31
then fat and therefore when you try to look at
00:33:35
statistical significance it drops out a significant big not because it
00:33:39
is actually but because you measured more precisely right
00:33:46
the other thing that came up yesterday that surprised me is that uh no
00:33:51
one mentioned the largest most comprehensive evaluation
00:33:55
of dietary core white hydration how
00:33:59
this was the u. k. scientific advisory committee and nutrition report that came out a few years ago
00:34:05
and this report was nine years in the making
00:34:09
it contains the report itself which is like two hundred and seventy five
00:34:13
pages and all the supporting systematic reviews that went into that
00:34:18
which is which are nearly three thousand pages so this is all on the web you can download it but i warn
00:34:24
you if you go to download the supporting material prepared to go away for a while it's a big file
00:34:30
so no one's gonna do this again in not gonna do this at the
00:34:36
dietary guidelines to come nine years to do and publish the stuff right
00:34:41
and this is the biggest most comprehensive davis on you know so that
00:34:47
so the u. k. e. scientific advisory committee report on carbohydrates intel said
00:34:53
they could find no association with or effect on should resign
00:34:57
corner events just delegate biased all that blood pressure
00:35:01
totally l. or h. p. l. testing for eclipse rights fasting blood glucose and insulin
00:35:07
or the risk of type two diabetes those were associations that it's uh
00:35:16
okay they looked also have randomised control trials
00:35:21
and the data on randomised control trial so there was insufficient evidence
00:35:26
of sure this weekend beverage effect on body weight weight gain
00:35:31
ah energy into it i think i see me or in so many me an an insulin resistance and sensitivity
00:35:37
okay and i senior the aura of uh the parapet glucose tolerance
00:35:44
like oscillator things like like gasoline middlemen and vascular compliance and c. r. p.
00:35:53
all these things are widely believed to occur this is a very
00:35:56
comprehensive report that says the evidence for those things is insufficient
00:36:02
they did damage they the committee said the reason the fact that they
00:36:05
can find an from observation all day it on an association between
00:36:09
should this when beverage consumption and the higher incidence of type two diabetes but there was a problem they couldn't find
00:36:17
any evidence of total sugar intake and type two diabetes or any association between
00:36:23
shoes when beverage intake and body weight and body fat which is
00:36:27
account for ninety something percent of the verification in developing type two diabetes
00:36:33
and it will the arms of a well throughout this report
00:36:37
for every association they look at the the committee stays repeatedly they
00:36:41
can exclude confounding and they don't show cause and effect
00:36:47
now we have to this extensive report that i thought is rather remarkably in you know uh
00:36:53
on the side of the innocence of sugar okay what happened to the recommendations so
00:37:00
like the w. h. o. they recommended that sure was intake be about five percent
00:37:05
of energy now the w. h. o. report okay rip was based largely on
00:37:12
the dental carries issue marion mention that yesterday the orders of those studies
00:37:17
say it's very low quality evidence now the w. h. o. said
00:37:21
well we made a conditional recommendation and i say if you don't have any evan and you should make any recommendation at all
00:37:27
right i mean to suggest that you have some reason to think it's five percent
00:37:35
they have no evidence offer that and gordon guy and others you know ah put them to
00:37:40
tasks and now they're doing some of the grading of evidence and things of this sort
00:37:44
precisely because of the snow the u. k. scientific advisory committee
00:37:48
and aggression which did this nine your extensive data analysis
00:37:53
when it came to make the final recommendation okay they'll use
00:37:57
this energy balance target from a previous can maybe
00:38:02
so they went to a subject or opinion of a previous committee after all
00:38:08
this objective data analysis it makes no sense to me at all right
00:38:12
i i personally see that there was an agenda there okay they wanted five percent they down the reason to come to that
00:38:19
now the other problem that we have is with that we see all the time is
00:38:26
you know oh no more than ninety percent of studies reported and certainly and biomedical literature but in depression
00:38:34
our show positive finding now this is another use a big data years you
00:38:39
know two million but most central abstraction wait million men align abstracts
00:38:45
ninety percent of the papers report accosted a fact finding
00:38:52
and on the face of it this is impossible right we're just not that good
00:38:57
we can't possibly guess right ninety percent of the time we're lucky if it's fifty percent of the time okay
00:39:03
so you know as marion said yesterday that the industry studies okay we're both the sponsor
00:39:10
and what i see in the a. g. c. n. is the academic academics that is
00:39:15
i usually say verbal to the legions biases are the people who are reporting them
00:39:19
because they have from years build on these particularly weaknesses and no matter
00:39:24
what happens no one can say nothing happened in this paper
00:39:31
so we have this vast preponderance of positive studies and uh rubber tables
00:39:39
and said in his book a statistics as a as a
00:39:43
structured arguments something of the you know as a prince principal argument you know
00:39:46
over confidence aborts uncertainty well that's true of all the investigators and
00:39:51
simmons who said this another way when faced with ambiguous
00:39:55
analytical decisions researchers will tend to conclude with convincing
00:39:58
self justification uh you pulling themselves that the appropriate
00:40:02
decisions are those that result in statistical significance
00:40:05
because they can't write a paper that said we did all of this and we work hard and we spend a huge amount of money nothing
00:40:13
okay so uh this is i've summarised human one slide
00:40:19
what i can spend an hour on the various investigator degrees of freedom that would be
00:40:26
to the nature of our positive you know literature and from this if you
00:40:33
haven't seen that any of these these principles come from the broccoli in probability principle
00:40:38
by david and which is a wonderful thing to read if you haven't seen
00:40:47
the kind of literature we have the first thing you need to do is you need to invoke the wall of
00:40:51
inevitability that is something must happen if i do a
00:40:56
study or you do is study something must happen
00:41:00
this is why we have publication bias in the nature that ninety percent
00:41:05
positive studies white negative studies gonna drawer blah blah blah i
00:41:10
we have to make sure that you count on the wall of cruelly large
00:41:14
numbers if you measure and nothing's likely things will happen just by chance
00:41:22
if you ask more than sixty one question you haven't ninety five percent chance of showing
00:41:27
at least one positive that the p. less than point o. five level and many
00:41:33
you know nutritional observation visual studies have hundreds of observations in the same study
00:41:39
so one expects more than one thing to be single begin at the point of five love
00:41:45
you have to master the last selection which is that you should be our linear room
00:41:49
where land you draw the balls like around right so here's where we have
00:41:55
all the post hoc p. hacking which is as far as i can tell pervasive we have what is
00:42:04
i think equally pervasive when not notice by the community is the baiting and switching of
00:42:08
primary and secondary endpoints which is why you need to read the registration documents first
00:42:14
before you look at the paper and you will discover that
00:42:18
either the secondary endpoints are reported as the primary endpoint
00:42:22
or the primary endpoint you know in the secondary endpoints switched positions during the course of the
00:42:27
study now i don't know about you but i think someone was looking at the results
00:42:32
we figured out which one should be primary right we need to apply
00:42:36
the the laws of near enough which is things like implying causality
00:42:41
or taking numbers that are similar and even though they're not
00:42:46
statistically significant are similar enough that we're gonna say
00:42:50
they're equal word okay exaggerating effects size and on all the time especially with use of relative risk
00:42:57
and then multiple publications from the same study which has a variety
00:43:02
of things one of which is to completely obscure the
00:43:05
primary and secondary endpoints or the number of observations it's actually been done in the study so it's very hard to
00:43:11
you know compute any corrections and okay then we have
00:43:19
you know smaller facts with big stand uh i'm not gonna want then of phi
00:43:24
this except to show you this is a study that's used as a positive
00:43:30
frequently tattered as a positive study in the in in an intervention that effects by way now
00:43:39
you know ah you need your statistician here right i mean just look at those
00:43:44
numbers can you tell me which is the control in which is the
00:43:47
intervention group this is unwilling journal paper by the way is one point out
00:43:52
you know with this is we have a lot of spin and duration
00:43:57
okay what about industry sponsorship it's actually hard to get enough studies and this but there are
00:44:04
true that came out recently from lisa barrows group and one in which appeared in uh the uh
00:44:13
it was jammer drum at it like this
00:44:17
i somehow missed or maybe it's on the bottom so this is looking at group studies
00:44:23
that this side favour is you know let's variable conclusions were industry more fable conclusions
00:44:30
and in this study you can see that of this six
00:44:34
or so things here almost all of them across one
00:44:39
meaning that none of those study shows significant difference right the
00:44:44
point estimate of the uh of the total is also when
00:44:49
significant barely so with the p. l. point o. sex
00:44:53
but what i'm the point here is that we have plenty of evidence and studies
00:44:57
in which there is no difference between the industry sponsored study and the
00:45:01
nine industry sponsored studies for the results now they just put online
00:45:07
several days ago an additional study that deal with industries
00:45:11
body sponsor studies in obesity in public health depression
00:45:15
and i had a little trouble getting this and you know blah blah blah but like
00:45:20
at the conclusion so the uproar apparatuses that would be a statistically significant differences
00:45:26
in the research topics that were funded by the fruit industry in
00:45:29
studies because the the compare with other funders was not confer
00:45:34
okay so one can certainly find studies where there are some differences you know for example this
00:45:39
one here but on the whole it's really in that i don't believe it's you know
00:45:45
completely obvious that there is a big difference
00:45:49
so where do we go in the literature this you know i have to spend the this is my night job that's taken over my day job
00:45:56
uh we need to have all studies registered all studies every study all study
00:46:03
twenty percent of the studies in clinical trials dot gov more than forty thousand
00:46:07
or observation all but only twenty percent of observation studies are reversed
00:46:12
we need to report the primary endpoints together
00:46:16
and we need to make sure people explicitly identify secondary endpoints which is almost never done
00:46:23
we need to have an a priori analytical plan so that we can
00:46:26
deal with post type p. hawking p. hacking and related manipulations
00:46:32
and we need to put some perspective in reporting results are not only relative risk but absolute risks
00:46:38
number needed to treat number need it on a fax sizes and things of this sort
00:46:45
we also need to make sure that everyone uses mandatory reporting violence because that
00:46:51
those allow you to have in the manuscript essential elements
00:46:56
we need to have some way of dealing with alternative analyses no models that are
00:47:00
as complicated as the ones that we see in large studies have a single
00:47:05
uh you know uniquely identifiable solution there are multiple solutions that
00:47:09
fit the models with the same degree of statistical confidence
00:47:14
and then we need to have the original data available scrutiny now some of you know as i said or the out were
00:47:21
lecture you know some of you don't that there's an ad in the us congress switches are already passed the house
00:47:27
and is now in the senate it's called the kindest that that's an acronym for something i don't remember
00:47:32
it's an entirely politically motivated that it's against the e. p. a.
00:47:37
so the motivation for this that is anti u. p. a. no question right
00:47:43
what the act says is
00:47:46
if you are using studies that are gonna
00:47:50
right policy recommendations or things like that you know they need to be
00:47:54
published right and they need to be available for alternative assessment
00:48:00
no i think that should be the case for essentially everything
00:48:05
and then we need to have some penalties for non compliance with the journal editors think
00:48:09
that no clout you know i mean we don't have any way dealing with this
00:48:13
the n. h. frankly doesn't seem to care sort of somebody that we have to have some
00:48:18
way of having penalties so in the end you know don't believe everything you read
00:48:24
especially if it has a picture of a famous scientist a company you know
00:48:28
uh yeah things of that sort and then finally uh
00:48:34
a quote by jonathan swift sort of four hundred years ago or so is
00:48:38
that a man should never be ashamed to own that he's been wrong
00:48:42
which is saying in other words that use wiser today than it was yesterday and
00:48:46
i think we have to use that principle and evaluating the literature so thanks