18.6 Exercises

  1. Cognitive neuroscientists investigate which areas of the brain are active during particular mental tasks. In many situations, researchers observe that a certain region of the brain is active and infer that a particular cognitive function is therefore being carried out; [372] cautioned that such inferences are not necessarily firm and need to be made with Bayes’ rule in mind. The same paper reports the following frequency table of previous studies that involved any language-related task (specifically phonological and semantic processing) and whether or not a particular region of interest (ROI) in the brain was activated (see table below). Suppose that a new study is conducted and finds that the ROI is activated (\(A\)). If the prior probability that the task involves language processing is \(P(L)=0.5\), what is the posterior probability, \(P(L\mid A)\), given that the ROI is activated?
Language \((L)\) Other \((\overline{L})\)
Activated \((A)\) 166 199
Not Activated \((\overline{A})\) 703 2154
  1. Suppose that, in 1975, 52% of UK voters supported the Labour Party and 48% the Conservative Party. Suppose further that 55% of Labour voters wanted the UK to remain part of the EEC and 85% of Conservative voters were also in favour. What is the probability that a person voting “Yes” (in favour of remaining in the EEC) in the 1975 referendum is a Labour voter? [373]

  2. Given the following statistics, what is the probability that a woman over 50 years of age has breast cancer if she receives a positive mammogram result? Bayes’ Theorem Problems, Definition and Examples

    • 1% of women over 50 have breast cancer;

    • 90% of women over 50 who have breast cancer test positive on mammograms.

    • 8% of women over 50 will obtain a false positive result on a breast cancer test.

  3. What would it take for you to update …

    • your belief in the existence/non-existence of a deity?

    • your belief in the shape of the Earth?

    • your political affiliation?

    • your allegiance to a sport team? (Go Sens!)

    • your belief in the effectiveness of homeopathic remedies?

    • your belief in the effectiveness of Bayesian analysis?

  4. Suppose that a test for a particular disease has a very high success rate. When a patient has the disease, the test accurately reports a ‘positive’ with probability 0.99; when they do not, the test accurately reports a ‘negative’ with probability 0.95. Assume further that only 0.1% of the population has the disease. What is the probability that a patient who tests positive does not in fact have the disease? Is this problem any different from problem 3?

  5. A road safety analyst has access to a dataset of fatal vehicle collisions (such as Canada’s National Collision Database) on roads in a specific region. The dataset is built using police reports, and it contains relevant collision information such as: the severity of the collision, the age of the drivers, the number of passengers in each vehicle, the date and time of the collision, weather and road conditions, blood alcohol content (BAC), etc. Let us further assume that the analyst has access to aggregated weather data and R.I.D.E. (sobriety checkpoint) reports for that region.

    Some information may be missing from the police reports at a given moment (perhaps the coroner has not yet had the chance to determine the BAC level, or some of the data may have been mistakenly erased and/or corrupted).

    For some collisions, we may need to answer either or both of the following questions: did alcohol play a role in the collision? did “bad” weather play a role in the collision? As usual, let \(I\) denote all relevant information relating to the situation, such as the snowy months of the year, the incidence of impaired driving in that region, etc.

    The analyst will consider 3 propositions:

    • \(A\): a fatal collision has occurred

    • \(B\): the weather and road conditions were bad

    • \(C\): the BAC level of one of the drivers involved in a collision was above 0.08% per volume

    The analysts may have an interest in \(P(B\mid A;I)\), \(P(C\mid A;I)\), \(P(B,C\mid A;I)\), \(P(B,-C\mid A;I)\), or \(P(-B,C\mid A;I)\).

    Derive an expression to compute the probability that “bad” weather and road conditions were present at the time of the collision.

  6. A Mild Winter scenario (we use the set-up of question 6): during a mild winter, “bad” weather affected regional road conditions 5% of the time. The analyst knows from other sources that the probabilities of fatal collisions given “bad” and “good” weather conditions in the region over the winter are 0.01% and 0.002%, respectively. If a fatal collision occurred on a regional road that winter, what is the probability that the weather conditions were “bad” on that road at that time? Is the result surprising?

  7. Not Quite as Mild a Winter scenario (we use the set-up of questions 6 and 7): assume that the winter was not quite as mild (perhaps “bad” weather affected regional road conditions 10% of the time, say). If a fatal collision occurred on a regional road that winter, what is the probability that the weather conditions were “bad” on that road at that time? How much of a jump are you expecting compared to question 7?

  8. Use the set-up of questions 6-8. Just how rough of a winter would be necessary before we conclude that a given fatal collision was more likely to have occurred in “bad” weather?

  9. Use the set-up of questions 6-9. In what follows, we assume that the analyst does not have access to other sources from which to derive the individual probabilities of fatal collisions given “bad” and “good” weather conditions in the region. Instead, the analyst has access to data that suggests that the probability of a fatal collision in “bad” weather is \(k\) times as high as the probability of a fatal collision in “good” weather. Let the probability of “bad” weather be \(w\in (0,1)\). Derive an expression for the probability that the weather conditions were “bad” on that road at that time, given that a fatal collision occurred, in terms of \(k\) and \(w\).

  10. Really Rough Winter scenario (we use the set-up of questions 6-10): during a really rough winter, “bad” weather affected road conditions with probability \(w=0.2\). Determine the probabilities that there were “bad” weather conditions given a fatal collision under 4 different values: \(k=0.1,1,10,100\). Which of these scenarios is most likely?

  11. Use the set-up of questions 6-11. In the next scenario, we assume that the traffic flow changes depending on the weather; while some individuals need to be on the roads no matter the conditions, others might tend to avoid the roads when the conditions are “bad”. Make whatever assumptions are necessary and analyze the situation as you have done in the previous questions.

  12. Use the set-up of questions 6-12. Repeat the process for the other conditional probabilities of interest.

  13. A lifetime’s supply of poutine is placed randomly behind one of three identical doors. The other two doors lead to empty rooms. You are asked to pick a door. One of the doors you have not selected is opened, revealing an empty room. You are given the option of changing your pick. What is your optimal strategy?

    1. Determine the ideal strategy using a simulation.

    2. Analyze a similar situation (for 100 doors instead of 3) using Bayes’ Theorem.

    3. Analyze the situation using Bayes’ Theorem.

  14. How many heads in a row would you need to observe before you would start doubting whether a coin is fair or not?

  15. Estimate the parameters \((\mu_i,\sigma_i)\) for \(i=1,\ldots, 12\) for the salary problem.

  16. Play with the parameters and implement new scenarios for the Money (Dollar Bill Y’All) example.

  17. Play with the BernBeta() function. Do you spot anything surprising?

  18. Suppose you have in your possession a coin that you know was minted by the federal government and for which you have no reason to suspect tampering of any kind. Your prior belief about fairness of the coin is thus strong. You flip the coin 10 times and record 9 H(eads). What is your predicted probability of obtaining 1H on the 11th flip? Explain your answer carefully; justify your choice of prior. How would your answer change (if at all) if you use a frequentist viewpoint?

  19. A mysterious stranger hands you a different coin, this one made of some strange-to-the-touch material, on which the words “Global Tricksters Association” You flip the coin 10 times and once again record 9H. What is your predicted probability of obtaining 1H on the 11th flip? Explain your answer carefully; justify your choice of prior. Hint: what would be a reasonable prior for this scenario?

  20. A group of adults are doing a simple learning experiment: when they see the two words “radio” and “ocean” appear simultaneously on a computer screen, they are asked to press the F key on the keyboard; whenever the words “radio” and “mountain” appear on the screen, they are asked to press the J key. After several practice repetitions, two new tasks are introduced: in the first, the word “radio” appears by itself and the participants are asked to provide the best response (F or J) based on what they learned before; in the second, the words “ocean” and “mountain” appear simultaneously and the participants are once again asked to provide the best response. This is repeated with 50 people. The data shows that, for the first test, 40 participants answered with F and 10 with J; while for the second test, 15 responded with F and 35 with J. Are people biased toward F or toward J for either of the two tests? To answer this question, assume a uniform prior, and use a 95% HDI to decide which biases can be declared to be credible.

  21. Suppose that the marketing group of a company are testing a new web page, with the hope of increasing the conversion rate (proportion of visitors who sign up or take some other action). The data is collected in the file ab_data.csv, which lists user visits with whether they were sent to the new page or the old page, and whether there was a conversion.

  1. Explore and visualize the dataset.

  2. We conduct Bayesian A/B testing, by defining and updating independent priors on the old and new conversion rates, to arrive at respective posterior distributions for the old page and the new page. Try a prior of Beta(alpha=2, beta=20) for the old rate, which represents what has been observed in the past. Start with a subset of 100 data points and perform inference. Find the posterior probability that the new page has a higher conversion rate. Hint: use random samples from the independent posteriors to estimate the probability. Update the posteriors with another 100 data points. At what data size do the priors become irrelevant?

  1. Sometimes we don’t just want to estimate a dependent variable, we want a probability distribution for it. For instance, if one’s life expectancy is 80 years, we might want to know whether it’s a 50/50 spli between 0 years and 160 years, or some other distribution.

    1. Load the mimic3d.csv dataset which lists the length of stay in a hospital (LOSdays) along with a number of other variables. Explore and visualize this dataset.

    2. Construct a dataset patients.csv containing information about 10 or so (or more) “patients”, for all but the LOSdays variable (you may use friends and family members, classmates, etc. as a basis for your observations).

    3. Predict the length of the hospital stay for the patients in the dataset by conducting a Bayesian linear regression analysis. What’s the probability of staying longer than 2 days and therefore definitely missing work? Use normal priors for simplicity.

References

[372]
R. A. Poldrac, “Can cognitive processes be inferred from neuroimaging data?” Trends Cogn.Sci., 2006.
[373]
D. Hitchcock, Introduction to Bayesian Data Analysis (course notes). Department of Statistics, University of South Carolina, 2014.