Archive for the 'JSM 2011' Category

Data science vs. statistics: has “statistics” become a dirty word?

(by John Johnson)

Revolution Analytics recently published the results of a poll indicating that JSM 2011 attendees consider themselves “data scientists.” Nancy Geller, President of the ASA, asks statisticians not to “Shun the ‘S’ word.” Yet a third take on the matter is the top tweet from JSM 2011 with Dave Blei’s quote “‘machine learning’ is how you say ‘statistics’ to a computer scientist.”

Comments about selection bias from Revolution’s poll aside (it was conducted as part of the free wifi connection in the expo), the shift from “statistics” to “analytics,” “machine learning,” “data science,” and other terms seems to reflect that calling oneself a “statistician” is just not cool or scares our colleagues. So I open the floor up to the question: has “statistics” become a dirty word?

Why Going to JSM?

(by Julien Cornebise)

For my final post about JSM, based on three year’s attendance in a row (DC, Vancouver, Miami), a recap for next year potential attendants: Why Going to JSM? When is it worth it, when is it not?

First, the obvious wrong reasons for going: such a massive monster, with its 15-20 minutes talks barely allowing for anything but an extended abstract, and with 50 sessions in parallel, you rarely go to JSM for its scientific presentations. JSM is not the place:

  • to learn on recent developments in your field: not enough precise content in 20 minutes.
  • to get to know better someone’s work: same problem.
  • to get advertisement and visibility for your work: same problem, plus, empty sessions do happen way too much — you can’t compete with a panel of world famous speakers, especially when all you offer is a skewer of 20 minutes talks.
  • to see a wide overview of your optic: conflicting sessions on a same topic make it a frustrating experience.

For all those, specific small conferences (such as MCMCSki in the MCMC field) are way better: more focused interaction, more time for work sessions, more time for exposure of ideas, for constructive feedback. So why the heck coming? What makes 5,000 people fly here and spend a whole week? Why am I so glad I attended?

Of course, JSM offers some important community events, most noticeably its awards sessions and lectures (COPPS, Neyman, Wolf, and Medallion Lectures, …) where great contributors to our fields are honored by all their peers. Even though we’re all in there for the science, I won’t hide that I, for one, appreciate such public displays of recognition: it is not because we are scientists that we should never tell those who completely wow us that, indeed, we do think they do amazingly and that we want to thank them for that! Still, this would not be a sufficient reason by itself to hold such a gigantic and costly meeting.

But JSM incredible strength is truly its social side:

  • Nowhere else can you meet all of your US-based colleagues face to face at the same time in the same place, exchanging scientific ideas or just spending some great time in an informal context, getting to know each other better in a relaxed setting.
  • Nowhere else can you see former and new people from all the institutions you’ve worked at, keeping up with what they’re up to, keeping them up with what you’re up to!
  • Never else can you go for dinner with people from all those, getting them to meet, meeting their new colleagues, learning about their recent interests, what’s hot in the field, who’s moving where, why this or that department suddenly busted, how this or that other one is about to double its size and go on a hiring spree, what interesting specialized workshop is in preparation, etc. JSM is the largest grapevine concentrated over three days.

JSM is like iterating the adjacency matrix of your graph by several steps: not only do you strengthen your links with colleagues/friends you already know and appreciate, but you also get to know those they know, and find great matches! With the obvious caveat: if you don’t know anyone, then it will be quite difficult to meet new people. I’d recommend going there with a few colleagues from your institution for the first time. The less easy profile: the isolated statistician from a foreign country; his geographical attaches (Alma mater, former employer) won’t even compensate for his lack of people to hang out with — with the noticeable exception of seizing the occasion to meet someone you’ve only interacted with remotely. The best profile: pretty much any other!

Of course, all of the above is by no mean as formal/opportunistic as it may sound. Most of this happens while going to the beach with friends (after sessions…), going to dinner, sampling terriblific junk food (Five Guys Burgers, 15th and Espanola… I will miss you), living crazy nights on Ocean Drive — note to funding agencies: this never happens, I am just pretending, we are an extremely serious bunch, all of us, no exception. Simple: most of this is essentially hanging out with friends. With the noticeable difference: those friends are also our colleagues, lots of colleagues are also our friends.

And that’s why, in spite of all its flaws, this massive meeting is so enjoyable: work and fun do mix, friends and colleagues do mix, and real long-term highlights come out of it. After all, we’re all in here for the different faces of a common passion! See you next year.

JSM treat for the road: Significance Magazine

(by Julien Cornebise)

That’s it. It’s over. Done. Gone. RIP JSM 2011. ’til next year. A great week!
Yesterday’s convention center was a mix between an airport and the ghost town of Saturday: a fraction of the people were still here, most of them carrying suitcases. There should not be any talks on the last day 😉 And, although there were not big 2 hours Lecture to attend, I still had a hard time choosing between

The 15-minutes shortness of the former’s talks put me off, and the curiosity about this magazine that Xian blogged about, the challenges to talk stats to non-statisticians, and my own will for a steroid-version of “Popular science” decided me into picking the latter.

Boy was I glad: after a short introduction outlining the aim of Significance and calling for contributors (think of it, for you or your PhD students, it looks like a great experience!), we were treated to three very enjoyable talks by authors of recent cover papers:

Howard Wainer on how missing data can lead to dire policies, and how just a few extra data will be of precious help to avoid dramatic mistakes, with striking illustrations in Education that are also available in his book. This was thought-provoking: in a first move, I might tend to integrate out the missing data using using EM algorithm or Data Augmentation, hence assuming that the missing data is distributed similarly to the non-missing. Wrong! Howard’s examples were some of those “ah-ah!” moments, where you just realize that the original strategy amounted to standing on your head. Three examples:

  • Allowing the students to pick a subset of possible questions in a test, so as to make it fairer. Wrong. A quick study on one class showed that it tends to worsen the inequality: weak students are impaired in their choice and pick the hardest questions, failing them. Consequence of assuming random missing data: augmenting the score gap with the better students who picked the easiest questions.
  • Eliminating tenure for teachers to save money. Wrong. Looking back to 1991’s suppression of tenure for super-intendants showed that the salaries increased massively. Most likely explanation: tenure is a job benefit that costs nothing to the employer; removing it requires to increase the salary to compensate. Consequence of assuming random missing data: augmenting the expenses.
  • Making SAT scores disclosure optional to enter college>. Wrong. Studying withheld SAT scores for the one college who has done so for 40 years shows that students choose rationally to disclose their score or not: very few “I did very well at SAT, but so what?”, many “I scored less than the average entry score, disclosing it won’t help my chances to enter”. Consequence of assuming random missing data: those students picked classes that they failed, as they lacked too many prerequisites. A thought here: it would also have been interesting to compare them not only with students who divulged their score as Howard did, but with other students with similar scores who went to other universities: did getting access to harder classes than they would have usually been allowed to helped them on the long term?

Andrew Solow on the Census of Marine Life (2000-2010): how many species, and is a species extinct? There were some striking statistical problems, again due to non-uniform missing data: it is missing because the species is harder to observe in our usual surroundings! So there is more to it than the abstract problem of estimating the number of classes in multinomial sampling, and of estimating the end-point of a distribution (a tricky problem in itself already).

Finally, most anchored in recent actuality, Ian MacDonald brilliant talk on the BP Discharge in the Gulf of Mexico (I learned it’s a more precise term than “Deepwater oil spill”: it’s not Deepwater in charge but BP, and it is not an overboard spill but a discharge from a reservoir).
This one was one for the records: a precise and scientific study of the estimates of the size of the discharge, based on the speaker’s experience with natural oil seeps occurring everyday in the Gulf. Beyond the beautiful/appalling before/after pictures, and the pleasant feeling of the modest scientist being (sadly) proved true vs the massive corporation, there was a fascinating scientific chase to the source of the discrepancies amongst the estimates. Ian brilliantly chased it down to the table linking thickness of the surface oil spread with its color (rainbow, metallic, light-brown, dark), which is multiplied by the surface to estimate the volume: while all of the scholar’s studies use one table, oil companies (BP, Exxon) use one provided by US Coast Guards with a 100-fold downward error for the thickest levels — precisely the ones needed when drama occurs!

The dramatic consequences of this error are well-know: we’re not talking indemnities, but dramatic error on the pressure escaping the well leading to failure of the blockage attempts — an error confirmed when the videos of the leak were finally released and particle-velocity expert scholars were able to confirm overnight that the flow was much more than officially stated.

Ian concluded not in an obvious “who’s to blame” that would have been too easy (and obvious…), but focused on the question: what will be the long-lasting impact? His study of the spatial distribution of the natural seeps, much different than that of the BP discharge, puts at rest the idea that the ecosystem is somehow immunized. We’re left with the challenge of designing a statistical test to that unwanted massive experiment. Ian calls for two concrete measure:

  • Identify and monitor key habitats and population to check ecosystem health.
  • Put the repayment of the ecosystem in the front of the line, using BP’s fine to that effect.

In conclusion, a much pleasant session, a treat for those of us who could stay this last day, and a much interesting magazine: I’ll definitely think of contributing!

Stay tuned for a final post later tonight, before I hand back the keys of the blog to its editor.

JSM impressions (day 4)

(by Christian Robert)

Another early day at JSM 2011, with a series of appointments at the Loews Hotel, whose only public outcome is that the vignettes on Bayesian statistics I called for in a previous post could end up being published in Statistical Science… I still managed to go back to the conference centre (almost) in time for Chris Holmes’ talk. Although I am sure Julien will be much more detailed about this Medallion Lecture talk, let me say that this was a very enjoyable and informative talk about the research Chris has brilliantly conducted so far! I like very much the emphasis on decision-theory, subjective Bayesianism, and hidden Markov models, while the application section was definitely impressive in the scope of the problems handled and the rich outcome of Chris’ statistical analyses, especially in connection with cancer issues…

In the afternoon I attended a Bayesian non-parametric session, before joining many others for the COPSS Awards session, where the awards were given to

seeing the same person Nilanjan Chatterjee being awarded two rewards twice for the first time.

Reflections on JSM – dusting the dusty corners

(by John Johnson)

 

The talks that everyone is talking about are of course very cool, and we can learn a lot from them. However, I came to this Joint Statistical Meetings in search of some of something a little different. I attended many fewer talks than I have in the past (where I would diligently attend something every session except maybe Thursday morning when I would check out and go. What I found were a lot of devils in the details.

On Saturday I attended a continuing education course on the analysis of register data. Register data is administrative data such as what a government would collect. For example, birth and death data are register data in the US and almost every other country with a functioning government. This data is a challenge to work with for the following reasons:
  • It is collected on the whole population, as a census, but is longitudinal in nature
  • It is very difficult to curate, and is collected and curated through administrative processes rather than sampling
  • It is difficult to quality control, and that control is best done through merging with other data
  • Its analysis value increases in merging with other data
  • The only source of error is transcription
While I don’t work with register data, I can appreciate the hardships that come from working with administrative data, or data that is collected as an artifact of a transaction. The challenges in merging come from the subtleties in defining the variables, and making sure that variable definitions are consistent across data. It got me to wondering whether many of the challenges and inefficiencies we have in working with this data comes from our sample-based approach to handling it.
Speaking of data, a late Sunday session on CDISC data standards was well received, and in fact we ran over by over half an hour with consent from the audience. This talk was sponsored by the statistical programming section, but there was something in there for statisticians as well especially regarding the planning of analysis of clinical trial data. Statisticians would do well to learn these standards to some degree, because they will become more of a centerpiece of statistical analysis of clinical trials.
More generally, I am curious how many statistics departments have a class on data cleaning and handling, and, if so, if it is required or a choice for a required track. I was almost completely unprepared for this aspect when I came into the industry, having only managed messy data a little bit during a consulting seminar. In planning data collection, it is important for the statistician to look ahead and thing about how the data will have to be organized for the desired method, and that requires some data handling experience.
On Monday I attended part of the session on reproducible research, and concluded that at least in the pharma/bio industry we have no clue what reproducible research is.  We have an excellent notion that research needs to be repeatable, and that documentation needs to accompany analysis to tell someone else how to interpret the findings. However, we don’t really integrate it as closely as is expected in a true reproducible research settings. Maybe CDISC data standards (as discussed above) will eliminate that need at least from the point of view of an FDA reviewer. However, it won’t within companies, or in studies that are not done with CDISC compliant data.
Monday night, I partied with the stat computing and graphics crowd, and had a mostly delightful time. Maybe they can run their raffle and business more efficiently next year. Hint hint.
On Tuesday I supported a colleague in a poster presentation describing challenges in a registry of chronic pain management, and gained a new appreciation for the poster format. Much of the discussion was thoughtful and insightful, and we were able to explain the challenges. It was at least validating that the attendees who stopped by agreed with our challenges and gave some suggestions along the lines we were thinking, and the depth of discussion was stimulating. Off the success of that, I made a point to stop by the posters and found some really good material. I would encourage more posters, and I found that most of the benefit I get from JSM is from small group discussions (and occasionally from the larger talks as well).
It was somewhere in here that Andrew forwarded me an email with a disturbing statistic about the number of investigators who cannot describe a clinical trial or the data, nor can the consulting statistician explain the trial. I think this is a topic we will return to in this blog, and I think I will submit this idea as a biopharm-sponsored invited session next year. I know that the consulting section has sponsored quality sessions on leadership in the past, and I saw a very good session on leadership at ENAR this year. I think it is time to bring it to a wider audience.
Tuesday night and Wednesday were mostly focused on catching up with old and new friends and going to posters. I’m fairly tired by Wednesday on the week of JSM, and even more so given that I got in on Friday this time, so I debated whether I would get anything out of sitting in talks. I found a couple of fascinating posters on using tumor burden to assess cancer drugs and whether safety monitoring of drug trials has an impact on Type II error rate (it does, and it’s nasty). On the basis of this, I hope to see more well-done posters submitted at next year’s meeting. I love the discussion they generate.
I ended up in a fascinating discussion about evidence needed for FDA drug approval, whether subjective Bayes has any role, and the myth and illusion of objectivity. Some of this discussion relates back to “the difference between statistically significant and not significant is not statistically significant,” but I think there are some deeper philosophical problems with the drug evidence evaluation that keep getting swept under the rug, such as the fact that we assume that drug efficacy and safety are static parameters that do not change over time. (There are obvious exceptions to this treatment, such as antibiotics.) This is a true can of worms, and I’ll let them crawl a bit. And yes, practical considerations come into play such as the fact that the choice of software is either do something that is hard to write and verify it is correct, or spend thousands of dollars on software.
Tomorrow is the last day of the conference, and I’ll try to catch a talk or two before I leave. I hope to see you next year, and before!

JSM Day 4, almost there

(by Julien Cornebise)

Intense pace and social life (on which I’ll dwell in tomorrow’s post) are starting to take their toll: morning sessions were far less attended this morning than on the two former days. The dance party yesterday probably did not help. Not claiming to be better than any other, I shamelessly skipped the early sessions, so as to be fully focused for Chris Holmes’ Medallion Lecture at 10:30 — preceded by a luxurious breakfast: M&M’s cookies from 7/11, yay! Long live the US and their food(ish).

Chris’s lecture was a model of clarity. With an unhurried pace leaving no room for boredom (which could have hit after this many sessions), he took the audience all the way from hidden Markov models (HMM) to specially designed Loss Functions to his applications in genome-based oncology. Ideas were flowing smoothly and naturally, striking a perfect balance between formalism and intuition.

He started by showing the kind of changepoint-detection/HMM problems he was facing, which actually stem from a hybrid supervised/unsupervised problem: you know the class number and characteristics, but have no training data. He then showed how off-the-shelf Viterbi Maximum A Posteriori estimate and Forward-Backward Marginal Maximum perform well but are not flexible enough — no possibility to tune the false-positive and false-negative discovery. He then explained what the ideal loss function on the whole path for his problem would be, just to shatter it to pieces by an algorithmic complexity analysis, which made his following solution pitch-perfect: use a k-th order Markovian loss function.

All of this was progressively illustrated on a running-example dataset with simple yet informative graphs — much better than any final recapitulating table would have done. He then spent the last third of his lecture on his real-world applications on Colon cancer genomics, going from the algorithmic considerations to the more classical statistic questions of distributions used, dealing with the mixture of population subtypes, and extended to Sequential (longitudinal) model, tracking the changes in a same patient over time and treatment.

On a side-note I especially appreciated his introductory memories of finding the theoretical chapters of Bernardo-Smith’s book mysterious and too abstract when he first came to Imperial — only to, years later, gain more and more appreciation for those very aspects of decision theory who lead him to attack those challenging problems in this winning angle. As more of a theoretician/algorithmician than an applied statistician, I am convinced that, once a practical problem is defined, taking a step back to a more abstract level can bring tremendous gains: first focus on the problem, then step back to an abstract, bigger picture, develop in this generic setting, and finally zoom back on the concrete problem — and then all his neighbors that can now be tackled with the same tools!
Really a lecture that I was glad to attend.

I then aimed at the afternoon session Beyond Pharmacokinetics: Recent Advances in Science and Methodology, a great follow-up to last summer’s SAMSI program on Pharmacokinetics/Pharmacodynamics, with a non-empty intersection of speakers! Although I missed the first talk and a half (didn’t see the time pass while in a discussion), I heard enough of the second talk to realize the amount of work that went into it: when the speaker thanks this and that person for “setting up the robotics that were used in the sequencing”, you know you just missed a good talk! The third and last talk was on familiar topics, I was struck by its links to the former talks of the week, especially Sylvia Richardson’s Medallion Lecture yesterday: just like her, Michele Guindani aims at clustering the profiles of the patients, but does so by using Non Parametric Bayes and Dirichlet Process. Of course, now that he mentions it, I recall that Sylvia also mentioned NPB in the part of her talk about mixtures, and I also remember Peter Mueller tutorial about using NPB to cluster the patients into subpopulations — but it only just now clicked all into place. That’s an advantage of having all those talks concentrated over 3 days: there’s not far from one to another!

A disheartening point in the Q&A session, though: while, during the whole congress, I have been impressed by the way usual controversies were smoothed out or, better, bridged, this carebear-spree ended when one of the attendants tried to pry his own method into the talk, with what seemed the biggest oratory forceps ever known to academics. The method seemed to me so prehistoric and limited in its applications that I am truly wondering which of him and I totally missed the point! Anyway, this is anecdotal in view of the great talks and debates I attended this week, and I still am impressed.

Tomorrow is the last day of this JSM 2011. Exhibit Hall is already disbanded, and it is unclear if the free wifi will still be there! Indeed one of the biggest anguishes I can live through, but I’ll nonetheless end up posting eventually, even if that means having to go steal wifi on the beach near the big hotels… Life is hard 😉

JSM Day 3 – Session on Afghanistan and Iraq

(by Julien Cornebise)

Determined to broaden my mind, I strayed from the familiar fields of mine to more of a mine field: yesterday’s session on The Human Cultural and Social Landscape of Afghanistan and Iraq, organized by the Section on Statistics in Defense and National Statistics. Defense and military are not very much my strong suit, so I was doubly curious. And I was really glad I went, on several levels, both scientific, cultural, political and human.

Starting with the human level: midway through the last talk, mathematically interested, I was suddenly hit by the following down-to-earth thought:

Every single one of those dots in the curve are actual dead human beings.

Between 1 and 10,000 dead people. From casualty to massacre. Left of the axis: industrial butchery. Right of the axis: artisanal hand-crafted death. While not a faint of heart, this sudden surge of empathy in what I was so far seeing as a purely intellectual challenge was mind-shaking. I was suddenly miles away from JSM, and from the usual numbers I’m manipulating. I was far from my theorems, my algorithms, and my comfy chairs. And although yesterday’s session on Ethics in Statistics was nowhere close to the topic, I felt strongly for the question of empathy in statistics, and how do you deal with such topics. Although years ago I really enjoyed Mark Yor‘s talk intervention years ago in a plenary lecture to high-schoolers that “there’s more to do with maths than finance!“, I am not sure that either him or I were thinking of such topics. Thought-provoking. As a side note, it gave me a better understanding of Kristian Lum‘s enthusiasm for her work on disappearances in Colombia at Benetech.

Getting back to more mainstream considerations. On the scientific and cultural level, the first talk by Y. H. Said was a sociological description of the social structure of the tribal-patrilineal Afghan culture, very different from our own — a great incentive to re-read the history of the last 10 years. It shed new light on the relationship between local military/tribal chiefs and foreign forces, and on why the Talibans refused to turn in Bin Laden, on ground of the “Nanatwey“, the code of honor dictating to never refusing asylum even if you do not like the person you’re hosting. It also explained how trying to break down the drug trade is actually trying to break down the “qawn“, a flexible and evolving social network with encompassing links between very different actors that, in western society, we wouldn’t think to associate. The kind of talk that would be a blessing to spread to a broader audience, e.g. via the New York Times or popular journals.

The second talk took a more quantitative approach, with object modelling and UML diagrams to simulate precisely such networks. I was a bit less impressed, maybe because I am not sure we can yet apply such hard models to populations –Hari Seldon might not yet be there. An interesting try, though!

The third and last talk of the session generated the most questions and my most interest. Tim Gulden presented an exploratory data analysis via power-laws on the number of deaths in “incidents” in Iraq. He does not claim to be a statistician, but an expert of the application field, and started by a comparison with Guatemalan conflict (which involved acts of war and acts of genocide, with two different statistical patterns) and with the war in Kenya: this was as clear as it was convincing. He then moved on to studying the same kind of data for the Iraq conflict,

  1. showing what where the discrepancy between the data and his model  for  specific years: a lack of high-valued points, i.e. incidents with very large amount of casulaties;
  2. what manipulation of the data lead to a better fit: scaling the ranks of the incidents up by 1%, i.e. saying that there should be additional 1% of incidents with highest death toll;
  3. and which reality could those manipulation correspond to: a suppression of the three top-level structures of command in Iraqi army, precisely what the US was focusing on in the first years of the conflict, hence leading to several small subgroups without the manpower for the largets offensives tpically leading to those 1% highest death tolls.

Of course, there is a question of “if you look for something, you will find something”: how much of those findings are a nice hypothesis that happen to be a posteriori backed up by data, and how much are genuine patterns discovered by analysis? How much is it a mathematical construct, how much is it a statistical analysis? I would love to know what Andrew thinks of it, especially given his Introductory Lecture. Again, Tim Gulden does not pretend by any means to be a statistician — and I am sure, given the passion that he transmitted, that he would very much welcome those willing to help! (his email is in his CV)

Tim also mentioned his data source, the IBC database, an extensive database of press articles on the conflict. He honestly highlighted what where his caveats in this regard, how he was absolutely unable to overturn or assess the possible sampling bias. At this point on, a very useful intervention from the attendance: a (former?) military who was in Iraq and is writing his own book on the topic said that, from his experience, this database is the most accurate you can find publicly available, as compared to the classified data to which he had access but was in no mean to publish.

This lead to my political teaching of the day: I candidly asked whether the Wikileaks Iraq Warlogs could be used for such an analysis. And although both agreed that, indeed, it would be possible, Tim underlined that he was funded by US Navy, and hence would be worried that they would not be happy for him to look at them — note that he precises, and I understand, that military data might not bring more precision than journalist’s data, at least on the body count, in terms of lack of coverage of the most dangerous regions.

Of course, nothing surprising in this, the problem is the same with many sources of fundings. However, this is an issue that I rarely run into personally, and it was interesting to see it pop up here, bringing this session to a well-round experience. Althgouh understandable and logical, I nevertheless twitch that a researcher so conscientious and passionate as Tim seems to be can fear for his funding to use data that have now been published in international newspapers! (Guardian, New York Times, Spiegel). Those are only my opinions, and I can imagine the alternate view that at least his funding allows him to do this research so far — but in my naive youth I am nevertheless inclined to ask for a full freedom for researchers: you are funded to do research on a topic, without being limited  in your inputs or your outputs. Again, this is my  very personal opinion, by no means ASA’s or anyone else’s here, not even Tim’s whom opinion on this I did not ask — I am very clear on this latter point. Just my own food for thought, and I would love to hear what ASA’s Commitee on Professional Ethics would have to say.

JSM Day 3

(by Julien Cornebise)

JSM is now rolling at full steam in its third day yesterday. I kicked off the day with a great panel with three massive speakers: Christian “Xian” Robert, Jim Berger, and Andrew Gelman, on Controversies in the Philosophy of Bayesian statistics. It turned out to be less about philosophy than about controversies (past and present), which in a sense suited better what I expecting from it!

Large attendance, as was to be expected. Appart from the fact that Xian is “getting old and hence less critical” (yeah, right… although he might indeed be getting old, as Susie Bayarri spotted gleefully that he had his Bayes formula wrong in his slides! Reinsuring to see that it can even happen to him 😉 ), several salient topics were covered, with deep and subtle discussions such that it is hard to do them full justice here.
Here is for an all-too-brief summary of what I most recall.

On the choice of the prior (first topic to pop up):

  1. Jim opened the dance with Objective Bayes vs Subjective Bayes, with the thought-catching “if everyone in the application agrees on a prior, it’s not subjective any more”.
  2. Andrew chipped in about how frequentist linear regression is not objective: your subjective prior knowledge is hidden in the structure of the subgroups you choose to look at. Besides, on the topic of sensitivity analysis, sure, but then do it on your likelihood too!
  3. To which Xian added that the sensitivity analysis can conveniently be turned into a hierarchical model: if you check the impact of your prior for such or such range of hyperparameters, then you ought as well put this information in an upper level in the hierarchy.

The conversation — to which the audience played a great part — then moved on to model selection and Jeffrey’s Scale of Evidence for Bayes Factor:

  1. how its arbitrary nature is a drag to precision: Jim argued that everyone knows what a 10 to 1 chance mean, and that we ought to teach to those who don’t; no need for such a truncative scale.
  2. the three also agreed that this scale, for convenient that it may seem, should actually really depend on the application field: a physicist will not consider the same level of evidence as a sociologist.

On the opposition with Frequentism:

  1. I was much impressed to see how Jim bridged the gap, clearly stating that the community made some mistakes 15 years ago, and that since then many new developments conciliated the two approaches, especially on multiple testing;
  2. All three converged on the fact that frequentist makes sense when you are expecting long-run behaviour to stay stable,
  3. and even though Xian argued that in some cases long-run makes no sense as the data is here, it won’t be growing to infinity,
  4. I, for myself, am glad to use some frequentist tools (e.g. EM to design proposal kernels) within Monte-Carlo algorithms, where precisely I can make my sample grow on an on! (nb: please sample responsibly)

Finally, on Non-Parametric Bayes (NPB):

  1. Jim stated that, as we’ve seen in Mike Jordan’s impressive talk the day before, those are brilliant for discovery and learning, flexible and powerful;
  2. however, he is not convinced of their use for statistical inference, as there is no theoretical long run insurance of consistency (no Von Mises theorem),
  3. and as it is hard to understand what the prior is doing: although NPB indeed does less assumptions, it is not better than more classical priors because of this.
  4. I would add that, from what I understood from Mike’s talk, the assumptions are still present, just not in the form of distributions, but in the hierarchy.

Overall an intense and enlightening discussion: I learned a lot, finding echos to my half baked questions, essentially getting out of it the equivalent of reading through hundreds of pages of the speaker’s books and blogs!

The next session I attended was Silvia Richardson’s Medallion lecture on Recent Developments in Bayesian Methods for Discovering Regression Structures: Applications in the Health Sciences. This was a very full 2 hours lecture on some of her most recent developments, especially how to combine:

  • dimension reduction by profile regression and clustering by means of finite mixture models — with an illuminating forumlation of appropriate post-processing analysis to get a good grasp of the large amount of output, with functionals invariant to relabelling; this latter step is often overlooked, but nevertheless mandatory to gain the full power of the flexibility of mixtures, as she brilliantly showed on appealing graphs with high investment/high reward.
  • variable selection going beyond plain 0-1 switches on the variables, moving from “0-1 useful covariate for such or such profile cluster” to “useful covariate with such probability for clustering” — a subtle difference that has a large impact on the quality of the inference.
  • Bayesian sparse regression, especially her free Evolutionary Monte-Carlo C++ software ESS++, optimized so as to reach circa one million evaluation of very intricate models in less than 10 hours — meaning you have a really significant sample in a day or two, a feat for such intricate applications.

My afternoon session was then such an experience that it deserves a post by itself.

Also, Dick De Veaux very kindly sent me the slides of CounterExample talk mentioned on day 1! Really worth a check — even though they’re only part of the story, missing the equally-counter-example way he actually gave his talk.

JSM impressions (day 3)

(by Christian Robert)

A new day at JSM 2011, admittedly not as tense as Monday, but still full. We started the day with the Controversies in the philosophy of Bayesian statistics with Jim Berger and Andrew Gelman, Rob Kass and Cosma Shalizi being unable to make it. From my point of view it was a fun session, even though I wish I had been more incisive! But I agreed with most of Jim said, so… It is too bad we could not cover his last point about the Bayesian procedures that were not Bayesianly justified (like posterior predictives) as I was quite interested in the potential discussion in this matter (incl. the position of the room on ABC!). Anyway, I am quite thankful to Andrew for setting up this session.

While there were many possible alternatives, I went to attend Sylvia Richardson’s Medallion Lecture. This made sense on many levels, the primary one being that Sylvia and I worked and are working on rather close topics, from mixtures of distributions, to variable selection, to ABC. So I was looking forward the global picture she would provide on those topics. I particularly enjoyed the way she linked mixtures with more general modelling structures, through extensions in the distribution of the latent variables. (This is also why I am attending Chris Holmes’ Memorial Lecture tomorrow, with the exciting title of Loss, Actions, Decisions: Bayesian Analysis in High-Throughput Genomics.)

In the afternoon, I only attended one talk by David Nott, Efficient MCMC Schemes for Computationally Expensive Posterior Distribution, which involved hybrid Monte Carlo on complex likelihoods. This was quite interesting, as hybrid Monte Carlo is indeed the solution to diminish the number of likelihood evaluations, since it moves along iso-density slices… After this, we went working on ABC model choice with Jean-Michel Marin and Natesh Pillai. Before joining the fun at the Section for Bayesian statistical mixer, where the Savage and Mitchell and student awards were presented. This was the opportunity to see friends, meet new Bayesians, and congratulate the winners.

JSM 3rd day highlights

(by Garvesh Raskutti)

I enjoyed today much more than yesterday since I am slowly figuring out how to navigate the mass of talks. I found that when I am unsure what to go to, the invited sessions usually end up being a better choice. The talks are longer meaning the speakers have a chance to give proper background and explain the results in more detail. The 15 minute talks are mainly useful just to find out what people are working on.

I attended the discussion on Bayesian analysis hosted by Professor Gelman, Professor Berger and Professor Robert. They all presented interesting points of view on the difference between ‘Subjective Bayesians,’ ‘Objective Bayesians’ and ‘Frequentists.’ In particular I liked some of the points Professor Berger made about non-parametric Bayesian methods. The fact that the models are difficult to interpret I think is a genuine concern. Furthermore, the common belief that non-parametric methods make fewer assumptions is simply not true, a point made by Professor Berger. I also liked the fact that Professor Gelman highlighted the importance of the underlying research question and conclusions drawn from the methods being more critical than the choice of methods. In general, I feel it is difficult to discuss which methods are better or worse without discussing them with respect to a specific scientific goal.

I also attended the invited session on Sparse Regression and High-Dimensional Statistical Analysis. I especially liked the talk by Professor Rigollet on a new method that achieves optimal performance for prediction error under the sparse regression model. The presentation was extremely clear and transparent and completely solved the problem of optimal prediction error for the sparse linear models.


About

The Statistics Forum, brought to you by the American Statistical Association and CHANCE magazine, provides everyone the opportunity to participate in discussions about probability and statistics and their role in important and interesting topics.

The views expressed here are those of the individual authors and not necessarily those of the ASA, its officers, or its staff. The Statistics Forum is edited by Andrew Gelman.

A Magazine for People Interested in the Analysis of Data