Friam

Theory, and Why It's Time Psychology Got One

Classic

List

Threaded

24 messages Options

Eric Charles

Re: Theory, and Why It's Time Psychology Got One

Roger, your final question is spot on:
"Is this the behavior of a community of researchers that collectively seeking a consensus of reproducible observations?"

No, it is not. You skewer mainstream psychology most effectively. What is this behavior then?

It is the behavior of a group that is not working towards consensus, and that is not clear on what the value of specific replicable results would be. It is the behavior of a group that vies for prestige through popularity contests and through bean counting publications regardless of replicability or actual progress being made. It is self-serving behavior, well adapted to the landscape of a field that lacks a core theory.

A core theory wouldn't change this overnight, but it is likely a key component of any long term efforts to change the culture.

Eric

P.S. On a related note, my suspicion is that you find much more willingness to share data amongst the sub-disciplines of psychology with more clearly defined cores. This is because the core provides plausible "positive" reasons why someone would ask for your data. You would also see less fabrication of data in these areas, because colleagues would quickly notice if a lab made a habit of publishing non-replicable data.

On Sun, Nov 13, 2011 07:35 PM, Roger Critchlow <[hidden email]> wrote:

On Sat, Nov 12, 2011 at 7:29 PM, ERIC P. CHARLES <epc2@...> wrote:

Roger,
You are correct that it might seem like psychology should have other things to worry about, but frankly the problems you mention (rampant misuse of statistics and the rare forged data scandals) would be a lot easier to deal with if we had a more unified theoretical base.

Eric --

Well, admittedly, it's been a bad few weeks for psychology in the news, not the sort of run of luck one would want to generalize too far.

But I don't see how having a theory helps if the practice doesn't involve sharing observations made under reproducible conditions so they can be independently verified.

Forget the statistical faux pas, and look at the PLOS paper: 49 papers from the Journal of Personality and Social Psychology and Journal of Experimental Psychology: Learning, Memory, and Cognition published in the second half of 2004, "all corresponding authors had signed a statement that they would share their data for such verification purposes", the data was requested in the summer of 2005, and

Responses to Data Requests

Of the 49 corresponding authors, 21 (42.9%) had shared some data with Wicherts et al. Thirteen corresponding authors (26.5%) failed to respond to the request or any of the two reminders. Three corresponding authors (6.1%) refused to share data either because the data were lost or because they lacked time to retrieve the data and write a codebook. Twelve corresponding authors (24.5%) promised to share data at a later date, but have not done so in the past six years (we did not follow up on it). These authors commonly indicated that the data were not readily available or that they first needed to write a codebook.

In more than half of the papers the supporting data effectively doesn't exist? And more than a quarter of the authors don't even feel obliged to make excuses? Is this the behavior of a community of researchers collectively seeking a consensus of reproducible observations?

-- rec --
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org

Eric Charles

Professional Student and
Assistant Professor of Psychology
Penn State University
Altoona, PA 16601

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org

Prof David West

Re: Theory, and Why It's Time Psychology Got One

On Monday, November 14, 2011 12:43 AM, "ERIC P. CHARLES" <[hidden email]> wrote:

"It is the behavior of a group that is not working towards consensus, and that is not clear on what the value of specific replicable results would be. It is the behavior of a group that vies for prestige through popularity contests and through bean counting publications regardless of replicability or actual progress being made. It is self-serving behavior, well adapted to the landscape of a field that lacks a core theory."

At the risk of annoying everyone (except perhaps Nick) - I would suggest that, with regard to the preceding paragraph, physics is no different from psychology. Feyeraband, Knorr-Certina, Christopher Alexander ("self conscious process") and many other observers of how science is really done as opposed to self serving reports of how it is supposed to be done.

How fast a discipline's thinking ossifies to a consensual theory is a function of the need to protect one's research funding and repelling challengers with outre ideas - not the substantiveness of the "core theory."

dave west

Nick Thompson

Re: Theory, and Why It's Time Psychology Got One

Dave,

I hate to bite the hand that feeds me, but I think there really IS a difference. It’s not that hard scientists are less venal than soft ones; Something in the state of play of the soft sciences themselves just does not reward rigor and head down, bull ahead normal science, in the way that it is rewarded in the hard sciences. I think being superficially uninteresting to the public goes a long way to protecting one from the kind of crap that goes on in the social sciences. By the way, statistics itself is one of those tortured fields. If you look at its history, you find that the statistics we were all taught in graduate school is an incoherent mélange of Spearman and Pearson (I think).

From: [hidden email] [mailto:[hidden email]] On Behalf Of Prof David West
Sent: Monday, November 14, 2011 12:02 AM
To: The Friday Morning Applied Complexity Coffee Group
Subject: Re: [FRIAM] Theory, and Why It's Time Psychology Got One

On Monday, November 14, 2011 12:43 AM, "ERIC P. CHARLES" <[hidden email]> wrote:

"It is the behavior of a group that is not working towards consensus, and that is not clear on what the value of specific replicable results would be. It is the behavior of a group that vies for prestige through popularity contests and through bean counting publications regardless of replicability or actual progress being made. It is self-serving behavior, well adapted to the landscape of a field that lacks a core theory."

At the risk of annoying everyone (except perhaps Nick) - I would suggest that, with regard to the preceding paragraph, physics is no different from psychology. Feyeraband, Knorr-Certina, Christopher Alexander ("self conscious process") and many other observers of how science is really done as opposed to self serving reports of how it is supposed to be done.

How fast a discipline's thinking ossifies to a consensual theory is a function of the need to protect one's research funding and repelling challengers with outre ideas - not the substantiveness of the "core theory."

dave west

Prof David West

Re: Theory, and Why It's Time Psychology Got One

Two points:

First, what are you doing up answering email in the wee hours of the AM? My excuse is that I am in India and it is midday here.

Second, I did say "in terms of the preceding paragraph ... I see no difference." And, behaviorally, I really do not see any difference. In semi-hard sciences like software engineering, the described behaviors are even more evident - aspect-oriented programming being a prime example. Statistics may very well be in this category as well - visualization of quantative information definitely is.

I heard once, that newly graduated particle physicists could look forward to doing ONE major experiment in their career - because of the cost of such experiments and the lack of places to conduct them. The only chance they have of even getting the opportunity for that one experiment is to conform to the prevailing "core theory" - thereby making it more and more "core." AND, it is the conformance that counts not the merits of the experiment or the core theory.

In the "soft sciences" the experiments cost far far less and almost anyone can do them. So you can have multiple theories in play simultaneously and no economic motive to force convergence to a single core.

Comparing economics and anthropology might be interesting. Econ aspires to be as hard a science as physics (sociology does as well - Harry Seldon is alive and well in that discipline) but anthropology is more content (barring the odd Marvin Harris) with "descriptive" theory with only statistical predictive capabilities. Econ makes predictions that are, more often than not, wrong - precise but wrong. Anthro makes predictions but expects them to be ballpark, not precise. Does this mean that economists have real theories and anthros do not?

davew

On Monday, November 14, 2011 12:54 AM, "Nicholas Thompson" <[hidden email]> wrote:

Dave,

I hate to bite the hand that feeds me, but I think there really IS a difference. It’s not that hard scientists are less venal than soft ones; Something in the state of play of the soft sciences themselves just does not reward rigor and head down, bull ahead normal science, in the way that it is rewarded in the hard sciences. I think being superficially uninteresting to the public goes a long way to protecting one from the kind of crap that goes on in the social sciences. By the way, statistics itself is one of those tortured fields. If you look at its history, you find that the statistics we were all taught in graduate school is an incoherent mélange of Spearman and Pearson (I think).

From: [hidden email] [mailto:[hidden email]] On Behalf Of Prof David West
Sent: Monday, November 14, 2011 12:02 AM
To: The Friday Morning Applied Complexity Coffee Group
Subject: Re: [FRIAM] Theory, and Why It's Time Psychology Got One

On Monday, November 14, 2011 12:43 AM, "ERIC P. CHARLES" <[hidden email]> wrote:

"It is the behavior of a group that is not working towards consensus, and that is not clear on what the value of specific replicable results would be. It is the behavior of a group that vies for prestige through popularity contests and through bean counting publications regardless of replicability or actual progress being made. It is self-serving behavior, well adapted to the landscape of a field that lacks a core theory."

At the risk of annoying everyone (except perhaps Nick) - I would suggest that, with regard to the preceding paragraph, physics is no different from psychology. Feyeraband, Knorr-Certina, Christopher Alexander ("self conscious process") and many other observers of how science is really done as opposed to self serving reports of how it is supposed to be done.

How fast a discipline's thinking ossifies to a consensual theory is a function of the need to protect one's research funding and repelling challengers with outre ideas - not the substantiveness of the "core theory."

dave west
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
lectures, archives, unsubscribe, maps at http://www.friam.org