Friam

speaking of analytics

Classic

List

Threaded

33 messages Options

gepr

speaking of analytics

The case against big data: "It’s like you’re being put into a cult, but you don’t actually believe in it"
http://www.salon.com/2016/09/08/the-case-against-big-data-it-is-like-youre-being-put-into-a-cult-but-you-dont-actually-believe-in-it/

> But it’s opaque right? Which is also what a lot of these things have in common.
>
> It’s opaque, and it’s unaccountable. You cannot appeal it because it is opaque. Not only is it opaque, but I actually filed a Freedom of Information Act request to get the source code. And I was told I couldn’t get the source code and not only that, but I was told the reason why was that New York City had signed a contract with this place called VARK in Madison, Wisconsin. Which was an agreement that they wouldn’t get access to the source code either. The Department of Education, the city of New York City but nobody in the city, in other words, could truly explain the scores of the teachers.
>
> It was like an alien had come down to earth and said, "Here are some scores, we’re not gonna explain them to you, but you should trust them. And by the way you can’t appeal them and you will not be given explanations for how to get better."

--
☣ glen

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

uǝʃƃ ⊥ glen

Marcus G. Daniels

Re: speaking of analytics

Racial profiling is a single dimensional predictor. It's bad because it is regressive, not because race is a useless predictor.
There are lots of attributes like that, and big data is just puts them together to predict aggregate behaviors about people without really having a theory of mind of that individual or a theory of mind at all. Like trying to learn from Google without understanding the reading and writing of human language. I think the FOIA type concerns should be fixable in principle. But in practice, these databases and algorithms are tightly held intellectual property that the government licenses from companies. Without sweeping legislation, the government can't get their hands on it, and the people interested in applying these systems, like law enforcement, aren't necessarily the most curious people in the world to begin with. Push a button and get an authoritative answer. What could be better? You're guilty because the system said so.

-----Original Message-----
From: Friam [mailto:[hidden email]] On Behalf Of glen ?
Sent: Thursday, September 08, 2016 4:54 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: [FRIAM] speaking of analytics

The case against big data: "It’s like you’re being put into a cult, but you don’t actually believe in it"
http://www.salon.com/2016/09/08/the-case-against-big-data-it-is-like-youre-being-put-into-a-cult-but-you-dont-actually-believe-in-it/

> But it’s opaque right? Which is also what a lot of these things have in common.
>
> It’s opaque, and it’s unaccountable. You cannot appeal it because it is opaque. Not only is it opaque, but I actually filed a Freedom of Information Act request to get the source code. And I was told I couldn’t get the source code and not only that, but I was told the reason why was that New York City had signed a contract with this place called VARK in Madison, Wisconsin. Which was an agreement that they wouldn’t get access to the source code either. The Department of Education, the city of New York City but nobody in the city, in other words, could truly explain the scores of the teachers.
>
> It was like an alien had come down to earth and said, "Here are some scores, we’re not gonna explain them to you, but you should trust them. And by the way you can’t appeal them and you will not be given explanations for how to get better."

--
☣ glen

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Roger Critchlow-2

Re: speaking of analytics

See the result of the AI judged beauty contest? Apparently the training set needed more curation. Very teachable moment.

-- rec --

On Sep 8, 2016 7:10 PM, "Marcus Daniels" <[hidden email]> wrote:

Racial profiling is a single dimensional predictor. It's bad because it is regressive, not because race is a useless predictor.
There are lots of attributes like that, and big data is just puts them together to predict aggregate behaviors about people without really having a theory of mind of that individual or a theory of mind at all. Like trying to learn from Google without understanding the reading and writing of human language. I think the FOIA type concerns should be fixable in principle. But in practice, these databases and algorithms are tightly held intellectual property that the government licenses from companies. Without sweeping legislation, the government can't get their hands on it, and the people interested in applying these systems, like law enforcement, aren't necessarily the most curious people in the world to begin with. Push a button and get an authoritative answer. What could be better? You're guilty because the system said so.

-----Original Message-----
From: Friam [mailto:[hidden email]] On Behalf Of glen ?
Sent: Thursday, September 08, 2016 4:54 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: [FRIAM] speaking of analytics

The case against big data: "It’s like you’re being put into a cult, but you don’t actually believe in it"
http://www.salon.com/2016/09/08/the-case-against-big-data-it-is-like-youre-being-put-into-a-cult-but-you-dont-actually-believe-in-it/

> But it’s opaque right? Which is also what a lot of these things have in common.
>
> It’s opaque, and it’s unaccountable. You cannot appeal it because it is opaque. Not only is it opaque, but I actually filed a Freedom of Information Act request to get the source code. And I was told I couldn’t get the source code and not only that, but I was told the reason why was that New York City had signed a contract with this place called VARK in Madison, Wisconsin. Which was an agreement that they wouldn’t get access to the source code either. The Department of Education, the city of New York City but nobody in the city, in other words, could truly explain the scores of the teachers.
>
> It was like an alien had come down to earth and said, "Here are some scores, we’re not gonna explain them to you, but you should trust them. And by the way you can’t appeal them and you will not be given explanations for how to get better."

--
☣ glen

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Nick Thompson

Re: speaking of analytics

Hi, Roger.

That was some hurricane, huh? I thought of you in Boston Harbor, battened against the lashing gales.

Speaking of analytics, I was struck by the notion of having a prediction without a theory. I am wondering if that is actually possible. I know that theories are really useful for making predictions, but can one actually make a prediction without one? Perhaps meteorology would be a good domain in which to think this through. The lowest level of prediction (and one that works remarkably and embarrassingly well) is to predict that tomorrow’s weather will be the same as todays …. “persistence forecasting.” But even that entails a theory that the weather is stable. Then one can have dynamic persistence theories, which one would apply to the stuff floating down a river ... the river will continue to flow down to me. The jet stream is sometimes like that. And jet “stream” is, after all, a metaphor. And this is making me think that we ought perhaps to talk about “levels of theory”, rather than “theory/non-theory”, persistence forecasting being the application of a VERY low level theory.

Anyway, I am probably bending this thread horribly. Off on my own cloud. Age has addled my brain, and now the heat has cooked it. I am an omelet.

Take care and keep afloat.

Nick

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Roger Critchlow
Sent: Thursday, September 08, 2016 7:21 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

See the result of the AI judged beauty contest? Apparently the training set needed more curation. Very teachable moment.

-- rec --

On Sep 8, 2016 7:10 PM, "Marcus Daniels" <[hidden email]> wrote:

Racial profiling is a single dimensional predictor. It's bad because it is regressive, not because race is a useless predictor.
There are lots of attributes like that, and big data is just puts them together to predict aggregate behaviors about people without really having a theory of mind of that individual or a theory of mind at all. Like trying to learn from Google without understanding the reading and writing of human language. I think the FOIA type concerns should be fixable in principle. But in practice, these databases and algorithms are tightly held intellectual property that the government licenses from companies. Without sweeping legislation, the government can't get their hands on it, and the people interested in applying these systems, like law enforcement, aren't necessarily the most curious people in the world to begin with. Push a button and get an authoritative answer. What could be better? You're guilty because the system said so.

-----Original Message-----
From: Friam [mailto:[hidden email]] On Behalf Of glen ?
Sent: Thursday, September 08, 2016 4:54 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: [FRIAM] speaking of analytics

The case against big data: "It’s like you’re being put into a cult, but you don’t actually believe in it"
http://www.salon.com/2016/09/08/the-case-against-big-data-it-is-like-youre-being-put-into-a-cult-but-you-dont-actually-believe-in-it/

> But it’s opaque right? Which is also what a lot of these things have in common.
>
> It’s opaque, and it’s unaccountable. You cannot appeal it because it is opaque. Not only is it opaque, but I actually filed a Freedom of Information Act request to get the source code. And I was told I couldn’t get the source code and not only that, but I was told the reason why was that New York City had signed a contract with this place called VARK in Madison, Wisconsin. Which was an agreement that they wouldn’t get access to the source code either. The Department of Education, the city of New York City but nobody in the city, in other words, could truly explain the scores of the teachers.
>
> It was like an alien had come down to earth and said, "Here are some scores, we’re not gonna explain them to you, but you should trust them. And by the way you can’t appeal them and you will not be given explanations for how to get better."

--
☣ glen

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

Eric Charles-2

Re: speaking of analytics

Marcus,

That's an interesting distinction. Is it the case that by "theory" Nick was referring to something verbal and explicitly metaphorical, or would the results of data mining, which one sought to validate on a different sample, count as a "theory".

So, for example, if my data mining of Marine data found that tying shoes left-to-right predicted success at Officer Candidate School, and I then went to test for that "prediction" in a later sample of incoming officer candidates, to what extent is my prediction based on "a theory".

Of course, "data mining will be a useful way to uncover patterns" is itself a theory, applicable in some domains but not others (i.e., not all domains of inquiry will contain the sought after patterns in a long-term stable form).

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

[hidden email]

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

I would say a statistical inference is less generative than a theory. A theory in some sense asserts how things really work. Data mining may stumble across the crucial aspects of a mechanism (whether it is physical, sociological, etc.) but they may also just being seeing some derived quantity of other hidden variables. Perhaps there _is_ a reason why tying shoes one way or another is related to some mode of cognitive processing that is more efficient? Or maybe it arises because some parts of the country, folks tend to have that habit, and those parts of the country happen to cleaner water or less air pollution or better schools or have social constraints in their communities that lead individuals to navigate authoritarianism better than others?

I think that data mining could be elaborated (and automated) to begin to create theories. For example, if a regression had an especially simple form that was also predictive, describe the variables with some ontology that says why they ought to relate in a deterministic fashion. Instead of just “the weather will be rainy tomorrow”, report “the weather will be rainy tomorrow because there is a low pressure system coming in the from the west”, and then reference mathematical models for how weather systems behave, etc.

From: Friam [mailto:[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 9:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

gepr

Re: speaking of analytics

In reply to this post by Nick Thompson

tanstaafl, of course. The concept of "levels" is good but misleading. It's more useful to think in terms of layers. As Marcus hints, we can combine induction with abduction and triangulate our way towards hypotheses with more and more (layered) structure. The more structure assumed for a hypothesis, the more we tend to call it a "theory".

On 09/09/2016 07:46 AM, Nick Thompson wrote:
> And this is making me think that we ought perhaps to talk about “levels of theory”, rather than “theory/non-theory”, persistence forecasting being the application of a VERY low level theory.

On 09/09/2016 08:54 AM, Marcus Daniels wrote:
> I think that data mining could be elaborated (and automated) to begin to create theories. For example, if a regression had an especially simple form that was also predictive, describe the variables with some ontology that says why they ought to relate in a deterministic fashion. Instead of just “the weather will be rainy tomorrow”, report “the weather will be rainy tomorrow because there is a low pressure system coming in the from the west”, and then reference mathematical models for how weather systems behave, etc.

--
␦glen?

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

uǝʃƃ ⊥ glen

Nick Thompson

Re: speaking of analytics

In reply to this post by Eric Charles-2

And data “mining” is a metaphor.

Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

Fine, “statistical inference” then.

From: Friam [mailto:[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

gepr

Re: speaking of analytics

There's no doubt that any form of inference done by humans is subject to premature registration or even apophenia. But the inverted claim, that _all_ registration is premature (or imaginary) is way too strong, and perhaps a case of tu quoque.

On 09/09/2016 11:42 AM, Marcus Daniels wrote:

> Fine, “statistical inference” then.
>
> *From:*Friam [mailto:[hidden email]] *On Behalf Of *Nick Thompson
> *Sent:* Friday, September 09, 2016 12:38 PM
> *To:* 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
> *Subject:* Re: [FRIAM] speaking of analytics
>
> And data “mining” is a metaphor.
>
> Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

--
␦glen?

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

uǝʃƃ ⊥ glen

Nick Thompson

Re: speaking of analytics

In reply to this post by Marcus G. Daniels

M-

No statistic has ever made an inference.

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Marcus Daniels
Sent: Friday, September 09, 2016 2:42 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Fine, “statistical inference” then.

From: Friam [[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

Marcus,

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Frank Wimberly-2

Re: speaking of analytics

Have computers made inferences? I know the obvious answer but Nick uses language in a special way.

Frank

Frank Wimberly
Phone (505) 670-9918

On Sep 9, 2016 1:02 PM, "Nick Thompson" <[hidden email]> wrote:

M-

No statistic has ever made an inference.

N

Nicholas S. Thompson
Emeritus Professor of Psychology and Biology
Clark University
http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Marcus Daniels
Sent: Friday, September 09, 2016 2:42 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Fine, “statistical inference” then.

From: Friam [[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson
Emeritus Professor of Psychology and Biology
Clark University
http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,
That's an interesting distinction. Is it the case that by "theory" Nick was referring to something verbal and explicitly metaphorical, or would the results of data mining, which one sought to validate on a different sample, count as a "theory".

So, for example, if my data mining of Marine data found that tying shoes left-to-right predicted success at Officer Candidate School, and I then went to test for that "prediction" in a later sample of incoming officer candidates, to what extent is my prediction based on "a theory".

Of course, "data mining will be a useful way to uncover patterns" is itself a theory, applicable in some domains but not others (i.e., not all domains of inquiry will contain the sought after patterns in a long-term stable form).

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician
U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:
“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

He can look it up on Wikipedia if he wants to know.

From: Friam [mailto:[hidden email]] On Behalf Of Frank Wimberly
Sent: Friday, September 09, 2016 1:16 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Have computers made inferences? I know the obvious answer but Nick uses language in a special way.

Frank

Frank Wimberly
Phone (505) 670-9918

On Sep 9, 2016 1:02 PM, "Nick Thompson" <[hidden email]> wrote:

M-

No statistic has ever made an inference.

N

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Marcus Daniels
Sent: Friday, September 09, 2016 2:42 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Fine, “statistical inference” then.

From: Friam [[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,

That's an interesting distinction. Is it the case that by "theory" Nick was referring to something verbal and explicitly metaphorical, or would the results of data mining, which one sought to validate on a different sample, count as a "theory".

So, for example, if my data mining of Marine data found that tying shoes left-to-right predicted success at Officer Candidate School, and I then went to test for that "prediction" in a later sample of incoming officer candidates, to what extent is my prediction based on "a theory".

Of course, "data mining will be a useful way to uncover patterns" is itself a theory, applicable in some domains but not others (i.e., not all domains of inquiry will contain the sought after patterns in a long-term stable form).

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

lrudolph

Re: speaking of analytics

In reply to this post by Nick Thompson

Nick asks:

> How does thinking of data as
> encased in a non-dynamic subterranean matrix shape our (your) thinking for good
> or ill?

I'm astounded that *that* is the (most) salient part of the metaphor to your mind. I'd sooner
know, "how does talking of data as something that is susceptible not only to extraction but
to BEING USED UP shape your BEHAVIOR for good or ill?"

It occurs to me just now, typing, that the originators of the damned phrase might not have
been thinking of mining coal (say), but rather of jewels or gold--in which case the product
isn't used up, it is simply re-sequestered from its subterranean matrix to some other matrix;
or maybe of ore, in which case some "useful" knowledge will be refined and used for probably
private good or public ill, at--even more so than for coal, perhaps--enormous public expense
in the form of mine tailings, cyanide-poisoned water supplies, etc., etc.

You may not have noticed that I've always despised this metaphor. But I never did give it
even this much (a couple of minutes) of thought, to see perhaps why. So thanks, I guess.

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

The investment aspect of the term is not inaccurate (not unlike with Bitcoin mining). Tabulating and calculating trillions of p-values (or SHA256 hashes) is not cheap. It's different than trying out a few hypotheses, ur, propectin'.

-----Original Message-----
From: Friam [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Friday, September 09, 2016 2:37 PM
To: Nick Thompson <[hidden email]>; The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Nick asks:

> How does thinking of data as
> encased in a non-dynamic subterranean matrix shape our (your) thinking
> for good or ill?

I'm astounded that *that* is the (most) salient part of the metaphor to your mind. I'd sooner know, "how does talking of data as something that is susceptible not only to extraction but to BEING USED UP shape your BEHAVIOR for good or ill?"

It occurs to me just now, typing, that the originators of the damned phrase might not have been thinking of mining coal (say), but rather of jewels or gold--in which case the product isn't used up, it is simply re-sequestered from its subterranean matrix to some other matrix; or maybe of ore, in which case some "useful" knowledge will be refined and used for probably private good or public ill, at--even more so than for coal, perhaps--enormous public expense in the form of mine tailings, cyanide-poisoned water supplies, etc., etc.

You may not have noticed that I've always despised this metaphor. But I never did give it even this much (a couple of minutes) of thought, to see perhaps why. So thanks, I guess.

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Nick Thompson

Re: speaking of analytics

In reply to this post by Frank Wimberly-2

Frank,

I would say that one can program a computer to make inferences. It may use statistics to do so. So, you could give it a “persistence forecast” algorithm. Then you type in “it rained today” and the computer will type out “it will probably rain tomorrow.”

Nick

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Frank Wimberly
Sent: Friday, September 09, 2016 3:16 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Have computers made inferences? I know the obvious answer but Nick uses language in a special way.

Frank

Frank Wimberly
Phone (505) 670-9918

On Sep 9, 2016 1:02 PM, "Nick Thompson" <[hidden email]> wrote:

M-

No statistic has ever made an inference.

N

Nicholas S. Thompson
Emeritus Professor of Psychology and Biology
Clark University
http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Marcus Daniels
Sent: Friday, September 09, 2016 2:42 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Fine, “statistical inference” then.

From: Friam [[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson
Emeritus Professor of Psychology and Biology
Clark University
http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,
That's an interesting distinction. Is it the case that by "theory" Nick was referring to something verbal and explicitly metaphorical, or would the results of data mining, which one sought to validate on a different sample, count as a "theory".

So, for example, if my data mining of Marine data found that tying shoes left-to-right predicted success at Officer Candidate School, and I then went to test for that "prediction" in a later sample of incoming officer candidates, to what extent is my prediction based on "a theory".

Of course, "data mining will be a useful way to uncover patterns" is itself a theory, applicable in some domains but not others (i.e., not all domains of inquiry will contain the sought after patterns in a long-term stable form).

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician
U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:
“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Nick Thompson

Re: speaking of analytics

In reply to this post by gepr

Glen,

You wrote:

Narcissist that I am, I assume you are punishing me for all the weird language I have inflicted on the list over the last 12 years. I humbly acknowledge the punishment.

Now: Could you explain what you meant? (};-)]

Thanks,

Nick

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

-----Original Message-----
From: Friam [mailto:[hidden email]] On Behalf Of ?glen?
Sent: Friday, September 09, 2016 2:51 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

On 09/09/2016 11:42 AM, Marcus Daniels wrote:

> Fine, “statistical inference” then.

> *From:*Friam [[hidden email]] *On Behalf Of *Nick

> Thompson

> *Sent:* Friday, September 09, 2016 12:38 PM

> *To:* 'The Friday Morning Applied Complexity Coffee Group'

> <[hidden email]>

> *Subject:* Re: [FRIAM] speaking of analytics

> And data “mining” is a metaphor.

> Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

␦glen?

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Marcus G. Daniels

Re: speaking of analytics

In reply to this post by Nick Thompson

Same could be said of these algorithms.

http://www.cesm.ucar.edu/models/atm-cam/docs/description/node2.html

From: Friam [mailto:[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 8:09 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Frank,

Nick

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Frank Wimberly
Sent: Friday, September 09, 2016 3:16 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Have computers made inferences? I know the obvious answer but Nick uses language in a special way.

Frank

Frank Wimberly
Phone (505) 670-9918

On Sep 9, 2016 1:02 PM, "Nick Thompson" <[hidden email]> wrote:

M-

No statistic has ever made an inference.

N

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [mailto:[hidden email]] On Behalf Of Marcus Daniels
Sent: Friday, September 09, 2016 2:42 PM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Fine, “statistical inference” then.

From: Friam [[hidden email]] On Behalf Of Nick Thompson
Sent: Friday, September 09, 2016 12:38 PM
To: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

And data “mining” is a metaphor.

Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse. I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

*cf, Monte Python’s Flying Circus

Nick Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

From: Friam [[hidden email]] On Behalf Of Eric Charles
Sent: Friday, September 09, 2016 11:31 AM
To: The Friday Morning Applied Complexity Coffee Group <[hidden email]>
Subject: Re: [FRIAM] speaking of analytics

Marcus,

That's an interesting distinction. Is it the case that by "theory" Nick was referring to something verbal and explicitly metaphorical, or would the results of data mining, which one sought to validate on a different sample, count as a "theory".

So, for example, if my data mining of Marine data found that tying shoes left-to-right predicted success at Officer Candidate School, and I then went to test for that "prediction" in a later sample of incoming officer candidates, to what extent is my prediction based on "a theory".

Of course, "data mining will be a useful way to uncover patterns" is itself a theory, applicable in some domains but not others (i.e., not all domains of inquiry will contain the sought after patterns in a long-term stable form).

Eric

-----------
Eric P. Charles, Ph.D.
Supervisory Survey Statistician

U.S. Marine Corps

On Fri, Sep 9, 2016 at 10:51 AM, Marcus Daniels <[hidden email]> wrote:

“I know that theories are really useful for making predictions, but can one actually make a prediction without one?”

Yes, that’s what data mining is: Take a large corpus of data, find some statistically rare relationships, and then test for their predictive value on another large corpus of data. In this way one can predict things without really having any kind of theory or even domain knowledge.

Marcus

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com

Steve Smith

Re: speaking of analytics

In reply to this post by Nick Thompson

N -

I read this as "Glen being Glen" which I approve of...

... that doesn't mean you don't get credit for inflicting your own inner vocabulary (or simply the lexicon of your profession?) on us...

Some of us appreciate what might otherwise seem idiosyncratic.

I had to parse this one very carefully and seek references (especially for the tu quoque ) but that is (to use a golf metaphor of all damned things) par for the course!

- S

PS.. I had three ravens fledge in the cottonwood by my house this summer and I thought of you!

On 9/9/16 8:18 PM, Nick Thompson wrote:

Glen,

You wrote:

There's no doubt that any form of inference done by humans is subject to premature registration or even apophenia. But the inverted claim, that _all_ registration is premature (or imaginary) is way too strong, and perhaps a case of tu quoque.

Narcissist that I am, I assume you are punishing me for all the weird language I have inflicted on the list over the last 12 years.   I humbly acknowledge the punishment.

Now: Could you explain what you meant? (};-)]

Thanks,

Nick

Nicholas S. Thompson

Emeritus Professor of Psychology and Biology

Clark University

http://home.earthlink.net/~nickthompson/naturaldesigns/

-----Original Message-----
From: Friam [[hidden email]] On Behalf Of ?glen?
Sent: Friday, September 09, 2016 2:51 PM
To: The Friday Morning Applied Complexity Coffee Group [hidden email]
Subject: Re: [FRIAM] speaking of analytics

There's no doubt that any form of inference done by humans is subject to premature registration or even apophenia. But the inverted claim, that _all_ registration is premature (or imaginary) is way too strong, and perhaps a case of tu quoque.

On 09/09/2016 11:42 AM, Marcus Daniels wrote:

> Fine, “statistical inference” then.

>

> *From:*Friam [[hidden email]] *On Behalf Of *Nick

> Thompson

> *Sent:* Friday, September 09, 2016 12:38 PM

> *To:* 'The Friday Morning Applied Complexity Coffee Group'

> <[hidden email]>

> *Subject:* Re: [FRIAM] speaking of analytics

>

> And data “mining” is a metaphor.

>

> Now people claim to use metaphors “metaphorically”, by which they mean that they mean nothing by them. But it is my “teery”* (and it is all mine) that nobody uses a metaphor but that hizr thinking is influenced by it. The influence can be inexplicit, in which case the user is blind to its effects on himmr, or explicit, in which case the user’s imagination is enhanced by its use and less likely to be misled by its misuse.   I would like to explore this “teery” using “Data Mining” as an example. How does thinking of data as encased in a non-dynamic subterranean matrix shape our (your) thinking for good or ill?

--

␦glen?

============================================================

FRIAM Applied Complexity Group listserv

Meets Fridays 9a-11:30 at cafe at St. John's College to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com
============================================================
FRIAM Applied Complexity Group listserv
Meets Fridays 9a-11:30 at cafe at St. John's College
to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com