Has anybody been able to get anything useful out of that thing? Most of the items I've searched for return totally bizarre results. For example, searching on ibuprofen with the intent to see any correlations with influenza give this as one of the higher-correlated results:
WTF? Does this show that on-line religion causes headaches? Seriously, has anybody found this tool to be of any practical use? ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
It's because 'ibuprofen' 'causes constipation' making folks turn to
the 'gateway bible' for a solution.
:-)
Robert On 9/30/11 9:53 AM, Douglas Roberts wrote: Has anybody been able to get anything useful out of that thing? Most of the items I've searched for return totally bizarre results. For example, searching on ibuprofen with the intent to see any correlations with influenza give this as one of the higher-correlated results: -- Robert Cordingley Web Development cirrillian.com 505-471-4569 (office) 281-989-6272 (cell) ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Douglas Roberts-2
Is this any help? https://www.google.com/trends/correlate/
-tj On Fri, Sep 30, 2011 at 9:53 AM, Douglas Roberts <[hidden email]> wrote: Has anybody been able to get anything useful out of that thing? Most of the items I've searched for return totally bizarre results. For example, searching on ibuprofen with the intent to see any correlations with influenza give this as one of the higher-correlated results: -- ========================================== J. T. Johnson Institute for Analytic Journalism -- Santa Fe, NM USA www.analyticjournalism.com 505.577.6482(c) 505.473.9646(h) http://www.jtjohnson.com [hidden email] ========================================== ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
That's the one I was using, Tom.
--Doug
On Fri, Sep 30, 2011 at 2:34 PM, Tom Johnson <[hidden email]> wrote: Is this any help? https://www.google.com/trends/correlate/ Doug Roberts [hidden email] [hidden email] ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
I thought the example Doug sent was a correlation between the
frequency of search terms, not with any actual real data. I always
thought correlating time based data was difficult since as usage
rises in general one might expect search term frequency to rise
together too. Real statisticians can chime in here.
'santa fe' was interesting in showing an annual cycle presumably corresponding to people's interest in finding info on a tourist destination. Drawing was interesting in that it seems one could always find some search term frequencies to correlate with any shape curve. Conclusion: it was fun. Thanks Robert C On 9/30/11 2:36 PM, Douglas Roberts wrote: That's the one I was using, Tom. -- Robert Cordingley Web Development cirrillian.com 505-471-4569 (office) 281-989-6272 (cell) ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Tom Johnson
Tom
Nice tool for those who make decisions and design strategies and funny for profanes like me. It's interesting and fun to find patterns and trends using correlate tool with the words I'm listing. Seasonality in some cases is amazing. Maybe resulting plots show what people collectively think, live and want. September 11th World Trade Center Prozac Depressed Fashion Baseball Nascar Super Bowl God Complexity End of the world Science War Hurricane Obama Death Hope Cancer Job Recovery Universities Industry Sales Graduation Linux Fun 2011/9/30 Tom Johnson <[hidden email]> Is this any help? https://www.google.com/trends/correlate/ -- Alfredo ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Tom Johnson
BTW words must be written in lower case 2011/9/30 Tom Johnson <[hidden email]> Is this any help? https://www.google.com/trends/correlate/ -- Alfredo ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Robert J. Cordingley
Sorry about the delay; busy weekend.
Yes, google correlate maps search data, not actual real data, although you do have the option of loading your own time series and letting google correlate that for you.
Still, I fail to understand why searches for "ibuprofen" would map to searches for "gateway bible" with r=0.949. --Doug
On Fri, Sep 30, 2011 at 3:10 PM, Robert J. Cordingley <[hidden email]> wrote:
============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org google-trend-ibuprofen.png (67K) Download Attachment |
Administrator
|
In reply to this post by Robert J. Cordingley
On Fri, Sep 30, 2011 at 3:10 PM, Robert J. Cordingley <[hidden email]> wrote:
Just to make sure I understand, its the correlation between the time series of the use of the search terms .. i.e. when search terms x & y were used, right?
Yes indeed, it would be much appreciated!
It certainly is an interesting/fun use of a whole lota data! -- Owen ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Douglas Roberts-2
Given an extremely large data set, wherein the long term trend of the
data has a fair correlation (e.g. people may search more in one month than another ) it seems very likely that you'll get an overlap with something non-correlated, simply because you're drawing from such a large set and there are only so many possibilities for the high order components of the signal. I'm betting i could create a set of functions by throwing random polynomials or sine series for a bound range 10^8 times (similar to the entropy of a 2 word phrases drawn from a 10k word dictionary ) and find excellent correlation between 2 of most any normalized signal. **************************** Greg Sonnenfeld On Mon, Oct 3, 2011 at 10:19 AM, Roger Critchlow <[hidden email]> wrote: > And I fail to understand why searches for "advil" would map to searches for > "chai tea latte" and "pappardelle" with r>0.931, guess I need to read the > Comic Book documentation. > -- rec -- > > On Mon, Oct 3, 2011 at 9:55 AM, Douglas Roberts <[hidden email]> > wrote: >> >> Sorry about the delay; busy weekend. >> Yes, google correlate maps search data, not actual real data, although you >> do have the option of loading your own time series and letting google >> correlate that for you. >> Still, I fail to understand why searches for "ibuprofen" would map to >> searches for "gateway bible" with r=0.949. >> --Doug >> >> On Fri, Sep 30, 2011 at 3:10 PM, Robert J. Cordingley >> <[hidden email]> wrote: >>> >>> I thought the example Doug sent was a correlation between the frequency >>> of search terms, not with any actual real data. I always thought >>> correlating time based data was difficult since as usage rises in general >>> one might expect search term frequency to rise together too. Real >>> statisticians can chime in here. >>> >>> 'santa fe' was interesting in showing an annual cycle presumably >>> corresponding to people's interest in finding info on a tourist >>> destination. >>> >>> Drawing was interesting in that it seems one could always find some >>> search term frequencies to correlate with any shape curve. >>> >>> Conclusion: it was fun. >>> >>> Thanks >>> Robert C >>> >>> On 9/30/11 2:36 PM, Douglas Roberts wrote: >>> >>> That's the one I was using, Tom. >>> --Doug >>> >>> On Fri, Sep 30, 2011 at 2:34 PM, Tom Johnson <[hidden email]> wrote: >>>> >>>> Is this any help? https://www.google.com/trends/correlate/ >>>> >>>> -tj >>>> >>>> >>>> >>>> On Fri, Sep 30, 2011 at 9:53 AM, Douglas Roberts <[hidden email]> >>>> wrote: >>>>> >>>>> Has anybody been able to get anything useful out of that thing? Most >>>>> of the items I've searched for return totally bizarre results. For example, >>>>> searching on ibuprofen with the intent to see any correlations with >>>>> influenza give this as one of the higher-correlated results: >>>>> >>>>> http://www.google.com/trends/correlate/search?e=ibuprofen&e=gateway+bible&t=weekly >>>>> WTF? Does this show that on-line religion causes headaches? >>>>> Seriously, has anybody found this tool to be of any practical use? >>>>> --Doug >>>>> >>>>> -- >>>>> Doug Roberts >>>>> [hidden email] >>>>> [hidden email] >>>>> http://parrot-farm.net/Second-Cousins >>>>> 505-455-7333 - Office >>>>> 505-670-8195 - Cell >>>>> >>>>> ============================================================ >>>>> FRIAM Applied Complexity Group listserv >>>>> Meets Fridays 9a-11:30 at cafe at St. John's College >>>>> lectures, archives, unsubscribe, maps at http://www.friam.org >>>> >>>> >>>> >>>> -- >>>> ========================================== >>>> J. T. Johnson >>>> Institute for Analytic Journalism -- Santa Fe, NM USA >>>> www.analyticjournalism.com >>>> 505.577.6482(c) 505.473.9646(h) >>>> http://www.jtjohnson.com [hidden email] >>>> ========================================== >>>> >>>> ============================================================ >>>> FRIAM Applied Complexity Group listserv >>>> Meets Fridays 9a-11:30 at cafe at St. John's College >>>> lectures, archives, unsubscribe, maps at http://www.friam.org >>> >>> >>> >>> -- >>> Doug Roberts >>> [hidden email] >>> [hidden email] >>> http://parrot-farm.net/Second-Cousins >>> 505-455-7333 - Office >>> 505-670-8195 - Cell >>> >>> >>> ============================================================ >>> FRIAM Applied Complexity Group listserv >>> Meets Fridays 9a-11:30 at cafe at St. John's College >>> lectures, archives, unsubscribe, maps at http://www.friam.org >>> >>> -- >>> Robert Cordingley >>> Web Development >>> cirrillian.com >>> 505-471-4569 (office) >>> 281-989-6272 (cell) >> >> ============================================================ >> FRIAM Applied Complexity Group listserv >> Meets Fridays 9a-11:30 at cafe at St. John's College >> lectures, archives, unsubscribe, maps at http://www.friam.org > > > ============================================================ > FRIAM Applied Complexity Group listserv > Meets Fridays 9a-11:30 at cafe at St. John's College > lectures, archives, unsubscribe, maps at http://www.friam.org > ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
Right, and a million monkeys banging away at typewriters could replicate all of Shakespeare's works over the next 10 years.
But come on: a 0.949 correlation between "ibuprofen" searches and "gateway bible" searches over the past 7 years? Sorry, either google's data is fubar'd or something else has yet to be explained.
--Doug On Mon, Oct 3, 2011 at 4:24 PM, Greg Sonnenfeld <[hidden email]> wrote:
Given an extremely large data set, wherein the long term trend of the ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
Administrator
|
In reply to this post by Greg Sonnenfeld
On Mon, Oct 3, 2011 at 4:24 PM, Greg Sonnenfeld <[hidden email]> wrote:
... This sounds interesting, but my math chops aren't up to understanding the details. Could you expand on this a bit? Sounds interesting.
-- Owen ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
In reply to this post by Douglas Roberts-2
The comic book documentation is actually pretty good. It was formed out of
the original idea that one could predict outbreaks
of sickness by tracking things like searches for medicine. This makes
sense (and works pretty well).
That admitted, given a sufficient number of variables, many, many things will correlate due to chance. Undoubtedly lots of other things will also correlate with any given data set, and most will be nonsensical. Thus, I'm not sure this is the type of program that can be used casually and result in accidental amazing discoveries. When I was poking around I noticed that a search for the word "Honda" had several hits for Yamaha motorcycles high on the list. That seems like the type of info that might be useful for those two companies to know. Apparently several things that might seem to be immensely popular are also the result of simple trends in usage (for example web usage). The popularity of xkcd tracks almost perfectly the popularity of youtube song searches and several craig's list sites. That seems like the type of information that would be useful to someone trying to tease apart specific effects of website popularity from general increases in usage. Other interesting things: I noticed that a search for "Glenn Beck" had "Glen Beck is an idiot" as the 10th highest correlation (r = .91). The graph suggests that from 2009 onward, a good number of the people interested in Glenn were interested because they thought he was an idiot (the correlation before that is much weaker). I searched as far down as r = .65 for Jon Stewart and Stephen Colbert and found no similar correlations. I also noticed that Google has a pretty decent naughty word filter in place. Not only does mysteriously find nothing if you search for naughty words, it doesn't put naughty things in the correlated words list. For example, trying to correlate "glory" or "sanchez" doesn't pull up anything of interest, despite the fact that it clearly should. Also, searches for "full facial" mostly peaked around reports of "full face transplants". That doesn't seem useful, but I thought it was an interesting built in limitation. Eric On Mon, Oct 3, 2011 12:19 PM, Roger Critchlow <[hidden email]> wrote: Eric Charles Professional Student and Assistant Professor of Psychology Penn State University Altoona, PA 16601 ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
Shock but not surprise:
Google Correlates with "Gppg;e" or off-home left hand? https://www.google.com/trends/correlate/search?e=google&e=gppg+e&t=weekly# Most of my Correlate results show a strong annual periodicity... I'm curious if others see this and if so what their speculations might be about it.. https://www.google.com/trends/correlate/search?e=egypt&e=about+egypt&t=weekly# What might make egypt less interesting to global searchers during the summer and winter solstices? This observation lead me to try to use Trends opposite Corrolate: http://www.google.com/trends?q=egypt%2C+solstice&ctab=0&geo=all&date=all And to work with a term (solstice) which is clearly going to have a (bi)annual periodicity. I'm waiting for Google Speculate, I wonder what that will look like? ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
There is either something seriously flawed with the G-corr tool or something flawed with the underlying data.
. For instance if "gateway bible" has the highest correlation with "ibuprofen" then we could expect that the reverse would be true. I suspect the tool is OK but the data is off (or requires more filtering). IMHO a classic instance of GIGO. On Tue, Oct 4, 2011 at 10:03 PM, Steve Smith <[hidden email]> wrote: Shock but not surprise: ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
Free forum by Nabble | Edit this page |