Capturing Tweets
Locked
 
|
I was wondering if anyone on here has, or is familiar with, capturing large numbers of twitter tweets and how one would go about doing it. I need about 30k tweets, 10k from three different regions. It looks like there should be relatively simple way to do this. I've been making my way though this: https://dev.twitter.com/docs/streaming-api/methods#count but haven't found what I'm looking for.
What we're hoping to do is a type of sentiment analysis, and look at how that compares to some polling data taken from these regions (I have a feeling what we'll find, but the project itself is interesting). Any insight is appreciated, or if anyone might have an interest in collaboration on the project. Thanks Aaron Perls
Galisteo Consulting Group, Inc. 2403 San Mateo Blvd NE, Suite W-12 Albuquerque, NM 87110 505.917.6447 cell 505.889.3927 voice 505.889.3929 fax www.galisteoconsulting.com ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
|
So, the question is: Are the sentiments of birds that tweet a random sample of the sentiments of birds that breath? I confess I do find interesting the question of how one extracts a sentiment from a tweet! N From: [hidden email] [mailto:[hidden email]] On Behalf Of Aaron Perls I was wondering if anyone on here has, or is familiar with, capturing large numbers of twitter tweets and how one would go about doing it. I need about 30k tweets, 10k from three different regions. It looks like there should be relatively simple way to do this. I've been making my way though this: https://dev.twitter.com/docs/streaming-api/methods#count but haven't found what I'm looking for.
Aaron Perls ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
|
In reply to this post by Aaron Perls
Aaron Perls wrote circa 11-10-19 03:24 PM:
> I was wondering if anyone on here has, or is familiar with, capturing > large numbers of twitter tweets and how one would go about doing it. > I need about 30k tweets, 10k from three different regions. It looks > like there should be relatively simple way to do this. I've been > making my way though this: > https://dev.twitter.com/docs/streaming-api/methods#count but haven't > found what I'm looking for. > > What we're hoping to do is a type of sentiment analysis, and look at > how that compares to some polling data taken from these regions (I > have a feeling what we'll find, but the project itself is > interesting). > > Any insight is appreciated, or if anyone might have an interest in > collaboration on the project. This code may help: https://code.google.com/p/tircd/ I can imagine setting up 3 different personalities, following a sample of people from a given region for each personality, then logging them either directly from perl or with a plugin to your irc client. Selecting who to follow for each region would be manual and difficult, I think. But these might be helpful: https://dev.twitter.com/docs/places/finding-tweets-about-places https://dev.twitter.com/docs/api/1/get/geo/reverse_geocode -- glen e. p. ropella, 971-222-9095, http://tempusdictum.com ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
|
I've been working on learning how to capture an analyze Twitter data recently so this is timely. I highly recommend the book "Mining the Social Web" by Matthew Russell, as long as you're comfortable with Python (or programming in general, since the examples are enough to get started even if you have to learn Python in the process). You can find it on O'Reilly (http://shop.oreilly.com/product/0636920010203.do) or Amazon (http://www.amazon.com/gp/product/1449388345?tag=oreonbl-20). I'm going through it now. Another potential source for tweets is Infochimps (http://www.infochimps.com/). They've got a good bit of readily available data from Twitter, although I'm not sure if they have actual tweets
available for free. Some data they do charge for access to. Aaron, if you're interested, I can put you in touch with one of my professors who has a company that does a whole lot of analysis of Tweets (although they're located in the Washington, DC area). Brent From: glen e. p. ropella <[hidden email]> To: The Friday Morning Applied Complexity Coffee Group <[hidden email]> Sent: Wednesday, October 19, 2011 7:21 PM Subject: Re: [FRIAM] Capturing Tweets Aaron Perls wrote circa 11-10-19 03:24 PM: > I was wondering if anyone on here has, or is familiar with, capturing > large numbers of twitter tweets and how one would go about doing it. > I need about 30k tweets, 10k from three different regions. It looks > like there should be relatively simple way to do this. I've been > making my way though this: > https://dev.twitter.com/docs/streaming-api/methods#count but haven't > found what I'm looking for. > > What we're hoping to do is a type of sentiment analysis, and look at > how that compares to some polling data taken from these regions (I > have a feeling what we'll find, but the project itself is > interesting). > > Any insight is appreciated, or if anyone might have an interest in > collaboration on the project. This code may help: https://code.google.com/p/tircd/ I can imagine setting up 3 different personalities, following a sample of people from a given region for each personality, then logging them either directly from perl or with a plugin to your irc client. Selecting who to follow for each region would be manual and difficult, I think. But these might be helpful: https://dev.twitter.com/docs/places/finding-tweets-about-places https://dev.twitter.com/docs/api/1/get/geo/reverse_geocode -- glen e. p. ropella, 971-222-9095, http://tempusdictum.com ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
Administrator
|
Way useful info, thanks!
Tom Johnson has been working on the Open Data project and may be able to apply some of these methods to it. Also, several of us around the Santa Fe Complex have been following the Stanford Machine Learning and/or Artificial Intelligence courses so are interested in this sort of thing.
So keep us posted on your work, sounds very interesting.
-- Owen
On Wed, Oct 19, 2011 at 11:29 PM, Brent Auble <[hidden email]> wrote:
============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
|
In reply to this post by Aaron Perls
Nick,
Did anyone ever try to answer your question? My guess is that they are trying to do something fairly simple, like comparing the number of people who tweet "Go Obama!" vs. the number who tweet "F*@% Obama!". If that is not what "sentiment analysis" meant, I would be very interested in knowing as well. Also, I'd be interested to know if there is any attempt to identify sarcastic "positive" tweets. Eric On Wed, Oct 19, 2011 06:50 PM, "Nicholas Thompson" <[hidden email]> wrote: Eric Charles Professional Student and Assistant Professor of Psychology Penn State University Altoona, PA 16601 ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
|
|
Actually, I can't believe I forgot to mention this. A couple of researchers, Scott Golder and Michael Macy (a name that might be familiar to some folks here) at Cornell, did some large-scale mining of tweets and looked at patterns of word use throughout the day around the world. Here's the info about the paper:
Scott A. Golder and Michael W. Macy. (2011) Diurnal and Seasonal Mood
Vary with Work, Sleep and Daylength Across Diverse Cultures. Science. 30 Sep 2011.
[ main text http://www.sciencemag.org/cgi/rapidpdf/333/6051/1878?ijkey=FeSTyP0rXFpqA&keytype=ref&siteid=sci (pdf) | supplementary materials http://www.sciencemag.org/content/suppl/2011/09/28/333.6051.1878.DC1/Golder.SOM.pdf (pdf) | extras http://redlog.net/timeuse/ (web) ] Also, check out the site they put up allowing people to search for words and then graph out when those words are most frequently used: http://timeu.se/ By they way, if anyone is more comfortable with R than with Python, here are some links on how to use R for gathering Twitter data: Using R to search
Twitter for analysis http://www.franklincenterhq.org/2429/using-r-to-search-twitter-for-analysis/ Text Data Mining with Twitter and R http://heuristically.wordpress.com/2011/04/08/text-data-mining-twitter-r/ And Drew Conway has a bunch of good material on his blog about getting and analyzing Twitter data with both Python and R: http://www.drewconway.com/zia/ Brent From: ERIC P. CHARLES <[hidden email]> To: Nicholas Thompson <[hidden email]> Cc: 'The Friday Morning Applied Complexity Coffee Group' <[hidden email]> Sent: Thursday, October 20, 2011 8:00 PM Subject: Re: [FRIAM] Capturing Tweets Nick,
Did anyone ever try to answer your question? My guess is that they are trying to do something fairly simple, like comparing the number of people who tweet "Go Obama!" vs. the number who tweet "F*@% Obama!". If that is not what "sentiment analysis" meant, I would be very interested in knowing as well. Also, I'd be interested to know if there is any attempt to identify sarcastic "positive" tweets. Eric On Wed, Oct 19, 2011 06:50 PM, "Nicholas Thompson" <[hidden email]> wrote: Eric Charles Professional Student and Assistant Professor of Psychology Penn State University Altoona, PA 16601 ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org ============================================================ FRIAM Applied Complexity Group listserv Meets Fridays 9a-11:30 at cafe at St. John's College lectures, archives, unsubscribe, maps at http://www.friam.org |
«
Return to Friam
|
1 view|%1 views
| Free forum by Nabble | Edit this page |