PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Saturday, July 9, 2022

[FIXED] How can I python twitter crawling (scraping) several keyword

 July 09, 2022     keyword, python, twitter, web-crawler     No comments   

Issue

I wrote the code.

But I don't think it's going to work.

I want to extract words from the concept of " or " rather than the concept of " and ".

It seems like only ' keyword 1 ' is extracted.

How do I make corrections?

import tweepy
import time
import os

search_term = 'keyword1'
search_term2= 'keyword2'

lat = "37.6"
lon = "127.0"
radius = "200km"
location = "%s,%s,%s" % (lat, lon, radius)


API_key = "11111"
API_secret = "22222"
Access_token = "33333"
Access_token_secret = "444"


auth = tweepy.OAuthHandler(API_key, API_secret)
auth.set_access_token(Access_token, Access_token_secret)

api = tweepy.API(auth)

c=tweepy.Cursor(api.search,
            q=(search_term or search_term2),
            rpp=1000,
            geocode=location,
            include_entities=True)

data = {}
i = 1
for tweet in c.items():
    data['text'] = tweet.text
    print(i, ":", data)
    i += 1
time.sleep(1)


wfile = open(os.getcwd()+"/twtw2.txt", mode='w')   
data = {}   
i = 0       

for tweet in c.items():
    data['text'] = tweet.text   
    wfile.write(data['text']+'\n')  
    i += 1
    time.sleep(1)

wfile.close()

Solution

Maybe change this line

q=(search_term or search_term2),

to

q="{}+OR+{}".format(search_term,search_term2),
  • Case matters here for the OR operator
  • enter q as a string, not as an expression that is short-circuit evaluated

By the way, your credentials (from your post) also work for me.



Answered By - knb
Answer Checked By - Cary Denson (PHPFixing Admin)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing