|||

Python Script to Parse Blog Feeds Version 2

I’m going to share an update to my original python script. It’s super simple and it completely automates parsing multiple blog feeds for you to autopost on Twitter.

The goal: automation!!!

Call the Python Modules!

Create a new empty python script. Call it awesomescript.py or something else. You’ll need to have feedparser and twython installed first. If you don’t go and do pip install feedparser’ and pip install Twython.’


import feedparser
import sys
import os
import random

from twython import Twython, TwythonError

from keys import dict

The one thing you’ll notice is I’m calling a python file called keys’ and importing a dictionary. All that this does is call my Twitter API keys from one central file. I do this because I have multiple scripts that call the same keys and instead of pasting them in each an every script, I just centralized it. Plus, it makes it easier to change the keys if I ever rate block myself, which is a fairly common occurrence with all my tinkering!

Twitter API Key dictionary

Ok, you need to get your API keys from Twitter. Do that first. Google how to do it. Then create a file called keys.py’ and paste the following into it. Then paste in your Consumer Key, Consumer Secrete, Access Token, and Access Secret into where the XXXXXXXXXXXXXXXs are.


dict = {'ckey': 'XXXXXXXXXXXXXXX', 'csecret': 'XXXXXXXXXXXXXXX', 'atoken': 'XXXXXXXXXXXXXXX', 'asecret':'XXXXXXXXXXXXXXX}

Calling the Dictonary of keys

Switch back to your main script and call in the key.py file by doing this:

CONSUMER_KEY = dict['ckey']
CONSUMER_SECRET = dict['csecret']
ACCESS_KEY = dict['atoken']
ACCESS_SECRET = dict['asecret']

Create the Feedlist you want to parse the RSS feed from

You’ll have to create a list of RSS feeds from where you want to parse the posts from. To do that you need to create a list in Python by doing something like myvariable = []’

feedlist = ['https://www.someurl.com/feed',
           'https://www.someurl2.com/feed',
           'https://www.someurl3.com/feed',
           'https://www.someurl4.com/feed',
           'https://www.someurl5.com/feed']

Make a Random Feed Selection and Parse the Feed

Next we’re going to randomly select a feed and then parse it via the feedparser library.

select_list = random.choice(feedlist)

d = feedparser.parse(select_list)

Extract the feed length and randomly select a Feed

This part is where I made the biggest change. I now parse the number of feeds and then automatically assign that number to a variable. This way I can create the correct feed range on the fly’ because each feed is different. Some sites only give you 5 current feeds, others give you 20, etc. In my old feed, I just hard coded in a range value. That worked but wasn’t very dynamic.

feedlen = len(d['entries'])

num = random.randint(0,feedlen)

Initialize Twitter API, Write the Status and Tweet Out!

The rest didn’t change much at all. You just initialize the Twitter API and write a status that appends the parsed feed information.

api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)

status_text = d['entries'][num]['title'] + ' link: '+ d['entries'][num]['link']

try: 
    api.update_status(status=status_text)
except TwythonError as e:
    print (e)

There you have it. A better version of my original script. The next goal is create a list of hashtags to randomly select against.

As always, please drop me a comment if you have questions.

Up next Unsupervised Machine Learning Translation Millennials are Bailing on their jobs I find this somewhat older post fascinating. It would appear that more and more millennials are waking up and realizing that there’s more to life
Latest posts The Ye Old Blog List Motorola: Then and Now EWM Redux Testing for mean reversion with Python & developing simple VIX system - Talaikis unsorted - Tadas Talaikis Blog Steps to calculate centroids in cluster using K-means clustering algorithm - Data Science Central Basics of Statistical Mean Reversion Testing - QuantStart Algorithmic trading in less than 100 lines of Python code - O’Reilly Media Interpreting Machine Learning Models Microsoft the AI Powerhouse Investing in the S&P500 still beats AI Trading Microsoft makes a push to simplify machine learning | TechCrunch 10 Great Articles On Python Development — Hacker Noon Introduction to Keras Democratising Machine learning with H2O — Towards Data Science Getting started with Python datatable | Kaggle Phone Addiction Version 12 Launches Today! Machine Learning Making Pesto Tastier 5 Dangerous Things You Should Let Your Kids Do The Pyschology of Writing Investing in 2019 and beyond TensorFlow and High Level APIs Driving Marketing Performance with H2O Driverless AI Machine Learning and Data Munging in H2O Driverless AI with datatable Making AI Happen Without Getting Fired Latest Musings from a Traveling Sales Engineer The Night before H2O World 2019 Why Forex Trading is Frustrating Functional Programming in Python Automatic Feature Engineering with Driverless AI Ray Dalio's Pure Alpha Fund