|||

Automate Feed Extraction and Posting it to Twitter

I wrote a small Python script to automate feed extraction and posting it to Twitter. The feed? My feed burner feed for this site. The script is super simple and uses the Twython, Pandas, and Feedparser libraries. The rest, I believe, comes stock with your Python 3 installation.

How it works

It’s really a simple process. First it takes a feed, then parses it, does a few things to it, saves it to a CSV file, and text file. Then it opens the text file (not elegant , I know) and randomly selects a row (blog post), and posts it to Twitter. I use this script to compile all my blog posts into a handy link formatted list that I use in my Newsletter.

There are a few things you need to have to run the Twitter part of the script. You’ll need to create an API key from Twitter and use your Consumer and Secret tokens to run the Twitter posting.

A lot of this script came from my original one here. I just added in some time filtering procedures. For example:

today = DT.date.today()
week_ago = today - DT.timedelta(days=7)
month_ago = today - DT.timedelta(days=60)

Sets up the time for today, a week ago and a month ago from today’s date. I use them to set my time filter in section 4 below and get a list of blog posts posted in the last 7 days or 60 days. You may change these values to your heart’s content. The script is open source and free to use anyway you want.

Python Code


# coding: utf-8

# In[1]:


#!/usr/bin/env python
import sys
import os
import random

from twython import Twython, TwythonError

from keys import dict
import datetime as DT
import feedparser

import pandas as pd
import csv


# In[2]:


#Set Time Deltas for tweets

today = DT.date.today()
week_ago = today - DT.timedelta(days=7)
month_ago = today - DT.timedelta(days=60)

t = today.timetuple()
w = week_ago.timetuple()
m = month_ago.timetuple()


# In[3]:


#Parse the Feed
d = feedparser.parse('https://www.neuralmarkettrends.com/feed/')


# In[4]:


#Create List of Feed items and iterate over them. 
output_posts = []
for pub_date in d.entries:
    date = pub_date.published_parsed
    #I need to automate this part below
    if date >= m and date <= t:
        print (pub_date.title   ' : '   pub_date.link)
        tmp = pub_date.title,pub_date.link
        output_posts.append(tmp)


# In[5]:


#print(output_posts)

#Create Dataframe for easy saving later
df = pd.DataFrame(output_posts, columns = ["Title", "RawLink"])


# In[6]:


date_f = str(DT.date.today())
#f = open (date_f   '-posts.md', 'w')
f = open ('60daytweet.txt', 'w')
for t in output_posts:
    line = ' : '.join(str(x) for x in t)
    f.write(line   '\n')
f.close()


# In[7]:


#Create Preformated link to export to CSV
df['Hyperlink'] = '<a href="' df['RawLink'] '">' df['Title'] '</a>'


# In[8]:


#Write to CSV for Newsletter
df.to_csv("formated_link_list.csv", quoting=csv.QUOTE_NONE)


# In[9]:


#Initialize automatic Twitter posting of random blog article

#Add your Twitter API keys by replacing 'dict['ckey|csecret|atoken|asecret'] with the applicable values like so 'XXXXXX'

CONSUMER_KEY = dict['ckey'] 
CONSUMER_SECRET = dict['csecret']
ACCESS_KEY = dict['atoken']
ACCESS_SECRET = dict['asecret']

f = open('60daytweet.txt', 'r')

lines = f.readlines()
f.close()

post = random.choice(lines)

api = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)

try: 
    api.update_status(status=post)
except TwythonError as e:
    print (e)

Note: This code was first sent out as part of my monthly newsletter. You can sign up for it here.

Up next Flux: A Machine Learning Framework for Julia There was a HUGE announcement on the Julia blog a few days ago. The convergence of a language for machine learning and marrying it with a compiler Changing Pinboard Tags with Python Welcome to another automation post! This is a super simple Python script for changing misspelled or wrong tags in your Pinboard account. I started
Latest posts The Ye Old Blog List Motorola: Then and Now EWM Redux Testing for mean reversion with Python & developing simple VIX system - Talaikis unsorted - Tadas Talaikis Blog Steps to calculate centroids in cluster using K-means clustering algorithm - Data Science Central Basics of Statistical Mean Reversion Testing - QuantStart Algorithmic trading in less than 100 lines of Python code - O’Reilly Media Interpreting Machine Learning Models Microsoft the AI Powerhouse Investing in the S&P500 still beats AI Trading Microsoft makes a push to simplify machine learning | TechCrunch 10 Great Articles On Python Development — Hacker Noon Introduction to Keras Democratising Machine learning with H2O — Towards Data Science Getting started with Python datatable | Kaggle Phone Addiction Version 12 Launches Today! Machine Learning Making Pesto Tastier 5 Dangerous Things You Should Let Your Kids Do The Pyschology of Writing Investing in 2019 and beyond TensorFlow and High Level APIs Driving Marketing Performance with H2O Driverless AI Machine Learning and Data Munging in H2O Driverless AI with datatable Making AI Happen Without Getting Fired Latest Musings from a Traveling Sales Engineer The Night before H2O World 2019 Why Forex Trading is Frustrating Functional Programming in Python Automatic Feature Engineering with Driverless AI Ray Dalio's Pure Alpha Fund