Automate your daily market update

Using python to generate a table of news for your search /study/market updates

Photo by Jess Torre on Unsplash

News articles hold many useful data on the driving forces that are impacting the markets we are interested in. If you are trading the stock market, you would like to know what is out there in the company you follow. If you study hydrogen development, you will be interested in the news articles that discuss the number of new projects planned and their capacities

Motivation

Recently, I was asked to monitor updates on some oil and gas assets in a whole region. The assets were many and getting updates on every single one of them through eh conventional search would have been killing and time-consuming. That is why I went to develop this python tool to extract news required to update my assets.

How o use Python to generate news articles in a clean format table.

What libraries do we need?

• GoogleNews: to search and extract search results.
• pandas: to format the results of your google news search into a dataframe.

How does the algorithm work?

1. search each item required to update and extract the results of the search
2. Save the search results in a dataframe
3. loop over the search items and concatenate the resulting dataframe with the new search articles.

What functions did we use?

First: the news_search function

• Function input: the function takes in the item we would like to search
• Function output: generate a dataframe of the resulting news.

def search_news(item):
    
    """
    function that takes in a search word and return a dataframe
    """
    googlenews = GoogleNews()
    googlenews.search(item)
    result=googlenews.result()
    df=pd.DataFrame(result)
    return df

Second: news_on_units

the purpose of the function is to loop the names of units or fields(search items) and search news

function input = list of column names for the resulting dataframe and list of the names of search items

output = dataframe with news and names of units

def news_on_units(col_names, search_list):    
    
    """
    the purpose of the function is to get names of units or field and search news 
    
    input = list of column names 
            search list that has the entities/fields/units we want to search
    output = dataframe with news and names of units 
    
    """
    
    #create the dataframe considering the name of columns
    final_df = pd.DataFrame(columns = col_names)
    
    #name the googlenews function
    googlenews=GoogleNews()
    
    #construct the search loop
    for item in search_list:
#apply the search news function on each item
        df = search_news(item)
# add a column to represent the name of the field
        #df['field'] = gas_field
        
        #print(df.head())
        #combine the resulting dataframe for each search into the final dataframe
        final_df = pd.concat([final_df, df], axis = 0, ignore_index = True)
# return the final dataframe
    x = final_df
    return x

The outcome is as below:

Image by the author

You can find the python notebook here.

You can go one step further and analyze the results of the titles of the news articles of your interest which we will present in the coming article.