Live Blogging the 2020 Election with Scalable Google Trends

7:15 pm PDT update

Well, things are moving right along in this election. I’m going to try to keep these short and sweet to avoid devolving into political commentary lol.

First observation, people are really focusing on “battleground” states and North Carolina, Michigan, Wisconsin are all in focus. Also people are focusing on the Senate and who will have control of that after the election is over (see below.)

Also, a civil unrest query has popped into the top rising queries as the election starts to shake out. Since it’s looking at current civil unrest it seems people just want to check in and make sure everything is still going “smoothly.”

Will be back in a bit as I’m kind curious how this develops, though made need some whiskey and a cheeseburger. In which case I will see you tomorrow.

6:28 pm PDT update

Just wanted to see how quickly the data was changing and boy is it fast!!!! Don’t know how people keep up with this without robots. Anyway, in the national data Texas election results have started to come online as strongly as Ohio. While Florida may Florida, it seems like national attention has turned towards this two battleground states.

Looking at individual rising queries New Zealand based Predictit has been showing strong all day. Everyone wants predictions it seems! I just want to watch the data come in. Also, like I mentioned before, people are brand devoted to Fox News. Lastly, Arizona has already started showing up in individual queries, so I would expect to start surging as it’s early results start to come online (given how recent polling has been there.)

6:03 pm PDT update

Famous last words!!! So I vroomed this into a bunch of errors, but am back in businesses now as our team built a robust system and now I know to scale our instances before doing that 🙂

Alright, so first let’s talk about state level data! This is the first time I’m going to report on the state election query set I ran through trends. Though suffice it to say that people care a lot about swing states as you can see by their disproportionate representation to population like Virginia, Georgia and Vermont even being in the same ballpark as Florida when it comes to population

Also it seriously feels like deja vu with Florida

Now let’s talk about the national data set. This stuff is starting to look really cool! Nationally, the things that are seeing off the charts, turn it up to 11 growth are all local. First up, Ohio. It’s showing up big in terms of both the count and growth of it’s terms as illustrated below:

But also, we are seeing two Texas counties showing up as queries that are skyrocketing (gonna run out of hyperbole to discuss query growth soon.). I have never heard of Bexar or Tarrant county in Texas before but Heckler assures me they exist! They are seeing an increase, per Google Trends, of almost 500,000% growth at the query level. Bonkers.

Also, Broward county Florida, famous as the problem of every national election in recent memory, is also showing up in query data

A couple of last general observations:

The growth of top rising queries is several times bigger than it was even this morning. Basically, A LOT OF PEOPLE are searching right now.

Fox News has some crazy “loyalty”

4:03 pm PDT update

Alright well it’s after work on the east coast and things are starting to pop in the data. Let’s start with general trends with a nice word cloud

Local issues are starting to pop in the national election data set. Here I have drawn attention to prop 22, a proposition here in California over the regulation of gig economy workers as well as governors races. However, the big dominating theme is Trump’s rally tonight, but more on that later.

This is also the first time I have looked a set of ~1k state election specific terms. Nothing much to report on that data set yet except that there is a lot of searching for historical electoral maps that got surfaced here (screenshot below). I guess people are trying to game this out at home?

And in term of just the top rising queries in the data set, well that is all coming up YUGE for Trump. Also, I think it’s kinda funny that Lil’ Pump has finally surfaced in this trends data so late in the day. Curious how this will progress as the night goes on, but my guess is Trump will work hard to drive search demand. We shall see!

1:17 pm PDT update

Alright party people, back after some delay as I wanted to give the data some time to change as user behavior and the events on the group have changed. And I’m glad I did because the data is radically different then it was previously. A quick reminder that before Biden vs. Trump was big in the rising ngrams (see previous update below.) Well know that trend has shifted to the Vice Presidential Candidates and Kamala Harris is cleaning up (go CA)

Also, if you back out into the top level word cloud you can see how different things have changed between

Now

This morning

So I expect things to continue to get super interesting as we head into this evening. Speaking of interesting, lots of Trump queries are highly searched queries, including “fox news trump” which could be in relation to his Fox & Friends this morning or a navigation query to Fox’s Trump related coverage.

Not feeling good that 538 continues to insinuate itself into the American election psyche. Oh also, the fly is back

10:29 am PDT update

Alright, well I have some news! First of all as we get later in the day I’m going to start segmenting out a seed keyword list of state and congressional election results so we can track what is going on with those as well.

Speaking of the data, you heard it hear first: Biden’s numbers are better then next to Trump’s. There are more Biden name related queries that are being searched at a greater rate of increase than Trump.

Also, queries around political violence continue to show up, but since these aren’t breaking through in terms of individual queries trending my guess is that is a self-perpetuating fear where people are doom searching.

Lastly, just look at these rising queries. First of all the Drudge and Real Clear are the two top trending news sources in this data set is very troubling. Second, who cares about either Barron Trump or Kim Kardashian just like in general but especially today. Lastly, “when will the next presidential election be” is a top 15 trending query. If only we planned these things at a regular clip!

8:17 am PDT update

Alright, things are really starting to get popping now. The midwest has come fully online and we can see people searching for their states specific results starting to bubble up in the trends data (Hi Michigan and Kentucky!)

Are some of the top queries showing the state specific results

And here is the rising in the ngrams (e.g. full topic areas)

I would expect to see state specific results start to dominate as the day goes on. Though of course that could be overtaken by whatever narrative is happening around the election itself as well.

Speaking of, violence has finally broken through to be able to be visualized in a top ngram (bottom right) as well as civil war (top left). American’s are keeping it spicy this am!

For anyone wanting to play around at home, feel free to check our GDS report here

7:18 am pdt update

This is probably going to be the least interesting update of the day. There isn’t enough movement in the midwest yet for that to really crack into trends. The trending ngrams are relatively the same. Most interesting pieces are Florida has officially entered the chat

and that election riots is trending downward (thankfully)

With the next update things should start to get interesting as the west coast is going to start coming online. I’m also going to release a version of the GDS dashboard for people to play with that will have our 2 test data pulls from yesterday and the first two data pulls of this am.

6:14 pdt update

Well, now that the east coast has started to full come online the main rising ngrams are dominated by the polls. No real surprise there as a morning baseline check-in for what is sure to be a wild day makes a tone of sense.

Also, individual rising queries have seen some drop off of net worth and age related questions as individual polling queries start to dominate. No idea what the Trump Train Song 2020 is ¯_(ツ)_/¯.

6:40am PDT update

There are some pretty good deep cuts in the top 50 rising queries in the data set. I think the middle class question is really interesting as it’s generally a deeper level of question then I would expect today. And of course we are seeing our first hint of people searching about civil unrest.

5:17am  PST update

Just ran the tool to capture 8am EST trends data. So far early searcher trends are looking to catch the latest poll numbers, which is unsurprising. Though I do think it curious how focused they are on candidates ages and net worth. As this beginning to develop I expect this data to be rather…interesting.

Wow, it’s November 3rd 2020! That means election day in the US, and what an election. I’m going to eschew politics except to say GO VOTE. We have the day of at LSG, but I figured this would be a great time to deploy a new tool and to do so i’m going to live blog and tweet the election.

Specifically we hooked up Google Trends to our data analysis stack, built an elastic load balancer to be able to scale jobs with cost and now it’s just party time. And when I say party time I mean we can pass thousands of keywords into Google Trends, get them back hourly, process the keywords as NGRAMs to better understand what they are about.

Okay, first start here with these two fantastic articles by Ruth Everett:

An Introduction to Using Google Trends with Python

Visualizing Python and Google Trends

Okay, now that you have read those articles let me just show you around. First we started with a manually curated seed keyword list, then ran it through a Google scraper of ours to pull related searches. We took that list and ran it through Google Trends and pulled back the top and rising terms as well as their scores. We then ran that through the NLTK ngram library to be able to get categories/topics/themes. Then we data warehouse and dumped it into GDS. Screens below:

This is a view of the ngrams of rising queries from our seed list visualized in Google Data Studio

Ngrams of top queries in Google Data Studio

As you can see themes and topics!

Also, we have the ability to drill down to see the terms that make up the ngrams

This screenshot below is a drill down into “results 2020”

Cool right?

Anyway, I’m going to be updating this every hour and sharing any cool info on this liveblog and on twitter. Also for those wondering, the data set contains ~3,000 keywords.