Category: MIB Mood Table

(Halfway) Around the World in 18 Days

(Halfway) Around the World in 18 Days

Greetings! I’m excited for some of the technical posts that I’m working on, but before I’m able to publish those I wanted to share the details of my speaking schedule for May. I’m incredibly fortunate to have these speaking opportunities, and I’m incredibly excited to share them with you and to share my presentations with the attendees at these events!

My epic May begins this weekend with SQL Saturday Jacksonville on May 5th. I’m looking forward to catching up with some friends in the area, but I’m also looking forward to my first presentation in Florida! I’ll be presenting my “How to Keep Your Database Servers Out of the News” session. I really enjoy this session because it lends itself to a lot of interactivity with the group as we talk through various challenges people have had and the questions those challenges bring to their mind. If you’re attending, I look forward to seeing you and hearing your questions. If you’re not attending, click here to register and I’ll see you there!

The following weekend, on May 12th, I’ll be presenting at SQL Saturday Finland in Helsinki. It is an understatement to say that I’m excited for this one. My wife has requested that I bring Kimi Raikkonen home with me, and while I’m pretty sure she’s going to be disappointed in my failure to do that, I’m thrilled that I’m meeting my goal by speaking there! I set a personal goal to do at least one international presentation in 2018 and I’m incredibly grateful to the organizers of SQL Saturday Finland for selecting my session on “New Features and New Speed in SQL Server 2016 (and 2017) Always On Availability Groups”. I last presented this session at SQL Saturday Cleveland in February and it went really well and seemed to help some folks with challenges they were having, so I’m excited to bring this one to an international audience. If you’d like to register, click here to do that.

Following my presentation in Finland, I’m hopping a quick 3-hour flight to England to present “Feelings Quantified – Ranking Football Clubs By Supporter Sentiment” to Tech Nottingham. I’m thrilled that I was able to work this out with the organizers and they’ve been absolutely wonderful to me as we’ve worked to get this setup. This will be the second time I present on the Azure Logic Apps and Azure SQL DB guts of the Men in Blazers Mood Table I blogged about here in December and the first time it will be to a crowd who calls it football instead of soccer. 🙂 When I arrive in London on 5/13 I’m taking a few hours out, before hopping the train to Nottingham, to catch Tottenham’s (my favorite English football club) final match of the season and final match at Wembley Stadium before moving to their new stadium in the fall, so it’s going to be a soccer-ful/football-ful couple of days! Come On You Spurs! If you’re interested in learning more about this event, information can be found here.

After that journey, I head back stateside for a couple days off in New York City before presenting my “Data To Impress Those That Sign The Checks – Azure Logic Apps, Social Media, and Sentiment Analysis” session at SQL Saturday New York City. This session is the American-ized version of my mood table presentation (less soccer emphasis and slightly more technical focus) so it will be interesting for me to present both versions of this talk a few days apart. Also, it is no exaggeration to say that attending SQL Saturday NYC in 2015 changed the course of my career, so I definitely encourage you to register. Click here to do that. The organizers do a great job with this event, it’s in a great city, and I’m very appreciative of being invited to speak at an event that’s been so significant in my professional growth. I hope to see you there!

Lastly, I wrap up my journey right where I’m sitting as I finish this blog: my home office. IDERA Software has been kind enough to invite me to present a Geek Sync on 5/23 with my “Where Should My Data Live (and Why)?”. This session is great for data professionals in an environment where they’re being encouraged to expand the organization’s data estate to the cloud. It offers several real-world examples of how cloud and on-premises deployments can work together and complement each other. We also go over some pros and cons of the cloud vs. on-premises and dispel some myths as well. I hope to “see” you there. Click here to register and hear my run my mouth for an hour on May 23rd!

I know I keep saying it, but I am grateful to the organizers of all of these events for allowing me to speak to their groups. I can’t wait to meet #sqlfamily from other parts of the world, see places I’ve never been, and hopefully share a little knowledge along the way. Thanks for reading and hope to see you at one of these events!

Men In Blazers PL Mood Table – How It Works

Men In Blazers PL Mood Table – How It Works

Odds are, if you’re reading this, you represent the part of the Venn diagram where my Microsoft data platform buddies and Premier League soccer fans intersect or you clicked the link in today’s Men In Blazers Raven because you were curious how the Premier League mood table works. Rest assured, if you can think of different ways to do this and tweaks to make, I’d love to hear about them in the comments (unless you’re an Arsenal supporter – mostly kidding)! I’m passionate about data solutions and am always trying to stay abreast of rapidly changing platforms and technologies. Feel free to reach out to correct me, congratulate me, or just to connect with me professionally!

Before we go any farther, I owe my buddy Bradley Ball (b|t) a tip of the cap and several cold beverages for the original blog that provided the inspiration behind the Mood Table. If you’re interested in a deeper dive than this high-level overview, click through to my deep dive post here or his original post here.

First off, the foundation of the mood table begins with creating an Azure Logic App (basically a workflow container) for each of the twenty Premier League clubs. Those logic apps contain a Twitter connector that, when active (more on that later), searches Twitter for tweets containing each club’s official Twitter handle. Those tweets are collected by the Twitter connector and fed into the Sentiment Analysis function of the Cognitive Services API found within Azure. The Sentiment Analysis function adds a sentiment score to the tweet (0 is completely negative and 1 is completely positive) and passes it on to the final couple of steps in the process.

Once the sentiment score of the tweet is calculated, the tweet and its corresponding data (text, score, originating account, date created, etc.) is fed into a loop container that separates each club handle in the tweet and connects the tweet to every club tagged within the tweet. Finally, all of this data is stored in an Azure SQL Database (basically a database in the cloud) for persistent storage, querying, and analysis.

It is important to point out that each club’s logic app runs for a 10-minute window around the end of each match. It captures roughly the last 2-3 minutes of match action as well as the 7-8 minutes immediately following the full time whistle. As Rog told me, the intention is to capture the sentiment of tweets akin to when you turn to your mates (or buddies, in ‘Merican) at the pub and offer your instant analysis on your club’s success, failure, or utter mediocrity in the match.

Before I get into a brief description of the query I use to provide this critical information to Rog and Davo, I do want to mention that there was originally a real-time component of the mood table as well. I had written a Power BI report that showed a live table throughout the matches as tweets and their data streamed into the Power BI streaming dataset. After chatting with Rog, though, that was deemed far too optimal for MIB (not to mention quite pricey for my personal Azure account) and we settled on the current sentiment information surrounding the final whistle. It may make an appearance at some point, though!

Once all this Azure coolness has done the heavy lifting, the actual query that compiles the results is fairly straightforward. It takes the average of the sentiment scores tagged to a specific club and then ranks those clubs (and their corresponding average scores) from the highest average to the lowest average. The only data it excludes is tweets that contain only an image (as the sentiment analysis cannot score images – yet). Sadly, your hilarious soccer GIFs are not yet understood by the mood table.

In a nutshell, that’s it. I look forward to comments, questions, and connections. As I mentioned, I have a much more detailed technical deep dive into that that you can read here. This has been a blast to work on and we have some potentially interesting plans for it moving forward. Finally, there is no truth to the rumor that Rog is paying the Russians to create bots to elevate Everton to the top spot. That is just the natural optimism and enthusiasm of Evertonians shining through! To the football…

Men In Blazers Premier League Mood Table – Deep Dive

Men In Blazers Premier League Mood Table – Deep Dive

Welcome nerdy GFOPs! If you’ve made it this far, you’re likely prepared for the nerdiness (and lengthy post) that you are about to dive into. While our assumption was that some GFOPs would be curious, at a high level, about how the Premier League mood table works, my hope was that some GFOPs who work in analytics, data science, or anywhere in the data platform realm would be curious enough about this to dig into what I’ve done, discuss it, and hopefully build upon it to do cool things with the mood table or in our professional work. With that greeting, let me get to a couple acknowledgements and then we’ll get nerdy.

There are two blogs that provided much of the foundation for the mood table. First, the inspiration for the mood table came from a blog written by a friend/former boss of mine that now works at Microsoft, Brad Ball (b|t). Please go check out his blog post to better understand the foundation of the mood table. The second blog that proved to be an immense help can be found here. It was written by Toon Vanhoutte. I don’t know Toon, but I owe him a beer or three for a wonderful blog post laying out, in necessary detail, how to schedule the execution of the Azure Logic Apps. Without being able to schedule the enabling and disabling of these logic apps, the mood table would likely be too expensive to run regularly. So, cheers to you, Brad and Toon! Let’s get into the guts of the mood table.

The foundation of the mood table is an Azure Logic App created for each of the 20 clubs. Because of character limits on some of the object names, I was forced to use creative abbreviations for some of the clubs. Apologies for the name below, Arsenal supporters, but I am a Spurs supporter after all! This screenshot shows what is inside each logic app – I’ll go into more detail about each component below the screenshot.


The first component (labeled “When a new tweet is posted”) is the Twitter connector itself. It signs into Twitter and searches for tweets containing the club’s official Twitter handle. It passes those tweets into the second component (labeled “Detect Sentiment”). That is where the sentiment analysis actually takes place. That component analyzes the text (not images – yet) of the tweet and scores it from 0 (most negative) to 1 (most positive). The sentiment score is added to that dataset and passed to the third component – the for each loop container.

The loop container (labeled “For each 2”) parses each tweet and tags it with all clubs mentioned within the tweet. It then inserts that data into an Azure SQL Database (via the “Insert row” component in red) for storage, querying, and analysis.

When I spun up the mood table for the first time, this is all it was. There was no scheduling component and it was just streaming tweets into my Azure SQL DB during all the matches. That proved to be prohibitively expensive, so let me walk you through some gotchas before we get into the scheduling components of the mood table.

Gotchas (i.e. how to not bankrupt yourself in Azure)

Gotcha 1: Twitter Connector Does Not Die (unless you kill it)

As I first started using the Twitter connector (screenshot below) I noticed that it asked a relatively simple question: “How often do you want to check for items?” Based on my research and discussion with others, our understanding was that the connector would wake up on that defined interval, check for tweets meeting the search criteria, and then shut down again until the next interval.


Unfortunately, that understanding was incorrect. The first full weekend I ran the mood table it cost several hundred dollars as the connector never stopped running because there were always tweets for it to find. Because all of this is billed at a fixed cost per operation, the cost increased exponentially throughout the weekend, especially for matches involving large clubs with supporters throughout the world. My wife was relatively unimpressed by my mistake, so I wanted to mention it here to save you from a similar fate should you begin experimenting with this.

Gotcha 2: Scheduling Logic Apps Always Works (Eventually)

After a discussion with Azure Support to clarify exactly how the connector works, I dug into trying to figure out how to schedule these logic apps so they would enable and disable on command. That would allow me to do two things: 1) capture the time intervals Rog and Davo wanted during or after the match and 2) keep the cost manageable so I would not have to sell a limb or one of my children to continue financing the mood table.

That led me to Azure Job Collections and Azure Scheduler and Toon’s wonderful blog on how to wire this up. I will not rehash his blog here, but please go read it if you want a step-by-step walkthrough of what I did. One note I will add to Toon’s blog is that, wherever he references <workflow name>, that is referring to the name of the logic app itself. It took some trial and error for me to discover that so I want to save my readers that time and effort.

My job collection contains an enable and disable job for each club’s logic app (for a total of 40 jobs). They run on set schedules (that I manually update) to capture the 2-3 minutes before the final whistle and the 7-8 minutes after the final whistle. Rog wants those instant post-match sentiments, the gut reaction you have to your club’s match as the final whistle blows.


This all seemed a brilliant idea until the next set of matches rolled around and I realized that some of the disable jobs failed their initial attempt – leading to yet more money accidentally donated to Microsoft. The Azure team’s holiday party likely got a bit better as a result of my mistakes!

Luckily, you can apply retry logic to these jobs (see below). I strongly recommend this as it will save you many disapproving looks from your significant other for making it rain in the Azure bill once again.


Gotcha 3: Having A Laptop At The Pub Is Dorky (use the Azure iOS app)

The first weekend or two I ran the mood table stuff I was monitoring it via a laptop while watching matches at the pub. I’m told, from reliable sources, that that made me look like a dork. It also meant that I could not reliably monitor the jobs enabling and disabling on schedule unless I was somewhere with 1) room for a laptop and 2) decent Wi-Fi.

To use an obviously hypothetical example relayed to me by a friend of mine (wink wink, nudge nudge), let’s say a big match ended in the middle of a Sunday church service and “my friend” wanted to make sure the jobs kicked off on schedule. Taking out a laptop during the service would be deemed inappropriate by my friend’s family, his priest, and likely incurs a risk of him being struck by lightning were the Good Lord to catch a glimpse of such activity. How could my friend solve this issue?

He used the Azure Portal iOS app found here (it’s also available for Android). That allowed “my friend” to ensure that the club’s logic apps enabled and disabled on schedule and that the tweets were properly captured. I’m also delighted to report that “my friend” was not struck by lightning while checking this during church.

Things Deemed Too Optimal For The Mood Table

Originally the mood table had a live component with it: a Power BI report that accepted the tweet information via a streaming dataset and showed a live table throughout the matches with up-to-the-second rankings. While it was fascinating, Rog wanted to focus on the end-of-match sentiments for the static table.

The ability to analyze streaming sentiment data in this way, however, may make an appearance for individual Premier League matches or, possibly, a major soccer tournament taking place this summer (no, not the Alternative Facts World Cup).


If you’ve read all the way to the end, I salute you! I’ve enjoyed working with the guys on the Premier League mood table and I’ve enjoyed putting together this deep dive post for those of you that are interested. As I mentioned, I look forward to questions or corrections (throw them in the comments) or connections via Twitter or LinkedIn. Cheers (and come on you Spurs)!






Coming 12/8/17 – Premier League Mood Table How-to

Coming 12/8/17 – Premier League Mood Table How-to

Greetings GFOPs (and others who have clicked on this post)! There were a few mentions on social media that I’d be publishing two blogs here this afternoon. The first was going to be a high-level, layman’s explanation for how the Premier League Mood Table I did for Men In Blazers (b|t) works and the second was going to be a deep dive into the nuts and bolts of how I built it, the mistakes I made, the ways I fixed those mistakes, and just how flipping cool Azure Logic Apps and Social Sentiment Analysis are. That second blog will form the foundation for a SQL Saturday/conference talk I intend to submit and present throughout 2018 (feel free to contact me if this interests you and your user group/conference/organization).

These blogs were originally timed with the 12/1 release of MIB’s latest Raven (their newsletter for readers who don’t listen to their podcast – register here if you’re not a subscriber) containing an interview with me, thus the handful of tweets and other social posts mentioning today as the day to take a look at my site.

Unfortunately, in typically suboptimal MIB fashion, the 12/1 Raven has been delayed until 12/8, thus Mauricio Pochettino (and I) make the sad face above. My blogs will be released in conjunction with that Raven, so we’ll see you back here next Friday. If you’re interested in soccer and/or data and/or social media analysis potentially relevant to your company, it will be worth a read. Have a great weekend and see you back here Friday, December 8!