Still Life with Data Points

Reading time ~4 minutes

I’ve spent my life recording numbers to generate graphs to make sense of the world.  The numbers are what’s important, but humans do not really gain comprehension staring at rows and columns of numbers.  The human brain is made for visual processing.  A glance at a well plotted graph can present more value than attempting to read through the millions of data points that made up the graph.  The plot illuminates the data and should allow a reasonable person to intuit meaning.  I’ve probably tossed many kilograms of notebooks filled with data sets and plotted graphs.  Almost no one ever cared about the numbers, just the resulting graphs.

Figures at Work

I’ve used GIFs generated with ancient versions of  GNU Plot  to analyze performance data.  The pain of creating the graphs turned justified when insights were gained from simple visualizations.

File transfer comparisons
From:

File Distribution Efficiencies: cfengine vs. rsync

Certainly, in my line of work pretty pictures help explain what’s going on to a wider audience than the raw numbers.  You can immediately look at a picture and notice when something appears out of the ordinary.

Server CPU Utilization
Maybe we should look at why the CPU load just spiked there.

Or…

Elastic Search EPS
Maybe things are happily cycling along.

Other times, I’ve needed to translate the data to justify costs or ways to lower costs.  The following graph helped start a conversation with finance to purchase Amazon AWS Reserved Instances.  A full multi-sheet spreadsheet explaining the financial breakdown was still needed for full justification.  But without this and few other pretty pictures, the discussion and the approval of the up front costs would never have happened.

AWS Instances
How many EC2 Instances are we running?

But that’s just work and part of the expected drudgery of justifying your department’s daily existence.  You do them like annual reviews and power point presentations; get them done to prove your point, then try to get back to real work…  Unless you are a scientist (data or otherwise), in which case, collecting data points and graphing the results might be your real job.

Zen and the Art of Keeping Track of My Car

But even before I ever had to plot a graph in school, I watched my dad keep a small notebook in every car to record the mileage, fuel added, fuel costs, etc.  He’d calculate the MPG since the last fuel stop and record that, and then reset the tripmeter.  He’d also indicate when he’d perform an old change or some other service on the car.  It was a valuable service record for the car.  Noticing a change in MPG could also indicate that something .  I continued the tradition when I started driving my own cars.  And it didn’t take me long to realize there was so much more information in those numbers if I could plot them.

Automotive gas prices clearly cycle over time, but over the years have been trending down in Northern California.  The slight uptick at the end could mean summer prices are coming.  But in this case, it’s more a matter of which town I needed to get gas.

Premium Gas Prices ($/gal) Over Time
Premium Gas Prices ($/gal) Over Time

How’s my driving?  So as I drive more, a higher percentage of my time spent in this car is commuting.  Commuting seems to be so much more fuel efficient than track days and auto-crossing.

MPG
Fuel Efficiency (or lack therof)

I’m driving this car a whole lot more now.

Trip-o-meter vs Odometer readings
Trips vs Odometer

Example

Google Sheets Logbook

Always Look on the Bright Side of Life

Maybe all this tracking, accounting, and graphing is sign of some compulsion on my part.  Certainly when it comes to question of my personal health, I become obsessive about tracking as many variables as possible.  There have been a few times when I’ve been not entirely healthy and the cause was not immediately apparent.  When a doctor or group of doctors become interested in you, life becomes less than pleasant.  On more than one occasion, after months of specialists and testing my body has sorted itself out.  Whatever was wrong resolved itself with no definitive answer for a cause.  The interminable wait between lab results and doctors visits, though, left for too much time for fidgeting and speculation.

Until wearable medical devices came around, I kept log books.  During one of these months long investigative sessions my log filled dozens of composition book pages.  Over three months I documented when symptoms would occur, the severity, duration, what remediation steps were taken, and their efficacy.  One specialist upon reading through my notes suggested I was adding to my general stress levels.  He did not come right out and suggest this additional stress was the cause of my ills, but I suspect that was on his mind.  I won’t discount that possibility entirely.  But sitting around waiting for something to happen and someone else to do something felt worse.

With a wearable, especially one that can track heart rates, the paper log book becomes a bit less necessary.

Resting Pulse over Months
Resting Pulse over Months

There’s still a lot of manual data entry than has to go on, but apps like Apple Health and

My Fitness Pal make it less tedious.  Apple’s Health app has horrible graphing in my opinion and without paying for it, it’s difficult to get all your data out of FitBit.  Using QS Access on the iPhone helps to move the data points into a spreadsheet like Google Apps or Excel where you can manipulate and visualize the data.

Blood Pressure Measurements over Months
Blood Pressure Measurements over Months

I still don’t know what’s going on with my current malady.  At least by recording data points and making graphs I feel like I’m doing something for my treatment while I wait for the next test or visit.

Alcohol content: measurements not recorded

Leaving Apple: part 1

## Leaving the MacBook...The choice by Apple to abandon "legacy" interfaces (USB-A, analogaudio, Ethernet, HDMI, MagSafe, etc) in favor o...… Continue reading

Bad Day on the Road

Published on June 24, 2019

piTravel Clock Build

Published on November 24, 2018