March 4, 2017 / by Brent / Tutorial

Creating a Scaled-Size List Visualization – Part 1

Scaled-Size List: Overview and Credits

Matt Chandler first noticed this per capita breakdown of budget spending, and suggested that we do something similar at the city level on OpenBudgetOKC. It’s basically a vertical running list where data values are propotionately coded.

We want others to be able to share this work, so we have created a reusable, Javascript-based component that you could use for your data. This tutorial walks you through how it was created. You can get the final source code for this component on GitHub. Most of the code is original, but the magic formula to calculate font sizes based on a string’s content came from the NYT original article.

Breaking Down The Visual

Before starting any code, we need to know what our desired markup end state is. You can also see a live preview of this on CodePen.

Markup

Looking at the HTML, it appears that every entry makes up a row. The rows are sorted first alphabetically by a high-level category (i.e. “Agriculture”, “Commerce” , Education”…) Then they are sorted based on the size of the value within that higher level category. So this visualization is encoding two pieces of information:

  1. The absolute weight of each value to the absolute set (encoded by font size).
  2. The relative weight of each value within its category (encoded by sort order).

We can think of the high-level category as the Level 1 grouping of the data (e.g. “Agriculture”) and the lowest-level detail associated to the measure as Level 2. Notice that the list also the Total for that Level 2 line item. The value shows the per capita cost, which is roughly the total spend divided by 318.9 million, or the estimated US population in 2014.

Screenshot of original viz broken down

Each row is placed in a block div with an o-row class. That row is divided into two spans. The number, which we want to vary in size, is classed o-measure. The font is sized explictly with an element level attribute, to override all CSS. The detail that appears to its right in three lines gets a o-detail tag.

<div id="Mayor and Council" class="o-row salaries-and-wages">
  <span class="o-measure" style="font-size: 12 px;">
     <span class="o-cash">$</span> 
      <span class="o-value">18</span>
    </span>
  <span class="o-detail">
    <p class="o-l1">MAYOR</p>
    <p class="o-l2">Salaries and Wages</p>
    <p class="o-total">Total: $178,428<p>
  </span>
</div>

CSS

The CSS controls the positioning with percentage-based widths. The measure is given 75%, while the details are granted 23%. The remaining 2% serves as a gutter.

.o-row {
  /* vertical-align:  bottom; */
  display: block;
  /* align-items: baseline; */
}

.o-row span {
  display: inline-block;
}

.o-row .o-measure{
  width: 75%;
  text-align: right;
  margin-bottom: 10px;
  font-weight: 300;
  position: relative;

}

.o-row .o-detail{
  width: 23%;
  margin-left: 1%;
}

Annotated image of markup to show positioning

Javascript

Now let’s look at the code that dynamically reads the data and generates the markup.

Reading the JSON data

Conveniently, our data is already in a JSON file like this. We can start to identify that Its font size should be scaled proportionately to its relative value among all the values. We also want to find the subtotal of a particular category.

We can identify the levels in our data as follows:

  • Level 1 – program
  • Level 2 – key
  • Measure – value

Note: This is only test data.

[
{
    "agency": "Mayor and Council",
    "fund": "GENERAL FUND",
    "lob": "Mayors Office",
    "program": "MAYOR",
    "key": "BUDGET-VACANCY DISCOUNT",
    "value": "47"
  },
  {
    "agency": "Mayor and Council",
    "fund": "GENERAL FUND",
    "lob": "Mayors Office",
    "program": "MAYOR",
    "key": "SALARIES AND WAGES",
    "value": "178"
  },
 ...
]

To read this data, we use jQuery’s getJSON function, which accepts a callback to invoke when it’s “done” promise is reached.

    function readData(callback) {
       $.getJSON("./data/sampleData.json")
       .done(function(results) {
           callback(results);
       })
       .fail(function (jqxhr, textStatus, error) {
           output("Had a problem getting the data: " + error);
       });
    }

The callback we provide is to the processData function. This function is really the “main” function for our script. Calling this function directly allows us to bypass reading the JSON data from an external source.

// master function to begin after data retrieval
function processData(data) {
    var options = getOptions();
    var structured = structureData(data, options);
    var sorted = sortData(structured);
    var subTotaled = subTotalData(sorted);
    var rootElement = getRootElement();

    rootElement = renderList(rootElement, subTotaled);
    resizeList();
}

Structuring the data

First, we want to shape the data to remove unnecessary elements, and send it along to the rendering functions in a predictable format. This function accepts the original data and a single object called “options” which contains the names of the properties to be read as level 1 (L1), level 2 (L2) and the measure.

We use Javascript’s native Array.map function to create a new array by calling a function on every element of our original array. The function checks for the existence of such a property on every element. If found, it sets strings to lower case and converts numbers to float.

// Create a new array of elements with L1, L2, measure structure
function structureData(data, options) {
    // get only the attributes you need
    var l1Key = options.L1;
    var l2Key = options.L2;
    var measureKey = options.measure;
    var totalKey = options.total;

    var structuredData = data.map(function (element) {
       var l1 = element.hasOwnProperty(l1Key) ? element[l1Key] : null;
       var l2 = element.hasOwnProperty(l2Key) ? element[l2Key] : null;
        var measure = element.hasOwnProperty(measureKey) ? parseFloat(element[measureKey]): null;
        var total = element.hasOwnProperty(totalKey) ? parseFloat(element[totalKey]): null;

        return {
            L1: l1.toString().toLowerCase(),
            L2: l2.toString().toLowerCase(),
            measure: measure,
            total: total
        };
    });

    return structuredData;
}

Note – right now the options are hardcoded in the getOptions function. Going to make that more conifgurable in the future.

Sorting the data

The data is then sorted, first by the Level 1 attribute, alphabetically. Then by the measure in descending order. This uses the compare function of Javascript’s Array.sort function.

// Sort data by L1 ASC, measure DESC
// assumes the data is already structured with L1, L2 & measure
function sortData(data) {   
    var sortedData = data.sort(function(a, b) {
        // compare L1 first, then measure descending
        if (a.L1 < b.L1) {
            return -1;
        }
        else if (a.L1 > b.L1) {
            return 1;
        }
        else {                            // a.L1 == b.L1
            return b.measure - a.measure; // descending order
        }
    });

    return sortedData;
}

Finding subtotals

Originally, I thought that the “total” field contained the subtotal of Level 1. During the writing of this post, I realized that it really shows the total of Level 2. Still, showing a subtotal of the L1 category is a useful feature to add, so it’s included below. This should probably be made an option for the viz, whether to simply show a different value as the total, or subtotal L1.

// calculates the total at the L1 group level
// adds a new attribute to each element called "subtotal"
function subTotalData(data) {
    subTotals = {};
    for ( i = 0; i < data.length; i++) {
        var key = data[i].L1;
        var value = data[i].measure;

        if(!subTotals.hasOwnProperty(key))
            subTotals[key] = 0;
        subTotals[key] += value;
    }

    // apply subtotals to each element
    var subTotaled = data.map(function (element) {
       element.l1SubTotal = subTotals[element.L1];
        return element; 
    });

    return subTotaled;
}

Rendering the Output

Now that the data is shaped, sorted, and subtotaled, we are ready to write out the desired HTML. This is accomplished by using jQuery to append elements to some given root element. We use the toLocaleString function to add the culture-aware separators and decimal markers to numbers.

//Render the elements in the data as a series of divs with the 'o-row' class applied
function renderList(rootElement, data) {
    data.forEach(function (element) {
        rowDiv = $("<div class='o-row'></div>");
        rowDiv.className = "o-row";
        rowDiv.id = element.L2;

        spanMeasure = $("<span class='o-measure'></span>");
        spanMeasure.append("<span class='o-cash'>$</span");
        spanMeasure.append("<span class='o-value'>" + element.measure.toLocaleString() + "</span>");

        spanDetail = $("<span class='o-detail'></span>");
        spanDetail.append("<p class='o-l1'>" + element.L1 + "</p>");
        spanDetail.append("<p class='o-l2'>" + element.L2 + "</p>");
        spanDetail.append("<p class='o-total'>Total: $" + element.total.toLocaleString() + "</p>");

        rowDiv.append(spanMeasure);
        rowDiv.append(spanDetail);
        rootElement.append(rowDiv); 
    });    

    return rootElement;
}

Scaling the fonts

At this point, the markup is basically what we want. Everything is perfect – except the fonts aren’t scaled. Here we come to the magic formula. You can find the original code for this embedded in the markup of the inspiration article. I borrowed heavily from it, though I did clean it up a bit.

It uses a scaler value to become a common multiplication factor based on the window’s actual width, within a given maximum and minimum window size.

You reach the font sizing function (line 35), which seems to plumb the depths of typographical sorcery. It divides the value of the number you want to scale by a ratio how many numeric and non-numeric characters it contains. Then we take the square root of that figure. (See note on why we need the square root here.)

// Resize the list based on window size
function resizeList() {
    rootElement = getRootElement();
    elementsToResize = getResizeElements(rootElement);

    var maxWidth = 1800;
    var defaultScaler = 10.5; //scaler - multiplication factor for fonts
    var minScaler = 2.2;


    var newWidth = Math.min($(window).width(), maxWidth); // sets max for width calc
    var scaler = Math.max(defaultScaler * newWidth / maxWidth, minScaler); // min scale

    elementsToResize.each(function(k,v) {
        var valSpan = $(v).children('.o-value')[0];
        var value = $(valSpan).text();
        var fontSize = getFontSize(value, scaler);
        $(v).css('font-size', fontSize + 'px');
    });

}

function getFontSize(val, scaler) {
    var minSize = 11; // minimum font size


    var pc = String(val);
    var str = pc.replace(',', ''); // "100.01"
    var val = Number(str); // 100.01
    var roundNum = Math.round(val);
    var periodCount = (str.match(/\./g) || []).length;
    var numeralCount = (str.match(/[0-9]/g) || []).length;
    var nonNumerals = Math.floor((String(roundNum).length-1) / 3) + periodCount; // count of periods and commas

    var size = Math.sqrt((val) / (.7*( (.56 * numeralCount) + (.27*nonNumerals) ))); // font size function
    var fontSize = scaler * size;

    return Math.max(fontSize, minSize);

}

Responsive design

We want the font scaling to look right on many sizes of screen. More than that, we want it to adjust if the screen size is changed. To support this responsiveness, we can tie into jQuery’s $(window).resize function and scale the fonts every time the window is resized.

// run on load
$(function () {
    // find root element
    // TODO: make root element dynamic
    readData(processData);
    // resize fonts when window resizes
    $(window).resize($.debounce(250, resizeList));
});

Pay special attention to the $.debounce() function. Debouncing is a technique that delays a call to an event handler if the event is raised many times in a given span of time. Without debouncing, our resize will be called for literally every window resize event. This event may be raised hundreds or thousands of times as a user drags and drops a corner. Debouncing raises the event only once, after the specified 250ms delay, so font resizing only happens when the window settles down.

This uses an implementation by Ben Alman. The script I used can be downloaded here.

Next steps

Node and Jade

The initial effort was to make a more reusable script. But there are many ways to improve. In Part 2 of this series, I’ll rearrange the code to add options for users, and make the root element more configurable.

To make the code much more reusable, I’ll also wrap it as a node package that can be easily incorporated into other node modules with a simple require statement. This will become more important to integrate this into the OpenBudgetOKC source, which is based on node and also uses Jade for HTML templating.

When that’s ready to go, I’ll open a pull request (a new experience for me).

I hope this code and walkthrough can help you to create your visualization!

Taking the square root with fonts

Taking the square root is important when scaling fonts. Comparing the sizes of fonts is like comparing the areas of two circles. The font-size setting is like controlling the areas of these circles with the radius. Say we want to compare two values: 1 and 2. If we simply set the radius of two circles to 1 and 2, we actually produce circles with these areas (remember A = \pi * r^2 ):

  • Area with radius 1: 3.14 * 1^2 = 3.14
  • Area with radius 2: 3.14 * 2^2 = 12.56

In trying to show that 2 is twice as large as 1, we will inadvertently create a circle that is actually 4 times larger. Taking the square root addresses this issue:

  • Using square root: 3.14 * \sqrt(2)^2 = 6.28

That’s the size comparison we want.