Better D3 Charts with TDD

Eventbrite Logo

Online

Next up

  • D3 Intro
  • Usual Workflow
  • Reusable Chart API
  • TDD Way
  • How to get started
  • Q&A

What is D3.js?

  • Data-Driven Documents
  • Low level, General Purpose Visualization Library
  • Manipulates data based documents
  • Open web standards (SVG, HTML, CSS and now Canvas)
  • Allows interactions with your graphs

How does it work?

  • Loads data
  • Binds data to elements
  • Transforms those elements
  • Transitions between states

D3 Demo

Obama Budget Proposal

D3 Niceties

  • Styling of elements with CSS
  • Transitions and animations baked in
  • Total control over our graphs
  • Amazing community
  • Decent amount of publications

D3 v4 Update

  • More modular
  • API improvements
  • Breaking changes
  • Highly adopted

Contracting story

Marketing guy: Hey, I saw this nice chart, could we do something like that?

Bump Chart

He loved it!

Usual workflow

Idea

Search for an example

Idea

Read and adapt code

Add/remove features

Polish it up

Wax on, Wax off

Usual workflow

  • Idea or requirement
  • Search for an example
  • Adapt the code
  • Add/remove features
  • Polish it up

The standard way

Code example

Bar chart example by Mike Bostock

Prepare container


var svg = d3.select("svg"),
    margin = {top: 20, right: 20, bottom: 30, left: 40},
    width = +svg.attr("width") - margin.left - margin.right,
    height = +svg.attr("height") - margin.top - margin.bottom;

var g = svg.append("g")
    .attr("transform", "translate(" + margin.left + "," + margin.top + ")");
                        
Reference: Margin Convention

Setting up scales


// Creates scales and gives them ranges
var x = d3.scaleBand().rangeRound([0, width]).padding(0.1),
    y = d3.scaleLinear().rangeRound([height, 0]);
                        
Reference: Scales tutorial

Loading data


d3.tsv("data.tsv", function(d) {
    d.frequency = +d.frequency;
    return d;
}, function(error, data) {
    if (error) throw error;
    // Chart Code here
});
                        

Drawing axes


// Updates domain of scales
x.domain(data.map(function(d) { return d.letter; }));
y.domain([0, d3.max(data, function(d) { return d.frequency; })]);

// Draws X axis
g.append("g")
    .attr("class", "axis axis--x")
    .attr("transform", "translate(0," + height + ")")
    .call(d3.axisBottom(x));

// Draws Y axis
g.append("g")
    .attr("class", "axis axis--y")
    .call(d3.axisLeft(y).ticks(10, "%"))
  .append("text")
    .attr("transform", "rotate(-90)")
    .attr("y", 6)
    .attr("dy", "0.71em")
    .attr("text-anchor", "end")
    .text("Frequency");
                        

Drawing bars


g.selectAll(".bar")
    .data(data)
    .enter()
      .append("rect")
        .attr("class", "bar")
        .attr("x", function(d) { return x(d.letter); })
        .attr("y", function(d) { return y(d.frequency); })
        .attr("width", x.bandwidth())
        .attr("height", function(d) { return height - y(d.frequency); });
                        

Output

Standard D3: drawbacks

  • Monolithic functions
  • Chained method calls
  • Hard to change code
  • Impossible to reuse
  • Delicate

Story continues...

Bump Chart

Marketing guy: What if we change this thing here...

Trial and error

Done!

M-guy: nice, let’s change this other thing!

Done!

M-guy: Great! I love it so much I want it on the product!

M-guy: So good you have it almost ready, right?

I was hating myself!

Possible outcomes

  • You keep on refactoring as needed
  • You dump it and start all over again
  • You avoid refactoring

What if you could work with charts the same way you work with the rest of the code?

Reusable Chart API

jQuery VS MV*

Where did it start?

Towards Reusable Charts by Mike Bostock

Reusable Chart API - code


return function module(){
    // @param  {D3Selection} _selection  Container(s) to render chart in
    function exports(_selection){
        // @param {object} _data    Data to generate the chart
        _selection.each(function(_data){
            // Build chart
        });
    }

    // @param  {object} _x          Margin object to get/set
    // @return { margin | module}   Current margin or Bar Chart module to chain calls
    exports.margin = function(_x) {
        if (!arguments.length) return margin;
        margin = _x;
        return this;
    };

    return exports;
}
                        

Reusable Chart API - use


// Creates bar chart component and configures its margins
barChart = chart()
    .margin({top: 5, left: 10});

container = d3.select('.chart-container');

// Calls bar chart with the data-fed selector
container.datum(dataset).call(barChart);
                        

Reusable Chart API - benefits

  • Modular
  • Composable
  • Configurable
  • Consistent
  • Teamwork Enabling
  • Testable

The TDD way

The "before" block


container = d3.select('.test-container');
dataset = [
    {   letter: 'A',
        frequency: .08167
    },{
        letter: 'B',
        frequency: .01492
    },...
];
barChart = barChart();

container.datum(dataset).call(barChart);
                        

Test: basic chart


it('should render a chart with minimal requirements', function(){
    expect(containerFixture.select('.bar-chart').empty()).toBeFalsy();
});
                        

Code: basic chart


return function module(){
    var margin = {top: 20, right: 20, bottom: 30, left: 40},
        width = 960, height = 500,
        svg;

    function exports(_selection){
        _selection.each(function(_data){
            var chartWidth = width - margin.left - margin.right,
                chartHeight = height - margin.top - margin.bottom;

            if (!svg) {
                svg = d3.select(this)
                    .append('svg')
                    .classed('bar-chart', true);
            }
        });
    };
    return exports;
}
                        
Reference: Towards Reusable Charts

Test: containers


it('should render container, axis and chart groups', function(){
    expect(containerFixture.select('g.container-group').empty()).toBeFalsy();
    expect(containerFixture.select('g.chart-group').empty()).toBeFalsy();
    expect(containerFixture.select('g.x-axis-group').empty()).toBeFalsy();
    expect(containerFixture.select('g.y-axis-group').empty()).toBeFalsy();
});
                       

Code: containers


function buildContainerGroups(){
    var container = svg
      .append('g')
        .classed('container-group', true)
        .attr('transform', 'translate(' + margin.left + ',' + margin.top + ')');

    container.append("g").attr("class", "chart-group");
    container.append("g").attr("class", "x-axis-group axis");
    container.append("g").attr("class", "y-axis-group axis");
}
                       

Test: axes


it('should render an X and Y axes', function(){
    expect(containerFixture.select('.x-axis-group.axis').empty()).toBeFalsy();
    expect(containerFixture.select('.y-axis-group.axis').empty()).toBeFalsy();
});
                        

Code: scales


function buildScales(){
    xScale = d3.scaleBand()
        .rangeRound([0, chartWidth]).padding(0.1)
        .domain(data.map(getLetter));

    yScale = d3.scaleLinear()
        .rangeRound([chartHeight, 0])
        .domain([0, d3.max(data, getFrequency)]);
}
                        

Code: axes


function buildAxis(){
    xAxis = d3.axisBottom(xScale);

    yAxis = d3.axisLeft(yScale)
        .ticks(10, '%');
}
                        

Code: axes drawing


function drawAxis(){
    svg.select('.x-axis-group.axis')
        .append("g")
        .attr("class", "axis axis--x")
        .attr("transform", "translate(0," + chartHeight + ")")
        .call(xAxis);

    svg.select(".y-axis-group")
        .append("g")
        .attr("class", "axis axis--y")
        .call(yAxis)
      .append("text")
        .attr("transform", "rotate(-90)")
        .attr("y", 6)
        .attr("dy", ".71em")
        .style("text-anchor", "end")
        .text("Frequency");
}
                        

Test: bars drawing


it('should render a bar for each data entry', function(){
    var numBars = dataset.length;

    expect(containerFixture.selectAll('.bar').size()).toEqual(numBars);
});
                        

Code: bars drawing


function drawBars(){
    // Setup the enter, exit and update of the actual bars in the chart.
    // Select the bars, and bind the data to the .bar elements.
    var bars = svg.select('.chart-group').selectAll(".bar")
        .data(data);

    // Create bars for the new elements
    bars.enter()
      .append('rect')
        .attr("class", "bar")
        .attr("x", function(d) { return xScale(d.letter); })
        .attr("y", function(d) { return yScale(d.frequency); })
        .attr("width", xScale.rangeBand())
        .attr("height", function(d) { return chartHeight - yScale(d.frequency); });
}
                        
Reference: Thinking with joins, General Update Pattern

Test: margin accessor


it('should provide margin getter and setter', function(){
    var previous = barChart.margin(),
        expected = {top: 4, right: 4, bottom: 4, left: 4},
        actual;

    barChart.margin(expected);
    actual = barChart.margin();

    expect(previous).not.toBe(expected);
    expect(actual).toBe(expected);
});
                        

Code: margin accessor


exports.margin = function(_x) {
    if (!arguments.length) return margin;
    margin = _x;

    return this;
};
                        

Looks the same, but is not

Final code: standard way


var svg = d3.select("svg"),
    margin = {top: 20, right: 20, bottom: 30, left: 40},
    width = +svg.attr("width") - margin.left - margin.right,
    height = +svg.attr("height") - margin.top - margin.bottom;

var x = d3.scaleBand().rangeRound([0, width]).padding(0.1),
    y = d3.scaleLinear().rangeRound([height, 0]);

var g = svg.append("g")
    .attr("transform", "translate(" + margin.left + "," + margin.top + ")");

d3.tsv("data.tsv", function(d) {
    d.frequency = +d.frequency;
    return d;
}, function(error, data) {
    if (error) throw error;

    x.domain(data.map(function(d) { return d.letter; }));
    y.domain([0, d3.max(data, function(d) { return d.frequency; })]);

    g.append("g")
        .attr("class", "axis axis--x")
        .attr("transform", "translate(0," + height + ")")
        .call(d3.axisBottom(x));

    g.append("g")
        .attr("class", "axis axis--y")
        .call(d3.axisLeft(y).ticks(10, "%"))
      .append("text")
        .attr("transform", "rotate(-90)")
        .attr("y", 6)
        .attr("dy", "0.71em")
        .attr("text-anchor", "end")
        .text("Frequency");

    g.selectAll(".bar")
        .data(data)
        .enter()
          .append("rect")
            .attr("class", "bar")
            .attr("x", function(d) { return x(d.letter); })
            .attr("y", function(d) { return y(d.frequency); })
            .attr("width", x.bandwidth())
            .attr("height", function(d) { return height - y(d.frequency); });
});
                        

Final code: TDD way


return function module(){
    var margin = {top: 20, right: 20, bottom: 30, left: 40},
        width = 960, height = 500,
        chartWidth, chartHeight,
        xScale, yScale,
        xAxis, yAxis,
        data, svg;

    function exports(_selection){
        _selection.each(function(_data){
            chartWidth = width - margin.left - margin.right;
            chartHeight = height - margin.top - margin.bottom;
            data = _data;

            buildScales();
            buildAxis();
            buildSVG(this);
            drawBars();
            drawAxis();
        });
    }

    function buildContainerGroups(){ ... }

    function buildScales(){ ... }

    function buildAxis(){ ... }

    function drawAxis(){ ... }

    function drawBars(){ ... }

    // Accessors to all configurable attributes
    exports.margin = function(_x) { ... };

    return exports;
};
                        

TDD way - benefits

  • Stress free refactors
  • Goal oriented
  • Deeper understanding
  • Improved communication
  • Quality, production ready output

How to get started

Some ideas

  • Test something that is in production
  • TDD the last chart you built
  • Pick a block, refactor it
  • TDD your next chart

Jumpstart Repository

http://tinyurl.com/fullstackd3
http://eventbrite.github.io/britecharts

What happened with my contracting gig?

I used the Reusable Chart API

Adding multiple dimensions?

I had tests!

Toogle dimensions, adding more y-axis?

Conclusions

  • Examples are great for exploration and prototyping, bad for production code
  • There is a better way of building D3 Charts
  • Reusable Chart API + TDD bring it to a Pro level
  • You can build your own library and feel proud!

Thanks for listening!

Learning resources

Example search

Books