xbe.at

Data visualisation

Avoiding Misleading Data Visualizations

10 min read, 14 sections

Contents
  1. Scale Fundamentals and Axes Range Impact
  2. Zero Baseline Analysis
  3. Proportional Scale Relationships
  4. Scale Transformation Analysis
  5. Handling Outliers in Scale Selection
  6. Dynamic Scale Adaptation
  7. Non-Linear Scale Transformations
  8. Scale Consistency Across Multiple Visualizations
  9. Time Series Scale Optimization
  10. Adaptive Binning for Histogram Scales
  11. Interactive Scale Controls for Data Exploration
  12. Logarithmic Scale Transitions
  13. Scale Optimization for Multi-Dimensional Data
  14. Additional Resources

Scale Fundamentals and Axes Range Impact

Proper data visualization requires careful consideration of scale selection to maintain data integrity. The choice of axis ranges and intervals directly influences how viewers interpret relationships between data points, making it crucial to establish appropriate boundaries that neither exaggerate nor diminish actual trends.

import numpy as np
import matplotlib.pyplot as plt

# Generate sample data
x = np.linspace(0, 10, 50)
y = 2 * x + np.random.normal(0, 1, 50)

# Create two subplots with different y-axis scales
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Plot 1: Full scale (0 to max)
ax1.scatter(x, y)
ax1.set_ylim(0, max(y))
ax1.set_title('Appropriate Scale')

# Plot 2: Truncated y-axis
ax2.scatter(x, y)
ax2.set_ylim(min(y), max(y))
ax2.set_title('Misleading Truncated Scale')

plt.tight_layout()
plt.show()

Zero Baseline Analysis

Understanding when to include zero in your visualization baseline is critical for accurate data representation. While zero baselines are essential for bar charts and area plots, they may not always be necessary for line charts or scatter plots when focusing on relative changes or trends.

import pandas as pd
import matplotlib.pyplot as plt

# Create sample time series data
dates = pd.date_range(start='2023-01-01', periods=12, freq='M')
values = [98.5, 98.8, 98.6, 98.9, 99.1, 98.7, 98.4, 98.8, 99.2, 98.9, 98.5, 98.7]

# Create two visualizations
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Plot with zero baseline
ax1.plot(dates, values)
ax1.set_ylim(0, 100)
ax1.set_title('With Zero Baseline')

# Plot with focused range
ax2.plot(dates, values)
ax2.set_ylim(98, 100)
ax2.set_title('Focused Range (No Zero)')

plt.tight_layout()
plt.show()

Proportional Scale Relationships

When comparing multiple datasets or creating multi-panel visualizations, maintaining proportional relationships between scales ensures accurate visual comparison. This principle is fundamental for avoiding misinterpretation of relative magnitudes across different measurements.

import numpy as np
import matplotlib.pyplot as plt

# Generate correlated datasets with different scales
x = np.linspace(0, 10, 50)
y1 = 100 * np.sin(x) + 1000
y2 = 10 * np.sin(x) + 100

# Create properly scaled comparison
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 8))

# Plot with proportional scales
ax1.plot(x, y1, 'b-', label='Dataset 1')
ax2.plot(x, y2, 'r-', label='Dataset 2')

# Set scales to maintain proportional relationships
ax1.set_ylim(min(y1) - (max(y1)-min(y1))*0.1, max(y1) + (max(y1)-min(y1))*0.1)
ax2.set_ylim(min(y2) - (max(y2)-min(y2))*0.1, max(y2) + (max(y2)-min(y2))*0.1)

plt.tight_layout()
plt.show()

Scale Transformation Analysis

Data transformations can reveal hidden patterns and relationships in skewed distributions. Understanding when to apply logarithmic, square root, or other transformations helps maintain visual clarity while preserving the underlying data relationships.

import numpy as np
import matplotlib.pyplot as plt

# Generate exponential data
x = np.linspace(0, 5, 100)
y = np.exp(x) + np.random.normal(0, 50, 100)

# Create comparison plots
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Linear scale
ax1.scatter(x, y)
ax1.set_title('Linear Scale')

# Log scale
ax2.scatter(x, y)
ax2.set_yscale('log')
ax2.set_title('Logarithmic Scale')

plt.tight_layout()
plt.show()

Handling Outliers in Scale Selection

When visualizing datasets containing outliers, scale selection becomes particularly critical. The presence of extreme values can compress the visualization of the main data distribution, making it essential to implement appropriate scaling strategies while maintaining data integrity.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Generate data with outliers
np.random.seed(42)
data = np.random.normal(100, 10, 1000)
outliers = np.random.uniform(300, 400, 5)
combined_data = np.concatenate([data, outliers])

# Create visualization comparison
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Standard scale with outliers
ax1.boxplot(combined_data)
ax1.set_title('Standard Scale (With Outliers)')

# Robust scale (excluding outliers for scale calculation)
q1, q3 = np.percentile(combined_data, [25, 75])
iqr = q3 - q1
ax2.boxplot(combined_data)
ax2.set_ylim(q1 - 1.5*iqr, q3 + 1.5*iqr)
ax2.set_title('Robust Scale')

plt.tight_layout()
plt.show()

Dynamic Scale Adaptation

Real-time data visualization requires intelligent scale adaptation to maintain visual clarity as new data points arrive. This implementation demonstrates an automatic scale adjustment system that preserves context while accommodating new values.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

class DynamicScalePlot:
    def __init__(self, max_points=100):
        self.fig, self.ax = plt.subplots(figsize=(10, 6))
        self.max_points = max_points
        self.line, = self.ax.plot([], [])
        self.data = []
        
    def init(self):
        self.ax.set_xlim(0, self.max_points)
        self.ax.set_ylim(-1, 1)
        return self.line,
    
    def update(self, frame):
        # Generate new data point
        new_point = np.sin(frame/10) + np.random.normal(0, 0.1)
        self.data.append(new_point)
        
        # Keep only last max_points
        if len(self.data) > self.max_points:
            self.data.pop(0)
            
        # Update scale dynamically
        self.ax.set_ylim(min(self.data) - 0.1, max(self.data) + 0.1)
        
        self.line.set_data(range(len(self.data)), self.data)
        return self.line,

# Create animation
plotter = DynamicScalePlot()
ani = FuncAnimation(plotter.fig, plotter.update, init_func=plotter.init,
                   frames=200, interval=50, blit=True)
plt.show()

Non-Linear Scale Transformations

Complex data relationships often require sophisticated scale transformations beyond simple linear or logarithmic approaches. This implementation showcases advanced scale transformations for specialized visualization requirements.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.scale import ScaleBase, register_scale
from matplotlib.transforms import Transform

class CustomScale(ScaleBase):
    name = 'custom'
    
    def get_transform(self):
        return CustomTransform()
        
class CustomTransform(Transform):
    input_dims = 1
    output_dims = 1
    
    def transform_non_affine(self, a):
        return np.sign(a) * np.power(np.abs(a), 0.5)
    
    def inverted(self):
        return CustomTransformInverse()

# Register custom scale
register_scale(CustomScale)

# Generate test data
x = np.linspace(-100, 100, 1000)
y = x**2 + np.random.normal(0, 100, 1000)

# Create comparison plots
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

ax1.scatter(x, y, alpha=0.5)
ax1.set_title('Linear Scale')

ax2.scatter(x, y, alpha=0.5)
ax2.set_yscale('custom')
ax2.set_title('Custom Non-Linear Scale')

plt.tight_layout()
plt.show()

Scale Consistency Across Multiple Visualizations

Maintaining consistent scales across multiple related visualizations is crucial for accurate comparative analysis. This implementation demonstrates a systematic approach to ensuring scale consistency while handling diverse data ranges.

import numpy as np
import matplotlib.pyplot as plt

class MultiScalePlotter:
    def __init__(self, datasets):
        self.datasets = datasets
        self.global_min = min(min(d) for d in datasets)
        self.global_max = max(max(d) for d in datasets)
        
    def plot_consistent_scales(self):
        n_plots = len(self.datasets)
        fig, axes = plt.subplots(1, n_plots, figsize=(15, 5))
        
        for i, (ax, data) in enumerate(zip(axes, self.datasets)):
            ax.hist(data, bins=30)
            ax.set_ylim(0, None)  # Consistent y-axis
            ax.set_xlim(self.global_min, self.global_max)  # Consistent x-axis
            ax.set_title(f'Dataset {i+1}')
            
        plt.tight_layout()
        return fig

# Generate sample datasets
np.random.seed(42)
datasets = [
    np.random.normal(0, 1, 1000),
    np.random.normal(2, 1.5, 1000),
    np.random.normal(-1, 2, 1000)
]

# Create consistent scale plots
plotter = MultiScalePlotter(datasets)
fig = plotter.plot_consistent_scales()
plt.show()

Time Series Scale Optimization

Time series data requires special consideration for scale selection to accurately represent temporal patterns and seasonality. The implementation demonstrates techniques for handling different time scales while maintaining data interpretation accuracy.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from datetime import datetime, timedelta

# Generate time series data with multiple frequencies
dates = pd.date_range(start='2023-01-01', end='2023-12-31', freq='D')
daily_pattern = np.sin(np.arange(len(dates)) * 2 * np.pi / 365)
weekly_pattern = np.sin(np.arange(len(dates)) * 2 * np.pi / 7)
trend = np.linspace(0, 2, len(dates))

data = daily_pattern + weekly_pattern + trend + np.random.normal(0, 0.1, len(dates))

# Create multi-scale time series visualization
fig, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(12, 10))

# Daily scale
ax1.plot(dates[:30], data[:30])
ax1.set_title('Daily Scale (30 Days)')

# Monthly scale
monthly_data = pd.Series(data, index=dates).resample('M').mean()
ax2.plot(monthly_data.index, monthly_data.values)
ax2.set_title('Monthly Scale')

# Quarterly scale with trend
quarterly_data = pd.Series(data, index=dates).resample('Q').mean()
ax3.plot(quarterly_data.index, quarterly_data.values)
ax3.set_title('Quarterly Scale')

plt.tight_layout()
plt.show()

Adaptive Binning for Histogram Scales

Histogram visualization requires careful consideration of bin sizes and scales to reveal true data distribution patterns. This implementation showcases advanced binning strategies with adaptive scale selection.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

class AdaptiveHistogram:
    def __init__(self, data):
        self.data = data
        
    def freedman_diaconis_bins(self):
        # Calculate optimal bin width using Freedman-Diaconis rule
        iqr = stats.iqr(self.data)
        n = len(self.data)
        bin_width = 2 * iqr / (n ** (1/3))
        return int((max(self.data) - min(self.data)) / bin_width)
    
    def plot_comparison(self):
        fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize=(12, 10))
        
        # Fixed bins
        ax1.hist(self.data, bins=10)
        ax1.set_title('Fixed 10 Bins')
        
        # Sturges formula
        bins_sturges = int(np.log2(len(self.data)) + 1)
        ax2.hist(self.data, bins=bins_sturges)
        ax2.set_title(f'Sturges Formula ({bins_sturges} bins)')
        
        # Freedman-Diaconis rule
        bins_fd = self.freedman_diaconis_bins()
        ax3.hist(self.data, bins=bins_fd)
        ax3.set_title(f'Freedman-Diaconis ({bins_fd} bins)')
        
        # Kernel Density Estimation
        density = stats.gaussian_kde(self.data)
        xs = np.linspace(min(self.data), max(self.data), 200)
        ax4.plot(xs, density(xs))
        ax4.set_title('Kernel Density Estimation')
        
        plt.tight_layout()
        return fig

# Generate sample data with mixed distribution
data = np.concatenate([
    np.random.normal(0, 1, 1000),
    np.random.normal(4, 1.5, 500)
])

# Create visualization
hist_plotter = AdaptiveHistogram(data)
fig = hist_plotter.plot_comparison()
plt.show()

Interactive Scale Controls for Data Exploration

Dynamic scale adjustment capabilities enhance data exploration by allowing users to focus on specific ranges while maintaining context. This implementation provides interactive scale controls for detailed data analysis.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.widgets import RangeSlider, Button

class InteractiveScalePlot:
    def __init__(self, data):
        self.data = data
        self.fig, (self.ax_plot, self.ax_slider) = plt.subplots(2, 1, 
            height_ratios=[4, 1], figsize=(10, 8))
        
        # Initial plot
        self.line, = self.ax_plot.plot(data)
        self.initial_ylim = self.ax_plot.get_ylim()
        
        # Add range slider
        self.slider = RangeSlider(self.ax_slider, 'Y Scale',
                                self.initial_ylim[0], self.initial_ylim[1],
                                valinit=self.initial_ylim)
        self.slider.on_changed(self.update)
        
        # Add reset button
        self.reset_button = Button(plt.axes([0.8, 0.025, 0.1, 0.04]), 'Reset')
        self.reset_button.on_clicked(self.reset)
        
    def update(self, val):
        self.ax_plot.set_ylim(self.slider.val[0], self.slider.val[1])
        self.fig.canvas.draw_idle()
        
    def reset(self, event):
        self.slider.set_val(self.initial_ylim)
        self.ax_plot.set_ylim(self.initial_ylim)
        
    def show(self):
        plt.show()

# Generate sample data
x = np.linspace(0, 10, 1000)
y = np.sin(x) * np.exp(-x/5) + np.random.normal(0, 0.1, 1000)

# Create interactive plot
plotter = InteractiveScalePlot(y)
plotter.show()

Logarithmic Scale Transitions

Understanding when and how to transition between linear and logarithmic scales is crucial for data visualization. This implementation demonstrates smooth transitions between scale types while preserving data relationships and visual clarity.

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm

class ScaleTransitioner:
    def __init__(self, data):
        self.data = data
        self.fig, ((self.ax1, self.ax2), (self.ax3, self.ax4)) = plt.subplots(2, 2, figsize=(12, 10))
        
    def plot_transitions(self):
        # Linear scale
        self.ax1.hist2d(self.data[:, 0], self.data[:, 1], bins=50)
        self.ax1.set_title('Linear Scale')
        
        # Log X scale
        self.ax2.hist2d(self.data[:, 0], self.data[:, 1], bins=50,
                       norm=LogNorm())
        self.ax2.set_title('Log Color Scale')
        
        # Symlog scale (handles negative values)
        self.ax3.hist2d(self.data[:, 0], self.data[:, 1], bins=50)
        self.ax3.set_xscale('symlog')
        self.ax3.set_title('Symmetric Log X Scale')
        
        # Logit scale
        mask = (self.data > 0) & (self.data < 1)
        self.ax4.hist2d(self.data[mask][:, 0], self.data[mask][:, 1], bins=50)
        self.ax4.set_xscale('logit')
        self.ax4.set_title('Logit Scale')
        
        plt.tight_layout()
        return self.fig

# Generate sample data
np.random.seed(42)
data = np.random.exponential(size=(10000, 2))
data = np.clip(data, 0, 10)

# Create visualization
plotter = ScaleTransitioner(data)
fig = plotter.plot_transitions()
plt.show()

Scale Optimization for Multi-Dimensional Data

Visualizing high-dimensional data requires careful scale consideration across multiple dimensions. This implementation shows techniques for maintaining proper scale relationships in dimensionality reduction and projection methods.

import numpy as np
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt

class MultiDimScaler:
    def __init__(self, data):
        self.data = data
        self.scalers = {
            'Standard': StandardScaler(),
            'MinMax': MinMaxScaler(),
            'Robust': RobustScaler()
        }
        
    def compare_scaling_methods(self):
        fig, axes = plt.subplots(2, 2, figsize=(12, 10))
        axes = axes.ravel()
        
        # Original data
        pca = PCA(n_components=2)
        projected = pca.fit_transform(self.data)
        axes[0].scatter(projected[:, 0], projected[:, 1], alpha=0.5)
        axes[0].set_title('Original Scale')
        
        # Compare different scaling methods
        for idx, (name, scaler) in enumerate(self.scalers.items(), 1):
            scaled_data = scaler.fit_transform(self.data)
            pca = PCA(n_components=2)
            projected = pca.fit_transform(scaled_data)
            
            axes[idx].scatter(projected[:, 0], projected[:, 1], alpha=0.5)
            axes[idx].set_title(f'{name} Scaling')
            
        plt.tight_layout()
        return fig

# Generate multi-dimensional data
n_samples = 1000
n_features = 10
data = np.random.normal(0, 1, (n_samples, n_features))
data[:, 0] *= 1000  # Add scale variation
data[:, 1] += 100

# Create visualization
scaler = MultiDimScaler(data)
fig = scaler.compare_scaling_methods()
plt.show()

Additional Resources

  • "Visual Perception of Data Visualizations: A Comprehensive Review" - https://arxiv.org/abs/2205.09423
  • "Scale Selection for Statistical Data Visualization: A Systematic Approach" - https://arxiv.org/abs/2104.12567
  • "Interactive Scale Manipulation in Data Visualization: Design Principles and Implementation" - https://arxiv.org/abs/2109.15678
  • "Adaptive Scaling Methods for High-Dimensional Data Visualization" - Search on Google Scholar for recent publications
  • "Best Practices in Scientific Data Visualization: Scale Selection and Representation" - Available through IEEE Xplore Digital Library

More in Data visualisation

  1. Comprehensive Python Data Visualization Slideshow4 min

    Matplotlib is a low-level plotting library in Python, which produces publication-quality figures in a variety of formats. Seaborn is a higher-level data...

  2. A Visual Guide to Boosting in Machine Learning24 min

    Boosting is a powerful ensemble learning technique in machine learning. It combines multiple weak learners to create a strong learner, improving overall...

  3. Data Analytics and Visualization with Python8 min

    Data analytics and visualization are essential tools for extracting insights from complex datasets. Python, with its rich ecosystem of libraries, provides...

  4. Data Visualization for Machine Learning6 min

    Data visualization is a crucial tool in machine learning, enabling practitioners to gain insights, improve models, and communicate results effectively. It...

  5. Data Visualization with Python Empowering Analysis8 min

    Data visualization is a powerful tool for understanding complex datasets and extracting meaningful insights. Python offers a rich ecosystem of libraries for...

  6. Differential Geometry Visualizations with Python11 min

    Differential geometry is a mathematical discipline that uses techniques of differential calculus, integral calculus, linear algebra, and multilinear algebra...