Datalens.ai
← All algorithms

OPTIMISATION

Gradient Descent

Iteratively update parameters to reduce an objective.

Example of usage

Fit model parameters by minimising error.

Usage / limitations

+Use when: Optimise differentiable objectives.

−Limitations: Sensitive to learning rate and may reach local minima.

Mathematical formula

theta_next = theta - learning_rate * gradient(loss)
Sample dataShow 
{
  "variable": "design_case",
  "domain": "Engineering parameter estimation and optimisation",
  "feature_names": [
    "measurement_time_s",
    "response_mm"
  ],
  "records": [
    {
      "measurement_time_s": 0,
      "response_mm": 1
    },
    {
      "measurement_time_s": 1,
      "response_mm": 3
    },
    {
      "measurement_time_s": 2,
      "response_mm": 5
    },
    {
      "measurement_time_s": 3,
      "response_mm": 7
    },
    {
      "measurement_time_s": 4,
      "response_mm": 9
    }
  ],
  "target_value": 4.0,
  "lower_bound": 0.0,
  "upper_bound": 4.0,
  "method": "gradient-descent"
}

Code example (Python)

# DataLens.ai example — Gradient Descent
# Domain: engineering parameter estimation and optimisation.
# The sample data are included by “Copy input + code + test”.
import numpy as np
from scipy import optimize
# Read the named engineering measurements.
times = np.array([row["measurement_time_s"] for row in design_case["records"]], dtype=float)
responses = np.array([row["response_mm"] for row in design_case["records"]], dtype=float)
# Use a known target, making the numerical assertion independently meaningful.
target = design_case["target_value"]

# Minimise a squared calibration error with an explicit iterative algorithm.
setting = 0.0
for iteration in range(100):
    # Derivative of (setting - 2)^2 is 2*(setting - 2).
    setting -= 0.2 * 2 * (setting - 2.0)
assert np.isclose(setting, 2.0, atol=1e-6)
print("Expected output verified")