News‎ > ‎

Submitted Paper: A New Line Search Speeds Up LBFGS for Parallel Logistic Regression in Spark

posted Oct 31, 2015, 6:09 AM by Hans De Sterck   [ updated Jan 20, 2016, 1:36 AM ]
Accepted for SIAM Data Mining conference 2016 - We submitted "A polynomial expansion line search for large-scale unconstrained minimization of smooth L2-regularized loss functions, with implementation in Apache Spark" (Hynes and De Sterck). See http://arxiv.org/abs/1510.08345.

Our new line search uses a simple polynomial expansion idea, but for smooth objective functions (like logistic regression) it makes the line search more accurate than commonly used approaches, which decreases the number of iterations required by LBFGS and leads to speed-up in parallel of 30% and more.
Speed-up as a function of accuracy.
(This figure shows speedup as a function of accuracy for some large problems on 16 nodes/256 cores.)

Comments