Category Archives: Publication

NEW PAPER: Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket Search

T. Arora and C. Teuscher, Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket Search, 2025 Under review. https://arxiv.org/abs/2512.07142

Abstract:

The Lottery Ticket Hypothesis asserts the existence of highly sparse, trainable subnetworks (‘winning tickets’) within dense, randomly initialized neural networks. However, state-of- the-art methods of drawing these tickets, like Lottery Ticket Rewinding (LTR), are computationally prohibitive, while more efficient saliency-based Pruning-at-Initialization (PaI) techniques suffer from a significant accuracy-sparsity trade-off and fail basic sanity checks. In this work, we argue that PaI’s reliance on first-order saliency metrics, which ignore inter-weight dependencies, contributes substantially to this performance gap, especially in the sparse regime. To address this, we introduce Concrete Ticket Search (CTS), an algorithm that frames subnetwork discovery as a holistic combinatorial optimization problem. By leveraging a Concrete relaxation of the discrete search space and a novel gradient balancing scheme (GRADBALANCE) to control sparsity, CTS efficiently identifies high-performing subnetworks near initialization without requiring sensitive hyperparameter tuning. Motivated by recent works on lottery ticket training dynamics, we further propose a knowledge distillation-inspired family of pruning objectives, finding that minimizing the reverse Kullback-Leibler divergence between sparse and dense network outputs (CTSKL) is particularly effective. Experiments on varying image classification tasks show that CTS produces subnetworks that robustly pass sanity checks and achieve accuracy comparable to or exceeding LTR, while requiring only a small fraction of the computation. For example, on ResNet-20 on CIFAR10, CTSKL produces subnetworks of 99.3% sparsity with a top-1 accuracy of 74.0% in just 7.9 minutes, while LTR produces subnetworks of the same sparsity with an accuracy of 68.3% in 95.2 minutes. However, while CTS outperforms saliency-based methods in the sparsity-accuracy tradeoff across all sparsities, such advantages over LTR emerge most clearly only in the highly sparse regime.

Undergraduate summer research interns publish book with their work

Interns from three summer undergraduate research programs collaborated to produce a 448-page book showcasing their collective work: “Summer Proceedings 2025 – A Summer of Discovery: How Students Use Computational Modeling to Benefit Society.”

The book is available for purchase on Amazon at https://www.amazon.com/dp/B0FSL57NQ3 or as a free download from PDXScholar at https://archives.pdx.edu/ds/psu/44150. All proceeds support Oregon TechStart.

New paper published: “Against the Current: Introducing Reversibility to Superscalar Processors via Reversible Branch Predictors”

tlab PhD student Byron Gregg presented both a paper and a poster on “Against the Current: Introducing Reversibility to Superscalar Processors via Reversible Branch Predictors” at “The 15th International Green and Sustainable Computing Conference,” Austin, TX, 2024.

IGSCC proceedings: https://www.computer.org/csdl/proceedings/igsc/2024/22gEnJUWwMg

Citation:

B. Gregg and C. Teuscher, “Against the Current: Introducing Reversibility to Superscalar Processors via Reversible Branch Predictors,” 2024 IEEE 15th International Green and Sustainable Computing Conference (IGSC), Austin, TX, USA, 2024, pp. 135-141, doi: 10.1109/IGSC64514.2024.00033.

Abstract:

Although highly energy efficient, adiabatic and reversible systems suffer from performance drawbacks inherent to the physical operations that make them so efficient. Superscalar processors provide high performance through out-of-order speculative work of which an effective branch predictor is a key component in those performance gains. In the context of reversibility, a branch predictor is a design focal point because any fully reversible system must also be able to predict branch outcomes when in reverse mode. Taking advantage of Temporal Streaming techniques, this paper introduces several reversible branch predictor implementations which enable reversible and out-of-order instruction execution. These first-of-their-kind designs allow for a superscalar architecture that would maintain both a high level of performance and a high level of energy efficiency with the ability to un-compute obsolete data stored in memory. Testing our designs using the SimpleScalar out-of-order simulator, we estimate possible additional savings of 24 fJ per MB of data recovered at room temperature and at reverse prediction rates 2.27% higher than the forward. This work opens new avenues for designing and developing what we are calling Fully Adiabatic, Reversible, and Superscalar (FARS) Processor Architectures and is the first of many adaptations of conventional superscalar components to a reversible system.