Fast post-hoc method for updating moments of large datasets
December 21, 2018 Β· Declared Dead Β· π arXiv.org
"No code URL or promise found in abstract"
Evidence collected by the PWNC Scanner
Authors
Benjamin J. Q. Woods
arXiv ID
1812.09372
Category
cs.DS: Data Structures & Algorithms
Citations
0
Venue
arXiv.org
Last Checked
5 months ago
Abstract
Moments of large datasets utilise the mean of the dataset; consequently, updating the dataset traditionally requires one to update the mean, which then requires one to recalculate the moment. This means that metrics such as the standard deviation, $R^2$ correlation, and other statistics have to be `refreshed' for dataset updates, requiring large data storage and taking long times to process. Here, a method is shown for updating moments that only requires the previous moments (which are computationally cheaper to store), and the new data to be appended. This leads to a dramatic decrease in data storage requirements, and significant computational speed-up for large datasets or low-order moments (n $\lesssim$ 10).
Community Contributions
Found the code? Know the venue? Think something is wrong? Let us know!
π Similar Papers
In the same crypt β Data Structures & Algorithms
π
π
The Cartographer
R.I.P.
π»
Ghosted
Route Planning in Transportation Networks
R.I.P.
π»
Ghosted
Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration
R.I.P.
π»
Ghosted
Hierarchical Clustering: Objective Functions and Algorithms
R.I.P.
π»
Ghosted
Graph Isomorphism in Quasipolynomial Time
π
π
The Cartographer
Simulation optimization: A review of algorithms and applications
Died the same way β π» Ghosted
R.I.P.
π»
Ghosted
Federated Learning: Strategies for Improving Communication Efficiency
R.I.P.
π»
Ghosted
In-Datacenter Performance Analysis of a Tensor Processing Unit
R.I.P.
π»
Ghosted
Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
R.I.P.
π»
Ghosted