A simple implementation of a KDE with cross validation to find the best bandwith parameter.
This is a simple implementation of a KDE - Kernel Density Extimator in Odin that finds automatically or manually the bandwith parameter. It find a smooth N Gaussian PDF - Probability Density Function for the N dimensional data. Then you can sample has many points has you like from the PDF.
Currently the method os sampling calculates the contribution ( sum ) of the gaussian of all points with the corresponding bandwith, to the final sample value for each dimension in the N dimensional sapace, but one can do better then that with a KDT-Tree to spacially separate the gaussians and so that the complexity don't be O( n * d ) where n is the number of datapoints and d is the simensions. That can be made because after 5 standard deviations of distance the constribution of a nearby gaussian is virtually zero for the current sample point. For low dimensions space hashtables can also be used.
$ time ./kde_kernel_density_extimator.exe
Kernel Density Extimator - Extimate the PDF of the samples derivated
from a Gaussian smoth model, with with bandwith extimated by cross validation....
==> Original Samples to learn the PDF - Probability Density Function.
sample 1: (-5.100, -5.000)
sample 2: (-4.900, -5.100)
sample 3: (-5.000, -4.900)
sample 4: (-5.200, -5.000)
sample 5: (-5.000, -5.200)
sample 6: (5.100, 5.000)
sample 7: (4.900, 5.100)
sample 8: (5.000, 4.900)
sample 9: (5.200, 5.000)
sample 10: (5.000, 5.200)
sample 11: (0.100, 5.000)
sample 12: (-0.100, 5.100)
sample 13: (0.000, 4.900)
sample 14: (0.200, 5.000)
sample 15: (0.000, 5.200)
==> Starting Cross-Validation.
Testing h = 0.1000 ... Score = -4.59
Testing h = 0.2000 ... Score = -6.97
Testing h = 0.5000 ... Score = -27.10
Testing h = 0.8000 ... Score = -40.27
Testing h = 1.0000 ... Score = -46.75
Testing h = 1.2000 ... Score = -52.10
Testing h = 1.5000 ... Score = -58.65
Testing h = 2.0000 ... Score = -66.71
Testing h = 3.0000 ... Score = -76.63
==> Cross-Validation Complete.
===>Fitting N-Dimensional KDE Model.
Original data points: 15
Dimensions: 2
Best Bandwidth ( from CV ): 0.100
Model fitted successfully.
==> Generating 20 New 2-D Samples
Sample 1: (5.005, 5.032)
Sample 2: (-4.825, -4.978)
Sample 3: (-5.014, -4.966)
Sample 4: (0.066, 4.942)
Sample 5: (-4.915, -4.692)
Sample 6: (-5.180, -4.920)
Sample 7: (4.962, 4.870)
Sample 8: (4.876, 5.287)
Sample 9: (-0.001, 4.871)
Sample 10: (-5.000, -5.084)
Sample 11: (5.016, 4.892)
Sample 12: (-0.062, 5.212)
Sample 13: (5.030, 5.083)
Sample 14: (4.981, 4.823)
Sample 15: (-0.196, 4.982)
Sample 16: (5.065, 4.954)
Sample 17: (0.073, 5.294)
Sample 18: (5.122, 5.110)
Sample 19: (4.973, 5.087)
Sample 20: (0.047, 5.183)
...end of Kernel Density Extimator - Extimate the PDF of the samples derivated
from a Gaussian smoth model, with with bandwith extimated by cross validation....
MIT Open Source License
Best regards, Joao Carvalho