This a very simple method with really high precision.
I do this repository in Odin, for self documentation reasons and to have a working example to play with the technique and learn from it. My favorite programming language after several decades of programming in many programming languages :-)
Sometime ago I bought some courses from Udemy from a German Physics Professor with that were something like Computational Mathematics or Computational Physics, I can't remember. They are very good courses, and sometime ago I was learning from there some kind of more evolved methods of differences that produce better aproximations to the calculation of the derivatives. I was very happy to learn about those methods, but today I was thinking, "What is the best method ( a precise, simples and fast method ) of calculating the first derivative of a 1D real function?" .
And after some time searching, I have found the first reference from MathOverflow, I read it and it spoke of a technique that was very simple but very accurate, I was currious and intrigged to learn more about it! So I followed the links and read all of the following references, the method is quite simple, but incredible precise. After reading all the references I wanted to make a simple repository to test it and document for future me with code that replicates in the Odin programming language, the article code in Matlab, that is present in that last reference ( see below ).
fd = imag( f( x + sqrt( -1 ) * h ) ) / h
package main
import "core:fmt"
import "core:math"
import "core:math/cmplx"
f64_2_complex :: #force_inline proc (
real_part : f64,
imag_part : f64 ) ->
( c : complex128 ) {
c = complex128( complex( real_part , imag_part ) )
return c
}
My_Real_Func :: proc ( x : f64 ) -> f64
My_Complex_Func :: proc ( x : complex128 ) -> complex128
// Computes the complex step approximation to the derivative of f at point x,
// using step h ( default 1e-9 )
//
// fd = imag( f( x + sqrt( -1 ) * h ) ) / h
//
complex_step :: proc ( f : My_Complex_Func,
x : f64,
h : f64 = 1e-8 ) ->
( func_derivative : f64 ) {
real_part := x
imag_part := h
complex_num := f64_2_complex( real_part, imag_part )
func_derivative = cmplx.imag( f( complex_num ) ) / h
return func_derivative
}
// Calculate the aproximation of the derivative of f,
// by using the finite diferences.
finite_diff :: proc ( f : My_Real_Func,
x : f64,
h : f64 = 1e-8 ) ->
( func_derivative : f64 ) {
func_derivative = ( f( x + h ) - f( x - h ) ) / ( 2 * h )
return func_derivative
}
// f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )
the_real_func :: proc ( x : f64 ) ->
( f_x : f64 ) {
cos_x : f64 = math.cos( x )
sin_x : f64 = math.sin( x )
f_x = math.exp( x ) /
( ( cos_x * cos_x * cos_x ) +
( sin_x * sin_x * sin_x ) )
return f_x
}
// f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )
the_complex_func :: proc ( x : complex128 ) ->
( f_x : complex128 ) {
cos_x : complex128 = cmplx.cos( x )
sin_x : complex128 = cmplx.sin( x )
f_x = cmplx.exp( x ) /
( ( cos_x * cos_x * cos_x ) +
( sin_x * sin_x * sin_x ) )
return f_x
}
./derivative_complex_step.exe
Numerical Diferentiation
Comparision of error, true value VS complex step VS finite differences ...
Equation:
f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )
Analitical derivative:
f_drv = ( exp( x ) * ( cos( 3 * x ) + sin( 3 * x ) / 2 +
( 3 * sin( x ) ) / 2 ) ) / ( cos( x )^3 + sin( x )^3 )^2
Evaluated at point x = PI / 4
Simplifing:
exact = 2^( 1 / 2 ) * exp( PI / 4 )
exact = 3.101766393836051
p | h | complex_step | finite diferences
----------------------------------------------------------------
01 0.1000000000000000 3.144276040634560 3.061511866568116
02 0.0100000000000000 3.102180075411272 3.101352937655876
03 0.0010000000000000 3.101770529535844 3.101762258158169
04 0.0001000000000000 3.101766435192939 3.101766352480161
05 0.0000100000000000 3.101766394249620 3.101766393398540
06 0.0000010000000000 3.101766393840187 3.101766393509563
07 0.0000001000000000 3.101766393836095 3.101766399282722
08 0.0000000100000000 3.101766393836054 3.101766399282722
09 0.0000000010000000 3.101766393836054 3.101766399282722
10 0.0000000001000000 3.101766393836052 3.101758849766155
11 0.0000000000100000 3.101766393836053 3.101785495118746
12 0.0000000000010000 3.101766393836053 3.101519041592836
13 0.0000000000001000 3.101766393836052 3.101963130802685
14 0.0000000000000100 3.101766393836053 3.108624468950436
15 0.0000000000000010 3.101766393836053 3.108624468950436
16 0.0000000000000001 3.101766393836052 4.440892098500622
Conclusion:
Complex Step Method is Better
then Finite Differences Method!
The end
-
MathOverflow
Numerical differentiation. What is the best method?
https://mathoverflow.net/questions/64302/numerical-differentiation-what-is-the-best-method -
Differentiation With(out) a Difference
by By Nicholas J. Higham
https://www.siam.org/publications/siam-news/articles/differentiation-without-a-difference -
Github higham - complex_step_example.m
https://gist.github.com/higham/6668082d7d51442474f11ae96130d2db -
Github higham - complex_step.m
https://gist.github.com/higham/5c32cce0532ed5c863615d588548518d -
Complex Step Differentiation
by Cleve Moler
https://blogs.mathworks.com/cleve/2013/10/14/complex-step-differentiation/
MIT Open Source License
Best regards,
Joao Carvalho