o
odinpkg.dev
packages / library / Calculate_high_precision_first_derivative_using_the_complex_step_method

Calculate_high_precision_first_derivative_using_the_complex_step_method

c775122library

This a very simple method with really high precision.

No license · updated 7 months ago

Calculate high precision first derivative using the complex step method

This a very simple method with really high precision.

Desciption

I do this repository in Odin, for self documentation reasons and to have a working example to play with the technique and learn from it. My favorite programming language after several decades of programming in many programming languages :-)
Sometime ago I bought some courses from Udemy from a German Physics Professor with that were something like Computational Mathematics or Computational Physics, I can't remember. They are very good courses, and sometime ago I was learning from there some kind of more evolved methods of differences that produce better aproximations to the calculation of the derivatives. I was very happy to learn about those methods, but today I was thinking, "What is the best method ( a precise, simples and fast method ) of calculating the first derivative of a 1D real function?" .
And after some time searching, I have found the first reference from MathOverflow, I read it and it spoke of a technique that was very simple but very accurate, I was currious and intrigged to learn more about it! So I followed the links and read all of the following references, the method is quite simple, but incredible precise. After reading all the references I wanted to make a simple repository to test it and document for future me with code that replicates in the Odin programming language, the article code in Matlab, that is present in that last reference ( see below ).

The Complex Step Method

fd = imag( f( x + sqrt( -1 ) * h ) ) / h

package main

import "core:fmt"
import "core:math"
import "core:math/cmplx"

f64_2_complex :: #force_inline proc (
                    real_part  : f64,
                    imag_part  : f64 ) ->
                  ( c : complex128  ) {

    c = complex128( complex( real_part , imag_part ) )

    return c
}

My_Real_Func :: proc ( x : f64 ) -> f64

My_Complex_Func :: proc ( x : complex128 ) -> complex128

// Computes the complex step approximation to the derivative of f at point x,
// using step h ( default 1e-9 )
//
// fd = imag( f( x + sqrt( -1 ) * h ) ) / h
//
complex_step :: proc ( f : My_Complex_Func,
                       x : f64,
                       h : f64 = 1e-8 ) ->
                     ( func_derivative : f64 ) {

    real_part := x
    imag_part := h
    complex_num := f64_2_complex( real_part, imag_part )
    func_derivative = cmplx.imag( f( complex_num ) ) / h

    return func_derivative
}

// Calculate the aproximation of the derivative of f,
// by using the finite diferences.
finite_diff :: proc ( f : My_Real_Func,
                      x : f64,
                      h : f64 = 1e-8 ) ->
                     ( func_derivative : f64 ) {

    func_derivative = ( f( x + h ) - f( x - h ) ) / ( 2 * h )

    return func_derivative
}

// f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )
the_real_func :: proc ( x : f64 ) ->
                      ( f_x : f64 ) {

    cos_x : f64 = math.cos( x )
    sin_x : f64 = math.sin( x )
    f_x = math.exp( x ) /
          ( ( cos_x * cos_x * cos_x ) +
            ( sin_x * sin_x * sin_x ) )

    return f_x
}

// f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )
the_complex_func :: proc ( x : complex128 ) ->
                         ( f_x : complex128 ) {

       cos_x : complex128 = cmplx.cos( x )
       sin_x : complex128 = cmplx.sin( x )
       f_x = cmplx.exp( x ) /
             ( ( cos_x * cos_x * cos_x ) +
               ( sin_x * sin_x * sin_x ) )

       return f_x
}

Output results

./derivative_complex_step.exe

Numerical Diferentiation
Comparision of error, true value VS complex step VS finite differences ...

Equation:
    f( x ) = e^x / ( ( cos( x ) )^3 + ( sin( x ) )^3 )

Analitical derivative:
   f_drv = ( exp( x ) * ( cos( 3 * x ) + sin( 3 * x ) / 2 +
           ( 3 * sin( x ) ) / 2 ) ) / ( cos( x )^3 + sin( x )^3 )^2

Evaluated at point x = PI / 4

Simplifing:
  exact = 2^( 1 / 2 ) * exp( PI / 4 )

exact = 3.101766393836051

p  |          h         |   complex_step    | finite diferences
----------------------------------------------------------------
01   0.1000000000000000   3.144276040634560   3.061511866568116
02   0.0100000000000000   3.102180075411272   3.101352937655876
03   0.0010000000000000   3.101770529535844   3.101762258158169
04   0.0001000000000000   3.101766435192939   3.101766352480161
05   0.0000100000000000   3.101766394249620   3.101766393398540
06   0.0000010000000000   3.101766393840187   3.101766393509563
07   0.0000001000000000   3.101766393836095   3.101766399282722
08   0.0000000100000000   3.101766393836054   3.101766399282722
09   0.0000000010000000   3.101766393836054   3.101766399282722
10   0.0000000001000000   3.101766393836052   3.101758849766155
11   0.0000000000100000   3.101766393836053   3.101785495118746
12   0.0000000000010000   3.101766393836053   3.101519041592836
13   0.0000000000001000   3.101766393836052   3.101963130802685
14   0.0000000000000100   3.101766393836053   3.108624468950436
15   0.0000000000000010   3.101766393836053   3.108624468950436
16   0.0000000000000001   3.101766393836052   4.440892098500622


Conclusion:
  Complex Step Method is Better
  then Finite Differences Method!


The end

References:

  1. MathOverflow
    Numerical differentiation. What is the best method?
    https://mathoverflow.net/questions/64302/numerical-differentiation-what-is-the-best-method

  2. Differentiation With(out) a Difference
    by By Nicholas J. Higham
    https://www.siam.org/publications/siam-news/articles/differentiation-without-a-difference

  3. Github higham - complex_step_example.m
    https://gist.github.com/higham/6668082d7d51442474f11ae96130d2db

  4. Github higham - complex_step.m
    https://gist.github.com/higham/5c32cce0532ed5c863615d588548518d

  5. Complex Step Differentiation
    by Cleve Moler
    https://blogs.mathworks.com/cleve/2013/10/14/complex-step-differentiation/

License

MIT Open Source License

Have fun

Best regards,
Joao Carvalho