[FIXED] How can I read values in an exponential format from csv into a numpy array?

Issue

I have values in this format, where e+002 and so on is the exponent:

 1.25663709640503E+0000    2.33967334032059E-0001
 2.51327419281006E+0000    4.85565841197968E-0001
 3.76991128921509E+0000    3.30846726894379E-0001
 5.02654838562012E+0000    5.50593174993992E-0002
 6.28318548202515E+0000    3.12543800100684E-0003
 7.53982257843018E+0000    4.11923155188560E-0002
 8.79645919799805E+0000    1.35717853903770E-0001
 1.00530967712402E+0001    1.26785650849342E-0001
 1.13097343444824E+0001    2.28818021714687E-0002
 1.25663709640502E+0001    3.12676839530468E-0003
 1.38230075836181E+0001    2.36203446984291E-0002
 1.50796451568603E+0001    7.74327516555786E-0002
 1.63362827301025E+0001    7.97238126397133E-0002
 1.75929183959961E+0001    1.36453993618488E-0002
 1.88495559692383E+0001    3.12899192795157E-0003

I tried to read them into a numpy array using this code:

import numpy as np

with open(r"fft_in.XY") as file_name:
    array = np.loadtxt(file_name, delimiter=",")

print(array)

However, it does not work, I get the following errors:

Traceback (most recent call last):
  File "D:\Python_projects\rt_fft\main.py", line 4, in <module>
    array = np.loadtxt(file_name, delimiter=",")
  File "C:\Users\achim\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\lib\npyio.py", line 1148, in loadtxt
    for x in read_data(_loadtxt_chunksize):
  File "C:\Users\achim\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\lib\npyio.py", line 999, in read_data
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "C:\Users\achim\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\lib\npyio.py", line 999, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "C:\Users\achim\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\lib\npyio.py", line 736, in floatconv
    return float(x)
ValueError: could not convert string to float: ' 0.00000000000000E+0000    0.00000000000000E+0000'

Does someone maybe know, whether there is a built-in function to read numbers in this format from csv into a numpy array? Thanks a lot for any replies and ideas!

Solution

The issue is not your code, that actually works fine. The issue is your data.

Notice how in this line:

array = np.loadtxt(file_name, delimiter=",")

You have delimiter="," but you do not have any , in your data? If you replace your data with:

 1.25663709640503E+0000,2.33967334032059E-0001
 2.51327419281006E+0000,4.85565841197968E-0001
 3.76991128921509E+0000,3.30846726894379E-0001
 5.02654838562012E+0000,5.50593174993992E-0002
 6.28318548202515E+0000,3.12543800100684E-0003
 7.53982257843018E+0000,4.11923155188560E-0002
 8.79645919799805E+0000,1.35717853903770E-0001
 1.00530967712402E+0001,1.26785650849342E-0001
 1.13097343444824E+0001,2.28818021714687E-0002
 1.25663709640502E+0001,3.12676839530468E-0003
 1.38230075836181E+0001,2.36203446984291E-0002
 1.50796451568603E+0001,7.74327516555786E-0002
 1.63362827301025E+0001,7.97238126397133E-0002
 1.75929183959961E+0001,1.36453993618488E-0002
 1.88495559692383E+0001,3.12899192795157E-0003

Then your script works fine because it uses the delimiter that numpy expects.

To make it work with the data you have, you should change:

array = np.loadtxt(file_name, delimiter=",")

To:

array = np.loadtxt(file_name, delimiter="    ")

Although you have to be sure that the delimiter always contains the same number of white spaces

Answered By - Lars Nielsen

Answer Checked By - Senaida (PHPFixing Volunteer)