Issue
I'm saving data to CSV files using pandas. I have one important column with dtype: datetime64[ns].Somehow the datatype is changed to object when I read the data back from CSV file. How can I write, read while keeping the same datatype? Is this related to the encoding type?
df = pd.io.sql.read_sql(sql, cnxn)
df.to_csv(fileName)
df.TimeSeries
Name: TimeSeries, Length: 10000, dtype: datetime64[ns]
DF = pd.read_csv(fileName, sep=',')
DF.TimeSeries
Name: TimeSeries, Length: 10000, dtype: object
Solution
CSV files do not store data types. Data in CSV files is stored as text.
Your best options are:
- Store in a serialized or other type-aware format (pickle, HDF5) if this is appropriate for your use case.
- Use the
parse_dates
argument ofpd.read_csv
, e.g.df = pd.read_csv(filename, sep=',', parse_dates=['Date'])
. Seepd.read_csv
documentation for more details.
The second option is just a workaround. It will convert text back to datetime
when you read the data into a dataframe.
Answered By - jpp Answer Checked By - Dawn Plyler (PHPFixing Volunteer)
0 Comments:
Post a Comment
Note: Only a member of this blog may post a comment.