PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Tuesday, November 1, 2022

[FIXED] How to prevent "pandas.read_csv" convert index column to float with arg 'dtype=np.float32'?

 November 01, 2022     csv, indexing, pandas, python, type-conversion     No comments   

Issue

I have a CSV file to be read by pandas, and it has the form as following:

name,   quart2c,    p_rat,  other_col
avg,    1,          2,      3
std,    1,          2,      3

I want to pandas.read_csv() guarantee that all cells have the type of float32, except the first column('name') because that is the index column.

Hence I pass two args to it like this:

pandas.read_csv(file_path, index_col=0, dtype=np.float32)

# or like this, both failed
pandas.read_csv(file_path, index_col='name', dtype=np.float32)

But pandas still tries to convert the first column to float, and raises a exception:

ValueError: could not convert string to float: 'avg'

What I want:

  1. The CSV file is made by another program coded by myself. If the structure is wrong, I can adjust it easily.
  2. I want to always specify the arg dtype=np.float32, so as to check whether there are any error values. I don't want the values be interpreted to integer type also.
  3. The index column "name" should be reserved as index_col, since it will be used later. This column should NOT be cut off anyway.

How should I get it?


Solution

Best is to first read in the csv with default args, giving index col, and then convert the entire df (which will not affect the index):

pd.read_csv(file_path, index_col='name').astype(float)


Answered By - Josh Friedlander
Answer Checked By - Robin (PHPFixing Admin)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing