[FIXED] How to truncate csv file to n of rows not reading the whole file

Issue

I have big csv(12 gb), so I can't read it in memory, and I need only 100 rows of them and save it back(truncate). Has java such api?

Solution

The other answers create a new file from the original file. As I understand it, you want to truncate the original file instead. You can do that quite easily using RandomAccessFile:

    try (RandomAccessFile file = new RandomAccessFile(FILE, "rw")) {
        for (int i = 0;  i < N && file.readLine() != null; i++)
            ;  // just keep reading
        file.setLength(file.getFilePointer());
    }

The caveat is that this will truncate after N lines, which is not necessarily the same thing as N rows, because CSV files can have rows that span multiple lines. For example, here is one CSV record that has a name, address, and phone number, and spans multiple lines:

Joe Bloggs, "1 Acacia Avenue,
Naboo Town,
Naboo", 01-234 56789

If you are sure all your rows only span one line, then the above code will work. But if there is any possibility that your CSV rows may span multiple lines, then you should first parse the file with a suitable CSV reader to find out how many lines you need to retain before you truncate the file. OpenCSV makes this quite easy:

    final long numLines;
    try (CSVReader csvReader = new CSVReader(new FileReader(FILE))) {
        csvReader.skip(N);  // Skips N rows, not lines
        numLines = csvReader.getLinesRead();  // Gives number of lines, not rows
    }

    try (RandomAccessFile file = new RandomAccessFile(FILE, "rw")) {
        for (int i = 0; i < numLines && file.readLine() != null; i++)
            ;  // just keep reading
        file.setLength(file.getFilePointer());
    }

Answered By - k314159

Answer Checked By - Dawn Plyler (PHPFixing Volunteer)