[FIXED] How would you remove trailing comma when writing to a stream json in node.js?

Issue

Suppose I have code as below:

const fs = require('node:fs')
const csvParser = require('csv-parser')

const rs = fs.createReadStream(dir)
const ws = fs.createWriteStream(outDir)

ws.write('[' + '\n')
rs.pipe(
    csvParser({
      mapHeaders: mapper,
    }),
)
.on('data', (data) => {
      ws.write('\t' + JSON.stringify({ ...data, provider } + ',\n'))
})
.on('end', () => {
      ws.write(']')
      console.log('writing to a file has been completed >> ', OUT_DIR)
})

Basically what this code does is to read from large csv file then stream this into write a json file. As seen in the code, I append ',\n' as separator for each row such that all the json code is formatted once streaming has been completed.

The issue with such method is that it leaves comma at the end of each entry that you would have to manually format the file (i.e. removing trailing comma) each time document write is completed.

For instance, the result would come out be as such:

[
  {"key1":"value1", "key2":"value2"},
  {"key1":"value1", "key2":"value2"}, // <- note the trailing comma here. 
]

Any suggestion as to handling the trailing comma? Also, what would be alternative way of handling read and write stream better, please let me know. Thanks in advance.

Solution

I'd reverse the problem and include the comma before every item except the first.

let first = true;
ws.write('[' + '\n')
// ...
.on('data', (data) => {
  if (!first) {
    ws.write(',\n');
  } 
  first = false;
  ws.write('\t' + JSON.stringify({ ...data, provider }))
})

Answered By - Quentin

Answer Checked By - Terry (PHPFixing Volunteer)