[FIXED] How to unzip a zip with multiple files into my lambda node.js function tmp directory preserving the original filenames

Issue

I have a zip file in S3 that contains several txt files and an image.

I am needing to grab that file from S3 and then unzip all of the files with their original filenames into the tmp directory in lambda using node.js

I am not super familiar with node and while the S3 part of getting the file works fine, I frankensteined the zip part together from the interwebs and was wondering if someone could help me get it right.

const zlib = require('zlib');
const fs = require('fs');
    
try { 
  const s3Object = await s3
    .getObject({
      Bucket: 'mybucket',
      Key: `zip/${myzipfilename}`
    })
    .promise();

  console.log("s3 zip fetched");
                
  // write file to tmp          
  writeFileSync(`/tmp/${myzipfilename}`, s3Object.Body);
        
  //unzip files
  const fileContents = fs.createReadStream(`/tmp/${myzipfilename}`);

  //I am quite sure this part is incorrect and is currently giving me an error
  const writeStream = fs.createWriteStream(`./tmp/${filename.slice(0, -3)}`);

  const unzip = zlib.createGunzip();
  fileContents.pipe(unzip).pipe(writeStream);            
}

End result within the lambda tmp directory would be something like:

/tmp/myoriginaltxtfilename.txt
/tmp/myoriginaltxtfilename2.txt
/tmp/myoriginaltxtfilename3.txt
/tmp/myoriginalimagefilename.png

I don't need to rezip anything.

Solution

You have a couple of issues in your code. First of all, at this line:

const writeStream = fs.createWriteStream(`./tmp/${filename.slice(0, -3)}`);

filename is not defined.

Second, you're using nodejs zlib to extract a .zip file which contains multiple files which won't work. zlib module is only for streams and buffers that represent singular resources not zip archives. You could use node-stream-zip instead.

Let's say you've successfully downloaded the zip file from S3 and saved in /tmp directory. Using node-stream-zip extracting the files from the zip file without unzipping it would look something like this:

const StreamZip = require('node-stream-zip');
const zip = new StreamZip({
    file: `/tmp/${myzipfilename}`,
    storeEntries: true
});

zip.on('ready', () => {
  console.log('All entries read: ' + zip.entriesCount);
});

zip.on('entry', (entry) => {
  if ('/' === entry.name[entry.name.length - 1]) {
    console.log('[DIR]', entry.name);
    return;
  }

  console.log('[FILE]', entry.name);

  zip.stream(entry.name, (err, stream) => {
    if (err) {
      console.log('Error: ', err.toString());
      return;
    }

    stream.on('error', (err) => {
      console.log('[ERROR]', err);
      return;
    });

    stream.pipe(fs.createWriteStream(`/tmp/${entry.name}`));
  });
});

Answered By - Rahul Sharma

Answer Checked By - Robin (PHPFixing Admin)