[FIXED] AWS S3 file upload over PHP and HTTP

Issue

I'm wanting to use Amazons S3 service to store the files that users upload to my LAMP package. I'm wondering what would be the best (time, cost, security, etc) to do this. I'm familar with uploading files using HTTP with PHP processing but I've always saved it to the local storage. Should I have a tmp directory that I have the SDK upload from or should or even could I upload the file to S3 from a data variable? Also I would like to be able to handle 5 GB files but at the moment I'm only running a half a GB of ram, would this cause any issues while I'm in the alfa of my project? Keep in mind my web server is an EC2 server. Thanks for the help.

Solution

Q1: Uploading From memory without creating temporary file.

Yes, you can do it. The Amazon SDK has "putObjectFile" and "putObjectString" functions, the first creates an object from a temporary file, the second from a string.

Q2: Uploading large files (5GB).

While you can get a server with 5GB of memory, it's a bit overkill just to store the data to upload entriely in memory while the upload happens - so going for a temporary file for upload and streaming chunk by chunk from that file would probably be wise. To handle chunks in curl in PHP, you may need to add CURLOPT_READFUNCTION that reads a bit of the file at a time for upload.

The name of a callback function where the callback function takes three parameters. The first is the cURL resource, the second is a stream resource provided to cURL through the option CURLOPT_INFILE, and the third is the maximum amount of data to be read. The callback function must return a string with a length equal or smaller than the amount of data requested, typically by reading it from the passed stream resource. It should return an empty string to signal EOF.

You can find the curl functions in the Amazon SDK, function getResponse(). The class is labelled "final" so you'll need to actually modify the SDK to add this in.

Q3: Costs. A server with a slightly larger hard disk (to store temp files) is most likely cheaper than adding memory.

Q4: Security. You can store your temp files out of web root, so they will be as secure as your webserver is. If your webserver gets compromised, they'll get your Amazon Secret Key anyway - so this shouldn't really be any more concern than protecting the rest of your application.

Answered By - Robbie