Upload to Amazon S3 server directly from browser in chunk and resumable way

UPDATE: This works for AWS sdk version 2.4.0. AWS sdk version 2.4.1 throws body missing exception.

I have written some javascript and php code to make big local files to be uploaded in Amazon S3 server directly, in chunk of 5 MB (amazon web service says that the chunk size should be at least 5MB), so the upload is resumable and recover easily from upload or network error.

To make this work we need our browsers to be able to read local files and send the data to cross-domain server with XMLHttpRequest2. Most mainstream browsers’ recent versions support these operations (Firefox, Chrome, Opera, IE, etc. and some mobile browsers also)

So what do we need? We also need a signing server which will sign the request we will send to the S3 server from browser (Why signing needed? Because we should not allow anybody to upload or manipulate our S3 bucket, shall we?). In my demonstration I used php backend server, but any backend can be used.

If you need a demonstration for inspiration or a starting point for your project, you can look at these source codes here – https://github.com/ienzam/s3-multipart-upload-browser

Please put your valuable comments ans suggestions.

Thank you all for reading.

Advertisements

AWS S3, PHP & Javascript – Get it working

UPDATE: This works for AWS sdk version 2.4.0. AWS sdk version 2.4.1 throws body missing exception.

Let me explain what I was trying to do. I was trying to call amazon web services from javascript directly, but as I can’t (must not) embed aws secret key to javascript I have to use some kind of server to sign my requests for me. As I was working on php, I am trying to use aws-sdk-php to sign my requests. After a lot of digging on the sdk code here is the rough hack code I have came up with (you need to set cors permission on your bucket to access from javascipt)

<?php
require 'vendor/autoload.php';

use Aws\Common\Enum\DateFormat;
use Aws\S3\S3Client;
use Guzzle\Common\Event;

$client = S3Client::factory(array(
'key' => AWS_KEY,
'secret' => AWS_SECRET
));

// for all available commands, go to
// http://docs.aws.amazon.com/aws-sdk-php-2/guide/latest/service-s3.html
$command = $client->getCommand('GetBucketAcl', array(
'Bucket' => BUCKET_NAME
));

$request = $command->prepare();
$request->setHeader('x-amz-date', gmdate(DateFormat::RFC2822));

// searching for these event dispatchers wasted a great deal of my time 😡
$client->getEventDispatcher()->dispatch('command.before_send', new Event(array(
'command' => $command,
)));

$client->getEventDispatcher()->dispatch('request.before_send', new Event(array(
'request' => $request
)));
?>
<html>
<body>
<p>Open the browser console and network monitor to see the outputs</p>
<button onclick=&quot;doXMLHttpRequest()&quot;>XMLHttpRequest</button>
<button onclick=&quot;doJQueryRequest()&quot;>JQuery Request</button>
<script src=&quot;jquery-1.9.1.min.js&quot;></script>
<script>
var url = &quot;<?php echo($request->getUrl()) ?>&quot;;
var authHeader = &quot;<?php echo($request->getHeader('Authorization')) ?>&quot;;
var dateHeader = &quot;<?php echo($request->getHeader('x-amz-date')) ?>&quot;;
var methodType = &quot;<?php echo($request->getMethod()) ?>&quot;;

console.log(url);
console.log(authHeader);
console.log(methodType);

function doXMLHttpRequest() {
var request = new XMLHttpRequest();
request.open(methodType, url, true);
request.onreadystatechange = function() {
if (request.readyState === 4) {
console.log(&quot;Request complete from XMLHttpRequest, request object below&quot;);
console.log(request);
}
}
request.setRequestHeader(&quot;x-amz-date&quot;, dateHeader);
request.setRequestHeader(&quot;Authorization&quot;, authHeader);
request.send();
}

function doJQueryRequest() {
$.ajax({
url: url,
headers: {
Authorization: authHeader,
&quot;x-amz-date&quot;: dateHeader
},
type: methodType
}).done(function(data) {
console.log(&quot;Request complete from JQuery, response data below&quot;);
console.log(data);
});
}
</script>
</body>
</html>

Yes I know I have written very short description. But check out the code, that’s what matters, right? ( I am lazy 😛 )