Upload to Amazon S3 server directly from browser in chunk and resumable way

UPDATE: This works for AWS sdk version 2.4.0. AWS sdk version 2.4.1 throws body missing exception.

I have written some javascript and php code to make big local files to be uploaded in Amazon S3 server directly, in chunk of 5 MB (amazon web service says that the chunk size should be at least 5MB), so the upload is resumable and recover easily from upload or network error.

To make this work we need our browsers to be able to read local files and send the data to cross-domain server with XMLHttpRequest2. Most mainstream browsers’ recent versions support these operations (Firefox, Chrome, Opera, IE, etc. and some mobile browsers also)

So what do we need? We also need a signing server which will sign the request we will send to the S3 server from browser (Why signing needed? Because we should not allow anybody to upload or manipulate our S3 bucket, shall we?). In my demonstration I used php backend server, but any backend can be used.

If you need a demonstration for inspiration or a starting point for your project, you can look at these source codes here – https://github.com/ienzam/s3-multipart-upload-browser

Please put your valuable comments ans suggestions.

Thank you all for reading.

AWS S3, PHP & Javascript – Get it working

UPDATE: This works for AWS sdk version 2.4.0. AWS sdk version 2.4.1 throws body missing exception.

Let me explain what I was trying to do. I was trying to call amazon web services from javascript directly, but as I can’t (must not) embed aws secret key to javascript I have to use some kind of server to sign my requests for me. As I was working on php, I am trying to use aws-sdk-php to sign my requests. After a lot of digging on the sdk code here is the rough hack code I have came up with (you need to set cors permission on your bucket to access from javascipt)

<?php
require 'vendor/autoload.php';

use Aws\Common\Enum\DateFormat;
use Aws\S3\S3Client;
use Guzzle\Common\Event;

$client = S3Client::factory(array(
'key' => AWS_KEY,
'secret' => AWS_SECRET
));

// for all available commands, go to
// http://docs.aws.amazon.com/aws-sdk-php-2/guide/latest/service-s3.html
$command = $client->getCommand('GetBucketAcl', array(
'Bucket' => BUCKET_NAME
));

$request = $command->prepare();
$request->setHeader('x-amz-date', gmdate(DateFormat::RFC2822));

// searching for these event dispatchers wasted a great deal of my time 😡
$client->getEventDispatcher()->dispatch('command.before_send', new Event(array(
'command' => $command,
)));

$client->getEventDispatcher()->dispatch('request.before_send', new Event(array(
'request' => $request
)));
?>
<html>
<body>
<p>Open the browser console and network monitor to see the outputs</p>
<button onclick=&quot;doXMLHttpRequest()&quot;>XMLHttpRequest</button>
<button onclick=&quot;doJQueryRequest()&quot;>JQuery Request</button>
<script src=&quot;jquery-1.9.1.min.js&quot;></script>
<script>
var url = &quot;<?php echo($request->getUrl()) ?>&quot;;
var authHeader = &quot;<?php echo($request->getHeader('Authorization')) ?>&quot;;
var dateHeader = &quot;<?php echo($request->getHeader('x-amz-date')) ?>&quot;;
var methodType = &quot;<?php echo($request->getMethod()) ?>&quot;;

console.log(url);
console.log(authHeader);
console.log(methodType);

function doXMLHttpRequest() {
var request = new XMLHttpRequest();
request.open(methodType, url, true);
request.onreadystatechange = function() {
if (request.readyState === 4) {
console.log(&quot;Request complete from XMLHttpRequest, request object below&quot;);
console.log(request);
}
}
request.setRequestHeader(&quot;x-amz-date&quot;, dateHeader);
request.setRequestHeader(&quot;Authorization&quot;, authHeader);
request.send();
}

function doJQueryRequest() {
$.ajax({
url: url,
headers: {
Authorization: authHeader,
&quot;x-amz-date&quot;: dateHeader
},
type: methodType
}).done(function(data) {
console.log(&quot;Request complete from JQuery, response data below&quot;);
console.log(data);
});
}
</script>
</body>
</html>

Yes I know I have written very short description. But check out the code, that’s what matters, right? ( I am lazy 😛 )

A Database Query Problem

A database query problem for those who are interested:

There is a table with two columns: userId, time (many to many relationship)
For every pair of userIds, if any two time difference of two users is less than 20 minutes, then the pair will get a point.
Now need to find out the pair with highest points.

For example:

userId time
a 20
a 120
a 230
b 30
b 110
b 260
c 30
c 120
c 245

Then points of pair:
ab = 2
ac = 3
bc = 3

PS: This query may be used for data mining and/or artificial intelligence.

PS: It can also be used for trolling 3:)

PS: Idea courtesy – Nipa Afroz

How to add a system call in linux kernel (Ubuntu OS)

This is just a brief description, so read at your own risk.

I have tested & used it. So it should be working. Please install a fresh install of Ubuntu.

This tutorial is for both 32 and 64 bit x86 processors and operating system. I have assumed that you are working in Ubuntu 10.10 and using kernel version 2.6.37.3 . If you are using any other kernel version just replace 2.6.37.3 with your version. I am also assuming you have extracted the source code.

Now let the new system call’s name be “add2”.

1. Now you will find a “arch” folder in the source code folder. Open the file arch/x86/kernel/syscall_table_32.S in a text editor. Go to the end of the document and add this line –

	.long sys_add2		/* my code */

2. Now open arch/x86/include/asm/unistd_32.h and find out
#define __NR_prlimit64 340

Add a new line after this:

#define __NR_add2		341

Don’t just close yet. After 3-4 lines, you will find a line like
#define NR_syscalls 341
Change it to

#define NR_syscalls		342

4. Now edit arch/x86/include/asm/unistd_64.h
Find out:
#define __NR_prlimit64 302
__SYSCALL(__NR_prlimit64, sys_prlimit64)

Now after these two lines, add these two lines

#define __NR_add2				303
__SYSCALL(__NR_add2, sys_add2)

5. Now again in the source folder you will find a folder named include. Open the file include/linux/syscalls.h and go to the end of the file. Before the line
#endif
write this prototype definition line:

asmlinkage long sys_add2(int i,int j);

6. Now find out the kernel folder in the source directory. Create a new empty file in the kernel folder with the name “mysysteamcalls.c” . Add the following codes in the file:

#include<linux/linkage.h>
asmlinkage long sys_add2(int i,int j)
{
    return i+j;
}

7. Now open the Makefile in this folder(/kernel/Makefile) and find out
obj-y += groups.o
Add a new line before this line :

obj-y += mysysteamcalls.o

Ok, this is the edit you need to do to add a new system call. Now compile or recompile the source code and enjoy your new system call.

Here is a sample code to call the system call :

#include <stdio.h>
#include <linux/unistd.h>
#include <sys/syscall.h>

//comment the following line if you are using 64 bit, this number is the same used previously
#define sys_add2 341

//comment the following line if you are using 32 bit, this number is the same used previously
#define sys_add2 303

int main(void)
{
    int a,b,c;
    printf("Adding Two Numbers in Kernel Space\n");
    printf("Input a: ");
    scanf("%d",&a);
    printf("Input b: ");
    scanf("%d", &b);
    c = syscall(sys_add2, a, b);
    printf("System call returned %d\n", c);
    return 0;
}

Important note: To add a new system call, you don’t need to create a new file, you can just add a new function in the same “mysysteamcalls.c” file. And if you don’t create a new file you don’t have to do the step 7.

-Enzam

Sample GP SMS API Use With PHP

Below there is a sample code of using GP SMS Api with PHP.

<?php
function send_sms($mobile_no, $text) {
//creating a soap client
$soap=new SoapClient(“SMS WSDL.wsdl”);
//setting the variables of SendSMSRequest type
$req_vars=array(
“registrationID”    =>  “YOUR USER ID”,
“password”          =>  “YOUR PASSWORD”,
“sourceMsisdn”      =>  “88017********”,
“destinationMsisdn” =>  “88”.$mobile_no,
“smsPort”           =>  7424,
“msgType”           =>  4,
“charge”            =>  0.00,//If you want you can give some value, but if you are charged, I’m not responsible for that 😛
“chargedParty”      =>  “88017********”,
“contentArea”       =>  “gpgp_psms”,
“msgContent”        =>  $text,
);
//The request variable
$request=array(“SendSMSRequest”=>$req_vars);
//requesting now to send sms
$response=$soap->sendSMS($request);
return $response;
}

?>
<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN”>
<html>
<head>
<meta http-equiv=”Content-Type” content=”text/html; charset=UTF-8″>
<title>GP SMS Api Testing</title>
</head>
<body>
<pre>
<?php
//calling the function
$phone_no=”017********”;
$text=”Hi people! Just testing out! :-)”;
$response=send_sms($phone_no, $text);
print_r($response);
?>
</pre>
</body>
</html>

P.S. : If you get any SSL related error, go to your php.ini file,find out

“;extension=php_openssl.dll”

then delete the ; from the beginning

Update : A new updated post (with updated code) in this link: http://blog.muktosource.com/enzam/send-sms-example-with-gp-aloashbei-api-php.html