How to add a system call in linux kernel (Ubuntu OS)

This is just a brief description, so read at your own risk.

I have tested & used it. So it should be working. Please install a fresh install of Ubuntu.

This tutorial is for both 32 and 64 bit x86 processors and operating system. I have assumed that you are working in Ubuntu 10.10 and using kernel version 2.6.37.3 . If you are using any other kernel version just replace 2.6.37.3 with your version. I am also assuming you have extracted the source code.

Now let the new system call’s name be “add2”.

1. Now you will find a “arch” folder in the source code folder. Open the file arch/x86/kernel/syscall_table_32.S in a text editor. Go to the end of the document and add this line –

	.long sys_add2		/* my code */

2. Now open arch/x86/include/asm/unistd_32.h and find out
#define __NR_prlimit64 340

Add a new line after this:

#define __NR_add2		341

Don’t just close yet. After 3-4 lines, you will find a line like
#define NR_syscalls 341
Change it to

#define NR_syscalls		342

4. Now edit arch/x86/include/asm/unistd_64.h
Find out:
#define __NR_prlimit64 302
__SYSCALL(__NR_prlimit64, sys_prlimit64)

Now after these two lines, add these two lines

#define __NR_add2				303
__SYSCALL(__NR_add2, sys_add2)

5. Now again in the source folder you will find a folder named include. Open the file include/linux/syscalls.h and go to the end of the file. Before the line
#endif
write this prototype definition line:

asmlinkage long sys_add2(int i,int j);

6. Now find out the kernel folder in the source directory. Create a new empty file in the kernel folder with the name “mysysteamcalls.c” . Add the following codes in the file:

#include<linux/linkage.h>
asmlinkage long sys_add2(int i,int j)
{
    return i+j;
}

7. Now open the Makefile in this folder(/kernel/Makefile) and find out
obj-y += groups.o
Add a new line before this line :

obj-y += mysysteamcalls.o

Ok, this is the edit you need to do to add a new system call. Now compile or recompile the source code and enjoy your new system call.

Here is a sample code to call the system call :

#include <stdio.h>
#include <linux/unistd.h>
#include <sys/syscall.h>

//comment the following line if you are using 64 bit, this number is the same used previously
#define sys_add2 341

//comment the following line if you are using 32 bit, this number is the same used previously
#define sys_add2 303

int main(void)
{
    int a,b,c;
    printf("Adding Two Numbers in Kernel Space\n");
    printf("Input a: ");
    scanf("%d",&a);
    printf("Input b: ");
    scanf("%d", &b);
    c = syscall(sys_add2, a, b);
    printf("System call returned %d\n", c);
    return 0;
}

Important note: To add a new system call, you don’t need to create a new file, you can just add a new function in the same “mysysteamcalls.c” file. And if you don’t create a new file you don’t have to do the step 7.

-Enzam