December 30, 2011

Disable Windows Temporary IPv6 Address

In a "cmd" window with administrative rights, run the following commands:

netsh interface ipv6 set privacy state=disabled store=active
netsh interface ipv6 set privacy state=disabled store=persistent
netsh interface ipv6 set global randomizeidentifiers=disabled store=active
netsh interface ipv6 set global randomizeidentifiers=disabled store=persistent

Then Restart your machine.

December 29, 2011

Shrew Soft VPN client group auth key auth-mutual-psk

If you are provided with a shrew-soft Windows VPN client from your company, and you would like to run a VPN client on Linux. You can either use the Shrew VPN client Linux version or another open source VPN client "vpnc".

To run vpnc, you will need to know the group password. If you export your Shrew VPN profile which is a plain text file, you will see a line in the file that starts with:


This is the group password, encoded with BASE64 format. So just copy the value and paste it to any web-based BASE64 decoder such as the one (This link happens to be on the top when I searched and it works fine), then you get the group password in plain-text.

December 28, 2011

Running User Mode Linux as normal non-root user

User Mode Linux (UML) is running linux over linux. The guest linux OS runs as a regular process on the host CPU. UML has been around for a while, the famous windows-based "coLinux" is inspired by it.

An unique feature of UML is that you can run it on a powerful Linux server as a regular user. You do not need any sort of root permission to run it. In fact, this is the only feasible solution for a non-root user (other than QEMU running in emulation mode which is much much slower). UML in fast.

To get UML running, you need two things: the kernel and the file system. UML provides the kernel and the compilation is straightforward.

To get file system, you need to have root access on a Linux machine (does not have to be your final HOST machine) and do the following:

(root@host)# apt-get install debootstrap
(root@host)# cd /tmp
(root@host)# dd if=/dev/zero of=debian.bin bs=1M count=1 seek=4096
(root@host)# mkfs.ext3 debian.bin
(root@host)# mkdir -p /mnt; mount -o loop debian.bin /mnt
(root@host)# debootstrap squeeze /mnt

If you are in the year after 2012, Replace "squeeze" with whatever the latest debian stable version name is. 

(root@host)# chroot /mnt 
(chroot@host)# mkdir /dev/ubd
(chroot@host)# cd /dev/ubd
(chroot@host)# for i in 0 1 2 3 4 5 6 7; do mknod $i b 98 $[ $i * 16 ]; done
(chroot@host)# cat > /etc/fstab << EOF
/dev/ubd/0      /        xfs    defaults 0 0
/dev/ubd/1      none     swap   defaults 0 0
none            /proc    proc   defaults 0 0
sys             /sys     sysfs  defaults 0 0
none            /dev/pts devpts defaults 0 0

(chroot@host)# echo uml0 > /etc/hostname
(chroot@host)# exit
(root@host)# rm -f /mnt/root/.bash_history
(root@host)# umount /mnt

Now the file debian.bin has the latest debian base installation. This is your File System. 

Now you need to download and compile Slirp from Slirp is a cool hack that allows UML limited access to the network by 
tunneling it over regular UDP and TCP connections. This is really the key to making UML useful as a regular user. The other (better) methods for accessing the network need root access. Be sure to apply the latest patch, as otherwise it does not work. It is critical to edit config.h and uncomment the line that says #define FULL_BOLT. This will make Slirp go as fast as it can. Otherwise, slirp will only go 115kbps. Trying to do "apt-get" over that is painful.

Copy the slirp program you build to /home/YOU/bin/

./linux ubd0=debian6.bin con=pts con0=fd:0,fd:1 eth0=slirp,,/home/user/bin/slirp

I also add "single rw" to get linux boot into single mode and get me started. Once everything is running, I take out "single rw" so that all the daemons are started correctly.

I also have a file ~/.slirprc to direct TCP connections so that I can ssh into the UML. 

#cat ~/.slirprc
redir 2200 22

December 23, 2011

StrongSwan Configuration Guide

Recently I got a chance to study strongswan and its configurations. This document intends to record the findings, in the hope to help myself in the future and to help others too.

Strongswan is open-source IPSec/VPN software. It was based on FreeSwan, whose development is now stopped. Another descendent of FreeSwan is “OpenSwan”. I have no experience with OpenSwan, and therefore will be focusing on StrongSwan in this document.

By the way, I found the authors of StrongSwan (Andrea and Martini) very much helpful. The know strongswan inside out and was able to explain things really well in many of the mailinglist posts and also in the wiki documentation. This is a great strength of strongswan.

StrongSwan’s core VPN behavior is largely controlled by the configuration file /etc/ipsec.conf. There are many possible lines there you can put in this file. Some lines are extremely important, and a good understanding of what they mean is critical to the successful establishment of the VPN tunnels.

There are a few types of VPN Connections:

  1. Host to Host
  2. Net to Net
  3. Host to Net

Host to Host is fairly rare, and many of the things discussed here also apply to apply to it. So I will focus on “net-net” and “host-net”.

The following settings are tested with StrongSwan 4.5.3 and 4.6.1.

1.      Common Configuration

Common configuration lines in /etc/ipsec.conf

config setup
conn %default


“strictcrlpolicy” indicates whether CRL is mandatory or not. If CRL is not mandatory, put no. Otherwise, put yes.

“Charon” is the IKEv2 daemon, and “Pluto” is the IKEv1 daemon. In this document, we are only using “IKEv2” and will focus on IKEv2 options only.

“Mobike” stands for Mobile IKE. This is for the case where the public IP of the device may change. If Mobike is enabled, strongswan may float its communication port from UDP port 500 to UDP port 4500 and start telling the Linux kernel to use UDP encapsulation for ESP packets.  It is a good thing to enable if there is a chance that your device’s public IP may change.

“leftikepor” and “rightikeport” tells strongswan to always use UDP port 4500, from the very beginning of IKEv2 message exchange.

2.      Net to Net

2.1   VPN Server

A working net-net VPN SERVER configuration file

Conn myvpn

Left means “my side”. Right means “my peer’s side”.  You could switch it the other way, but this is how most people use it and is a common convention. Unless one really wants to be different and asking for troubles, it is strongly suggested that this common convention be followed.

“left” is your IP address. This can be set to “%defaultroute” where the system will figure out the value based on the “right” IP address.

“leftcert” is the certificate file of the left. The file path can be an absolute path (starting with /) or a relative path, in which case, the system will look for certs under /etc/ipsec.d/certs/

“leftsubnet”: This is “the” critical line that tells strongswan you want a subnet tunnel instead of a host tunnel. Without this line, the strongswan will try to make a host-only tunnel.  This is the subnet on “your” device’s side. Your peers will only be able to talk to IPs in this subnet.

“leftfirewall”: optional. Tells strongswan to automatically insert firewall rules (iptables rules) when a connection is up or down.

“right” is the peer’s address. For server, this can be “%any”.

“rightsubnet”: similar to “leftsubnet”, this line is critical to indicating that you want to connect to a “subnet”, not just a remote host. Unless “leftsubnet”, you can put “” indicating that you just accept the subnet that the peer defines.

“auto=add” means when you run “ipsec start”, the ipsec daemon just listens, not initiate a connection. “auto=start” means that when you run “ipsec start”, the ipsec daemon will actually try to initiate a call. So “auto=add” is good for servers, and “auto=start” is good for clients. “auto=start” is equivalent to “auto=add” plus “ipsec up MYCONNECTION”

We skipped “leftid=”. leftid by default is derived from the leftcert certificate file, using the Distinguished Name, in the format of “C=XX, O=XXX, CN=XXX, …”. There are any forms of ID that can be used by the leftid/rightid field, but in this document we chose to use this format. See the end of this document for more detailed description of leftid/rightid field.

A working net-net VPN CLIENT configuration file

Net-net is pretty much symmetrical. You can run the above same configuration file on client side and it will work. I chose to one more configuration line:

rightid="C=CH, O=strongSwan, CN=server"

This tells the client to check the server’s certificate ID and make sure it matches this ID. This is just to be safe so that I know I did not connect to some other server. Keep this mind this is after the server certificate is being authenticated by the CA certificate.

3.      Host to Net

3.1   VPN Server

A working host-net VPN SERVER configuration file
conn server

Notice that we simply removed “leftsubnet” and “rightsubnet” from the net-net VPN SERVER configuration, and here we have a host-net VPN SERVER.

We added one more line “rightsourceip”. This enables the server to “automatically assign a virtual IP address to the connecting peer”.

Note that in IKEv1 (Pluto) "rightsourceip" can be used to specify the internal side IP address. For IKEv2 (Charon) this is done automatically by the charon daemon and rightsourceip takes up the new meaning of requesting a virtual IP address. If the server does not have "rightsourceip" configured but client has "leftsourceip=x.x.x.x" configured, the tunnel establishment will fail because the server is rejecting the request for a virtual IP address.

3.2  VPN Client

A working host-net VPN CLIENT configuration file

rightid="C=CH, O=strongSwan, CN=server"

The only special line here is “leftsourceip”, which tells the client to obtain a virtual IP address from the VPN Server.

4.       leftid and rightid, what to use?

The ID by which a peer is identifying itself during IKE main mode can by any of the ID types IPV4_ADDR, FQDN, USER_FQDN or DER_ASN1_DN. If one of the first three ID types is used, then the accompanying X.509 certificate of the peer must contain a matching subjectAltName field of the type ipAddress (IP:), dnsName (DNS:) or rfc822Name (email:), respectively. With the fourth type DER_ASN1_DN, the identifier must completely match the subject field of the peer's certificate. One of the two possible representations of a Distinguished Name (DN) is the LDAP-type format
     rightid="C=CH,O=Linux strongSwan,"

Additional whitespace can be added everywhere as desired since it will be automatically eliminated by the X.509 parser. An exception is the single whitespace between individual words , like e.g. in Linux strongSwan, which is preserved by the parser.

The Relative Distinguished Names (RDNs) can alternatively be separated by a slash ( '/')  instead of a comma (',')

rightid="/C=CH/O=Linux strongSwan/"

This is the representation extracted from the certificate by the OpenSSL command line option

openssl x509 -in sunCert.pem -noout –subject

The following X.501 RDNs are supported by strongSwan
 Domain Component
 State or province
 Locality or town
 Organisational Unit
 Common Name
 Name Distinguisher, used with CN
 Given name
 Personal title
 Serial number
 Serial number
 User ID
 X.500 Unique Identifier
 [Siemens] Trust Center Global ID
 Unstructured Name
 Unstructured Name
 Employee Number
 Employee Number

5.      Recommended Documentation
  1. I’ve found this readme file very helpful, although it may be a little old. Information presented here still applies.  It is a good starting point, and gives you a good base understanding of everything. This should be the first read, and then you can move on to other documentation such as the wiki.
  2. Strong Swan WiKi, which contains “lots of” information. Here is a guide to get you started:
    1. IpsecConf
    2. ConnSection
    3. ConfigurationExamples:

December 15, 2011

ubuntu kvm KSM high cpu usage


If you use KVM virtualization, under Ubuntu 10.04 Server LTS, with more than one virtual host running at a time, you may have noticed ksmd eating a lot of CPU cycles.  This behavior was not present under 8.04 Server LTS.  Apparently, the kmsd functionality was enabled by default in 10.04.  KSM – Kernel Samepage Merging – merges memory pages between virtual hosts to save space.  Unfortunately, for some, it also utilizes a lot of CPU resources to perform this function.  For my use case, I don’t care about the amount of RAM occupied by my virtual hosts, indeed, I installed far more RAM than I need in my server so that I don’t have to worry about a lack of RAM. 

If you would like to disable ksmd, edit your /etc/default/qemu-kvm file as follows:

# To disable qemu-kvm’s page merging feature, set KSM_ENABLED=0 and
# sudo restart qemu-kvm

Then, as the comments in the file state, restart qemu-kvm. (stop your VM, then "sudo stop qemu-kvm", then "sudo start qemu-kvm")

December 13, 2011

Net-snmp debug token list

For example: register_mib is an useful token for debugging SNMP AGENTX subagents.

December 7, 2011

crontab for non-root user

If you could not get your crontab file to work with non-root user, and you have made sure the crond is running and your crontab syntax is correct, check this:

Make sure there is an empty line as last line in your crontab file.

This fixed my problem.

November 15, 2011

Samba shared drive on Ubuntu/Debian

This samba configuration file configures the samba to be a group shared drive (x: drive). All users access are guest.

File: /etc/samba/smb.conf

workgroup = DOCS
netbios name = DOCS_SRV2
security = share
name resolve order = wins lmhosts bcast host
wins server =
wins proxy = yes
guest account = nobody

        comment = Guest Share
        path = /data/x
        browseable = yes
        read only = no
        guest ok = yes
        guest account = nobody
        guest only = yes

November 9, 2011

cvs server setup in debian / ubuntu

 Setup CVS server on Debian / Ubuntu

1. Install CVS client and server:
apt-get install cvs cvsd

With the default installation, the root for cvsd is at /var/lib/cvsd.The default repositories demo and myrepos are defined in the configuration file /etc/cvsd/cvsd.conf.

By default, cvsd is installed in a chroot environment. To make it use the regular / environment, change this in cvsd.conf:

      RootJail none

Also, you may want to change the Listen statement to the following so that it does not listening on IPv6 sockets.

      Listen 2401

2. Start the CVS server:

        sudo /etc/init.d/cvsd start

3. Create myrepos and CVSROOT directories with following command (Note: the absolute path is needed to initialize the repose):

sudo cvs -d /var/lib/cvsd/myrepos init
sudo chown -R cvsd:cvsd /var/lib/cvsd/myrepos

The basic structures of cvsd and the repository $CVSROOT are ready.

4. Add user to the group cvsd and create the password for users.

From System->Administration->Users and Groups. Click Unlock button and enter the password, then click Manage Groups button. Highlight cvsd then click Properties and select the username from the Group Members list.

Create the user password for CVS login as shown below:

cvsd-passwd /var/lib/cvsd/myrepos username

5. Login to cvs server

cvs -d :pserver:username@localhost:/myrepos login

However, if the $CVSROOT is defined just simply enter "cvs login" instead. Add following line in ~/.bashrc is the easiest way to add it to the environment.

export CVSROOT=:pserver:$USER@localhost:/myrepos

If log in has no problem, CVS should be working fine.

Adapted from this post:

November 3, 2011

Windows SSH client with automatic reconnect capability

Putty does not automatic reconnect. There are a few that do:

1. MyEnTunnel  , Your background ssh tunnel software that runs on Windows.
2. Tunnelier  (Free for individual use)
3.  Putty-Tray 

What do you use for this purpose?

October 20, 2011

compiling / building colinux

I have been using colinux for the last 6 years under Windows XP and then Windows Vista. It has always worked great.  However, I have never tried to build coLinux from scratch. Today, due to the reason that I need a kernel module that is not in the module list provided by the stock colinux build image, I had to go through this exercise.

Overall it is a fairly straightforward process. But I need to make the notes so that later on other people can find it useful.

It is better to have a fast machine, a fast internet connection, and over 1GB of free disk space.

My host machine is Ubuntu 10.04 running in X86-64bit mode. The final image of colinux kernel is 32-bit. The build process automatically does that for you.

1. Download the latest snapshot release source tar ball from
2. untar it, cd to devel-coLinux...
3. ./configure
4. make
    This will download gcc,bin-utils,mingw32, kernel source, etc
5. You will find you kernel (vmlinux) and modules tar.gz file in the build directory,which should be displayed on the console when the build started.

--To rebuild the kernel with your modified config file--
1. go to the kernel build directory, where your vmlinux file is located.
2. backup your current config file:  cp .config good_config
3. change the config file: make ARCH=i386 menuconfig
4. go to your devel-coLinux-xxx directory, do "make kernel"

October 12, 2011

linux dynamic library executable and initialization

C file

/* initialization when .so is loaded */
static int (*libc_flock)(int fd, int operation)=NULL;
int flock(int fd, int operation){
    printf("my flock called.  fd=%d operation=%d\n",fd,operation);

    if (!handle){
    if (!libc_flock){
        if (libc_flock==NULL)
            return NULL;
    return libc_flock(fd,operation);

__attribute__((constructor)) static void mylib_init(void){
    printf("============>\nmylib is initialized\n================>\n");

/* this part of code make your .so executable */
const char my_interp[] __attribute__((section(".interp"))) = "/lib/";

void my_main(int argc, char **argv) {
    int i;

    printf("Called as:");
    for (i = 0; i < argc; i++)
        printf(" %s", argv[i]);


---------------- mylib.c
        $(CC) -shared -Wl,-soname,$@ -fPIC -O2 -s -o $@ $^ -ldl  -Wl,-e,my_main

October 7, 2011

To read chinese in gvim in Windows

set guifont=NSimSun:h12:cGB2312

October 4, 2011

C/C++ dbug library

There are many debug libraries available for C/C++.  Recently I started looking into the "dbug" library created by Fred Fish in 1988 and released to public. You can download the library at source forge .

The library has just one C file: dbug.c and one header file: dbug.h (another header file dbug_long.h has more comments can be used in place of dbug.h)

The concept is that you can turn debug on/off on the fly; enable per-function tracing; and other stuff such as profiling which usually you don't need.

To use it:

First, you initialize it by doing this:

  DBUG_PUSH ("d:t:O:L:");

Then in every function, you begin your function code with:
  DBUG_ENTER ("my_function_name");

End your function with either



if you return integer 0.

To use printf to debug something, use this macro:
   DBUG_PRINT ("info", ("Returned value: %d", ret));
where "info" is your keyword/category of debugs. In DBUG_PUSH, you can use "d,keyword1,keyword2..." to turn on/off a list of categories that you want to debug.

Note that there is a file "example.c" in the root directory of the downloaded package, which shows you how to use the library.

Reference for the controlling string in dbug_push:
this is copied from MySQL document Appendix C

The debug control string is a sequence of colon separated fields as follows:
Each field consists of a mandatory flag character followed by an optional "," and comma separated list of modifiers:

The currently recognized flag characters are:

  • d Enable output from DBUG_ macros for for the current state. May be followed by a list of keywords which selects output only for the DBUG macros with that keyword. A null list of keywords implies output for all macros.
  • D Delay after each debugger output line. The argument is the number of tenths of seconds to delay, subject to machine capabilities. I.E. -#D,20 is delay two seconds.
  • f Limit debugging and/or tracing, and profiling to the list of named functions. Note that a null list will disable all functions. The appropriate "d" or "t" flags must still be given, this flag only limits their actions if they are enabled.
  • F Identify the source file name for each line of debug or trace output.
  • i Identify the process with the pid for each line of debug or trace output.
  • g Enable profiling. Create a file called 'dbugmon.out' containing information that can be used to profile the program. May be followed by a list of keywords that select profiling only for the functions in that list. A null list implies that all functions are considered.
  • L Identify the source file line number for each line of debug or trace output.
  • n Print the current function nesting depth for each line of debug or trace output.
  • N Number each line of dbug output.
  • o Redirect the debugger output stream to the specified file. The default output is stderr.
  • O As O but the file is really flushed between each write. When needed the file is closed and reopened between each write.
  • p Limit debugger actions to specified processes. A process must be identified with the DBUG_PROCESS macro and match one in the list for debugger actions to occur.
  • P Print the current process name for each line of debug or trace output.
  • r When pushing a new state, do not inherit the previous state's function nesting level. Useful when the output is to start at the left margin.
  • S Do function _sanity(_file_,_line_) at each debugged function until _sanity() returns something that differs from 0. (Mostly used with safemalloc)
  • t Enable function call/exit trace lines. May be followed by a list (containing only one modifier) giving a numeric maximum trace level, beyond which no output will occur for either debugging or tracing macros. The default is a compile time option.
Some examples of debug control strings which might appear on a shell command line (the "-#" is typically used to introduce a control string to an application program) are:

For convenience, any leading "-#" is stripped off.

September 30, 2011

vim paste and indent

If you use "dd" to cut a line in vim and want to use "p" to paste it to another place. The original indent will remain. To make the intent fit with the destination indent, use ]p.

List unused C/C++ functions

You have a C/C++ project. It is more than a few files, including some libraries. Now you want to know whether you can get rid of / comment out some of the functions that are never used.

There is no well-known FREE and OPEN SOURCE tools that can do this for you as of 2011, but a not-very-well-known tool called "callcatcher" created by Caolan does the job beautifully.

First, to download the callcatcher, go to the official website:

It is written in Python. So you may want to install it by becoming a root or use sudo.

To use the tool, you need to change ALL your make files to prepend "callcatcher" to your "gcc/g++" command. For example, in your original Makefile, you have


Now change it to

CC=callcatcher gcc
AR=callarchive ar

When you are done compiling, to see which functions is not used, do:

callanalyse MY-EXE-FILE

Another caveat is that you cannot compile multiple times at the same time, like this

$(CC) -o main main.c lib1.c lib2.c  (This is bad for callcatcher)

Instead you need to change it to something like this, so that you only compile one file at a time, then link it:

   $(CC) $(CFLAGS) -c $^
main: $(OBJS)
    $(CC) -o $@ $(OBJS)

September 1, 2011

Setup your own (small / tiny) Certificate Authority (CA)

For more than 100 certificates, you could use: NewPKI, OpenCA, IDX PKI which
are opensource projects.

A smaller solution is TinyCA ( Note that this is just a GUI front-end to openssl.

From Wikipedia:

  • EJBCA (java-based)
  • OpenCA (the most popular one)
  • OpenSSL, which is really an SSL/TLS library, but comes with tools allowing its use as a simple certificate authority.
  • gnoMint
  • DogTag (This looks really good)
  • XCA

August 30, 2011

simple telnet server by using socat

On server side:

./socat exec:'bash -li',pty,stderr,setsid  tcp-listen:8999,reuseaddr

On client side:

./socat tcp-connect: file:`tty`,raw,echo=0

C, C++ source code tools to help you be productive

1. Source code beautifier: helps you with coding style compliance

2. Static anylysis
  pc-lint/flexelint (needs $$)
  splint (google code)

3. bug finder

August 26, 2011

gitignore diff_exclude

A list of files usually should be ignored by git and diff when you generate a patch:


August 25, 2011

git cheat sheet

Makefile Automatic Variables

Here is a table of automatic variables:
The file name of the target of the rule. If the target is an archive member, then ‘$@’ is the name of the archive file. In a pattern rule that has multiple targets (see Introduction to Pattern Rules), ‘$@’ is the name of whichever target caused the rule's recipe to be run.
The target member name, when the target is an archive member. See Archives. For example, if the target is foo.a(bar.o) then ‘$%’ is bar.o and ‘$@’ is foo.a. ‘$%’ is empty when the target is not an archive member.
The name of the first prerequisite. If the target got its recipe from an implicit rule, this will be the first prerequisite added by the implicit rule (see Implicit Rules).
The names of all the prerequisites that are newer than the target, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives).
The names of all the prerequisites, with spaces between them. For prerequisites which are archive members, only the named member is used (see Archives). A target has only one prerequisite on each other file it depends on, no matter how many times each file is listed as a prerequisite. So if you list a prerequisite more than once for a target, the value of $^ contains just one copy of the name. This list does not contain any of the order-only prerequisites; for those see the ‘$|’ variable, below.
This is like ‘$^’, but prerequisites listed more than once are duplicated in the order they were listed in the makefile. This is primarily useful for use in linking commands where it is meaningful to repeat library file names in a particular order.
The names of all the order-only prerequisites, with spaces between them.
The stem with which an implicit rule matches (see How Patterns Match). If the target is dir/ and the target pattern is a.%.b then the stem is dir/foo. The stem is useful for constructing names of related files. In a static pattern rule, the stem is part of the file name that matched the ‘%’ in the target pattern. In an explicit rule, there is no stem; so ‘$*’ cannot be determined in that way. Instead, if the target name ends with a recognized suffix (see Old-Fashioned Suffix Rules), ‘$*’ is set to the target name minus the suffix. For example, if the target name is ‘foo.c’, then ‘$*’ is set to ‘foo’, since ‘.c’ is a suffix. GNU make does this bizarre thing only for compatibility with other implementations of make. You should generally avoid using ‘$*’ except in implicit rules or static pattern rules. If the target name in an explicit rule does not end with a recognized suffix, ‘$*’ is set to the empty string for that rule.
$?’ is useful even in explicit rules when you wish to operate on only the prerequisites that have changed. For example, suppose that an archive named lib is supposed to contain copies of several object files. This rule copies just the changed object files into the archive:
lib: foo.o bar.o lose.o win.o
             ar r lib $?
Of the variables listed above, four have values that are single file names, and three have values that are lists of file names. These seven have variants that get just the file's directory name or just the file name within the directory. The variant variables' names are formed by appending ‘D’ or ‘F’, respectively. These variants are semi-obsolete in GNU make since the functions dir and notdir can be used to get a similar effect (see Functions for File Names). Note, however, that the ‘D’ variants all omit the trailing slash which always appears in the output of the dir function. Here is a table of the variants:
The directory part of the file name of the target, with the trailing slash removed. If the value of ‘$@’ is dir/foo.o then ‘$(@D)’ is dir. This value is . if ‘$@’ does not contain a slash.
The file-within-directory part of the file name of the target. If the value of ‘$@’ is dir/foo.o then ‘$(@F)’ is foo.o. ‘$(@F)’ is equivalent to ‘$(notdir $@)’.
The directory part and the file-within-directory part of the stem; dir and foo in this example.
The directory part and the file-within-directory part of the target archive member name. This makes sense only for archive member targets of the form archive(member) and is useful only when member may contain a directory name. (See Archive Members as Targets.)
The directory part and the file-within-directory part of the first prerequisite.
Lists of the directory parts and the file-within-directory parts of all prerequisites.
Lists of the directory parts and the file-within-directory parts of all prerequisites, including multiple instances of duplicated prerequisites.
Lists of the directory parts and the file-within-directory parts of all prerequisites that are newer than the target.
Note that we use a special stylistic convention when we talk about these automatic variables; we write “the value of ‘$<’”, rather than “the variable <” as we would write for ordinary variables such as objects and CFLAGS. We think this convention looks more natural in this special case. Please do not assume it has a deep significance; ‘$<’ refers to the variable named < just as ‘$(CFLAGS)’ refers to the variable named CFLAGS. You could just as well use ‘$(<)’ in place of ‘$<’.

August 23, 2011

vim global command

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":
:g/George Bush/ d
The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:
:g/George Bush/ .+5 d
To delete the DEBUG log entries:
:g/DEBUG/ .,+10 d
If you knew the stack trace was variable length but always ended at a blank line (or other regex):
:g/DEBUG/ .,/^$/ d
You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":
 :g!/sucks/ s/Bush/Obama/
The default command is to print the line to the message window. e.g. to list every line marked TODO:
This is also useful for checking the regex matches the lines you expect before you do something destructive.
You can chain multiple commands using "|". e.g. to change Bush to Obama AND George to Barack on every line that does not contain "sucks":
 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

vim global command

The :global command is your friend - learn it well. It lets you run arbitrary :ex commands on every line that matches a regex. It abbreviates to :g.
To delete all lines that match "George Bush":
:g/George Bush/ d
The command that follows can have its own address/range prefix, which will be relative to the matched line. So to delete the 5th line after George Bush:
:g/George Bush/ .+5 d
To delete the DEBUG log entries:
:g/DEBUG/ .,+10 d
If you knew the stack trace was variable length but always ended at a blank line (or other regex):
:g/DEBUG/ .,/^$/ d
You can also execute a command on every line that does NOT match with :g!. e.g. to replace "Bush" with "Obama" on every line that does not contain the word "sucks":
 :g!/sucks/ s/Bush/Obama/
The default command is to print the line to the message window. e.g. to list every line marked TODO:
This is also useful for checking the regex matches the lines you expect before you do something destructive.
You can chain multiple commands using "|". e.g. to change Bush to Obama AND George to Barack on every line that does not contain "sucks":
 :g!/sucks/ s/Bush/Obama/g | s/George/Barack/g

gcc default include path

this is how to find out the default include path:

method 1:

`gcc -print-prog-name=cc1` -v

method 2:

You can create a file that attempts to include a bogus system header. If you run gcc in verbose mode on such a source, it will list all the system include locations as it looks for the bogus header. $ echo "#include <bogus.h> int main(){}" > t.c; gcc -v t.c; rm t.c [..]
#include "..." search starts here:
#include <...> search starts here:
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
t.c:1:32: error: bogus.h: No such file or directory

August 19, 2011

FREE PHP IDE editors

1. aptana Studio
2. codeLobster
3. Komodo edit

August 8, 2011

Ubnt airos upgrade firmware

hold reset pin while power up the device, then

tftp -i put XS2.ar2316.v3.4-rc.4351.090504.2146.bin

August 4, 2011

virtualbox vboxheadless high cpu usage

If you are running RHEL / CentOS in virtualbox, and find that virtualbox uses high cpu even if the virtual machine is idling, it may be caused by the HZ=1000 timer interrupt issue in your linux kernel. The solution:

add divider=10 to the CentOS kernel parms in /boot/grub/grub.conf

August 2, 2011

Widows XP Remote Desktop IPv6

Windows XP Remote Desktop service does not listen on IPv6 addresses. Use the follow trick to fix it:

This command will do the trick.
netsh interface portproxy add v6tov4 listenport=3389 connectaddress= connectport=3389

After using it you'll be able to connect using a recent version of Remote Desktop Client over IPv6 to a WinXP/Win2k3 box.

July 28, 2011

Simulate unplug usb device on Linux (USB Reset)

/* usbreset -- send a USB port reset to a USB device */


and needs mounted usbfs filesystem

 sudo mount -t usbfs none /proc/bus/usb

There is a way to suspend a USB device.  In order to use it, 
you must have a kernel with CONFIG_PM_SYSFS_DEPRECATED turned on.  To 
suspend a device, do (as root):

 echo -n 2 >/sys/bus/usb/devices/.../power/state

where the "..." is the ID for your device.  To unsuspend, do the same 
thing but with a "0" instead of the "2" above.

Note that this mechanism is slated to be removed from the kernel within 
the next year.  Hopefully some other mechanism will take its place.

> To reset a
> device?

Here's a program to do it.  You invoke it as either

 usbreset /proc/bus/usb/BBB/DDD
 usbreset /dev/usbB.D

depending on how your system is set up, where BBB and DDD are the bus and
device address numbers.

Alan Stern


#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/ioctl.h>

#include <linux/usbdevice_fs.h>

int main(int argc, char **argv)
 const char *filename;
 int fd;
 int rc;

 if (argc != 2) {
  fprintf(stderr, "Usage: usbreset device-filename\n");
  return 1;
 filename = argv[1];

 fd = open(filename, O_WRONLY);
 if (fd < 0) {
  perror("Error opening output file");
  return 1;

 printf("Resetting USB device %s\n", filename);
 rc = ioctl(fd, USBDEVFS_RESET, 0);
 if (rc < 0) {
  perror("Error in ioctl");
  return 1;
 printf("Reset successful\n");

 return 0;

Tether Blackberry 9700 on Linux (Debian/Ubuntu)

With the fantastic open source software "barry", tethering to Blackberry ( I use 9700 and my carrier is AT&T) is very easy and straightforward. You don't need a Tethering plan, just the regular blackberry data plan.

Warning: be careful with your data usage once you are tethered.

My system is Debian 5.0.3. But this should work on all newer debians and reasonably new Ubuntus. Run all commands as "root":

1. Download and install barry for debian from the sourceforge website. You only need two files: libbarry0 and barry-utils. The files I use are:
            barry-util_0.17.1-0_i386.deb  libbarry0_0.17.1-0_i386.deb
2. For AT&T users:
     cd /etc/ppp/peers
     change the file 'barry-att_cingular' to comment out the following lines:
3. run "pppd call barry-att_cingular" to get you connected.

Notes about routes:
If your computer already have a default gateway, pppd will not change/update it. In that case, make sure you delete your default gateway before running pppd, or you can manually add the default gw.

To stop the connection, just hit "Ctrl-C". Or, you can do "killall pppd" and wait for a few seconds for it to disconnect.

I use this in Vmware Player virtual machine with Debian 5.0.3 and it works great.

July 20, 2011

Windows 2003 Server Remote Desktop Local Drive Map

I could not get the local drive map to work when I remote desktop to a windows 2003 server.

Turned out the the Windows 2003 server disabled the local drive map. So

  • Click Start, point to All Programs, point to Administrative Tools, and then click Terminal Services Configuration.

  • In the left pane, click Connections.

  • In the right pane, right-click RDP-tcp, and then click Properties.

  • Click around all the tabs, and make sure disk drive map is not disabled.
  • July 17, 2011

    airos persistent directory

    AirOS allow users to add scripts in the /etc/persistent directory of Ubiquiti device.
    These scripts can modify configuration by starting additional services and more.
    The standard scripts are:
    They are called before and after the standard boot and shutdown scripts start or stop services...

    Add static route via script

    • Reboot devices to clean all static routes and access via SSH/Telnet to AirOS device
    • Create a script /etc/persistent/rc.poststart and write here commands to add static route
    • Make sure that /etc/persistent/rc.poststart are executable
    • Check if your script work fine running the command line: ./etc/persistent/rc.poststart
      note: the command now start with a "dot" = run the script!
    • Run the command: cfgmtd -w -p /etc/ to make persistent
    • Reboot and verify that all it's working... 

    July 16, 2011

    openwrt make targets


    These were culled from IRC logs at on #wgt634u . The comments may not be correct. If so, please fix them. This page was put together to help with build problems related to dirty source trees.

    • make package/base-files-clean target/linux-clean target/lzma-clean target/utils-clean
      • good general cleaning and doesn't rebuild toolchain
      • clobbers certain changes you may have made to files in 'root' - (i.e. is it incompatible with WGT634U_USB_root building?)

    • make target/linux-clean
      • cleans kernel

    • make clean
      • duh, calls 'make dirclean'

    • make toolchain/uClibc-clean
      • clean uClibc only

    • make package/base-files-clean
      • always run before rebuild?
      • DANGER - remove package and unpacks the source archive again
      • aka: make package/package_name-clean (ie: make package/jpeg-clean)

    • make package/package name-rebuild
      • recompile the package after you make changes
      • if this still does a make clean (ie: deletes everything, including your changes) add this to your package/whatever/Makefile:

    • mostlyclean:
      • rm $(PKG_BUILD_DIR)/.built

     I can't seem to get the above to look right, so just remember to indent the rm with a tab.
    • make package/clean
      • cleans all packages, not always required

    • make target/clean
      • ?

    • for specific packages, e.g. courtesy of frop
      • ...well, if there's a new version of madwifi (from svn)... 'make package/madwifi-clean'

    • make target/install
      • rebuilds install image (and kernel too?)

    • make (any command) world
      • runs that command, and recompile (same as "make (command); make" (i think))
      • confirmed - yes, make takes multiple targets on the command line. Be careful when you type commands - if you miss a "-" and type make package/jpeg clean instead of make package/jpeg-clean it'll delete everything as it does the clean after a make package/jpeg. MUCH DIFFERENT!

    July 14, 2011

    sqlite 3 string to integer conversion, similar to C function atoi

    In sqlite3, if you have a column defined as string, but you want to use it as integer when doing comparison, you can do the following to convert it.

    CAST(expr AS type)

    For example: you have a column named "version", and you stored value "11" in it. Now you can convert that to integer by doing:

    Select * from mytable where CAST(version as integer)>=11;

    Hope this helps you.

    July 12, 2011

    Windows 2003, XP, recursive taking file and folders ownership and permissions

    I found that the security tab on the file properties did not work for me when taking ownership and setting permission for a full directories with multiple sub directories. The follow method did the trick:

    1. Download fileacl
    2. Run:  cd C:\Program Files\fileacl
    3. fileacl "My directory" /s myusername:RWD /FILES /SUB

    July 1, 2011

    Reliable UDP Library



    One of the best articles describing udp hole punching

    There is another article that goes into details on how to predict port on symmetrical NAT. Here is the article/IETF DRAFT 

    Internet Draft                                                   B. Ford
    Document: draft-ford-natp2p-00.txt                                M.I.T.
    Expires: October 2003                                         April 2003
       Network Address Translation and Peer-to-Peer Applications (NATP2P)
    Status of this Memo
       This document is an Internet-Draft and is subject to all provisions
       of Section 10 of RFC2026.  Internet-Drafts are working documents of
       the Internet Engineering Task Force (IETF), its areas, and its
       working groups.  Note that other groups may also distribute working
       documents as Internet-Drafts.
       Internet-Drafts are draft documents valid for a maximum of six months
       and may be updated, replaced, or obsoleted by other documents at any
       time.  It is inappropriate to use Internet- Drafts as reference
       material or to cite them other than as "work in progress."
       The list of current Internet-Drafts can be accessed at
       The list of Internet-Draft Shadow Directories can be accessed at
       Distribution of this document is unlimited.
    Copyright Notice
       Copyright (C) The Internet Society (2003).  All Rights Reserved.
       This document describes and recommends methods by which peer-to-peer
       (P2P) applications can operate efficiently in the presence of Network
       Address Translation (NAT).  This document also provides
       recommendations for the design of network address translators, in
       order for them to support P2P applications effectively without
       compromising security or performance.  This memo focuses on the
       interaction of P2P with NAT in the absence of any special proxy,
       gateway, or relaying protocols.  While not intending to preclude the
       use of such protocols, the goal of this memo is to enable P2P
       applications to function automatically without specific knowledge of
       the type, location, or configuration of the NAT.
    Ford                                                            [Page 1]
    draft-ford-natp2p-00.txt                                      April 2003
    1. Introduction
       The ubiquitous deployment of Network Address Translation (NAT) on the
       Internet has provided an effective if hopefully temporary means of
       working around the ongoing depletion of the IP address space.  At the
       same time, however, the asymmetric addressing and connectivity
       regimes established by NAT and firewall systems have created unique
       problems for peer-to-peer (P2P) applications and protocols, such as
       teleconferencing and multiplayer on-line gaming.  This document
       discusses these issues and how they can be addressed.  Familiarity is
       assumed with NAT terminology and conventions, as specified in [NAT-
       TRAD] and [NAT-TERM].  As used throughout this document, the term
       "NAT" refers to "Traditional NAT" in both of its standard variants:
       namely Basic NAT, in which only IP addresses are translated, as well
       as Network Address/Port Translation (NAPT), where both IP addresses
       and transport-level port numbers are translated.  In general, this
       document always assumes NAPT as the standard "worst-case" scenario.
       In the traditional client/server paradigm, for which current NAT and
       firewall mechanisms are primarily designed, network communication
       typically operates in terms of sessions that originate in a privately
       addressed network behind a network address translator, and are
       directed to a well-connected public server with a stable IP address
       and DNS mapping.  The client, or originator, of these connections
       often does not have its own routable IP address on the public
       Internet, but instead must share a single public IP address with a
       number of other hosts on the same private network using the NAT as a
       multiplexor.  The lack of a stable, dedicated public IP address is
       not a problem for most client software such as web browsers, because
       the client only needs to be addressable for the duration of a
       particular session.
       In the peer-to-peer paradigm, however, Internet hosts that would
       normally be considered to be "clients" need to establish
       communication sessions directly with each other.  In this case, both
       the initiator and responder in a communication session might lie
       behind a NAT/Firewall, without any permanent IP address or other form
       of public network presence.  A common on-line gaming architecture,
       for example, is for the participating application hosts to contact a
       well-known server for initialization and administration purposes, but
       then to establish direct connections with each other for fast and
       efficient propagation of updates during gameplay.  Similarly, a file
       sharing application might contact a well-known server for resource
       discovery or searching purposes, but establish direct connections
       with peer hosts for data transfer.  NAT creates a problem for peer-
       to-peer connections because unless the NAT is specially configured,
       hosts behind the NAT have no consistent, permanently usable ports to
       which incoming TCP or UDP connections from "the Internet at large"
    Ford                                                            [Page 2]
    draft-ford-natp2p-00.txt                                      April 2003
       can be directed.  Firewall functionality, which is typically but not
       always bundled with NAT, creates a similar problem because firewalls
       are generally designed as one-way filters: sessions initiated inside
       the protected network are allowed, but attempts by external hosts on
       the Internet at large to initiate communication sessions with hosts
       inside the firewall are blocked.  RFC 3235 [NAT-APPL] briefly
       addresses this issue, but does not offer any general solutions that
       do not compromise security; filling that gap is the purpose of this
    2. Techniques for P2P Communication with NAT
       This section describes in detail the currently known techniques for
       implementing peer-to-peer communication in the presence of NAT, from
       the perspective of the application or protocol designer.
    2.1. Relaying
       The most reliable, but least efficient, method of implementing peer-
       to-peer communication in the presence of NAT is to make the peer-to-
       peer communication look to the network like client/server
       communication.  For example, suppose two client hosts, A and B, have
       each initiated TCP or UDP connections with a well-known server S
       having a permanent IP address.  Clients A and B both reside on
       privately addressed networks behind network address translators,
       however, and neither of them have control over a public IP address or
       permanently stable TCP or UDP port to which incoming connections can
       be directed.
                                    Server S
                |                                             |
              NAT A                                         NAT B
                |                                             |
                |                                             |
             Client A                                      Client B
       Instead of attempting to establish a direct TCP or UDP connection
       between clients A and B, the two clients can simply use the server S
       to relay messages between them.  For example, to send a message to
       client B, client A simply sends the message to server S along its
       already-established client/server connection, and server S then sends
       the message on to client B using its existing client/server
       connection with B.  This method has the advantage that it will always
       work as long as both clients have connectivity to the server.  Its
       obvious disadvantages are that it consumes the server's processing
    Ford                                                            [Page 3]
    draft-ford-natp2p-00.txt                                      April 2003
       power and network bandwidth unnecessarily, and communication latency
       between the two clients is likely to be increased even if the server
       is well-connected.
    2.2. Connection Reversal
       The second technique works if only one of the clients is behind a
       NAT.  For example, suppose client A is behind a NAT but client B is
       not, as in the following diagram:
                                    Server S
                |                                             |
              NAT A                                           |                                    |
                |                                             |
                |                                             |
             Client A                                      Client B
       Client A has private IP address, and the application is
       using TCP port 1234.  This client has established a connection with
       server S at public IP address and port 1235.  NAT A has
       assigned TCP port 62000, at its own public IP address,
       to serve as the temporary public endpoint address for A's session
       with S: therefore, server S believes that client A is at IP address using port 62000.  Client B, however, has its own
       permanent IP address,, and the peer-to-peer application
       on B is accepting TCP connections at port 1234.
       Now suppose client B would like to initiate a peer-to-peer
       communication session with client A.  B might first attempt to
       contact client A either at the address client A believes itself to
       have, namely, or at the address of A as observed by
       server S, namely  In either case, however, the
       connection will fail.  In the first case, traffic directed to IP
       address will simply be dropped by the network because is not a publicly routable IP address.  In the second case,
       the TCP SYN request from B will arrive at NAT A directed to port
       62000, but NAT A will typically reject the connection request with a
       RST packet because only outgoing connections are allowed.
       After attempting and failing to establish a direct connection to A,
       client B can use server S to relay a request to client A to initiate
       a "reversed" connection to client B.  Client A, upon receiving this
    Ford                                                            [Page 4]
    draft-ford-natp2p-00.txt                                      April 2003
       relayed request through S, opens a TCP connection to client B at B's
       public IP address and port number.  NAT A allows the connection to
       proceed because it is originating inside the firewall, and client B
       can receive the connection because it is not behind a NAT.
       A variety of current peer-to-peer systems implement this technique.
       Its main limitation, of course, is that it only works as long as only
       one of the communicating peers is behind a NAT: if both peers are
       behind NATs, then the method fails.  Because connection reversal is
       not a general solution to the problem, it is NOT recommended as a
       primary strategy.  Applications may choose to attempt connection
       reversal, but should be table to fall back automatically on another
       mechanism such as relaying if neither a "forward" nor a "reversed"
       connection can be established.
    2.3. UDP Hole Punching
       The third technique, and the one of primary interest in this memo, is
       sometimes known as "UDP Hole Punching."  UDP hole punching relies on
       well-established NAT conventions to allow appropriately designed
       peer-to-peer applications to "punch holes" through NATs and firewalls
       and establish direct connectivity with each other, even when both
       communicating hosts may lie behind a NAT.  This technique was
       mentioned briefly in section 5.1 of RFC 3027 [NAT-PROT] and has been
       informally described elsewhere on the Internet [KEGEL].  As the name
       implies, unfortunately, this technique works reliably only with UDP.
       We will consider two specific scenarios, and how applications can be
       designed to handle both of them gracefully.  In the first situation,
       representing the common case, two clients desiring direct peer-to-
       peer communication reside behind different NATs.  In the second, the
       two clients actually reside behind the same NAT, but do not
       necessarily know that they do.
    2.3.1. Clients Behind Different NATs
       Suppose clients A and B both have private IP addresses and lie behind
       different network address translators.  The peer-to-peer application
       running on clients A and B and on server S each use UDP port 1234.  A
       and B have each initiated UDP communication sessions with server S,
       causing NAT A to assign its own public UDP port 62000 for A's session
       with S, and causing NAT B to assign its port 31000 to B's session
       with S, respectively.
    Ford                                                            [Page 5]
    draft-ford-natp2p-00.txt                                      April 2003
                                    Server S
                |                                             |
              NAT A                                         NAT B                  
                |                                             |
                |                                             |
             Client A                                      Client B
       Now suppose that client A wants to establish a UDP communication
       session directly with client B.  If A simply starts sending UDP
       requests to B's public address,, then NAT B will
       typically discard these incoming messages because the source address
       and port number does not match those of S, with which the original
       outgoing session was established.  Similarly, if B simply starts
       sending UDP requests to A's public address, then NAT A will discard
       these messages.
       Suppose A starts sending UDP requests to B's public address, however,
       and simultaneously relays a request through server S to B, asking B
       to start sending UDP requests to A's public address.  A's outgoing
       messages directed to B's public address ( will
       cause NAT A to open up a new communication session between A's
       private address and B's public address.  At the same time, B's
       messages to A's public address ( will cause NAT B
       to open up a new communication session between B's private address
       and A's public address.  Once the new UDP sessions have been opened
       up in each direction, client A and B can communicate with each other
       directly without further reference to or burden on the "introduction"
       server S.
       The UDP hole punching technique has several useful properties.  Once
       a direct peer-to-peer UDP connection has been established between two
       NATted clients, either party on that connection can in turn take over
       the role of "introducer" and help the other party establish peer-to-
       peer connections with additional peers, minimizing the load on the
       initial introduction server S.  The application does not need to
       attempt to detect explicitly what kind of NAT it is behind, if any
       [STUN], since the procedure above will establish peer-to-peer
       communication channels equally well if either or both clients do not
       happen to be behind a NAT.  The hole punching technique even works
       automatically under "Twice NAT", where one or both clients are
       removed from the public Internet via two or more levels of address
    Ford                                                            [Page 6]
    draft-ford-natp2p-00.txt                                      April 2003
    2.3.2. Clients Behind the Same NAT
       Now consider the scenario in which the two clients (probably
       unknowingly) happen to reside behind the same NAT, and are therefore
       located in the same private IP address space.  Client A has
       established a UDP session with server S, to which the common NAT has
       assigned public port number 62000.  Client B has similarly
       established a session with S, to which the NAT has assigned public
       port number 62001.
                                    Server S
                |                                             |
             Client A                                      Client B
       Suppose that A and B use the UDP hole punching technique as outlined
       above to establish a communication channel using server S as an
       introducer.  Then A and B will learn each other's public IP addresses
       and port numbers as observed by server S, and start sending each
       other messages at those public addresses.  The two clients will be
       able to communicate with each other this way as long as the NAT
       allows hosts on the internal network to open translated UDP sessions
       with other internal hosts and not just with external hosts.  For
       example, when A sends a UDP packet to B's public address, the packet
       initially has a source IP address and port number of and
       a destination of  The NAT receives this packet,
       translates it to have a source of (A's public
       address) and a destination of, and then forwards it on
       to B.  Even if supported by the NAT, this translation and forwarding
       step is obviously unnecessary in this situation, and is likely to add
       latency to the dialog between A and B as well as burdening the NAT.
       The solution to this problem is straightforward, however.  When A and
       B initially exchange address information through server S, they
       should include their own IP addresses and port numbers as "observed"
       by themselves, as well as their addresses as observed by S.  The
       clients then simultaneously start sending packets to each other at
       each of the alternative addresses they know about, and use the first
       address that leads to successful communication.  If the two clients
       are behind the same NAT, then the packets directed to their private
    Ford                                                            [Page 7]
    draft-ford-natp2p-00.txt                                      April 2003
       addresses are likely to arrive first, resulting in a direct
       communication channel not involving the NAT.  If the two clients are
       behind different NATs, then the packets directed to their private
       addresses will fail to reach each other at all, but the clients will
       hopefully establish connectivity using their respective public
       addresses.  It is important that these packets be authenticated in
       some way, however, since in the case of different NATs it is entirely
       possible for A's messages to B's private address to reach some other,
       unrelated node on A's private network, or vice versa.
    2.3.3. Consistent Port Mappings
       The hole punching technique has one main caveat: it works only if
       each of the NATs maintains a single, consistent mapping from a given
       (private IP address, private UDP port) pair to a translated (public
       IP address, public UDP port) pair, for as long as that UDP port is in
       use.  For a given UDP port on the private network, the NAT must
       ensure that the internal port is always mapped to the same public UDP
       port on the globally addressable Internet, even if communication
       occurs between that internal UDP port and multiple distinct external
       destinations on the Internet.  In particular, the NAT must not just
       naively allocate and assign a new public UDP port for each new
       session initiated from within the firewall boundary, where a
       "session" is defined by the addresses and port numbers of both
       communicating endpoints.  Assigning a new public port for each new
       session makes it impossible for a UDP application to reuse an
       already-established (public IP address, public UDP port) mapping for
       communication with different external destinations.
       While RFC 3022 [NAT-TRAD] suggests and explicitly allows NATs to
       maintain a single mapping from a (private IP address, private port)
       pair to a (public IP address, public port) pair, it unfortunately
       does not mandate this behavior.  Therefore, while many NATs implement
       the desirable behavior and therefore allow direct UDP-based P2P
       connections using the hole punching technique, other NATs currently
       do not support the technique.  Since this is the most efficient known
       method of establishing direct peer-to-peer communication between two
       nodes that are both behind NATs, and it works with a wide variety of
       existing NATs, it is recommended that applications use this technique
       if efficient peer-to-peer communication is required, but be prepared
       to fall back on simple relaying when direct communication cannot be
    2.4. UDP Port Number Prediction
       A variant of the UDP hole punching technique discussed above exists
       that allows peer-to-peer UDP sessions to be created in the presence
       of some NATs that do not maintain a consistent mapping between
    Ford                                                            [Page 8]
    draft-ford-natp2p-00.txt                                      April 2003
       private and public UDP ports.  This method, sometimes called the
       "N+1" technique [BIDIR], works if public port numbers selected by the
       NAT are not held constant across all sessions originating at a given
       private IP address and port, but are nevertheless predictable.
       Consider again the situation in which two clients, A and B, each
       behind a separate NAT, have each established UDP connections with a
       permanently addressable server S:
                                      Server S
                  |                                             |
                NAT A                                         NAT B
       A-S                        B-S
                  |                                             |
                  |                                             |
               Client A                                      Client B
       NAT A has assigned its own UDP port 62000 to the communication
       session between A and S, and NAT B has assigned its port 31000 to the
       session between B and S.  By communicating through server S, A and B
       learn each other's public IP addresses and port numbers as observed
       by S.  Client A now starts sending UDP messages to port 31001 at
       address (note the port number increment), and client B
       simultaneously starts sending messages to port 62001 at address  If NATs A and B assign port numbers to new sessions
       sequentially, and if not much time has passed since the A-S and B-S
       sessions were initiated, then a working bidirectional communication
       channel between A and B should result.  A's messages to B cause NAT A
       to open up a new session, to which NAT A will (hopefully) assign
       public port number 62001, because 62001 is next in sequence after the
       port number 62000 it previously assigned to the session between A and
       S.  Similarly, B's messages to A will cause NAT B to open a new
       session, to which it will (hopefully) assign port number 31001.  If
       both clients have correctly guessed the port numbers each NAT assigns
       to the new sessions, then a bidirectional UDP communication channel
       will have been established as shown below.
    Ford                                                            [Page 9]
    draft-ford-natp2p-00.txt                                      April 2003
                                      Server S
                  |                                             |
                NAT A                                         NAT B
       A-S                        B-S
       A-B                        B-A
                  |                                             |
                  |                                             |
               Client A                                      Client B
       Obviously there are many things that can cause this trick to fail.
       If the predicted port number at either NAT already happens to be in
       use by an unrelated session, then the NAT will skip over that port
       number and the connection attempt will fail.  If either NAT sometimes
       or always chooses port numbers non-sequentially, then the trick will
       fail.  If a different client behind NAT A (or B respectively) opens
       up a new outgoing UDP connection to any external destination after A
       (B) establishes its connection with S but before sending its first
       message to B (A), then the unrelated client will inadvertently
       "steal" the desired port number.  This trick is therefore much less
       likely to work when either NAT involved is under load.  For all of
       these reasons, it is NOT recommended that new applications implement
       this trick; it is described here for purely for historical and
       informational purposes.
    2.5. Simultaneous TCP Connection Initiation
       There is a method that can be used in some cases to establish direct
       peer-to-peer TCP connections between a pair of nodes that are both
       behind NATs.  Most TCP sessions start with one endpoint sending a SYN
       packet, to which the other party responds with a SYN-ACK packet.  It
       is possible and legal, however, for two endpoints to start a TCP
       session by simultaneously sending each other SYN packets, to which
       each party subsequently responds with a separate ACK.  This procedure
       is known as a "simultaneous open."
       If a NAT receives a TCP SYN packet from outside the private network
       attempting to initiate an incoming TCP connection, the NAT will
       normally reject the connection attempt by sending back a TCP RST
       (connection reset) packet.  If, however, the SYN packet arrives with
       source and destination addresses and port numbers that correspond to
       a TCP session that the NAT believes is already active, then the NAT
       will allow the packet to pass through.  In particular, if the NAT has
       just recently seen and transmitted an outgoing SYN packet with the
    Ford                                                           [Page 10]
    draft-ford-natp2p-00.txt                                      April 2003
       same addresses and port numbers, then it will consider the session
       active and allow the incoming SYN through.  If clients A and B can
       each correctly predict the public port number that its respective NAT
       will assign the next outgoing TCP connection, and if each client
       initiates an outgoing TCP connection with the other client timed so
       that each client's outgoing SYN passes through its local NAT before
       either SYN reaches the opposite NAT, then a working peer-to-peer TCP
       connection will result.
       Unfortunately, this trick is even more fragile and timing-sensitive
       than the UDP port number prediction trick described above.  First,
       all the same things can go wrong with each side's attempt to predict
       the public port numbers the NATs will assign to the new sessions.  In
       addition, if either client's SYN arrives at the opposite NAT too
       quickly, then the NAT will reject the SYN with a RST packet, causing
       the local NAT in turn to close the new session.  Finally, even though
       support for simultaneous open is technically a mandatory part of the
       TCP specification [TCP], it is not implemented correctly or at all in
       many common operating systems.  For this reason, this trick is
       likewise mentioned here only for historical interest; it is NOT
       recommended for use by applications.  Applications that require
       efficient, direct peer-to-peer communication should use UDP.
    3. NAT Design Guidelines
       This section discusses considerations for the design of network
       address translators, as they affect peer-to-peer applications.
    3.1. Maintaining Consistent Public/Private Mappings for UDP Ports
       The primary and most important recommendation of this document for
       NAT designers is that the NAT maintain a consistent and stable
       mapping between a given (internal IP address, internal UDP port) pair
       and a corresponding (public IP address, public UDP port) pair for as
       long as any active sessions exist using that mapping.  The NAT may
       filter incoming traffic on a per-session basis, by examining both the
       source and destination IP addresses and port numbers in each packet.
       When a node on the private network starts sending outgoing packets to
       a new external destination, however, while using the same source IP
       address and UDP port as an existing translated UDP session, the NAT
       should ensure that the new UDP session is given the same public IP
       address and UDP port numbers as the existing session.
       One method of implementing this behavior, meant to be only
       illustrative and not prescriptive, is outlined here.  The "critical-
       path" processing performed by a NAT on a packet flowing in either
       direction typically involves extracting the source and destination IP
       addresses from the IP header, and the source and destination TCP/UDP
    Ford                                                           [Page 11]
    draft-ford-natp2p-00.txt                                      April 2003
       port numbers from the TCP/UDP header, and using these four values to
       index into a table of active sessions.  When a packet is received for
       which no entry is found in the session table, the NAT must decide
       whether or not and how to establish a new session, and this is where
       the typical "outgoing sessions only" firewall policy comes into
       If the unknown packet was received from the public network, then it
       is usually dropped (or perhaps rejected with a RST in the case of
       TCP).  If a UDP packet for an unknown session arrives from the
       private network, however, and the NAT is configured in the typical
       way to allow outgoing UDP sessions, the then NAT looks up the
       packet's (source IP address, source UDP port) in a second internal
       table: a "mapping table."  For a given (private IP address, private
       UDP port) on the internal network, the NAT stores in the mapping
       table the corresponding (public IP address, public UDP port) it has
       assigned to represent that private endpoint.  If a mapping already
       exists, the NAT simply uses the existing mapping when constructing
       the new session.  If no such entry is present in the mapping table,
       then the NAT allocates a new public UDP port from its pool and
       creates a new mapping table entry along with the new session.  The
       NAT also maintains with each entry in the mapping table a list or
       count of the active sessions using that mapping, so that it can
       reassign the public UDP port to other purposes once all of the
       outstanding sessions for the mapping are deemed inactive.
    3.1.1. Preserving Port Numbers
       Some NATs, when establishing a new UDP session, attempt to assign the
       same public port number as the corresponding private port number, if
       that port number happens to be available.  For example, if client A
       at address initiates an outgoing UDP session with a datagram
       from port number 1234, and the NAT's public port number 1234 happens
       to be available, then the NAT uses port number 1234 at the NAT's
       public IP address as the translated endpoint address for the session.
       This behavior might be beneficial to some legacy UDP applications
       that expect to communicate only using specific UDP port numbers, but
       it is not recommended that applications depend on this behavior since
       it is only possible for a NAT to preserve the port number if at most
       one node on the internal network is using that port number.
       In addition, a NAT should NOT try to preserve the port number in a
       new session if doing so would conflict with the goal of maintaining a
       consistent mapping between public and private endpoint addresses.
       For example, suppose client A at internal port 1234 has established a
       session with external server S, and NAT A has assigned public port
       62000 to this session because port number 1234 on the NAT was not
       available at the time.  Now suppose port number 1234 on the NAT
    Ford                                                           [Page 12]
    draft-ford-natp2p-00.txt                                      April 2003
       subsequently becomes available, and while the session between A and S
       is still active, client A initiates a new session from its same
       internal port (1234) to a different external node B.  In this case,
       because an active mapping has already been established between client
       A's port 1234 and the NAT's public port 62000, this mapping should be
       maintained and the new session should also use port 62000 as the
       public port corresponding to client A's port 1234.  The NAT should
       NOT assign public port 1234 to this new session just because port
       1234 has become available: that behavior would not be likely to
       benefit the application in any way since the application has already
       been operating with a translated port number, and it would break any
       attempts the application might make to establish peer-to-peer
       connections using the UDP hole punching technique.
    3.2. Maintaining Consistent Public/Private Mappings for TCP Ports
       For consistency with the behavior of UDP translation, it is suggested
       that NATs also maintain a consistent mapping between private and
       public (IP address, TCP port number) pairs for TCP connections, in
       the same way as described above for UDP.  Maintaining consistent
       mappings for TCP ports facilitates the operation of the simultaneous
       TCP open technique, which although not recommended in general for
       peer-to-peer applications, may be useful in controlled situations
       when the two endpoints are sufficiently well synchronized.
       Maintaining TCP endpoint mappings consistently may also increase the
       NAT's compatibility with other applications that initiate multiple
       TCP connections from the same source port.
    3.3. Proxy Protocols
       Besides adopting the above recommendations to make a NAT's basic
       "transparent-mode" operation as peer-to-peer friendly as possible, it
       is helpful for NATs also to support proxy protocols that allow
       applications to request an explicitly managed presence on the public
       side of the NAT.  Unfortunately, several alternative protocols have
       been proposed with varying characteristics [SOCKS, RSIP, MIDCOM,
       UPNP], and as of this writing none of them have achieved clear
       acceptance or dominance in the Internet community.  Furthermore, it
       is not clear yet how well these protocols will work in the
       increasingly common "Twice NAT" situation where clients are located
       behind multiple levels of NAT, especially if the NATs are from
       different vendors, support different features and policies, and are
       under different administrative domains.  (In the common case, one is
       owned and managed by the ISP and the other by the end user.)  For
       these reasons, this document makes no attempt to explore this issue
       in detail or to recommend specific proxy protocols for NATs to
    Ford                                                           [Page 13]
    draft-ford-natp2p-00.txt                                      April 2003
    4. Security Considerations
       Using the UDP hole punching technique in peer-to-peer applications
       and supporting it in NATs should not create any new security issues.
       In particular, the technique does not require a NAT firewall to be
       "promiscuous" in any way about acceping incoming UDP traffic.  As
       long as outgoing UDP sessions are enabled and the firewall maintains
       consistent mappings between internal and external UDP ports, the
       firewall can still filter out all incoming UDP packets except those
       with (source IP, source port, destination IP, destination port)
       tuples that exactly match those of active sessions initiated from
       within the enclave.  Filtering incoming traffic aggressively while
       maintaining consistent mappings thus allows a firewall to be "peer-
       to-peer friendly" without compromising the standard firewall security
       principle of rejecting all unsolicited incoming traffic.
       It might be argued that maintaining a consistent internal/external
       port mapping can "leak" some information to the outside about the
       applications on the internal network, particularly about the
       relationships between different UDP sessions that cross the firewall
       boundary.  If the security requirements are so critical that such a
       subtle information channel is of concern, however, then the firewall
       almost certainly should not be configured to allow unrestricted
       outgoing UDP traffic in the first place.  Controlling information
       flow to this degree generally requires that the firewall only allow
       communication only via tightly-controllable application-level
       gateways, in which case the firewall can either implement the proper
       peer-to-peer communication behavior itself or disallow it entirely.
    [BIDIR]    Peer-to-Peer Working Group, NAT/Firewall Working Committee,
               "Bidirectional Peer-to-Peer Communication with Interposing
               Firewalls and NATs", August 2001.
    [KEGEL]    Dan Kegel, "NAT and Peer-to-Peer Networking", July 1999.
    [MIDCOM]   P. Srisuresh, J. Kuthan, J. Rosenberg, A. Molitor, and
               A. Rayhan, "Middlebox communication architecture and
               framework", RFC 3303, August 2002.
    [NAT-APPL] D. Senie, "Network Address Translator (NAT)-Friendly
               Application Design Guidelines", RFC 3235, January 2002.
    [NAT-PROT] M. Holdrege and P. Srisuresh, "Protocol Complications
               with the IP Network Address Translator", RFC 3027,
    Ford                                                           [Page 14]
    draft-ford-natp2p-00.txt                                      April 2003
               January 2001.
    [NAT-TERM] P. Srisuresh and M. Holdrege, "IP Network Address
               Translator (NAT) Terminology and Considerations", RFC
               2663, August 1999.
    [NAT-TRAD] P. Srisuresh and K. Egevang, "Traditional IP Network
               Address Translator (Traditional NAT)", RFC 3022,
               January 2001.
    [RSIP]     M. Borella, J. Lo, D. Grabelsky, and G. Montenegro,
               "Realm Specific IP: Framework", RFC 3102, October 2001.
    [SOCKS]    M. Leech, M. Ganis, Y. Lee, R. Kuris, D. Koblas, and
               L. Jones, "OCKS Protocol Version 5", RFC 1928, March 1996.
    [STUN]     J. Rosenberg, J. Weinberger, C. Huitema, and R. Mahy,
               "STUN - Simple Traversal of User Datagram Protocol (UDP)
               Through Network Address Translators (NATs)", RFC 3489,
               March 2003.
    [TCP]      "Transmission Control Protocol", RFC 793, September 1981.
    [UPNP]     UPnP Forum, "Internet Gateway Device (IGD) Standardized
               Device Control Protocol V 1.0", November 2001.
    Author's Address
       Bryan Ford
       Laboratory for Computer Science
       Massachusetts Institute of Technology
       77 Massachusetts Ave.
       Cambridge, MA 02139
       Phone: (617) 253-5261
    Full Copyright Statement
       Copyright (C) The Internet Society (2003).  All Rights Reserved.
       This document and translations of it may be copied and furnished to
       others, and derivative works that comment on or otherwise explain it
       or assist in its implementation may be prepared, copied, published
       and distributed, in whole or in part, without restriction of any
       kind, provided that the above copyright notice and this paragraph are
    Ford                                                           [Page 15]
    draft-ford-natp2p-00.txt                                      April 2003
       included on all such copies and derivative works.  However, this
       document itself may not be modified in any way, such as by removing
       the copyright notice or references to the Internet Society or other
       Internet organizations, except as needed for the purpose of
       developing Internet standards in which case the procedures for
       copyrights defined in the Internet Standards process must be
       followed, or as required to translate it into languages other than
       The limited permissions granted above are perpetual and will not be
       revoked by the Internet Society or its successors or assigns.
       This document and the information contained herein is provided on an
    Ford                                                           [Page 16]