Reading on websockets the other day I ended in this article about websockets and vnc and it draw my attention immediately over the showmethedesktop project.

Now, given my interest on async servers and my recent interest on their implementation on python along with the rustiness of my perl I decided to try a reimplementation on async python as a way to introduce me to the gevent framework.

First impressions

• It is actually feasible to draw a whole screen using just Javascript through canvas and (put/get)ImageData
• Some browsers are remarkable faster than others doing this, chrome 15 has a 800×600 draw time of 0-1 msec while Firefox 7 has about 5-13 msec draw time
• It is perfectly usable unless you want to play videos or something like that

Project

Available in github, comments and patches are welcome! still in development though.

How did I do it

Very quickly actually, thanks to python and gevent, and specially thanks to other projects I took and adapted code from, such as showmethedesktop, python-vnc-viewer and gevent-websocket.

• Most of the html+javascript is just copied from showmethedesktop with some adaptations of my own
• I forked gevent-websocket on bitbucket and implemented the new hybi WS protocol on it, it is actually the only thing that current firefox and chrome speak so there was no option there, I also introduced some other small changes
• I copied all the rfb(Remote Frame Buffer, VNC protocol) implementation from python-vnc-viewer and then removed the twisted dependency and adapted everything to work on gevent.
• Finally thanks to gevent I created the “main” of my app, really just a gevent app

Issues

Gevent and greenlet have some kind of incompatibility on python 2.7, requirements.txt had to include this version http://bitbucket.org/snaury/greenlet/get/tip.zip#egg=greenlet

Howto

git clone git://github.com/arkaitzj/wsvnc.git &&
cd wsvnc &&
virtualenv --no-site-packages sandbox &&
sandbox/bin/pip install -r requirements.txt &&
sandbox/bin/python wsvnc.py


#adduser my_user cdrom

But this is not end of story. In order to get effective group permissions we need to logout and login again in the system for our group permissions to be refereshed, which is literaly a pain in the ass sometimes.

There is a quick trick that will save our time here, just do:

$newgrp cdrom

If we have been added to the group newgrp will give us instantly group permissions for the given group.
For mor info on the subject:

$man newgrp

// g++ 4.1
#include <string>
#include <iostream>
#include <tr1/tuple>
using std::tr1::tuple;
using std::tr1::tie;
using std::tr1::make_tuple;

tuple<std::string,int> parse_url(const std::string& url){
    return make_tuple("127.0.0.1",9000); //Cheat for parsing
}
int main(){
    std::string host;
    int port;
    tie(host,port) = parse_url("127.0.0.1:9000");
    std::cout << "Host: "<<host<< ":"<<port<<"\n";
}

At the first look, returning several values from a function/method in C++ may look awkward and contrary to the C++ way, being more usual in languages like python or perl.
However the fact is that template metaprogramming gives us the ability to do such good tricks and renders the rest for us in compile-time. This tuple templates are part of the TR1 and thats why we need to include them from <tr1/tuple> and use them from std::tr1 namespace, but they’ll be part of the next standard c++0x.

The logic behind the metaprogramming is simple, by the time you compile a tuple the compiler knows perfectly the size of the new type because you supply the template parameters.

tie() for example is a template that gets the given parameters and put references to them in an lvalue obj that can be assigned to, so its a painless way of getting results from a tuple_returning method.
make_tuple() goes the same way but creates a new tuple with copies of the provided parameters, so it is a quick way of creating tuples of any size, its like the current make_pair() but for tuples.

#pagebar { width:955px; }
#page { width:955px; }
#content { width:700px; }
#sidebar { margin:-60px 0 0 750px; }
#headerimg {
background-image:url('http://s2.wordpress.com/wp-content/themes/pub/contempt/images/blue_flower/head.jpg');
background-repeat:repeat-x;
}
#footer {width:955px; }

class Singleton{   
    Singleton(){
        // Thread-unsafe code
    }
    static Singleton* getInstance(){
        static Singleton instance;
        return &instance;
    }
};

This essentially reserves static memory for the size of the instance and executes the class constructor during the first call to getInstance(), this of course occurs only if we are dealing with a non-POD object.

The problem here is that we might run in problems if we get inside getInstance with 2 threads at the same time for the first time. Obviously the constructor is not thread-safe per se.

However, GCCs g++ gives us a help here and protects every local static variable construction with thread-safe guards, __cxa_guard_acquire and __cxa_guard_release do the job here protecting the constructor. Note that this only happens with local statics as global static construction occurs before any function of the translation unit gets executed.

Differences are obvious if we take a look at the generated code, PODs are just data so no construction is needed, non-PODs are properly protected

POD type, function body highlighted:

class pod_class {    public: 
        static pod_class* instance(){ 
            static pod_class once; 
            return &once;
        }};
int main(){ return (int)pod_class::instance; }

080483fd <_ZN9pod_class8instanceEv>:
 80483fd:       55                push   %ebp
 80483fe:       89 e5             mov    %esp,%ebp
 8048400:       b8 f8 95 04 08    mov    $0x80495f8,%eax
 8048405:       5d                pop    %ebp
 8048406:       c3                ret          

class nonpod_class {    public:
        nonpod_class(){}
        static nonpod_class* instance(){
            static nonpod_class once; return &once;
        }};
int main(){ return (int)nonpod_class::instance; }

080484b3 <_ZN12nonpod_class8instanceEv>:
 80484b3:  55                    push   %ebp
 80484b4:  89 e5                 mov    %esp,%ebp
 80484b6:  83 ec 08              sub    $0x8,%esp
 80484b9:  b8 f0 96 04 08        mov    $0x80496f0,%eax
 80484be:  0f b6 00              movzbl (%eax),%eax
 80484c1:  84 c0                 test   %al,%al
 80484c3:  75 2d                 jne    80484f2 <_ZN12nonpod_class8instanceEv+0x3f>
 80484c5:  c7 04 24 f0 96 04 08  movl   $0x80496f0,(%esp)
 80484cc:  e8 c7 fe ff ff        call   8048398 <__cxa_guard_acquire@plt>
 80484d1:  85 c0                 test   %eax,%eax
 80484d3:  0f 95 c0              setne  %al
 80484d6:  84 c0                 test   %al,%al
 80484d8:  74 18                 je     80484f2 <_ZN12nonpod_class8instanceEv+0x3f>
 80484da:  c7 04 24 f8 96 04 08  movl   $0x80496f8,(%esp)
 80484e1:  e8 c8 ff ff ff        call   80484ae <_ZN12nonpod_classC1Ev>
 80484e6:  c7 04 24 f0 96 04 08  movl   $0x80496f0,(%esp)
 80484ed:  e8 d6 fe ff ff        call   80483c8 <__cxa_guard_release@plt>
 80484f2:  b8 f8 96 04 08        mov    $0x80496f8,%eax
 80484f7:  c9                    leave         
 80484f8:  c3                    ret           

This behaviour can be avoided by using -fno-threadsafe-statics in the g++s commandline.

EDIT: Thanks to Tom in the comment below I updated the post with the involved code_comment from gcc project
http://gcc.gnu.org/viewcvs/branches/gcc-4_2-branch/gcc/cp/decl.c?view=markup&pathrev=129388

/* Emit code to perform this initialization but once. This code looks like:
static guard;
if (!guard.first_byte) {
    if (__cxa_guard_acquire (&guard)) {
        bool flag = false;
        try {
            // Do initialization.
            flag = true; __cxa_guard_release (&guard);
            // Register variable for destruction at end of program.
        } catch {
            if (!flag) __cxa_guard_abort (&guard);
        }
}

Basics

I’ve been recently playing around with lambda function support and looks pretty stable in the gcc compiler. I haven’t tried but I read that VC2010 has lambda support as well.

It gives support for in-place anonymous function definitions with context access, which basically helps us in a lot of situations, for example, if a given function, say wait_5minutes_call(callback_function), requires a callback we could easily do:

wait_5minutes_call( [ ](){ std::cout << "Callback called\n";} );

#include <iostream>
#include <functional>
#include <algorithm>
void print_function(int size, std::function funct){
    std::cout << funct(size) << std::endl;
}
int main(int argc, char* argv[])
{
    auto myfnc = [](int size){return size*size;};
    print_function(10,myfnc);
    return 0;
}

[howtogetcontext](ptype parameter){body} // implicit return type resolution
[howtogetcontext](ptype parameter) ->returntype {body} // explicit return type resolution

What is the context?

The context are the variables scoped in the place we define the lambda function.
Yes, we can access those variables even if we pass the lambda function far away of the current scope definition. Of course if any of those variables went out-of-scope or was freed and we refer to them by reference we’ll get a big crash, there’s no magic .
We can access everything that is in scope on definition, locals, even this pointer if the function was defined inside an object method, actually, any method defined inside an class method is a friend of the class.

Ok, how do I get those cool lambda functions?

Easy.
We need to download the gcc development branch for lambda, as the trunk doesn’t have lambda support.

svn co svn://gcc.gnu.org/svn/gcc/branches/cxx0x-lambdas-branch gcc-lambda
mkdir build
cd build
# Compilation can take several hours, depends on your machine
../gcc-lambda/configure --program-prefix=lambda --disable-libgcj --enable-languages=c++ --disable-werror
make -j3
make install

Now you have a lambdag++ binary in your system which you can use to compile c++ code with lambdas just like this:

lambdag++ --std=g++c0x main.cpp -o main

Don’t forget the –std=c++0x

#!/usr/bin/perl
use warnings;
use strict;
use LWP::Simple;

my $url="http://stackoverflow.com/feeds/tag/c%2b%2b";
my $max = 5;

my %newshash;
while(1){

    my @news;
    my @titles;
    my @urls;

    my $feed = get $url;
    if(not $feed){
        my $exec = "notify-send -t 10000 \"Stackoverflow C++\" \"Can't reach $url\"";
        system($exec);
        sleep(600);
        next;
    }
    @urls = $feed =~ m/<id>(.+)<\/id>/g;
    @titles = $feed =~ m/<title type="text">(.+)<\/title>/g;
    for( my $i=0; $i<(scalar @titles) ; $i++){
        push(@news,[$titles[$i],$urls[$i]]);
    }
    shift(@news);

    my $printed = 0;
    for my $new (@news){
        my($id) = $new->[1] =~ m/questions\/([0-9]+)\// or print "Faulty id in $new->[1]";

        if( not exists($newshash{$id})){
            $newshash{$id} = $new-[1];
            if($printed<$max){
                my $exec = "notify-send -t 10000 'Stackoverflow C++'  '<a href=\"".$new->[1]."\">".$new->[0]."</a>'";
                system($exec);
                print $new->[0] . "\n";
            }
            $printed++;
       }
    }
    sleep 300;
}

struct tm _tmbuf;
/* Return the `struct tm' representation of *T in local time,
   using *TP to store the result.  */
struct tm *
__localtime_r (t, tp)
     const time_t *t;
     struct tm *tp;
{
  return __tz_convert (t, 1, tp);
}
weak_alias (__localtime_r, localtime_r)

/* Return the `struct tm' representation of *T in local time.  */
struct tm *
localtime (t)
     const time_t *t;
{
  return __tz_convert (t, 1, &_tmbuf);
}
libc_hidden_def (localtime)

The localtime representation is returned in a chunk of memory than in the case of localtime_r is given by the user but in the case of the standard localtime every call returns the result in the same memory chunk, currently it uses the file scoped variable _tmbuf in localtime compilation unit.
So simultaneous calls to the localtime will produce non-desired results in every real concurrency scenario.

Required tools and libs

Prepare a place for the PSP development stuff:

mkdir -p $HOME/lib/pspdev

Make it available for your current session:

echo -e "export PSPDEV=~/lib/pspdev\nexport PATH=\$PATH:\$PSPDEV/bin" >> $HOME/.bashrc
bash

Get and install the toolchain:

svn co svn://svn.ps2dev.org/psp/trunk/psptoolchain
cd psptoolchain
./toolchain.sh

It’ll probably ask for some develpment packages, just look for them in apt, install them and run the last step again.

Get and install psp libraries

svn co svn://svn.ps2dev.org/psp/trunk/psplibraries
cd psplibraries
./libraries.sh

Ok, requirements installed

Hello world

main.cpp

#include <pspkernel.h>
#include <pspdebug.h>

PSP_MODULE_INFO("Hello World", 0, 1, 1);

#define printf pspDebugScreenPrintf 

/* Exit callback */
int exit_callback(int arg1, int arg2, void *common) {
          sceKernelExitGame();
          return 0;
}

/* Callback thread */
int CallbackThread(SceSize args, void *argp) {
          int cbid;

          cbid = sceKernelCreateCallback("Exit Callback", exit_callback, NULL);
          sceKernelRegisterExitCallback(cbid);

          sceKernelSleepThreadCB();

          return 0;
}

/* Sets up the callback thread and returns its thread id */
int SetupCallbacks(void) {
    int thid = 0;

    thid = sceKernelCreateThread("update_thread", CallbackThread, 0x11, 0xFA0, 0, 0);
    if(thid &gt;= 0) {
        sceKernelStartThread(thid, 0, 0);
    }

    return thid;
}

int main() {
    pspDebugScreenInit();
    SetupCallbacks();
    printf("Hello World");
    sceKernelSleepThread();
    return 0;
}

Makefile

TARGET = hello
OBJS = main.o

CFLAGS = -O2 -G0 -Wall
CXXFLAGS = $(CFLAGS) -fno-exceptions -fno-rtti
ASFLAGS = $(CFLAGS)

EXTRA_TARGETS = EBOOT.PBP
PSP_EBOOT_TITLE = Hello World

PSPSDK=$(shell psp-config --pspsdk-path)
include $(PSPSDK)/lib/build.mak

Ok, now compile it

saladino@Zack:~/projects/pspstuff$ make
psp-g++ -I. -I/home/saladino/lib/pspdev/psp/sdk/include -O2 -G0 -Wall -I. -I/home/saladino/lib/pspdev/psp/sdk/include -O2 -G0 -Wall -fno-exceptions -fno-rtti -D_PSP_FW_VERSION=150   -c -o main.o main.cpp
psp-gcc -I. -I/home/saladino/lib/pspdev/psp/sdk/include -O2 -G0 -Wall -D_PSP_FW_VERSION=150  -L. -L/home/saladino/lib/pspdev/psp/sdk/lib   main.o  -lpspdebug -lpspdisplay -lpspge -lpspctrl -lpspsdk -lc -lpspnet -lpspnet_inet -lpspnet_apctl -lpspnet_resolver -lpsputility -lpspuser -lpspkernel -o hello.elf
psp-fixup-imports hello.elf
mksfo 'Hello World' PARAM.SFO
psp-strip hello.elf -o hello_strip.elf
pack-pbp EBOOT.PBP PARAM.SFO NULL  \
		NULL NULL NULL  \
		NULL  hello_strip.elf NULL
[0]        408 bytes | PARAM.SFO
[1]          0 bytes | NULL
[2]          0 bytes | NULL
[3]          0 bytes | NULL
[4]          0 bytes | NULL
[5]          0 bytes | NULL
[6]      77508 bytes | hello_strip.elf
[7]          0 bytes | NULL
rm -f hello_strip.elf

That’s all, now copy it to the corresponding folder in your memory stick, in my case this was psp/game380/helloworld/EBOOT.PBP

• We need 2 repos, one for git and one for cvs but we don’t touch cvs one ever

• repos/MODULENAME
• cvs/MODULENAME

git cvsimport -d server:/path/to/cvs -C repo/MODULENAME -r cvs -k MODULENAME
cvs checkout -d cvs/MODULENAME MODULENAME

Create a new branch to work

git checkout -b non-master-branch-name

Tweaks for automatic updating

modulename, cvs and the $CVSROOT contents

git config cvsimport.module MODULENAME
git config cvsimport.r cvs
git config cvsimport.d server:/path/to/cvs

Updating the code

git checkout master
git cvsimport
git checkout non-master-branch-name
git rebase master

Use

• Create another branch, DON’T WORK ON MASTER branch,use it only for updates, rebases and commits.
• Steps:
  
  • Modify the code in non-master branch and commit there
  • We may have several commits in our non-master branch, some of them we want them in master and others maybe not, we need to tell to cvsexportcommit from which commit to which commit we want to export, so lets say the master and non-master branchs were synced and that we commited a single commit to the non-master branch and we want it in the cvs
  • First we need its commit-id and the commit-id of the previous one

  arkaitzj@zack:~/repos/module$ git log  --pretty=oneline -n2
  60e3c17262aa67adece8070e1b47e819353b778c Test improvements                           #Last commit
  3ee1411a952b81ff70bcbf0219852b70cccd67f0    Softwaretest needed Makefiles added  #Previous commit, exists in both branches

• Now we export to cvs the changes on those commit

• -w parameter marks the path of the cvs checked out code, no need to touch it but needs to be there for cvsexportcommit
• -u ensures the cvs repo is updated before diffs applied
• -p makes diff reject dubious patches Remember to always update before exportcommit
• -c if the patch was cleanly applied it is committed to cvs automatically

arkaitzj@zack:~/repos/modulename$ git cvsexportcommit -w ../../cvs/modulename -u -p -c   3ee1411a952b81ff70bcbf0219852b7 60e3c17262aa67adece8070e1b47e

• Committed, next time we do a cvs import on master branch we’ll get the changes from the CVS server, but still we need an extra step
• As cvsimport and cvsexportcommit don’t play well for the moment they don’t recognize each other commits, so once we’ve committed something from non-master branch we still have to do 3 things, remove the commit from non-master branch, cvsimport the last change to master branch and rebase non-master branch to master

Let 10 minutes pass from the cvsexportcommit to the cvsimport, if you don’t see your commit on master history after cvsimport wait a little more and cvsimport again

#non-master branch
git rebase -i HEAD~2 # Find the last commit(what we sent to CVS) and remove that line, this will erase that commit form history
#Checkout master branch and cvsimport
git checkout master
git cvsimport
#Now we have imported our commit from the CVS and it is valid history of the master branch, checkout non-master and rebase
git checkout non-master
git rebase master

• Use this script, it’ll warn you on commits and you’ll know wich branch are you committing to. place it in DEST_DIRECTORY/.git/hooks/pre-commit

arkaitzj@zack:~/repos/module$ cat .git/hooks/pre-commit
#!/bin/bash
NC='\e[0m' # No Color
RED='\e[1;31m'
BRANCHNAME=$(git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/')

exec < /dev/tty
echo -e "Committing to branch $RED$BRANCHNAME$NC. Commit (y/n): "
read response

if [ "$response" == "y" ]
then
       echo "Committing...."
       exit 0
else
       echo "Commit ABORTED"
       exit 1
fi