Category Archives: Engineering notes

Learning SWIG

Warning, I don’t know SWIG. Here’s my notes as a total neophyte.

All posts in the series

  1. Learning SWIG
  2. Anatomy of SWIG
  3. Automatic garbage collection of opaque pointers in SWIG

Here’s a simple one from sql.h

   %module sql
   SQLRETURN SQL_API SQLAllocEnv(SQLHENV *OUTPUT);

The above is clearly not going to work without some defines

   %module sql
   
   typedef short           SQLSMALLINT;
   typedef SQLSMALLINT     SQLRETURN;
   /* SQL_API is __stdcall on Windows */
   #define SQL_API
   
   SQLRETURN SQL_API SQLAllocEnv(SQLHENV *OUTPUT);

Taking a look at the generated code, and it is incorrect because we want to call the function without arguments. i.e. in python env = SQLAllocEnv().

if (!PyArg_ParseTuple(args,(char *)"O:SQLAllocEnv",&obj0)) SWIG_fail;

Sounds like we’ll need one of those dreaded typemaps. Here’s the result after a few attempts. The plan is as follows, we’ll define a local SQLHENV called env, and then assign it’s address to arg1, which will be passed to SQLAllocEnv. Observe the absence of semicolons.

%module sql

typedef short           SQLSMALLINT;
typedef SQLSMALLINT     SQLRETURN;
/* SQL_API is __stdcall on Windows */
#define SQL_API

%typemap(in, numinputs=0) SQLHENV *OUTPUT
(
   SQLHENV env
)
{
   $1 = &env
}

SQLRETURN SQL_API SQLAllocEnv(SQLHENV *OUTPUT);

The result looks a bit better.

SQLHENV *arg1 = (SQLHENV *) 0 ;
SQLHENV env1 ;
SQLRETURN result;

{
  arg1 = &env1
}
if (!PyArg_ParseTuple(args,(char *)":SQLAllocEnv")) SWIG_fail;
result = (SQLRETURN)SQLAllocEnv(arg1);
resultobj = SWIG_From_short((short)(result));
return resultobj;

We’re passing in the right arguments, but we are returning the SQLRETURN. SQLRETURN is an error code, which we should check and probably raise an exception. But first, we should probably cast the SQLHENV to a pointer type. (My preference would be a reference-counted pointer type, so that we can call SQLDealloc automatically, but I don’t know how yet).

To create a pointer to the SQLHENV, I used the SWIG_NewPointerObj incantation. Where did I get this from? I cheated. I created a prototype like this SQLHENV Dummy(); and observed what Swig generated. This is almost certainly not the right way to do this.

%typemap(argout) SQLHENV *OUTPUT
{
   $result = SWIG_NewPointerObj((SQLHENV *)memcpy((SQLHENV *)malloc(sizeof(SQLHENV)),$1,sizeof(SQLHENV)), SWIGTYPE_p_SQLHENV, SWIG_POINTER_OWN |  0 );
}

However, the wrapper code generated is almost decent.

result = (SQLRETURN)SQLAllocEnv(arg1);
resultobj = SWIG_From_short((short)(result));
{
  resultobj = SWIG_NewPointerObj((SQLHENV *)memcpy((SQLHENV *)malloc(sizeof(SQLHENV)),arg1,sizeof(SQLHENV)), SWIGTYPE_p_SQLHENV, SWIG_POINTER_OWN |  0 );
}

It would be a good time to check for exceptions.

/* I found it necessary to add the following include */
%include "exception.i"
%typemap(ret) SQLRETURN
{
   if (!SQL_SUCCEEDED($1)) SWIG_exception(SWIG_RuntimeError, "failed!");
}

Which gives us the following:

 #ifdef __cplusplus
 extern "C" {
 #endif
 SWIGINTERN PyObject *_wrap_SQLAllocEnv(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
   PyObject *resultobj = 0;
   SQLHENV *arg1 = (SQLHENV *) 0 ;
   SQLHENV env1 ;
   SQLRETURN result;

   {
     arg1 = &env1
   }
   if (!PyArg_ParseTuple(args,(char *)":SQLAllocEnv")) SWIG_fail;
   result = (SQLRETURN)SQLAllocEnv(arg1);
   resultobj = SWIG_From_short((short)(result));
   {
     resultobj = SWIG_NewPointerObj((SQLHENV *)memcpy((SQLHENV *)malloc(sizeof(SQLHENV)),arg1,sizeof(SQLHENV)), SWIGTYPE_p_SQLHENV, SWIG_POINTER_OWN |  0 );
   }
   {
     if (FAILED(result)) SWIG_exception(SWIG_RuntimeError, "failed!");
   }
   return resultobj;
 fail:
   return NULL;
 }

I don’t really like the SWIG_NewPointerObj bit. It seems a little too error-prone. Instead, I investigated an alternate approach of letting SWIG generate all its typemaps.

%module sql

typedef short           SQLSMALLINT;
typedef SQLSMALLINT     SQLRETURN;
/* SQL_API is __stdcall on Windows */
#define SQL_API

%rename(SQLAllocEnv) SQLAllocEnv_new;
%inline %{
SQLHENV SQLAllocEnv_new()
{
    SQLHENV hEnv;
    SQLRETURN result = SQLAllocEnv(&hEnv);
    if (SQL_SUCCEEDED(result))
        return hEnv;
}
%}

The result is better, but now I’ve lost my exception handling capability.

SWIGINTERN PyObject *_wrap_SQLAllocEnv(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  SQLHENV result;

  if (!PyArg_ParseTuple(args,(char *)":SQLAllocEnv")) SWIG_fail;
  result = SQLAllocEnv_new();
  resultobj = SWIG_NewPointerObj((SQLHENV *)memcpy((SQLHENV *)malloc(sizeof(SQLHENV)),&result,sizeof(SQLHENV)), SWIGTYPE_p_SQLHENV, SWIG_POINTER_OWN |  0 );
  return resultobj;
fail:
  return NULL;
}

With a little help from StackOverflow, we are now closer to a final solution.


%module sql

typedef short           SQLSMALLINT;
typedef SQLSMALLINT     SQLRETURN;
/* SQL_API is __stdcall on Windows */
#define SQL_API

%exception SQLAllocEnv_new {
    $action;
    if (!SQL_SUCCEEDED($1)) SWIG_exception(SWIG_RuntimeError, "failed!");
}

%rename(SQLAllocEnv) SQLAllocEnv_new;
%inline %{
SQLHENV SQLAllocEnv_new()
{
    SQLHENV hEnv;
    SQLRETURN result = SQLAllocEnv(&hEnv);
    if (SQL_SUCCEEDED(result))
        return hEnv;
}
%}

This gives us an almost equivalent result to the first attempts.

SQLHENV SQLAllocEnv_new()
{
    SQLHENV hEnv;
    SQLRETURN result = SQLAllocEnv(&hEnv);
    if (SQL_SUCCEEDED(result))
        return hEnv;
}

#ifdef __cplusplus
extern "C" {
#endif
SWIGINTERN PyObject *_wrap_SQLAllocEnv(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  SQLHENV result;
  
  if (!PyArg_ParseTuple(args,(char *)":SQLAllocEnv")) SWIG_fail;
  {
    result = SQLAllocEnv_new();;
    if (!SQL_SUCCEEDED($1)) SWIG_exception(SWIG_RuntimeError, "failed!");
  }
  resultobj = SWIG_NewPointerObj((SQLHENV *)memcpy((SQLHENV *)malloc(sizeof(SQLHENV)),&result,sizeof(SQLHENV)), SWIGTYPE_p_SQLHENV, SWIG_POINTER_OWN |  0 );
  return resultobj;
fail:
  return NULL;
}

Using Reactive Extensions to Delay Searches

Here’s some notes from experimenting with Reactive Extensions to clean up an otherwise messy tangle of state and event handlers.

The result required is a TextBox which acts as a filter on a set of results. However, we do not want the filter to execute immediately after typing, but instead wait until there are no more KeyDown events for a second or so.

Here’s a step by step account: (You will require the Reactive Extensions, downloadable via NuGet)


       IDisposable keyPressSubscription = null;

       public void InitSearchBox()
        {
            // Show a label 'Searching...' only when filtering is being performed
            // Hide the label for now
            this.SearchingLabel.Visibility = Visibility.Collapsed;

            // Create a stream of keyPressEvents
            var keyPressEvents =
                Observable.FromEventPatter<KeyEventHandler, KeyEventArgs>(
                (KeyEventHandler ev) => { SearchTextBox.KeyDown += ev; },
                (KeyEventHandler ev) => { SearchTextBox.KeyDown -= ev; });

            // Throttle the events so that we don't get more than one 
            // event every 1500 milliseconds. This will hold back the event
            // until there is a momentary pause in key presses.
            keyPressSubscription = keyPressEvents
                .Throttle(TimeSpan.FromMilliseconds(1500))
                .ObserveOnDispatcher()
                .Subscribe(InitiateSearch);
        }

        public void InitiateSearch(EventPattern<KeyEventArgs> e)
        {
            this.Searching.Visibility = Visibility.Visible;

            // Simulate searching for 1 second, then
            // hide the 'searching...' notification
            Observable.Timer(TimeSpan.FromMilliseconds(1000))
                .ObserveOnDispatcher()
                .Take(1)  // Timer is an infinite stream. By specifying we are 
                          // only interested in one value, this allows the timer
                          // to be disposed.
                .Subscribe((e) =>
                {
                    this.Searching.Visibility = Visibility.Collapsed;
                });
            
        }

        // IDisposable
        public void Dispose()
        {
           // Prevent memory leaks
           keyPressSubscription.Dispose();
        }

Fatal: git-http-push failed

I had trouble pushing changes to one of my git repositories. The error message was:

Fatal: git-http-push failed

Looking at server logs

60.xxx.xxx.205 - xxx.xxx [06/Mar/2012:21:37:56 -0500] "MKCOL /git/Test.git/objects/77/ HTTP/1.1" 403 489

It appears that file permissions were the issue.

In my case, the Git repository was created using virtualmin, and the owner was www-data. CGI programs were run under suexec, where the program was executed using the file owner’s permissions. For suexec to work, the owner needs write permissions to the directories. The following was sufficient to do the trick.

sudo chmod -R u+w Test.git

References:
Problem with HTTP Verbs
How to troubleshoot git server issues

How to block bots from seeking exploits on your web server with Fail2Ban

Source: Exploit scanner, secure your Apache

Steps:

  • sudo apt-get install fail2ban
  • edit /etc/fail2ban/jail.conf
    [apache-nohome]
    enabled = true
    port    = http,https
    filter  = apache-nohome
    logpath = /var/log/apache*/*error.log
    maxretry = 5
    
  • fix a bug in the filter /etc/fail2ban/filter.d/apache-nohome.conf
    failregex = [[]client (?P<host>\S*)[]] File does not exist:
    
  • Restart /etc/init.d/fail2ban restart
  • Check that fail2ban has started.
     sudo fail2ban-client status
    Status
    |- Number of jail:      2
    `- Jail list:           apache-nohome, ssh
    

Daydreaming about tablet development

I was day dreaming about an image-based development environment on the tablet where the developer works on a PC, and the commands are transmitted to the client via something like WebSockets…, and the client then renders the controls and the developer can iterate quickly by pushing more code down, e.g. load some scripting language on the fly.

Then I woke up and realized the key idea already exists. It is called web development.

Migrating from default profile provider to SqlTableProfileProvider in ASP.net

Steps

  1. Prepare database. Create a table called aspnet_Profile2. It must have at least two columns:
    1. UserID (uniqueidentifier – Primary Key)
    2. LastUpdatedDate (datetime)
    3. The rest of the columns are your profile columns
  2. Modify Web.config (see sample below)
  3. Write a migration script (see sample below)
  4. Modify Web.config one more time, renaming JobTitle2 to JobTitle, and delete the default provider

Web.config

   
        
            
        
      
          
          
          
        
        
        
     
    

Migration script

        Dim pc As ProfileInfoCollection = ProfileManager.GetAllProfiles(ProfileAuthenticationOption.All)
        For Each pi As ProfileInfo In pc
            Dim prof As Object = ProfileBase.Create(pi.UserName)
            prof.FriendlyName2 = prof.FriendlyName
            prof.JobTitle2 = prof.JobTitle
            prof.Phone2 = prof.Phone
        Next

Web.config

   
        
            
            
        
      
        
        
        
     
    

Notes on working with WinForms DataBinding

  1. Always handle BindingSource.ListChanged event to avoid unpleasant surprises with BindingSource.Current changing on its own upon EndEdit.
  2. In master detail, call master binding source EndEdit() before calling detail binding source EndEdit
  3. Use ComboBox SelectionChangeCommitted instead of SelectedIndexChanged. The latter will fire during initial databinding
  4. To set default values after BindingSource.AddNew() is called, use the AddingNewEvent
     AddingNewEventHandler h = (o, e) =&gt;
                {
                    WorkDataSet.SessionRow session = (sessionBindingSource.List as DataView).AddNew().Row as WorkDataSet.SessionRow;
                    session.UserID = Program.Server.UserID;
                    session.SessionID = Guid.NewGuid().ToString();
                    session.Duration = 0.5M;
                    sessionBindingSource.MoveLast();
                    e.NewObject = session;
                };
    
                sessionBindingSource.AddingNew += h;
                sessionBindingSource.AddNew();
                sessionBindingSource.AddingNew -= h;